Jakob Ackermann
0840700ffd
[Router] validate the callback argument
2020-06-05 10:59:01 +01:00
Jakob Ackermann
c6d08647c7
[misc] socket.io: use a secondary publicId for public facing usages
2020-06-04 17:18:07 +01:00
Jakob Ackermann
41debfae0f
[ChannelManager] rework (un)subscribing to redis
...
- send a subscribe request on every request
- wait for a pending unsubscribe request before subscribing
- wait for a pending subscribe request before unsubscribing
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2020-05-15 18:34:33 +02:00
Jakob Ackermann
b713beb7f0
Merge pull request #135 from overleaf/jpa-skip-leave-project-for-invalid-clients
...
[WebsocketController] skip leaveProject when joinProject didn't complete
2020-05-13 15:34:41 +02:00
Jakob Ackermann
55af5e502f
[WebsocketController] skip leaveProject when joinProject didn't complete
...
Also drop dead code:
- user_id bailout
There is a check on a completed joinProject call now. It will always
set a user_id, see Router.coffee which has a fallback `{_id:"..."}`.
- late project_id bailout
WebsocketLoadBalancer.emitToRoom will not work without a project_id.
We have to bail out before the call.
2020-05-12 17:15:08 +02:00
Jakob Ackermann
684cb3903c
[WebsocketController] handle redis subscribe error on joinProject
...
joinProject should not complete when the redis pub/sub subscribe request
failed.
2020-05-12 13:03:50 +02:00
Brian Gough
5c28da1031
add metric for pendingUpdates queue
2020-04-07 11:53:54 +01:00
Jakob Ackermann
56628a16c6
[misc] track redis pub/sub payload sizes on publish
2020-03-30 11:31:44 +02:00
Jakob Ackermann
af53d3b603
[misc] skip duplicate JSON serialization for size check
2020-03-24 11:22:28 +01:00
Jakob Ackermann
cb675d38c2
[misc] SafeJsonParse: align the size limit with the frontend->rt limit
...
frontend -> real-time and doc-updater -> real-time should be in sync.
Otherwise we can send a payload to doc-updater, but can not receive the
confirmation of it -- and the client will send it again in a loop.
Also log the size of the payload.
2020-03-24 09:14:15 +01:00
Jakob Ackermann
15244a54be
[misc] WebsocketController: limit the update size to 7mb
...
bail out early on -- especially do not push the update into redis for
doc-updater to discard it.
Confirm the update silently, otherwise the frontend will send it again.
Broadcast a 'otUpdateError' message and disconnect the client, like
doc-updater would do.
2020-03-24 09:12:12 +01:00
Brian Gough
64bd739a87
Revert "Merge pull request #91 from overleaf/spd-trycatch-all-the-things"
...
This reverts commit 2bf7f14f9d050c58f141f465633bb6e274b903dd, reversing
changes made to 989240812532ca43a52513339f4dda8f44a80a64.
2020-02-05 10:05:36 +00:00
Simon Detheridge
9fd4699098
Merge remote-tracking branch 'origin/bg-avoid-emit-when-client-not-connected' into spd-trycatch-all-the-things
2020-02-04 12:35:35 +00:00
Simon Detheridge
216a977922
Add try/catch around all client emissions
2020-02-04 12:13:03 +00:00
Brian Gough
ebb83e4633
use diconnected property, not connected
2020-02-04 11:59:25 +00:00
Brian Gough
7380d523d5
avoid emitting when client not connected
...
the emit is happening asynchronously after the client list is computed,
so clients may have disconnected in the intervening time.
2020-02-04 11:39:37 +00:00
Brian Gough
4f94110225
Merge pull request #84 from overleaf/spd-catch-errors
...
Bump to Node 10 and add error handlers for socket.io
2020-02-03 14:59:04 +00:00
Brian Gough
49a8e1214b
use a separate field for client errors
2020-02-03 14:47:45 +00:00
Brian Gough
e04b6e1e49
Update app/coffee/Router.coffee
...
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
2020-02-03 14:46:14 +00:00
Shane Kilkelly
6df88ebc49
Filter "comments" if restricted user.
2019-10-30 13:54:40 +00:00
Shane Kilkelly
403caa65e8
Revert "Revert "Track the isRestrictedUser flag on clients""
...
This reverts commit 651e392a7c644403f199e1b03e7494b61ce71d0c.
2019-10-30 13:52:36 +00:00
Simon Detheridge
925a8651c1
Revert "Track the isRestrictedUser flag on clients"
2019-10-22 10:17:38 +01:00
Simon Detheridge
7543f2fcbd
Catch errors from socket.io and attempt graceful cleanup
2019-10-17 12:45:56 +01:00
Shane Kilkelly
2cc2be3d9c
send messages to clients with concurrency of 2
2019-10-11 10:01:21 +01:00
Shane Kilkelly
06aa578bdc
Make it an error when we get no data from joinProject
2019-10-11 09:57:16 +01:00
Shane Kilkelly
df6cd4a054
Also block getConnectedUsers for restricted users.
...
Plus refactor to use a pass list instead of a deny list.
2019-10-04 13:41:49 +01:00
Shane Kilkelly
6765d03339
Track the isRestrictedUser flag on clients
...
Then, don't send new chat messages and new comments to those restricted clients.
We do this because we don't want to leak private information (email addresses
and names) to "restricted" users, those who have read-only access via a
shared token.
2019-10-04 10:30:24 +01:00
Simon Detheridge
21e294c6eb
Generate retryable error when hitting rate limits in web
2019-09-02 11:27:04 +01:00
Brian Gough
fe2e7b3065
minimal fix for undefined connected users
2019-08-16 10:07:30 +01:00
Henry Oswald
38ed780d80
add log line to draining
2019-08-15 14:41:22 +01:00
Brian Gough
a7a161556f
Merge branch 'bg-status-on-shutdown'
2019-08-15 09:42:00 +01:00
Brian Gough
fa94e3d5e3
Merge pull request #69 from overleaf/ho-drain-connections-timewindow
...
add shutdownDrainTimeWindow, drains all connections within time range
2019-08-15 09:22:10 +01:00
Brian Gough
022e47b5c8
Merge pull request #73 from overleaf/bg-connected-client-metrics
...
add connected client count metric
2019-08-15 09:20:03 +01:00
Brian Gough
f13e66b453
fix client count so that result is zero when all clients have left
2019-08-14 15:34:23 +01:00
Brian Gough
8270c14d86
add connected client count metric
2019-08-14 15:22:03 +01:00
Brian Gough
d3171e4e2e
remove unwanted argument
2019-08-14 13:03:06 +01:00
Brian Gough
20d442120f
notify docupdate if the flush is from a shutdown
2019-08-13 17:36:53 +01:00
Brian Gough
0708f717fd
reject connections when shutdown in progress
...
send a message to the client to reconnect immediately
2019-08-13 16:59:15 +01:00
Brian Gough
b3e5709b64
enforce a minimum drain rate
2019-08-13 16:15:30 +01:00
Henry Oswald
00cca29d9e
add shutdownDrainTimeWindow, drains all connections within time range
2019-08-13 14:21:47 +01:00
Brian Gough
2000f478a7
refresh the client list on demand
2019-08-13 10:40:03 +01:00
Brian Gough
478a727c61
ignore spurious requests to leave other docs
2019-07-29 15:19:08 +01:00
Brian Gough
04a171171f
fix async behaviour of join/leave
2019-07-29 11:54:02 +01:00
Brian Gough
22d722f3e8
add metric for RoomEvents listeners
2019-07-24 16:25:45 +01:00
Brian Gough
277ec71a5b
subscribe to doc updates before requesting doc content
2019-07-24 15:49:29 +01:00
Brian Gough
1c74cbbc4e
add comments
2019-07-24 15:49:29 +01:00
Brian Gough
273af3f3aa
refactor subscribe resolution
2019-07-24 14:30:48 +01:00
Brian Gough
e14a94906a
update naming from Set -> Map
2019-07-24 14:18:15 +01:00
Brian Gough
cb53bfafd6
remove unnecessary require
2019-07-24 09:52:31 +01:00
Brian Gough
61b3a000b4
fix whitespace
2019-07-24 09:52:20 +01:00
Brian Gough
159b39c491
ensure redis channel is subscribed when joining room
2019-07-23 17:02:09 +01:00
Brian Gough
84e6ff616f
whitespace fix
2019-07-22 12:25:41 +01:00
Brian Gough
92e6910180
cleanup
2019-07-22 11:23:33 +01:00
Brian Gough
a538d10488
extend comment re disconnection
2019-07-19 08:56:38 +01:00
Brian Gough
616014e05d
add comment about automatically leaving rooms
2019-07-19 08:50:43 +01:00
Brian Gough
3bf5dd5d6b
clarify errors for subscribe/unsubscribe
2019-07-18 14:25:25 +01:00
Brian Gough
f6f6f549d9
don't publish on individual channels until explicitly set
2019-07-18 12:55:23 +01:00
Brian Gough
804f4c2bd2
listen on separate channels for each project/doc
2019-07-18 12:55:23 +01:00
Brian Gough
9ecce32ff9
Merge pull request #63 from overleaf/bg-log-out-of-order-events
...
log out of order events now that the rate is lower
2019-07-17 13:47:45 +01:00
Brian Gough
8a7804f0a7
make event order check a configuration setting
2019-07-15 13:45:34 +01:00
Brian Gough
24a4709cff
log out of order events now that the rate is lower
2019-07-15 11:14:48 +01:00
Brian Gough
e632f9f29d
only create per-client metrics when there are multiple redis clients
2019-07-11 11:35:48 +01:00
Brian Gough
80f8f2465e
remove unused pubsub client
2019-07-11 11:10:33 +01:00
Brian Gough
689a75f397
add logging for redis clients at start up
2019-07-09 14:18:39 +01:00
Brian Gough
dd54789e2b
fix build problems
2019-07-09 12:20:59 +01:00
Brian Gough
580b100362
only publish to one redis client in WebsocketLoadBalancer
...
but listen to all of them
2019-07-09 12:03:13 +01:00
Brian Gough
999cbd8ee6
add a per-client metric
2019-07-09 12:01:58 +01:00
Brian Gough
cb289f2dec
make redis client list dynamic based on settings
2019-07-09 11:45:00 +01:00
Brian Gough
b5f9bc422b
support multple redis instances for pubsub
2019-07-08 15:56:25 +01:00
Henry Oswald
8042a415ec
move pubsub traffic over to a pubsub redis connection string
2019-07-04 13:38:39 +01:00
miguel
46dfe56b05
Downgraded unathorised log to warning
2019-06-21 07:30:12 +02:00
Brian Gough
a579a83389
Merge pull request #53 from overleaf/em-doc-updater-404s
...
Do not log error on doc updater 404
2019-06-03 10:55:04 +01:00
Brian Gough
d4f003fe0f
Merge pull request #55 from overleaf/bg-allow-fractional-drain-rate
...
allow fractional drain rate
2019-06-03 10:53:57 +01:00
Brian Gough
9da98614c7
Merge pull request #54 from overleaf/bg-use-background-flush-on-disconnect
...
use background flush on disconnect
2019-06-03 10:53:43 +01:00
Brian Gough
63f052192e
use background flush on disconnect
2019-06-03 09:42:12 +01:00
Eric Mc Sween
acf850bce9
Do not log error on doc updater 404
...
Document updater 404s are not indicative of a problem, but just of the client
trying to connect to a deleted document, which is easy to trigger. We
log a warning instead.
2019-05-31 17:32:21 -04:00
Henry Oswald
421a914e72
log out when health check manager fails a check
2019-05-31 09:15:49 +01:00
Brian Gough
74db743ffa
allow fractional drain rate
2019-05-24 15:23:01 +01:00
Brian Gough
20d5cc69a4
filter invalid updates
2019-05-24 10:19:02 +01:00
Brian Gough
2dbdcf5bc8
add health check to pubsub channels
2019-04-15 14:17:55 +01:00
Brian Gough
e72acacf17
downsample logging
2019-04-11 16:25:42 +01:00
Brian Gough
ef9e97e7d1
add metric for applied-ops events
2019-04-11 15:39:28 +01:00
Brian Gough
c6225d614e
add /debug/events endpoint
2019-04-11 15:00:25 +01:00
Brian Gough
2a31139254
log and skip duplicate events
2019-04-11 12:53:43 +01:00
Brian Gough
893515e83f
handle duplicate entries in io.sockets.clients
2019-04-09 14:48:00 +01:00
Brian Gough
6c71ae172b
return out of order events for consistency
2019-03-22 11:19:08 +00:00
Brian Gough
c5a09fcf0a
add comment about downsampling
2019-03-22 11:18:34 +00:00
Brian Gough
695472a8aa
fix event id parsing to allow for dashes in keys
2019-03-22 11:18:19 +00:00
Brian Gough
1ab5e52699
down-sample valid events by 1000
2019-03-21 15:52:53 +00:00
Brian Gough
e91b967bdb
use per-channel event metrics
2019-03-21 14:59:39 +00:00
Brian Gough
8c82faa966
check order of messages on applied-ops channel
2019-03-21 14:59:39 +00:00
Brian Gough
57c5ec14bd
check for a valid counter value in event log
2019-03-21 14:59:19 +00:00
Brian Gough
9b25374cd3
use time-based cleaning of event log
2019-03-21 14:59:19 +00:00
Brian Gough
befe4be517
add check for duplicate events
2019-03-19 14:22:11 +00:00
Henry Oswald
5119db43a7
Merge pull request #37 from sharelatex/ho-continual-pub-sub-traffic
...
setup continualPubsubTraffic to keep pub sub happy
2019-02-19 11:28:06 +00:00
Henry Oswald
26e903f384
setup continualPubsubTraffic
...
this keeps the pub sub channel ticking along happily
2019-02-15 17:04:49 +00:00
Chrystal Maria Griffiths
0b88a63444
Merge pull request #35 from sharelatex/cmg-anonymous-display-name
...
Nameless logged in user labelled as anonymous
2019-02-15 14:53:18 +00:00
Chrystal Griffiths
26acdfd072
Add comment explaining why not sending anon data up
2019-02-12 14:06:59 +00:00
Chrystal Griffiths
bb06f82e04
Still send cursorData for logged in users
2019-02-12 14:00:47 +00:00
Chrystal Griffiths
2ec760403f
Revert to method not sending cursorData because of duplication
2019-02-11 11:52:14 +00:00
Chrystal Griffiths
cb12e1c6f6
Send an empty string for every nameless user
2019-02-08 15:39:51 +00:00
Henry Oswald
d85bf5cedb
remove extra logging line
2019-02-06 15:26:12 +00:00
Henry Oswald
37d1c96051
add logging
2019-02-04 16:59:36 +00:00
Henry Oswald
d4e8bc1d4b
remove metrics.set as we don't support it atm
2018-12-05 15:29:23 +00:00
Henry Oswald
8073cdea75
improve logging
2018-12-04 14:17:59 +00:00
Henry Oswald
7f2decae4d
start drain after 3 hours
2018-12-04 14:06:39 +00:00
Henry Oswald
327fa79f1c
update build scripts, acceptence tests now pass. includes dockerfile 1.1.3
2018-05-23 15:00:46 +01:00
Shane Kilkelly
7295342ec2
fix existance checks for first_name and last_name
2017-12-18 11:13:19 +00:00
Shane Kilkelly
675814f1b1
Handle the case where the user has only a last_name set
2017-12-13 10:28:35 +00:00
Shane Kilkelly
06c8729ce7
If a user has only their first_name set, don't label as Anonymous
2017-12-12 15:27:50 +00:00
Brian Gough
b734f7a3f7
convert errors to warnings
2017-11-10 15:01:23 +00:00
Shane Kilkelly
3b39464aa5
Merge branch 'master' into sk-unlisted-projects
2017-11-02 10:58:41 +00:00
Shane Kilkelly
c5e602c0a9
Fix typo
2017-10-26 16:00:06 +01:00
Shane Kilkelly
76673d5f0a
Change anonToken to anonymousAccessToken
2017-10-20 10:10:58 +01:00
Shane Kilkelly
438bb28c04
Get anonToken from joinProject payload
2017-09-29 16:32:46 +01:00
Shane Kilkelly
d4c735c3ea
Pass anonymous-read token along as header to web-api
2017-09-26 14:21:41 +01:00
Alasdair Smith
a0505afb23
Be defensive on comment text
2017-09-22 09:34:10 +01:00
Alasdair Smith
c67150ea10
Ensure falsy value doesn't fail conditional
2017-09-22 09:33:29 +01:00
Alasdair Smith
937bf82a2f
Return callback with guard
2017-09-22 09:25:24 +01:00
Alasdair Smith
5d8e201732
Don't return obj
2017-09-21 16:58:03 +01:00
Alasdair Smith
a299d7335d
Fix incorrect var
2017-09-21 16:56:18 +01:00
Alasdair Smith
90d05dc6dd
Make args order consistent
2017-09-21 16:56:09 +01:00
Alasdair Smith
3966e2f85b
Make variadic options more explicit
2017-09-21 16:55:49 +01:00
Alasdair Smith
790b9ea8ed
Switch order of args
2017-09-21 15:19:19 +01:00
Alasdair Smith
55c880e1dd
DRY up a bit
2017-09-21 15:07:15 +01:00
Alasdair Smith
b796879c9f
Handle options not being passed
2017-09-21 14:58:49 +01:00
Alasdair Smith
aa6e0d0d69
Only encode ranges if option passed
2017-09-21 14:23:16 +01:00
Alasdair Smith
9f503f1e9f
First pass at encoding changes & comments in ranges
2017-09-21 13:25:55 +01:00
Brian Gough
ab6fe1d948
check for null bytes from JSON.stringify
2017-06-01 11:27:56 +01:00
James Allen
642134da79
Don't return a user if there is no entry
...
ioredis returns a blank object, {}, if there is no key with hgetall.
Previously, node-redis returned nil. So we need to check for a blank
object as well as a nil object.
2017-05-16 11:06:05 +01:00
James Allen
a8917b933f
Remove multi call to make compatible with redis-cluster
2017-05-11 17:27:28 +01:00
James Allen
720f24427a
Use new redis-sharelatex with support for cluster
2017-05-02 15:51:17 +01:00
James Allen
d8ff8ba098
Merge pull request #11 from sharelatex/ja-allow-readonly-comments
...
Allow readonly users to submit comments
2017-03-20 14:25:03 +00:00
James Allen
8766646149
Allow users to send a comment update if they are read-only
2017-03-15 15:45:52 +00:00
James Allen
d939f6cd65
Remove some old logging
2017-03-15 15:45:18 +00:00
Brian Gough
d468f662ac
handle disconnects of unauthenticated users
2017-02-23 12:04:36 +00:00
James Allen
9fd099c24f
Update getDoc signature to match reality
2017-01-16 17:09:44 +01:00
James Allen
f66d631886
Merge branch 'master' into ja-track-changes
2017-01-13 13:40:42 +01:00
Brian Gough
e5160d9a36
log client id when disconnecting on otUpdateError
...
only log errors for connected clients
2016-12-16 14:43:51 +00:00
James Allen
bf2620ee0c
Return ranges from docupdater to client
2016-12-08 11:37:31 +00:00
James Allen
4cff89becc
Fix acceptance tests
2016-12-08 11:12:07 +00:00
Brian Gough
2e0f5b74db
send connectionAccepted/Rejected events on connect
...
let the client know whether it has successfully authenticated
2016-10-28 15:41:57 +01:00
James Allen
6fa2a81baf
Make breakout logic after draining N clients more clear
2016-10-24 16:54:56 +01:00
James Allen
7107d9adcd
Delete logging that will be noisy in production
2016-10-24 16:40:10 +01:00
James Allen
59d042e264
Add end point to start draining clients
2016-10-24 16:36:09 +01:00
Shane Kilkelly
185bc7e635
Update session code
2016-09-07 08:58:35 +01:00
Brian Gough
8ffec68250
add comment about fallback case
2016-09-05 12:46:52 +01:00
Brian Gough
ef85bce3b8
track permissions when clients join and leave docs
2016-09-05 12:46:52 +01:00
Brian Gough
9ab19c5d03
avoid double callback
2016-09-02 16:34:14 +01:00
James Allen
51939512ac
Return semantic error if doc ops range is not loaded
2016-05-31 14:21:23 +01:00
