Jakob Ackermann
f82177a46a
[Errors] migrate to OError
2020-08-21 12:47:04 +01:00
Jakob Ackermann
ee59056c6e
[misc] forcefully disconnect stale clients from shutdown process
2020-08-13 13:39:22 +01:00
Brian Gough
831d794bf4
clean up join/leave handling
...
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
2020-08-12 10:54:22 +01:00
Jakob Ackermann
562375d351
[misc] fix express deprecations
2020-07-22 09:45:14 +01:00
Jakob Ackermann
9fff03bca5
[misc] optionally expose the hostname in the 'debug.getHostname' rpc
2020-07-07 11:07:28 +01:00
Jakob Ackermann
aa9d6c8dc9
[misc] reland decaff cleanup ( #166 )
...
* [misc] decaff cleanup: RoomManager
* [misc] decaff cleanup: RedisClientManager
* [misc] decaff cleanup: SafeJsonParse
* [misc] decaff cleanup: WebApiManager
* [misc] decaff cleanup: WebsocketController
* [misc] decaff cleanup: WebsocketLoadBalancer
* [misc] decaff cleanup: SessionSockets
* [misc] decaff cleanup: HttpController
* [misc] decaff cleanup: HttpApiController
* [misc] decaff cleanup: HealthCheckManager
* [misc] decaff cleanup: EventLogger
* [misc] decaff cleanup: Errors
o-error will eliminate most of it -- when we migrate over.
* [misc] decaff cleanup: DrainManager
* [misc] decaff cleanup: DocumentUpdaterManager
* [misc] decaff cleanup: DocumentUpdaterController: no-unused-vars
* [misc] decaff cleanup: DocumentUpdaterController: Array.from
* [misc] decaff cleanup: DocumentUpdaterController: implicit return
* [misc] decaff cleanup: DocumentUpdaterController: IIFE
* [misc] decaff cleanup: DocumentUpdaterController: null checks
* [misc] decaff cleanup: DocumentUpdaterController: simpler loops
* [misc] decaff cleanup: DocumentUpdaterController: move module name def
* [misc] decaff cleanup: ConnectedUsersManager: handle-callback-err
* [misc] decaff cleanup: ConnectedUsersManager: implicit returns
* [misc] decaff cleanup: ConnectedUsersManager: null checks
* [misc] decaff cleanup: ChannelManager: no-unused-vars
* [misc] decaff cleanup: ChannelManager: implicit returns
* [misc] decaff cleanup: ChannelManager: other cleanup
- var -> const
- drop variable assignment before return
* [misc] decaff cleanup: AuthorizationManager: handle-callback-err
Note: This requires a change in WebsocketController to provide a dummy
callback.
* [misc] decaff cleanup: AuthorizationManager: Array.from
* [misc] decaff cleanup: AuthorizationManager: implicit returns
* [misc] decaff cleanup: AuthorizationManager: null checks
* [misc] decaff cleanup: Router: handle-callback-err
* [misc] decaff cleanup: Router: standard/no-callback-literal
* [misc] decaff cleanup: Router: Array.from
* [misc] decaff cleanup: Router: implicit returns
* [misc] decaff cleanup: Router: refactor __guard__ wrapper
* [misc] decaff cleanup: Router: null checks
And a minor bug fix: user.id -> user._id
* [misc] decaff cleanup: Router: move variable declarations to assignments
* [misc] decaff cleanup: app: implicit returns
* [misc] decaff cleanup: app: __guard__
* [misc] decaff cleanup: app: null checks
* [misc] decaff cleanup: app: function definitions
* [misc] decaff cleanup: app: drop unused next argument
* [misc] decaff cleanup: app: var -> const
2020-07-07 11:06:02 +01:00
Jakob Ackermann
a77222470b
[misc] WebsocketControllerTests: always pass cb to updateClientPosition
...
...and fix the inconsistent async behaviour for logged-in vs anonymous
users in the app.
2020-06-29 18:11:33 +01:00
decaffeinate
817844515d
prettier: convert app/js decaffeinated files to Prettier format
2020-06-23 18:29:44 +01:00
decaffeinate
04a85a6716
decaffeinate: rename app/coffee dir to app/js
2020-06-23 18:29:41 +01:00
decaffeinate
a397154e18
decaffeinate: Run post-processing cleanups on AuthorizationManager.coffee and 18 other files
2020-06-23 18:29:38 +01:00
decaffeinate
7335084c26
decaffeinate: Convert AuthorizationManager.coffee and 18 other files to JS
2020-06-23 18:29:34 +01:00
decaffeinate
90eafa388a
decaffeinate: Rename AuthorizationManager.coffee and 18 other files from .coffee to .js
2020-06-23 18:29:29 +01:00
Jakob Ackermann
205efa8812
[WebsocketController] reset the client context before (re)populating it
...
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2020-06-22 17:04:31 +01:00
Jakob Ackermann
1fcf534dcf
[Router] revert preserving of client.ol_context
...
We do not enter this line twice, it would result in multiple event
handlers too.
2020-06-22 17:04:31 +01:00
Jakob Ackermann
d17ef183d0
[Router] gracefully set and do not reset the ol_context
2020-06-22 17:04:30 +01:00
Jakob Ackermann
ce4f9148c3
[perf] WebsocketLoadBalancer: move back to a sync loop for msg fan out
2020-06-22 17:04:15 +01:00
Jakob Ackermann
a70c1e1fa2
[perf] WebsocketLoadBalancer: check is_restricted_message once
...
...and filter clients early on.
2020-06-22 17:04:15 +01:00
Jakob Ackermann
5282f8f531
[misc] synchronous client store using an Object at .ol_context
2020-06-22 17:04:08 +01:00
Jakob Ackermann
5f7841526f
[misc] RoomManager: emitOnCompletion: properly handle Promise rejections
...
```
result = Promise.all([<Promise that rejects eventually>]) # rejection 1
result.then () -> RoomEvents.emit(eventName) # rejection 2
result.catch (err) -> RoomEvents.emit(eventName, err) # handle r1
```
As shown above, the second rejection remains unhandled. The fix is to
chain the `.catch()` onto the `.then()` Promise.
2020-06-17 09:29:12 +01:00
Jakob Ackermann
10eb30e572
Merge pull request #154 from overleaf/jpa-backport-httpcontroller-tests
...
[backport] 111 HttpController: 404 for missing clients and add tests
2020-06-10 11:28:17 +02:00
Jakob Ackermann
45ac2bdd97
Merge pull request #153 from overleaf/jpa-backport-drainmanager-tests
...
[backport] 108 and 112: DrainManager acceptance tests
2020-06-10 11:28:08 +02:00
Jakob Ackermann
cfe37dcbb5
Merge pull request #152 from overleaf/jpa-vendor-cookie
...
[misc] vendor a patched session.socket.io middleware
2020-06-10 11:27:58 +02:00
Jakob Ackermann
bc44494466
[HttpController] return 404 in case of a missing client and add tests
...
Add acceptance tests for the client view.
2020-06-10 09:41:36 +01:00
Jakob Ackermann
de35fc5ecf
[HttpApiController] implement the disconnection of a single client
...
The http route returns as soon as the client has fully disconnected.
2020-06-09 18:01:08 +01:00
Jakob Ackermann
acb7d7df5a
[misc] add test cases for the validation of the callback argument
...
When the user provides a function as last argument for socket.emit,
socket.io will flag this as an RPC and add a cb as the last argument
to the client.on('event', ...) handler on the server side.
Without a function as last argument for socket.emit, the callback
argument on the server side is undefined, leading to invalid function
calls (`undefined()`) and an unhandled exception.
The user can also provide lots of other arguments, so the 2nd/3rd ...
argument is of arbitrary type, again leading to invalid function calls
-- e.g. `1()`.
2020-06-09 16:30:03 +01:00
Jakob Ackermann
dc553c4150
[misc] vendor a patched session.socket.io middleware
2020-06-09 15:21:33 +01:00
Jakob Ackermann
1c9eaf574a
Merge pull request #147 from overleaf/jpa-backport-141
...
[backport] 141: Router: validate the callback argument
2020-06-09 14:41:24 +02:00
Jakob Ackermann
5238e6ad36
Merge pull request #145 from overleaf/jpa-backport-140
...
[backport] 140: stop processing requests as we detect a client disconnect
2020-06-09 14:41:12 +02:00
Jakob Ackermann
32af7001fc
[misc] Router: prefix the publicId with 'P.' for easy differentiation
2020-06-08 11:29:40 +01:00
Jakob Ackermann
0b2cccf1e0
[misc] apply review feedback: adjust metric names
...
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
(cherry-picked from commit 67674b83efb452ece05cdc39525ee3a5eeb8a4d7)
2020-06-05 11:33:00 +01:00
Jakob Ackermann
ddcb9cf8c8
[misc] downgrade a warning message from clients leaving non-joined rooms
...
This can now happen all the time, as we skip the join for clients that
disconnect before joinProject/joinDoc completed.
(cherry-picked from commit f357931de74e088800f3cced3898cce4f251dad0)
2020-06-05 11:32:43 +01:00
Jakob Ackermann
7fa9061015
[misc] stop processing requests as we detect a client disconnect
...
v2 expose `client.connected`; v0 exposes `client.disconnected`
(cherry-picked from commit a9d70484343ca9be367d45bf7bb949e4be449647)
2020-06-05 11:32:13 +01:00
Jakob Ackermann
0840700ffd
[Router] validate the callback argument
2020-06-05 10:59:01 +01:00
Jakob Ackermann
c6d08647c7
[misc] socket.io: use a secondary publicId for public facing usages
2020-06-04 17:18:07 +01:00
Jakob Ackermann
41debfae0f
[ChannelManager] rework (un)subscribing to redis
...
- send a subscribe request on every request
- wait for a pending unsubscribe request before subscribing
- wait for a pending subscribe request before unsubscribing
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2020-05-15 18:34:33 +02:00
Jakob Ackermann
b713beb7f0
Merge pull request #135 from overleaf/jpa-skip-leave-project-for-invalid-clients
...
[WebsocketController] skip leaveProject when joinProject didn't complete
2020-05-13 15:34:41 +02:00
Jakob Ackermann
55af5e502f
[WebsocketController] skip leaveProject when joinProject didn't complete
...
Also drop dead code:
- user_id bailout
There is a check on a completed joinProject call now. It will always
set a user_id, see Router.coffee which has a fallback `{_id:"..."}`.
- late project_id bailout
WebsocketLoadBalancer.emitToRoom will not work without a project_id.
We have to bail out before the call.
2020-05-12 17:15:08 +02:00
Jakob Ackermann
684cb3903c
[WebsocketController] handle redis subscribe error on joinProject
...
joinProject should not complete when the redis pub/sub subscribe request
failed.
2020-05-12 13:03:50 +02:00
Brian Gough
5c28da1031
add metric for pendingUpdates queue
2020-04-07 11:53:54 +01:00
Jakob Ackermann
56628a16c6
[misc] track redis pub/sub payload sizes on publish
2020-03-30 11:31:44 +02:00
Jakob Ackermann
af53d3b603
[misc] skip duplicate JSON serialization for size check
2020-03-24 11:22:28 +01:00
Jakob Ackermann
cb675d38c2
[misc] SafeJsonParse: align the size limit with the frontend->rt limit
...
frontend -> real-time and doc-updater -> real-time should be in sync.
Otherwise we can send a payload to doc-updater, but can not receive the
confirmation of it -- and the client will send it again in a loop.
Also log the size of the payload.
2020-03-24 09:14:15 +01:00
Jakob Ackermann
15244a54be
[misc] WebsocketController: limit the update size to 7mb
...
bail out early on -- especially do not push the update into redis for
doc-updater to discard it.
Confirm the update silently, otherwise the frontend will send it again.
Broadcast a 'otUpdateError' message and disconnect the client, like
doc-updater would do.
2020-03-24 09:12:12 +01:00
Brian Gough
64bd739a87
Revert "Merge pull request #91 from overleaf/spd-trycatch-all-the-things"
...
This reverts commit 2bf7f14f9d050c58f141f465633bb6e274b903dd, reversing
changes made to 989240812532ca43a52513339f4dda8f44a80a64.
2020-02-05 10:05:36 +00:00
Simon Detheridge
9fd4699098
Merge remote-tracking branch 'origin/bg-avoid-emit-when-client-not-connected' into spd-trycatch-all-the-things
2020-02-04 12:35:35 +00:00
Simon Detheridge
216a977922
Add try/catch around all client emissions
2020-02-04 12:13:03 +00:00
Brian Gough
ebb83e4633
use diconnected property, not connected
2020-02-04 11:59:25 +00:00
Brian Gough
7380d523d5
avoid emitting when client not connected
...
the emit is happening asynchronously after the client list is computed,
so clients may have disconnected in the intervening time.
2020-02-04 11:39:37 +00:00
Brian Gough
4f94110225
Merge pull request #84 from overleaf/spd-catch-errors
...
Bump to Node 10 and add error handlers for socket.io
2020-02-03 14:59:04 +00:00
Brian Gough
49a8e1214b
use a separate field for client errors
2020-02-03 14:47:45 +00:00
Brian Gough
e04b6e1e49
Update app/coffee/Router.coffee
...
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
2020-02-03 14:46:14 +00:00
Shane Kilkelly
6df88ebc49
Filter "comments" if restricted user.
2019-10-30 13:54:40 +00:00
Shane Kilkelly
403caa65e8
Revert "Revert "Track the isRestrictedUser
flag on clients""
...
This reverts commit 651e392a7c644403f199e1b03e7494b61ce71d0c.
2019-10-30 13:52:36 +00:00
Simon Detheridge
925a8651c1
Revert "Track the isRestrictedUser
flag on clients"
2019-10-22 10:17:38 +01:00
Simon Detheridge
7543f2fcbd
Catch errors from socket.io and attempt graceful cleanup
2019-10-17 12:45:56 +01:00
Shane Kilkelly
2cc2be3d9c
send messages to clients with concurrency of 2
2019-10-11 10:01:21 +01:00
Shane Kilkelly
06aa578bdc
Make it an error when we get no data from joinProject
2019-10-11 09:57:16 +01:00
Shane Kilkelly
df6cd4a054
Also block getConnectedUsers for restricted users.
...
Plus refactor to use a pass list instead of a deny list.
2019-10-04 13:41:49 +01:00
Shane Kilkelly
6765d03339
Track the isRestrictedUser
flag on clients
...
Then, don't send new chat messages and new comments to those restricted clients.
We do this because we don't want to leak private information (email addresses
and names) to "restricted" users, those who have read-only access via a
shared token.
2019-10-04 10:30:24 +01:00
Simon Detheridge
21e294c6eb
Generate retryable error when hitting rate limits in web
2019-09-02 11:27:04 +01:00
Brian Gough
fe2e7b3065
minimal fix for undefined connected users
2019-08-16 10:07:30 +01:00
Henry Oswald
38ed780d80
add log line to draining
2019-08-15 14:41:22 +01:00
Brian Gough
a7a161556f
Merge branch 'bg-status-on-shutdown'
2019-08-15 09:42:00 +01:00
Brian Gough
fa94e3d5e3
Merge pull request #69 from overleaf/ho-drain-connections-timewindow
...
add shutdownDrainTimeWindow, drains all connections within time range
2019-08-15 09:22:10 +01:00
Brian Gough
022e47b5c8
Merge pull request #73 from overleaf/bg-connected-client-metrics
...
add connected client count metric
2019-08-15 09:20:03 +01:00
Brian Gough
f13e66b453
fix client count so that result is zero when all clients have left
2019-08-14 15:34:23 +01:00
Brian Gough
8270c14d86
add connected client count metric
2019-08-14 15:22:03 +01:00
Brian Gough
d3171e4e2e
remove unwanted argument
2019-08-14 13:03:06 +01:00
Brian Gough
20d442120f
notify docupdate if the flush is from a shutdown
2019-08-13 17:36:53 +01:00
Brian Gough
0708f717fd
reject connections when shutdown in progress
...
send a message to the client to reconnect immediately
2019-08-13 16:59:15 +01:00
Brian Gough
b3e5709b64
enforce a minimum drain rate
2019-08-13 16:15:30 +01:00
Henry Oswald
00cca29d9e
add shutdownDrainTimeWindow, drains all connections within time range
2019-08-13 14:21:47 +01:00
Brian Gough
2000f478a7
refresh the client list on demand
2019-08-13 10:40:03 +01:00
Brian Gough
478a727c61
ignore spurious requests to leave other docs
2019-07-29 15:19:08 +01:00
Brian Gough
04a171171f
fix async behaviour of join/leave
2019-07-29 11:54:02 +01:00
Brian Gough
22d722f3e8
add metric for RoomEvents listeners
2019-07-24 16:25:45 +01:00
Brian Gough
277ec71a5b
subscribe to doc updates before requesting doc content
2019-07-24 15:49:29 +01:00
Brian Gough
1c74cbbc4e
add comments
2019-07-24 15:49:29 +01:00
Brian Gough
273af3f3aa
refactor subscribe resolution
2019-07-24 14:30:48 +01:00
Brian Gough
e14a94906a
update naming from Set -> Map
2019-07-24 14:18:15 +01:00
Brian Gough
cb53bfafd6
remove unnecessary require
2019-07-24 09:52:31 +01:00
Brian Gough
61b3a000b4
fix whitespace
2019-07-24 09:52:20 +01:00
Brian Gough
159b39c491
ensure redis channel is subscribed when joining room
2019-07-23 17:02:09 +01:00
Brian Gough
84e6ff616f
whitespace fix
2019-07-22 12:25:41 +01:00
Brian Gough
92e6910180
cleanup
2019-07-22 11:23:33 +01:00
Brian Gough
a538d10488
extend comment re disconnection
2019-07-19 08:56:38 +01:00
Brian Gough
616014e05d
add comment about automatically leaving rooms
2019-07-19 08:50:43 +01:00
Brian Gough
3bf5dd5d6b
clarify errors for subscribe/unsubscribe
2019-07-18 14:25:25 +01:00
Brian Gough
f6f6f549d9
don't publish on individual channels until explicitly set
2019-07-18 12:55:23 +01:00
Brian Gough
804f4c2bd2
listen on separate channels for each project/doc
2019-07-18 12:55:23 +01:00
Brian Gough
9ecce32ff9
Merge pull request #63 from overleaf/bg-log-out-of-order-events
...
log out of order events now that the rate is lower
2019-07-17 13:47:45 +01:00
Brian Gough
8a7804f0a7
make event order check a configuration setting
2019-07-15 13:45:34 +01:00
Brian Gough
24a4709cff
log out of order events now that the rate is lower
2019-07-15 11:14:48 +01:00
Brian Gough
e632f9f29d
only create per-client metrics when there are multiple redis clients
2019-07-11 11:35:48 +01:00
Brian Gough
80f8f2465e
remove unused pubsub client
2019-07-11 11:10:33 +01:00
Brian Gough
689a75f397
add logging for redis clients at start up
2019-07-09 14:18:39 +01:00
Brian Gough
dd54789e2b
fix build problems
2019-07-09 12:20:59 +01:00
Brian Gough
580b100362
only publish to one redis client in WebsocketLoadBalancer
...
but listen to all of them
2019-07-09 12:03:13 +01:00
Brian Gough
999cbd8ee6
add a per-client metric
2019-07-09 12:01:58 +01:00
Brian Gough
cb289f2dec
make redis client list dynamic based on settings
2019-07-09 11:45:00 +01:00