github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-12-29 05:43:30 +00:00
Code Issues Projects Releases Packages Wiki Activity
17792 commits 3 branches 7 tags 206 MiB
Commit graph

268 commits

Author SHA1 Message Date
Jakob Ackermann
9fff03bca5 [misc] optionally expose the hostname in the 'debug.getHostname' rpc 2020-07-07 11:07:28 +01:00
Jakob Ackermann
aa9d6c8dc9 [misc] reland decaff cleanup (#166) 
* [misc] decaff cleanup: RoomManager

* [misc] decaff cleanup: RedisClientManager

* [misc] decaff cleanup: SafeJsonParse

* [misc] decaff cleanup: WebApiManager

* [misc] decaff cleanup: WebsocketController

* [misc] decaff cleanup: WebsocketLoadBalancer

* [misc] decaff cleanup: SessionSockets

* [misc] decaff cleanup: HttpController

* [misc] decaff cleanup: HttpApiController

* [misc] decaff cleanup: HealthCheckManager

* [misc] decaff cleanup: EventLogger

* [misc] decaff cleanup: Errors

o-error will eliminate most of it -- when we migrate over.

* [misc] decaff cleanup: DrainManager

* [misc] decaff cleanup: DocumentUpdaterManager

* [misc] decaff cleanup: DocumentUpdaterController: no-unused-vars

* [misc] decaff cleanup: DocumentUpdaterController: Array.from

* [misc] decaff cleanup: DocumentUpdaterController: implicit return

* [misc] decaff cleanup: DocumentUpdaterController: IIFE

* [misc] decaff cleanup: DocumentUpdaterController: null checks

* [misc] decaff cleanup: DocumentUpdaterController: simpler loops

* [misc] decaff cleanup: DocumentUpdaterController: move module name def

* [misc] decaff cleanup: ConnectedUsersManager: handle-callback-err

* [misc] decaff cleanup: ConnectedUsersManager: implicit returns

* [misc] decaff cleanup: ConnectedUsersManager: null checks

* [misc] decaff cleanup: ChannelManager: no-unused-vars

* [misc] decaff cleanup: ChannelManager: implicit returns

* [misc] decaff cleanup: ChannelManager: other cleanup

- var -> const
- drop variable assignment before return

* [misc] decaff cleanup: AuthorizationManager: handle-callback-err

Note: This requires a change in WebsocketController to provide a dummy
 callback.

* [misc] decaff cleanup: AuthorizationManager: Array.from

* [misc] decaff cleanup: AuthorizationManager: implicit returns

* [misc] decaff cleanup: AuthorizationManager: null checks

* [misc] decaff cleanup: Router: handle-callback-err

* [misc] decaff cleanup: Router: standard/no-callback-literal

* [misc] decaff cleanup: Router: Array.from

* [misc] decaff cleanup: Router: implicit returns

* [misc] decaff cleanup: Router: refactor __guard__ wrapper

* [misc] decaff cleanup: Router: null checks

And a minor bug fix: user.id -> user._id

* [misc] decaff cleanup: Router: move variable declarations to assignments

* [misc] decaff cleanup: app: implicit returns

* [misc] decaff cleanup: app: __guard__

* [misc] decaff cleanup: app: null checks

* [misc] decaff cleanup: app: function definitions

* [misc] decaff cleanup: app: drop unused next argument

* [misc] decaff cleanup: app: var -> const
 2020-07-07 11:06:02 +01:00
Jakob Ackermann
a77222470b [misc] WebsocketControllerTests: always pass cb to updateClientPosition 
...and fix the inconsistent async behaviour for logged-in vs anonymous
 users in the app.
 2020-06-29 18:11:33 +01:00
decaffeinate
817844515d prettier: convert app/js decaffeinated files to Prettier format 2020-06-23 18:29:44 +01:00
decaffeinate
04a85a6716 decaffeinate: rename app/coffee dir to app/js 2020-06-23 18:29:41 +01:00
decaffeinate
a397154e18 decaffeinate: Run post-processing cleanups on AuthorizationManager.coffee and 18 other files 2020-06-23 18:29:38 +01:00
decaffeinate
7335084c26 decaffeinate: Convert AuthorizationManager.coffee and 18 other files to JS 2020-06-23 18:29:34 +01:00
decaffeinate
90eafa388a decaffeinate: Rename AuthorizationManager.coffee and 18 other files from .coffee to .js 2020-06-23 18:29:29 +01:00
Jakob Ackermann
205efa8812 [WebsocketController] reset the client context before (re)populating it 
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
 2020-06-22 17:04:31 +01:00
Jakob Ackermann
1fcf534dcf [Router] revert preserving of client.ol_context 
We do not enter this line twice, it would result in multiple event
 handlers too.
 2020-06-22 17:04:31 +01:00
Jakob Ackermann
d17ef183d0 [Router] gracefully set and do not reset the ol_context 2020-06-22 17:04:30 +01:00
Jakob Ackermann
ce4f9148c3 [perf] WebsocketLoadBalancer: move back to a sync loop for msg fan out 2020-06-22 17:04:15 +01:00
Jakob Ackermann
a70c1e1fa2 [perf] WebsocketLoadBalancer: check is_restricted_message once 
...and filter clients early on.
 2020-06-22 17:04:15 +01:00
Jakob Ackermann
5282f8f531 [misc] synchronous client store using an Object at .ol_context 2020-06-22 17:04:08 +01:00
Jakob Ackermann
5f7841526f [misc] RoomManager: emitOnCompletion: properly handle Promise rejections 
```
result = Promise.all([<Promise that rejects eventually>]) # rejection 1
result.then () -> RoomEvents.emit(eventName)              # rejection 2
result.catch (err) -> RoomEvents.emit(eventName, err)     # handle r1
```
As shown above, the second rejection remains unhandled. The fix is to
 chain the `.catch()` onto the `.then()` Promise.
 2020-06-17 09:29:12 +01:00
Jakob Ackermann
10eb30e572 Merge pull request #154 from overleaf/jpa-backport-httpcontroller-tests 
[backport] 111 HttpController: 404 for missing clients and add tests
 2020-06-10 11:28:17 +02:00
Jakob Ackermann
45ac2bdd97 Merge pull request #153 from overleaf/jpa-backport-drainmanager-tests 
[backport] 108 and 112: DrainManager acceptance tests
 2020-06-10 11:28:08 +02:00
Jakob Ackermann
cfe37dcbb5 Merge pull request #152 from overleaf/jpa-vendor-cookie 
[misc] vendor a patched session.socket.io middleware
 2020-06-10 11:27:58 +02:00
Jakob Ackermann
bc44494466 [HttpController] return 404 in case of a missing client and add tests 
Add acceptance tests for the client view.
 2020-06-10 09:41:36 +01:00
Jakob Ackermann
de35fc5ecf [HttpApiController] implement the disconnection of a single client 
The http route returns as soon as the client has fully disconnected.
 2020-06-09 18:01:08 +01:00
Jakob Ackermann
acb7d7df5a [misc] add test cases for the validation of the callback argument 
When the user provides a function as last argument for socket.emit,
 socket.io will flag this as an RPC and add a cb as the last argument
 to the client.on('event', ...) handler on the server side.
Without a function as last argument for socket.emit, the callback
 argument on the server side is undefined, leading to invalid function
 calls (`undefined()`) and an unhandled exception.
The user can also provide lots of other arguments, so the 2nd/3rd ...
 argument is of arbitrary type, again leading to invalid function calls
 -- e.g. `1()`.
 2020-06-09 16:30:03 +01:00
Jakob Ackermann
dc553c4150 [misc] vendor a patched session.socket.io middleware 2020-06-09 15:21:33 +01:00
Jakob Ackermann
1c9eaf574a Merge pull request #147 from overleaf/jpa-backport-141 
[backport] 141: Router: validate the callback argument
 2020-06-09 14:41:24 +02:00
Jakob Ackermann
5238e6ad36 Merge pull request #145 from overleaf/jpa-backport-140 
[backport] 140: stop processing requests as we detect a client disconnect
 2020-06-09 14:41:12 +02:00
Jakob Ackermann
32af7001fc [misc] Router: prefix the publicId with 'P.' for easy differentiation 2020-06-08 11:29:40 +01:00
Jakob Ackermann
0b2cccf1e0 [misc] apply review feedback: adjust metric names 
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>

(cherry-picked from commit 67674b83efb452ece05cdc39525ee3a5eeb8a4d7)
 2020-06-05 11:33:00 +01:00
Jakob Ackermann
ddcb9cf8c8 [misc] downgrade a warning message from clients leaving non-joined rooms 
This can now happen all the time, as we skip the join for clients that
 disconnect before joinProject/joinDoc completed.

(cherry-picked from commit f357931de74e088800f3cced3898cce4f251dad0)
 2020-06-05 11:32:43 +01:00
Jakob Ackermann
7fa9061015 [misc] stop processing requests as we detect a client disconnect 
v2 expose `client.connected`; v0 exposes `client.disconnected`

(cherry-picked from commit a9d70484343ca9be367d45bf7bb949e4be449647)
 2020-06-05 11:32:13 +01:00
Jakob Ackermann
0840700ffd [Router] validate the callback argument 2020-06-05 10:59:01 +01:00
Jakob Ackermann
c6d08647c7 [misc] socket.io: use a secondary publicId for public facing usages 2020-06-04 17:18:07 +01:00
Jakob Ackermann
41debfae0f [ChannelManager] rework (un)subscribing to redis 
- send a subscribe request on every request
- wait for a pending unsubscribe request before subscribing
- wait for a pending subscribe request before unsubscribing

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
 2020-05-15 18:34:33 +02:00
Jakob Ackermann
b713beb7f0 Merge pull request #135 from overleaf/jpa-skip-leave-project-for-invalid-clients 
[WebsocketController] skip leaveProject when joinProject didn't complete
 2020-05-13 15:34:41 +02:00
Jakob Ackermann
55af5e502f [WebsocketController] skip leaveProject when joinProject didn't complete 
Also drop dead code:
 - user_id bailout

   There is a check on a completed joinProject call now. It will always
    set a user_id, see Router.coffee which has a fallback `{_id:"..."}`.

 - late project_id bailout

   WebsocketLoadBalancer.emitToRoom will not work without a project_id.
   We have to bail out before the call.
 2020-05-12 17:15:08 +02:00
Jakob Ackermann
684cb3903c [WebsocketController] handle redis subscribe error on joinProject 
joinProject should not complete when the redis pub/sub subscribe request
 failed.
 2020-05-12 13:03:50 +02:00
Brian Gough
5c28da1031 add metric for pendingUpdates queue 2020-04-07 11:53:54 +01:00
Jakob Ackermann
56628a16c6 [misc] track redis pub/sub payload sizes on publish 2020-03-30 11:31:44 +02:00
Jakob Ackermann
af53d3b603 [misc] skip duplicate JSON serialization for size check 2020-03-24 11:22:28 +01:00
Jakob Ackermann
cb675d38c2 [misc] SafeJsonParse: align the size limit with the frontend->rt limit 
frontend -> real-time and doc-updater -> real-time should be in sync.
Otherwise we can send a payload to doc-updater, but can not receive the
 confirmation of it -- and the client will send it again in a loop.

Also log the size of the payload.
 2020-03-24 09:14:15 +01:00
Jakob Ackermann
15244a54be [misc] WebsocketController: limit the update size to 7mb 
bail out early on -- especially do not push the update into redis for
 doc-updater to discard it.

Confirm the update silently, otherwise the frontend will send it again.
Broadcast a 'otUpdateError' message and disconnect the client, like
 doc-updater would do.
 2020-03-24 09:12:12 +01:00
Brian Gough
64bd739a87 Revert "Merge pull request #91 from overleaf/spd-trycatch-all-the-things" 
This reverts commit 2bf7f14f9d050c58f141f465633bb6e274b903dd, reversing
changes made to 989240812532ca43a52513339f4dda8f44a80a64.
 2020-02-05 10:05:36 +00:00
Simon Detheridge
9fd4699098 Merge remote-tracking branch 'origin/bg-avoid-emit-when-client-not-connected' into spd-trycatch-all-the-things 2020-02-04 12:35:35 +00:00
Simon Detheridge
216a977922 Add try/catch around all client emissions 2020-02-04 12:13:03 +00:00
Brian Gough
ebb83e4633 use diconnected property, not connected 2020-02-04 11:59:25 +00:00
Brian Gough
7380d523d5 avoid emitting when client not connected 
the emit is happening asynchronously after the client list is computed,
so clients may have disconnected in the intervening time.
 2020-02-04 11:39:37 +00:00
Brian Gough
4f94110225 Merge pull request #84 from overleaf/spd-catch-errors 
Bump to Node 10 and add error handlers for socket.io
 2020-02-03 14:59:04 +00:00
Brian Gough
49a8e1214b use a separate field for client errors 2020-02-03 14:47:45 +00:00
Brian Gough
e04b6e1e49 Update app/coffee/Router.coffee 
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
 2020-02-03 14:46:14 +00:00
Shane Kilkelly
6df88ebc49 Filter "comments" if restricted user. 2019-10-30 13:54:40 +00:00
Shane Kilkelly
403caa65e8 Revert "Revert "Track the isRestrictedUser flag on clients"" 
This reverts commit 651e392a7c644403f199e1b03e7494b61ce71d0c.
 2019-10-30 13:52:36 +00:00
Simon Detheridge
925a8651c1 Revert "Track the isRestrictedUser flag on clients" 2019-10-22 10:17:38 +01:00
Simon Detheridge
7543f2fcbd Catch errors from socket.io and attempt graceful cleanup 2019-10-17 12:45:56 +01:00
Shane Kilkelly
2cc2be3d9c send messages to clients with concurrency of 2 2019-10-11 10:01:21 +01:00
Shane Kilkelly
06aa578bdc Make it an error when we get no data from joinProject 2019-10-11 09:57:16 +01:00
Shane Kilkelly
df6cd4a054 Also block getConnectedUsers for restricted users. 
Plus refactor to use a pass list instead of a deny list.
 2019-10-04 13:41:49 +01:00
Shane Kilkelly
6765d03339 Track the isRestrictedUser flag on clients 
Then, don't send new chat messages and new comments to those restricted clients.
We do this because we don't want to leak private information (email addresses
and names) to "restricted" users, those who have read-only access via a
shared token.
 2019-10-04 10:30:24 +01:00
Simon Detheridge
21e294c6eb Generate retryable error when hitting rate limits in web 2019-09-02 11:27:04 +01:00
Brian Gough
fe2e7b3065 minimal fix for undefined connected users 2019-08-16 10:07:30 +01:00
Henry Oswald
38ed780d80 add log line to draining 2019-08-15 14:41:22 +01:00
Brian Gough
a7a161556f Merge branch 'bg-status-on-shutdown' 2019-08-15 09:42:00 +01:00
Brian Gough
fa94e3d5e3 Merge pull request #69 from overleaf/ho-drain-connections-timewindow 
add shutdownDrainTimeWindow, drains all connections within time range
 2019-08-15 09:22:10 +01:00
Brian Gough
022e47b5c8 Merge pull request #73 from overleaf/bg-connected-client-metrics 
add connected client count metric
 2019-08-15 09:20:03 +01:00
Brian Gough
f13e66b453 fix client count so that result is zero when all clients have left 2019-08-14 15:34:23 +01:00
Brian Gough
8270c14d86 add connected client count metric 2019-08-14 15:22:03 +01:00
Brian Gough
d3171e4e2e remove unwanted argument 2019-08-14 13:03:06 +01:00
Brian Gough
20d442120f notify docupdate if the flush is from a shutdown 2019-08-13 17:36:53 +01:00
Brian Gough
0708f717fd reject connections when shutdown in progress 
send a message to the client to reconnect immediately
 2019-08-13 16:59:15 +01:00
Brian Gough
b3e5709b64 enforce a minimum drain rate 2019-08-13 16:15:30 +01:00
Henry Oswald
00cca29d9e add shutdownDrainTimeWindow, drains all connections within time range 2019-08-13 14:21:47 +01:00
Brian Gough
2000f478a7 refresh the client list on demand 2019-08-13 10:40:03 +01:00
Brian Gough
478a727c61 ignore spurious requests to leave other docs 2019-07-29 15:19:08 +01:00
Brian Gough
04a171171f fix async behaviour of join/leave 2019-07-29 11:54:02 +01:00
Brian Gough
22d722f3e8 add metric for RoomEvents listeners 2019-07-24 16:25:45 +01:00
Brian Gough
277ec71a5b subscribe to doc updates before requesting doc content 2019-07-24 15:49:29 +01:00
Brian Gough
1c74cbbc4e add comments 2019-07-24 15:49:29 +01:00
Brian Gough
273af3f3aa refactor subscribe resolution 2019-07-24 14:30:48 +01:00
Brian Gough
e14a94906a update naming from Set -> Map 2019-07-24 14:18:15 +01:00
Brian Gough
cb53bfafd6 remove unnecessary require 2019-07-24 09:52:31 +01:00
Brian Gough
61b3a000b4 fix whitespace 2019-07-24 09:52:20 +01:00
Brian Gough
159b39c491 ensure redis channel is subscribed when joining room 2019-07-23 17:02:09 +01:00
Brian Gough
84e6ff616f whitespace fix 2019-07-22 12:25:41 +01:00
Brian Gough
92e6910180 cleanup 2019-07-22 11:23:33 +01:00
Brian Gough
a538d10488 extend comment re disconnection 2019-07-19 08:56:38 +01:00
Brian Gough
616014e05d add comment about automatically leaving rooms 2019-07-19 08:50:43 +01:00
Brian Gough
3bf5dd5d6b clarify errors for subscribe/unsubscribe 2019-07-18 14:25:25 +01:00
Brian Gough
f6f6f549d9 don't publish on individual channels until explicitly set 2019-07-18 12:55:23 +01:00
Brian Gough
804f4c2bd2 listen on separate channels for each project/doc 2019-07-18 12:55:23 +01:00
Brian Gough
9ecce32ff9 Merge pull request #63 from overleaf/bg-log-out-of-order-events 
log out of order events now that the rate is lower
 2019-07-17 13:47:45 +01:00
Brian Gough
8a7804f0a7 make event order check a configuration setting 2019-07-15 13:45:34 +01:00
Brian Gough
24a4709cff log out of order events now that the rate is lower 2019-07-15 11:14:48 +01:00
Brian Gough
e632f9f29d only create per-client metrics when there are multiple redis clients 2019-07-11 11:35:48 +01:00
Brian Gough
80f8f2465e remove unused pubsub client 2019-07-11 11:10:33 +01:00
Brian Gough
689a75f397 add logging for redis clients at start up 2019-07-09 14:18:39 +01:00
Brian Gough
dd54789e2b fix build problems 2019-07-09 12:20:59 +01:00
Brian Gough
580b100362 only publish to one redis client in WebsocketLoadBalancer 
but listen to all of them
 2019-07-09 12:03:13 +01:00
Brian Gough
999cbd8ee6 add a per-client metric 2019-07-09 12:01:58 +01:00
Brian Gough
cb289f2dec make redis client list dynamic based on settings 2019-07-09 11:45:00 +01:00
Brian Gough
b5f9bc422b support multple redis instances for pubsub 2019-07-08 15:56:25 +01:00
Henry Oswald
8042a415ec move pubsub traffic over to a pubsub redis connection string 2019-07-04 13:38:39 +01:00
miguel
46dfe56b05 Downgraded unathorised log to warning 2019-06-21 07:30:12 +02:00
Brian Gough
a579a83389 Merge pull request #53 from overleaf/em-doc-updater-404s 
Do not log error on doc updater 404
 2019-06-03 10:55:04 +01:00