github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
23852 commits 3 branches 7 tags 206 MiB
Commit graph

202 commits

Author SHA1 Message Date
Jakob Ackermann
ee3d3b09ed [misc] wrap redis errors as tagging does not work with them 
ioredis may reuse the error instance for multiple callbacks. E.g. when
 the connection to redis fails, the queue is flushed with the same
 MaxRetriesPerRequestError instance.
 2020-08-24 10:12:20 +01:00
Jakob Ackermann
537e97be73 [misc] OError.tag all the errors in async contexts 
See the docs of OError.tag:
https://github.com/overleaf/o-error#long-stack-traces-with-oerrortag
(currently at 221dd902e7bfa0ee92de1ea5a3cbf3152c3ceeb4)

I am tagging all errors at each async hop. Most of the controller code
 will only ever see already tagged errors -- or new errors created in
 our app code. They should have enough info that we do not need to tag
 them again.
 2020-08-24 10:12:06 +01:00
Jakob Ackermann
4cb8cc4a85 [DocumentUpdaterManager] use a new ClientRequestedMissingOpsError 2020-08-21 12:47:07 +01:00
Jakob Ackermann
8abfdb87ff [DocumentUpdaterManager] use a new DocumentUpdaterRequestFailedError 2020-08-21 12:47:07 +01:00
Jakob Ackermann
68bc9d0d23 [WebApiManager] use a new WebApiRequestFailedError 2020-08-21 12:47:06 +01:00
Jakob Ackermann
af50f9b02c [DocumentUpdaterManager] use a new UpdateTooLargeError 2020-08-21 12:47:05 +01:00
Jakob Ackermann
5950b26a42 [SafeJsonParse] migrate to OError and use a new DataTooLargeToParseError 2020-08-21 12:47:05 +01:00
Jakob Ackermann
f82177a46a [Errors] migrate to OError 2020-08-21 12:47:04 +01:00
Brian Gough
831d794bf4 clean up join/leave handling 
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
 2020-08-12 10:54:22 +01:00
Jakob Ackermann
8601084a2e [misc] WebsocketControllerTests: remove debugging code 2020-07-01 11:36:36 +01:00
Jakob Ackermann
a77222470b [misc] WebsocketControllerTests: always pass cb to updateClientPosition 
...and fix the inconsistent async behaviour for logged-in vs anonymous
 users in the app.
 2020-06-29 18:11:33 +01:00
Jakob Ackermann
4d01c02946 [misc] WebsocketControllerTests: explicitly pass a cb to leaveProject 
...and fix the stubbing of `io.sockets.clients`.

We were running the assertions prior to the actual completion of the
 leaveProject. Which in turn shadowed the broken test setup --
 `io.sockets.clients()` was returning the `this.clientsInRoom=[]` from a
 previous context.
 2020-06-29 17:38:22 +01:00
Jakob Ackermann
bea0e9beb7 [misc] RoomManagerTests: explicitly pass a cb to joinProject/joinDoc 2020-06-29 17:12:26 +01:00
Jakob Ackermann
c76bcb7732 [misc] fix eslint errors in XMLHttpRequest.js 2020-06-24 10:52:37 +01:00
decaffeinate
53058452ee prettier: convert miscellaneous files to Prettier format 2020-06-24 10:28:28 +01:00
decaffeinate
8a7fc78dc8 prettier: convert test/acceptance decaffeinated files to Prettier format 2020-06-23 18:30:45 +01:00
decaffeinate
db98fdac0a decaffeinate: rename test/acceptance/coffee to test/acceptance/js 2020-06-23 18:30:37 +01:00
decaffeinate
5443450abb decaffeinate: Run post-processing cleanups on ApplyUpdateTests.coffee and 18 other files 2020-06-23 18:30:34 +01:00
decaffeinate
d318e4fd0e decaffeinate: Convert ApplyUpdateTests.coffee and 18 other files to JS 2020-06-23 18:30:29 +01:00
decaffeinate
30a9c6ed2c decaffeinate: Rename ApplyUpdateTests.coffee and 18 other files from .coffee to .js 2020-06-23 18:30:23 +01:00
decaffeinate
3eceb8a5f6 prettier: convert test/unit decaffeinated files to Prettier format 2020-06-23 18:30:16 +01:00
decaffeinate
68e2adebf5 decaffeinate: rename test/unit/coffee to test/unit/js 2020-06-23 18:30:06 +01:00
decaffeinate
93697a8c5c decaffeinate: Run post-processing cleanups on AuthorizationManagerTests.coffee and 13 other files 2020-06-23 18:30:03 +01:00
decaffeinate
2ca620e7a0 decaffeinate: Convert AuthorizationManagerTests.coffee and 13 other files to JS 2020-06-23 18:29:59 +01:00
decaffeinate
e5d07bd3af decaffeinate: Rename AuthorizationManagerTests.coffee and 13 other files from .coffee to .js 2020-06-23 18:29:53 +01:00
Jakob Ackermann
8ac973e767 [misc] help prettier/decaffeinate with scoping a comment into a fn body 2020-06-23 18:04:09 +01:00
Jakob Ackermann
5282f8f531 [misc] synchronous client store using an Object at .ol_context 2020-06-22 17:04:08 +01:00
Jakob Ackermann
5f7841526f [misc] RoomManager: emitOnCompletion: properly handle Promise rejections 
```
result = Promise.all([<Promise that rejects eventually>]) # rejection 1
result.then () -> RoomEvents.emit(eventName)              # rejection 2
result.catch (err) -> RoomEvents.emit(eventName, err)     # handle r1
```
As shown above, the second rejection remains unhandled. The fix is to
 chain the `.catch()` onto the `.then()` Promise.
 2020-06-17 09:29:12 +01:00
Jakob Ackermann
06a8dc8ec1 Merge pull request #155 from overleaf/jpa-backport-receiveupdate-tests 
[backport] 107 and 116: Improve ReceiveUpdate acceptance tests
 2020-06-10 11:28:25 +02:00
Jakob Ackermann
10eb30e572 Merge pull request #154 from overleaf/jpa-backport-httpcontroller-tests 
[backport] 111 HttpController: 404 for missing clients and add tests
 2020-06-10 11:28:17 +02:00
Jakob Ackermann
45ac2bdd97 Merge pull request #153 from overleaf/jpa-backport-drainmanager-tests 
[backport] 108 and 112: DrainManager acceptance tests
 2020-06-10 11:28:08 +02:00
Jakob Ackermann
cfe37dcbb5 Merge pull request #152 from overleaf/jpa-vendor-cookie 
[misc] vendor a patched session.socket.io middleware
 2020-06-10 11:27:58 +02:00
Jakob Ackermann
b2e4448992 [misc] test/acceptance: ReceiveUpdateTests: test remotely sent update 2020-06-10 09:53:48 +01:00
Jakob Ackermann
83e3ff0ed7 [misc] test/acceptance: ReceiveUpdateTests: add 2nd project/3rd client 
...and check for cross project leakage.
 2020-06-10 09:53:29 +01:00
Jakob Ackermann
56fda1f9b0 [misc] test/acceptance: use the correct redis instances 2020-06-10 09:43:15 +01:00
Jakob Ackermann
bc44494466 [HttpController] return 404 in case of a missing client and add tests 
Add acceptance tests for the client view.
 2020-06-10 09:41:36 +01:00
Jakob Ackermann
eabff1d6b2 [perf] test/acceptance: DrainManagerTests: cleanup previous clients 
...before starting to drain.
 2020-06-09 18:02:38 +01:00
Jakob Ackermann
91e296533f [misc] test/acceptance: add tests for the draining of connections 2020-06-09 18:00:44 +01:00
Jakob Ackermann
acb7d7df5a [misc] add test cases for the validation of the callback argument 
When the user provides a function as last argument for socket.emit,
 socket.io will flag this as an RPC and add a cb as the last argument
 to the client.on('event', ...) handler on the server side.
Without a function as last argument for socket.emit, the callback
 argument on the server side is undefined, leading to invalid function
 calls (`undefined()`) and an unhandled exception.
The user can also provide lots of other arguments, so the 2nd/3rd ...
 argument is of arbitrary type, again leading to invalid function calls
 -- e.g. `1()`.
 2020-06-09 16:30:03 +01:00
Jakob Ackermann
dc553c4150 [misc] vendor a patched session.socket.io middleware 2020-06-09 15:21:33 +01:00
Jakob Ackermann
5238e6ad36 Merge pull request #145 from overleaf/jpa-backport-140 
[backport] 140: stop processing requests as we detect a client disconnect
 2020-06-09 14:41:12 +02:00
Jakob Ackermann
589bedc3dd Merge pull request #146 from overleaf/jpa-downgrade-logging 
[misc] downgrade logging when running tests
 2020-06-09 14:40:59 +02:00
Jakob Ackermann
f40241a037 [misc] downgrade logging when running tests 2020-06-05 11:38:09 +01:00
Jakob Ackermann
0b2cccf1e0 [misc] apply review feedback: adjust metric names 
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>

(cherry-picked from commit 67674b83efb452ece05cdc39525ee3a5eeb8a4d7)
 2020-06-05 11:33:00 +01:00
Jakob Ackermann
ddcb9cf8c8 [misc] downgrade a warning message from clients leaving non-joined rooms 
This can now happen all the time, as we skip the join for clients that
 disconnect before joinProject/joinDoc completed.

(cherry-picked from commit f357931de74e088800f3cced3898cce4f251dad0)
 2020-06-05 11:32:43 +01:00
Jakob Ackermann
7fa9061015 [misc] stop processing requests as we detect a client disconnect 
v2 expose `client.connected`; v0 exposes `client.disconnected`

(cherry-picked from commit a9d70484343ca9be367d45bf7bb949e4be449647)
 2020-06-05 11:32:13 +01:00
Jakob Ackermann
c6d08647c7 [misc] socket.io: use a secondary publicId for public facing usages 2020-06-04 17:18:07 +01:00
Jakob Ackermann
d13acb8ca3 [ChannelManager] port related and still mostly valid test from v2 
I skipped the bulk of verifyConsistentBehaviour tests which are not
 valid for the new implementation -- there is no optimization and
 always cleanup.
 2020-05-15 18:39:33 +02:00
Jakob Ackermann
1095851dfe [misc] test/unit: fix typos and assertion of error messages 
Sinon does not check the contents of the passed error when checked via
 sinon.stub().calledWith.
```
callback = sinon.stub()
callback(new Error("some message"))
  .calledWith(new Error("completely different message"))
  === true
```

Cherry-pick plus an additional patch for the joinProject bail-out.
(cherry picked from commit d9570fee70701a5f431c39fdbec5f8bc5a7843fe)
 2020-05-15 14:46:05 +02:00
Jakob Ackermann
b713beb7f0 Merge pull request #135 from overleaf/jpa-skip-leave-project-for-invalid-clients 
[WebsocketController] skip leaveProject when joinProject didn't complete
 2020-05-13 15:34:41 +02:00
Jakob Ackermann
55af5e502f [WebsocketController] skip leaveProject when joinProject didn't complete 
Also drop dead code:
 - user_id bailout

   There is a check on a completed joinProject call now. It will always
    set a user_id, see Router.coffee which has a fallback `{_id:"..."}`.

 - late project_id bailout

   WebsocketLoadBalancer.emitToRoom will not work without a project_id.
   We have to bail out before the call.
 2020-05-12 17:15:08 +02:00
Jakob Ackermann
684cb3903c [WebsocketController] handle redis subscribe error on joinProject 
joinProject should not complete when the redis pub/sub subscribe request
 failed.
 2020-05-12 13:03:50 +02:00
Brian Gough
5c28da1031 add metric for pendingUpdates queue 2020-04-07 11:53:54 +01:00
Jakob Ackermann
56628a16c6 [misc] track redis pub/sub payload sizes on publish 2020-03-30 11:31:44 +02:00
Jakob Ackermann
af53d3b603 [misc] skip duplicate JSON serialization for size check 2020-03-24 11:22:28 +01:00
Jakob Ackermann
cb675d38c2 [misc] SafeJsonParse: align the size limit with the frontend->rt limit 
frontend -> real-time and doc-updater -> real-time should be in sync.
Otherwise we can send a payload to doc-updater, but can not receive the
 confirmation of it -- and the client will send it again in a loop.

Also log the size of the payload.
 2020-03-24 09:14:15 +01:00
Jakob Ackermann
15244a54be [misc] WebsocketController: limit the update size to 7mb 
bail out early on -- especially do not push the update into redis for
 doc-updater to discard it.

Confirm the update silently, otherwise the frontend will send it again.
Broadcast a 'otUpdateError' message and disconnect the client, like
 doc-updater would do.
 2020-03-24 09:12:12 +01:00
Shane Kilkelly
6df88ebc49 Filter "comments" if restricted user. 2019-10-30 13:54:40 +00:00
Shane Kilkelly
403caa65e8 Revert "Revert "Track the isRestrictedUser flag on clients"" 
This reverts commit 651e392a7c644403f199e1b03e7494b61ce71d0c.
 2019-10-30 13:52:36 +00:00
Simon Detheridge
925a8651c1 Revert "Track the isRestrictedUser flag on clients" 2019-10-22 10:17:38 +01:00
Shane Kilkelly
06aa578bdc Make it an error when we get no data from joinProject 2019-10-11 09:57:16 +01:00
Shane Kilkelly
df6cd4a054 Also block getConnectedUsers for restricted users. 
Plus refactor to use a pass list instead of a deny list.
 2019-10-04 13:41:49 +01:00
Shane Kilkelly
6765d03339 Track the isRestrictedUser flag on clients 
Then, don't send new chat messages and new comments to those restricted clients.
We do this because we don't want to leak private information (email addresses
and names) to "restricted" users, those who have read-only access via a
shared token.
 2019-10-04 10:30:24 +01:00
Simon Detheridge
21e294c6eb Generate retryable error when hitting rate limits in web 2019-09-02 11:27:04 +01:00
Brian Gough
fe2e7b3065 minimal fix for undefined connected users 2019-08-16 10:07:30 +01:00
Brian Gough
fa94e3d5e3 Merge pull request #69 from overleaf/ho-drain-connections-timewindow 
add shutdownDrainTimeWindow, drains all connections within time range
 2019-08-15 09:22:10 +01:00
Brian Gough
d57b229e17 update tests 2019-08-14 13:03:14 +01:00
Brian Gough
7db882f339 fix unit tests 2019-08-13 17:26:49 +01:00
Henry Oswald
00cca29d9e add shutdownDrainTimeWindow, drains all connections within time range 2019-08-13 14:21:47 +01:00
Brian Gough
478a727c61 ignore spurious requests to leave other docs 2019-07-29 15:19:08 +01:00
Brian Gough
04a171171f fix async behaviour of join/leave 2019-07-29 11:54:02 +01:00
Brian Gough
22d722f3e8 add metric for RoomEvents listeners 2019-07-24 16:25:45 +01:00
Brian Gough
e14a94906a update naming from Set -> Map 2019-07-24 14:18:15 +01:00
Brian Gough
61b3a000b4 fix whitespace 2019-07-24 09:52:20 +01:00
Brian Gough
159b39c491 ensure redis channel is subscribed when joining room 2019-07-23 17:02:09 +01:00
Brian Gough
bb629c27a1 rename unit test ChannelManager to ChannelManagerTests 2019-07-22 11:28:49 +01:00
Brian Gough
1afebd12a1 unit tests 2019-07-22 11:23:43 +01:00
Brian Gough
9f7df5f10c wip unit tests 2019-07-19 11:58:40 +01:00
Brian Gough
40353a410f fix unit tests 2019-07-19 08:49:57 +01:00
Brian Gough
dd54789e2b fix build problems 2019-07-09 12:20:59 +01:00
Brian Gough
b5f9bc422b support multple redis instances for pubsub 2019-07-08 15:56:25 +01:00
miguel
46dfe56b05 Downgraded unathorised log to warning 2019-06-21 07:30:12 +02:00
Brian Gough
a579a83389 Merge pull request #53 from overleaf/em-doc-updater-404s 
Do not log error on doc updater 404
 2019-06-03 10:55:04 +01:00
Brian Gough
9da98614c7 Merge pull request #54 from overleaf/bg-use-background-flush-on-disconnect 
use background flush on disconnect
 2019-06-03 10:53:43 +01:00
Brian Gough
40f3456b1c update unit test 2019-06-03 10:34:59 +01:00
Eric Mc Sween
acf850bce9 Do not log error on doc updater 404 
Document updater 404s are not indicative of a problem, but just of the client
trying to connect to a deleted document, which is easy to trigger. We
log a warning instead.
 2019-05-31 17:32:21 -04:00
Henry Oswald
2c9b222437 fix breaking tests from using ioredis with inbuilt promises 
because a promise is returned from ioredis it errors in mocha as it
can't take a promise and a callback
 2019-05-30 10:58:05 +01:00
Brian Gough
20d5cc69a4 filter invalid updates 2019-05-24 10:19:02 +01:00
Brian Gough
2dbdcf5bc8 add health check to pubsub channels 2019-04-15 14:17:55 +01:00
Brian Gough
6374a641d5 fix unit tests 2019-04-11 15:07:42 +01:00
Brian Gough
893515e83f handle duplicate entries in io.sockets.clients 2019-04-09 14:48:00 +01:00
Brian Gough
936311f1fb fix eventlogger tests to use name with dashes 2019-03-22 11:20:38 +00:00
Brian Gough
efa83b4cde stub out eventlogger in unit test 2019-03-22 11:19:31 +00:00
Brian Gough
befe4be517 add check for duplicate events 2019-03-19 14:22:11 +00:00
Chrystal Griffiths
507c4e5ce2 Fix test to accommodate changes to anonymous users 2019-02-15 15:18:17 +00:00
Chrystal Maria Griffiths
0b88a63444 Merge pull request #35 from sharelatex/cmg-anonymous-display-name 
Nameless logged in user labelled as anonymous
 2019-02-15 14:53:18 +00:00
Chrystal Griffiths
bb06f82e04 Still send cursorData for logged in users 2019-02-12 14:00:47 +00:00
Chrystal Griffiths
cb12e1c6f6 Send an empty string for every nameless user 2019-02-08 15:39:51 +00:00
Henry Oswald
a07e516f65 update redis key 2019-02-04 10:22:26 +00:00
Henry Oswald
bbd88e75eb fix broken tests 2018-12-05 15:41:12 +00:00