Jakob Ackermann
370baa06c3
[misc] bump the dev-env to 3.3.2
2020-08-10 17:10:31 +01:00
Jakob Ackermann
51e56dafcf
[misc] bump the dev-env to 3.3.1
2020-08-05 10:46:23 +01:00
Jakob Ackermann
1b52366c61
Merge pull request #174 from overleaf/jpa-bump-dev-env
...
[misc] bump the dev-env to 3.3.0
2020-08-04 12:29:22 +02:00
Jakob Ackermann
f4d9ae1150
[misc] bump the dev-env to 3.3.0
2020-08-04 11:07:19 +01:00
Jakob Ackermann
a91f78634b
[misc] bump socket.io and socket.io-client
2020-07-31 16:32:03 +01:00
Shane Kilkelly
3d92858939
Upgrade redis-sharelatex to 1.0.13
2020-07-24 10:42:46 +01:00
Jakob Ackermann
562375d351
[misc] fix express deprecations
2020-07-22 09:45:14 +01:00
Jakob Ackermann
ca4168ce90
[misc] use overleaf/socket.io and overleaf/socket.io-client forks
2020-07-08 15:07:19 +01:00
Jakob Ackermann
9fff03bca5
[misc] optionally expose the hostname in the 'debug.getHostname' rpc
2020-07-07 11:07:28 +01:00
Jakob Ackermann
aa9d6c8dc9
[misc] reland decaff cleanup ( #166 )
...
* [misc] decaff cleanup: RoomManager
* [misc] decaff cleanup: RedisClientManager
* [misc] decaff cleanup: SafeJsonParse
* [misc] decaff cleanup: WebApiManager
* [misc] decaff cleanup: WebsocketController
* [misc] decaff cleanup: WebsocketLoadBalancer
* [misc] decaff cleanup: SessionSockets
* [misc] decaff cleanup: HttpController
* [misc] decaff cleanup: HttpApiController
* [misc] decaff cleanup: HealthCheckManager
* [misc] decaff cleanup: EventLogger
* [misc] decaff cleanup: Errors
o-error will eliminate most of it -- when we migrate over.
* [misc] decaff cleanup: DrainManager
* [misc] decaff cleanup: DocumentUpdaterManager
* [misc] decaff cleanup: DocumentUpdaterController: no-unused-vars
* [misc] decaff cleanup: DocumentUpdaterController: Array.from
* [misc] decaff cleanup: DocumentUpdaterController: implicit return
* [misc] decaff cleanup: DocumentUpdaterController: IIFE
* [misc] decaff cleanup: DocumentUpdaterController: null checks
* [misc] decaff cleanup: DocumentUpdaterController: simpler loops
* [misc] decaff cleanup: DocumentUpdaterController: move module name def
* [misc] decaff cleanup: ConnectedUsersManager: handle-callback-err
* [misc] decaff cleanup: ConnectedUsersManager: implicit returns
* [misc] decaff cleanup: ConnectedUsersManager: null checks
* [misc] decaff cleanup: ChannelManager: no-unused-vars
* [misc] decaff cleanup: ChannelManager: implicit returns
* [misc] decaff cleanup: ChannelManager: other cleanup
- var -> const
- drop variable assignment before return
* [misc] decaff cleanup: AuthorizationManager: handle-callback-err
Note: This requires a change in WebsocketController to provide a dummy
callback.
* [misc] decaff cleanup: AuthorizationManager: Array.from
* [misc] decaff cleanup: AuthorizationManager: implicit returns
* [misc] decaff cleanup: AuthorizationManager: null checks
* [misc] decaff cleanup: Router: handle-callback-err
* [misc] decaff cleanup: Router: standard/no-callback-literal
* [misc] decaff cleanup: Router: Array.from
* [misc] decaff cleanup: Router: implicit returns
* [misc] decaff cleanup: Router: refactor __guard__ wrapper
* [misc] decaff cleanup: Router: null checks
And a minor bug fix: user.id -> user._id
* [misc] decaff cleanup: Router: move variable declarations to assignments
* [misc] decaff cleanup: app: implicit returns
* [misc] decaff cleanup: app: __guard__
* [misc] decaff cleanup: app: null checks
* [misc] decaff cleanup: app: function definitions
* [misc] decaff cleanup: app: drop unused next argument
* [misc] decaff cleanup: app: var -> const
2020-07-07 11:06:02 +01:00
Jakob Ackermann
8601084a2e
[misc] WebsocketControllerTests: remove debugging code
2020-07-01 11:36:36 +01:00
Jakob Ackermann
a77222470b
[misc] WebsocketControllerTests: always pass cb to updateClientPosition
...
...and fix the inconsistent async behaviour for logged-in vs anonymous
users in the app.
2020-06-29 18:11:33 +01:00
Jakob Ackermann
4d01c02946
[misc] WebsocketControllerTests: explicitly pass a cb to leaveProject
...
...and fix the stubbing of `io.sockets.clients`.
We were running the assertions prior to the actual completion of the
leaveProject. Which in turn shadowed the broken test setup --
`io.sockets.clients()` was returning the `this.clientsInRoom=[]` from a
previous context.
2020-06-29 17:38:22 +01:00
Jakob Ackermann
bea0e9beb7
[misc] RoomManagerTests: explicitly pass a cb to joinProject/joinDoc
2020-06-29 17:12:26 +01:00
Jakob Ackermann
5e191be171
[misc] replace console logging with logger-sharelatex
2020-06-24 11:12:22 +01:00
Jakob Ackermann
fa42166be3
[misc] ignore camelcase warning in settings (key_schema parameter)
2020-06-24 10:52:38 +01:00
Jakob Ackermann
9c7bfe020c
[misc] fix eslint errors in app.js
2020-06-24 10:52:38 +01:00
Jakob Ackermann
c76bcb7732
[misc] fix eslint errors in XMLHttpRequest.js
2020-06-24 10:52:37 +01:00
Jakob Ackermann
b8b3fb8b11
[misc] fix usage of deprecated node apis
2020-06-24 10:52:21 +01:00
decaffeinate
53058452ee
prettier: convert miscellaneous files to Prettier format
2020-06-24 10:28:28 +01:00
decaffeinate
92dede867f
prettier: convert individual decaffeinated files to Prettier format
2020-06-23 18:30:51 +01:00
decaffeinate
bdfca5f155
decaffeinate: convert individual files to js
2020-06-23 18:30:48 +01:00
decaffeinate
42f55c4651
decaffeinate: rename individual coffee files to js files
2020-06-23 18:30:46 +01:00
decaffeinate
8a7fc78dc8
prettier: convert test/acceptance decaffeinated files to Prettier format
2020-06-23 18:30:45 +01:00
decaffeinate
db98fdac0a
decaffeinate: rename test/acceptance/coffee to test/acceptance/js
2020-06-23 18:30:37 +01:00
decaffeinate
5443450abb
decaffeinate: Run post-processing cleanups on ApplyUpdateTests.coffee and 18 other files
2020-06-23 18:30:34 +01:00
decaffeinate
d318e4fd0e
decaffeinate: Convert ApplyUpdateTests.coffee and 18 other files to JS
2020-06-23 18:30:29 +01:00
decaffeinate
30a9c6ed2c
decaffeinate: Rename ApplyUpdateTests.coffee and 18 other files from .coffee to .js
2020-06-23 18:30:23 +01:00
decaffeinate
3eceb8a5f6
prettier: convert test/unit decaffeinated files to Prettier format
2020-06-23 18:30:16 +01:00
decaffeinate
68e2adebf5
decaffeinate: rename test/unit/coffee to test/unit/js
2020-06-23 18:30:06 +01:00
decaffeinate
93697a8c5c
decaffeinate: Run post-processing cleanups on AuthorizationManagerTests.coffee and 13 other files
2020-06-23 18:30:03 +01:00
decaffeinate
2ca620e7a0
decaffeinate: Convert AuthorizationManagerTests.coffee and 13 other files to JS
2020-06-23 18:29:59 +01:00
decaffeinate
e5d07bd3af
decaffeinate: Rename AuthorizationManagerTests.coffee and 13 other files from .coffee to .js
2020-06-23 18:29:53 +01:00
decaffeinate
817844515d
prettier: convert app/js decaffeinated files to Prettier format
2020-06-23 18:29:44 +01:00
decaffeinate
04a85a6716
decaffeinate: rename app/coffee dir to app/js
2020-06-23 18:29:41 +01:00
decaffeinate
a397154e18
decaffeinate: Run post-processing cleanups on AuthorizationManager.coffee and 18 other files
2020-06-23 18:29:38 +01:00
decaffeinate
7335084c26
decaffeinate: Convert AuthorizationManager.coffee and 18 other files to JS
2020-06-23 18:29:34 +01:00
decaffeinate
90eafa388a
decaffeinate: Rename AuthorizationManager.coffee and 18 other files from .coffee to .js
2020-06-23 18:29:29 +01:00
decaffeinate
20bb3540e7
decaffeinate: update .gitignore
2020-06-23 18:29:21 +01:00
decaffeinate
59083edb9e
decaffeinate: update build scripts to es
2020-06-23 18:29:17 +01:00
Jakob Ackermann
8ac973e767
[misc] help prettier/decaffeinate with scoping a comment into a fn body
2020-06-23 18:04:09 +01:00
Jakob Ackermann
205efa8812
[WebsocketController] reset the client context before (re)populating it
...
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2020-06-22 17:04:31 +01:00
Jakob Ackermann
1fcf534dcf
[Router] revert preserving of client.ol_context
...
We do not enter this line twice, it would result in multiple event
handlers too.
2020-06-22 17:04:31 +01:00
Jakob Ackermann
d17ef183d0
[Router] gracefully set and do not reset the ol_context
2020-06-22 17:04:30 +01:00
Jakob Ackermann
ce4f9148c3
[perf] WebsocketLoadBalancer: move back to a sync loop for msg fan out
2020-06-22 17:04:15 +01:00
Jakob Ackermann
a70c1e1fa2
[perf] WebsocketLoadBalancer: check is_restricted_message once
...
...and filter clients early on.
2020-06-22 17:04:15 +01:00
Jakob Ackermann
5282f8f531
[misc] synchronous client store using an Object at .ol_context
2020-06-22 17:04:08 +01:00
Miguel Serrano
f2cae26166
Merge pull request #157 from overleaf/msm-update-deps
...
Update Dependencies
2020-06-22 15:45:11 +02:00
Miguel Serrano
3ffc218b2c
Rolled back redis-sharelatex to 1.0.12
2020-06-22 13:13:44 +02:00
Miguel Serrano
ce3266c59c
Fixed sha downgrades
2020-06-19 17:29:31 +02:00
Miguel Serrano
1e1f6ca19f
Updated 'uid-safe' and minor/patch dependencies
2020-06-19 16:26:32 +02:00
Jakob Ackermann
5f7841526f
[misc] RoomManager: emitOnCompletion: properly handle Promise rejections
...
```
result = Promise.all([<Promise that rejects eventually>]) # rejection 1
result.then () -> RoomEvents.emit(eventName) # rejection 2
result.catch (err) -> RoomEvents.emit(eventName, err) # handle r1
```
As shown above, the second rejection remains unhandled. The fix is to
chain the `.catch()` onto the `.then()` Promise.
2020-06-17 09:29:12 +01:00
Jakob Ackermann
06a8dc8ec1
Merge pull request #155 from overleaf/jpa-backport-receiveupdate-tests
...
[backport] 107 and 116: Improve ReceiveUpdate acceptance tests
2020-06-10 11:28:25 +02:00
Jakob Ackermann
10eb30e572
Merge pull request #154 from overleaf/jpa-backport-httpcontroller-tests
...
[backport] 111 HttpController: 404 for missing clients and add tests
2020-06-10 11:28:17 +02:00
Jakob Ackermann
45ac2bdd97
Merge pull request #153 from overleaf/jpa-backport-drainmanager-tests
...
[backport] 108 and 112: DrainManager acceptance tests
2020-06-10 11:28:08 +02:00
Jakob Ackermann
cfe37dcbb5
Merge pull request #152 from overleaf/jpa-vendor-cookie
...
[misc] vendor a patched session.socket.io middleware
2020-06-10 11:27:58 +02:00
Jakob Ackermann
5b85e0154d
Merge pull request #151 from overleaf/jpa-perf-parser
...
[perf] add a few short cuts to the packet decoding
2020-06-10 11:27:50 +02:00
Jakob Ackermann
b2e4448992
[misc] test/acceptance: ReceiveUpdateTests: test remotely sent update
2020-06-10 09:53:48 +01:00
Jakob Ackermann
83e3ff0ed7
[misc] test/acceptance: ReceiveUpdateTests: add 2nd project/3rd client
...
...and check for cross project leakage.
2020-06-10 09:53:29 +01:00
Jakob Ackermann
56fda1f9b0
[misc] test/acceptance: use the correct redis instances
2020-06-10 09:43:15 +01:00
Jakob Ackermann
bc44494466
[HttpController] return 404 in case of a missing client and add tests
...
Add acceptance tests for the client view.
2020-06-10 09:41:36 +01:00
Jakob Ackermann
eabff1d6b2
[perf] test/acceptance: DrainManagerTests: cleanup previous clients
...
...before starting to drain.
2020-06-09 18:02:38 +01:00
Jakob Ackermann
de35fc5ecf
[HttpApiController] implement the disconnection of a single client
...
The http route returns as soon as the client has fully disconnected.
2020-06-09 18:01:08 +01:00
Jakob Ackermann
91e296533f
[misc] test/acceptance: add tests for the draining of connections
2020-06-09 18:00:44 +01:00
Jakob Ackermann
acb7d7df5a
[misc] add test cases for the validation of the callback argument
...
When the user provides a function as last argument for socket.emit,
socket.io will flag this as an RPC and add a cb as the last argument
to the client.on('event', ...) handler on the server side.
Without a function as last argument for socket.emit, the callback
argument on the server side is undefined, leading to invalid function
calls (`undefined()`) and an unhandled exception.
The user can also provide lots of other arguments, so the 2nd/3rd ...
argument is of arbitrary type, again leading to invalid function calls
-- e.g. `1()`.
2020-06-09 16:30:03 +01:00
Jakob Ackermann
853ee994a6
[perf] add a few short cuts to the packet decoding
2020-06-09 15:26:17 +01:00
Jakob Ackermann
dc553c4150
[misc] vendor a patched session.socket.io middleware
2020-06-09 15:21:33 +01:00
Jakob Ackermann
1c9eaf574a
Merge pull request #147 from overleaf/jpa-backport-141
...
[backport] 141: Router: validate the callback argument
2020-06-09 14:41:24 +02:00
Jakob Ackermann
5238e6ad36
Merge pull request #145 from overleaf/jpa-backport-140
...
[backport] 140: stop processing requests as we detect a client disconnect
2020-06-09 14:41:12 +02:00
Jakob Ackermann
589bedc3dd
Merge pull request #146 from overleaf/jpa-downgrade-logging
...
[misc] downgrade logging when running tests
2020-06-09 14:40:59 +02:00
Jakob Ackermann
32af7001fc
[misc] Router: prefix the publicId with 'P.' for easy differentiation
2020-06-08 11:29:40 +01:00
Jakob Ackermann
f40241a037
[misc] downgrade logging when running tests
2020-06-05 11:38:09 +01:00
Jakob Ackermann
0b2cccf1e0
[misc] apply review feedback: adjust metric names
...
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
(cherry-picked from commit 67674b83efb452ece05cdc39525ee3a5eeb8a4d7)
2020-06-05 11:33:00 +01:00
Jakob Ackermann
ddcb9cf8c8
[misc] downgrade a warning message from clients leaving non-joined rooms
...
This can now happen all the time, as we skip the join for clients that
disconnect before joinProject/joinDoc completed.
(cherry-picked from commit f357931de74e088800f3cced3898cce4f251dad0)
2020-06-05 11:32:43 +01:00
Jakob Ackermann
7fa9061015
[misc] stop processing requests as we detect a client disconnect
...
v2 expose `client.connected`; v0 exposes `client.disconnected`
(cherry-picked from commit a9d70484343ca9be367d45bf7bb949e4be449647)
2020-06-05 11:32:13 +01:00
Jakob Ackermann
0840700ffd
[Router] validate the callback argument
2020-06-05 10:59:01 +01:00
Jakob Ackermann
c6d08647c7
[misc] socket.io: use a secondary publicId for public facing usages
2020-06-04 17:18:07 +01:00
Brian Gough
f973b377f0
update node to v10.21.0
2020-06-03 09:12:21 +01:00
Tim Alby
94d57f50c0
add fake lint and format targets
...
Highly hacky!
Lint and format steps are coming very soon thanks to the decaf, but in the
meantime we need steps to pass CI. Updating the build scripts after the
decaf will undo this change.
2020-05-28 16:09:27 +02:00
Jakob Ackermann
d13acb8ca3
[ChannelManager] port related and still mostly valid test from v2
...
I skipped the bulk of verifyConsistentBehaviour tests which are not
valid for the new implementation -- there is no optimization and
always cleanup.
2020-05-15 18:39:33 +02:00
Jakob Ackermann
41debfae0f
[ChannelManager] rework (un)subscribing to redis
...
- send a subscribe request on every request
- wait for a pending unsubscribe request before subscribing
- wait for a pending subscribe request before unsubscribing
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2020-05-15 18:34:33 +02:00
Jakob Ackermann
1095851dfe
[misc] test/unit: fix typos and assertion of error messages
...
Sinon does not check the contents of the passed error when checked via
sinon.stub().calledWith.
```
callback = sinon.stub()
callback(new Error("some message"))
.calledWith(new Error("completely different message"))
=== true
```
Cherry-pick plus an additional patch for the joinProject bail-out.
(cherry picked from commit d9570fee70701a5f431c39fdbec5f8bc5a7843fe)
2020-05-15 14:46:05 +02:00
Jakob Ackermann
17d04b9041
[misc] bump sinon to 2.x for promise support with stubs
...
(cherry picked from commit 3c92b937f9430175d7c51660e03c507635448e88)
2020-05-15 12:01:22 +02:00
Jakob Ackermann
b713beb7f0
Merge pull request #135 from overleaf/jpa-skip-leave-project-for-invalid-clients
...
[WebsocketController] skip leaveProject when joinProject didn't complete
2020-05-13 15:34:41 +02:00
Jakob Ackermann
55af5e502f
[WebsocketController] skip leaveProject when joinProject didn't complete
...
Also drop dead code:
- user_id bailout
There is a check on a completed joinProject call now. It will always
set a user_id, see Router.coffee which has a fallback `{_id:"..."}`.
- late project_id bailout
WebsocketLoadBalancer.emitToRoom will not work without a project_id.
We have to bail out before the call.
2020-05-12 17:15:08 +02:00
Jakob Ackermann
684cb3903c
[WebsocketController] handle redis subscribe error on joinProject
...
joinProject should not complete when the redis pub/sub subscribe request
failed.
2020-05-12 13:03:50 +02:00
Brian Gough
5c28da1031
add metric for pendingUpdates queue
2020-04-07 11:53:54 +01:00
Brian Gough
5765884f38
Merge branch 'master' into jpa-pub-sub-metrics
2020-04-07 11:46:23 +01:00
Henry Oswald
8711abdb66
bump redis to 1.0.12
2020-03-31 14:04:33 +01:00
Jakob Ackermann
56628a16c6
[misc] track redis pub/sub payload sizes on publish
2020-03-30 11:31:44 +02:00
Jakob Ackermann
a9b8e9be3b
[misc] upgrade metrics-sharelatex to 2.6.2
2020-03-30 10:47:01 +02:00
Jakob Ackermann
69569e3571
[misc] config: add headroom for JSON serialization in maxUpdateSize
2020-03-24 16:21:29 +01:00
Jakob Ackermann
af53d3b603
[misc] skip duplicate JSON serialization for size check
2020-03-24 11:22:28 +01:00
Jakob Ackermann
cb675d38c2
[misc] SafeJsonParse: align the size limit with the frontend->rt limit
...
frontend -> real-time and doc-updater -> real-time should be in sync.
Otherwise we can send a payload to doc-updater, but can not receive the
confirmation of it -- and the client will send it again in a loop.
Also log the size of the payload.
2020-03-24 09:14:15 +01:00
Jakob Ackermann
15244a54be
[misc] WebsocketController: limit the update size to 7mb
...
bail out early on -- especially do not push the update into redis for
doc-updater to discard it.
Confirm the update silently, otherwise the frontend will send it again.
Broadcast a 'otUpdateError' message and disconnect the client, like
doc-updater would do.
2020-03-24 09:12:12 +01:00
Jakob Ackermann
d320c2d5f3
[misc] let proxys observe an upcoming shutdown before starting to drain
...
Otherwise clients may be routed to the same pod upon reconnecting.
2020-03-17 16:41:48 +01:00
Jakob Ackermann
2b1c5bf436
Merge pull request #100 from overleaf/jpa-dependencies-cleanup
...
[misc] cleanup dependencies
2020-02-14 10:56:11 +01:00
Jakob Ackermann
43013e0820
[misc] cleanup unused dependency on mongo
2020-02-12 14:44:01 +01:00
Jakob Ackermann
902b4fca46
[misc] rename npm-shrinkwrap.json to package-lock.json and run npm i
2020-02-12 14:39:53 +01:00
Brian Gough
1ad8315437
remove unused .travis.yml file
2020-02-12 12:37:00 +00:00