github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-10-31 21:21:03 -04:00
Code Issues Projects Releases Packages Wiki Activity
18276 commits 3 branches 7 tags 199 MiB
Commit graph

583 commits

Author SHA1 Message Date
Brian Gough
a55aa61d71 use .includes instead of .indexOf 2021-03-29 11:13:59 +01:00
Eric Mc Sween
61f43bf5d8 Upgrade metrics module to 3.5.1 2021-03-25 08:10:43 -04:00
Brian Gough
32184330d9 delay closing by 1 minute for deployment flip 2021-03-22 11:26:02 +00:00
Brian Gough
17e3068499 close real-time via a status file 2021-03-22 10:46:25 +00:00
Eric Mc Sween
b27cb4ec12 Upgrade to Node 12 2021-03-18 16:41:31 -04:00
Eric Mc Sween
b9e7cbf73b Add global test setup 
Configure chai and SandboxedModule in a central place. Configure
SandboxedModule globals that are needed in Node 12.
 2021-03-18 16:41:31 -04:00
Jakob Ackermann
17c3d7729e Merge remote-tracking branch 'origin/master' into jpa-let-mocha-catch-errors 2021-03-02 08:54:25 +00:00
Jakob Ackermann
a26a497743 [misc] let mocha catch errors when running acceptance tests 2021-03-01 15:51:55 +00:00
Jakob Ackermann
b7b13a90d1 [misc] ApplyUpdateTests: process all the pending-updates-lists 2021-03-01 15:26:18 +00:00
Brian Gough
22762f698c Merge pull request #201 from overleaf/bg-add-health-check-comment 
add comment about health checks on '/'
 2021-02-25 11:41:54 +00:00
Henry Oswald
27e7b77bdb bump default key shards to 10 2021-02-16 10:53:35 +00:00
Henry Oswald
49bed6595d added queueKey to error info when trying to push to redis 2021-02-09 12:42:58 +00:00
Henry Oswald
46b389e8b3 fix off by 1 error in key sharding 2021-02-09 10:48:40 +00:00
Henry Oswald
31e1808dd8 shard the pending-updates-list queue 2021-02-08 16:02:41 +00:00
Henry Oswald
c308e60b6b speed up DocumentUpdaterControllerTests by directly injecting redis into 
tests
 2021-02-08 11:13:22 +00:00
Jakob Ackermann
c2f8f26da4 [misc] bump the node version to 10.23.1 2021-01-05 18:26:52 +00:00
John Lees-Miller
8fec6723d8 Add correct license 2020-12-03 10:27:38 +00:00
Brian Gough
1ddcdf38ca add comment about health checks on '/' 2020-11-27 14:42:42 +00:00
Jakob Ackermann
8a5858f430 Merge pull request #200 from overleaf/jpa-bump-metrics-module 
[misc] bump metrics module to 3.4.1
 2020-11-26 13:13:50 +00:00
Jakob Ackermann
746c5aeb80 [misc] bump metrics module to 3.4.1 
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
 2020-11-25 11:57:22 +00:00
Eric Mc Sween
96126ecf34 Upgrade build-scripts to 3.4.0 
This version fixes docker-compose health checks for dependent services. See
https://github.com/overleaf/dev-environment/pull/409 for details.
 2020-11-24 08:03:29 -05:00
Jakob Ackermann
15af5c7977 [misc] bump @overleaf/redis-wrapper to version 2.0.0 2020-11-11 16:24:22 +00:00
Jakob Ackermann
dea6968fa9 [misc] explicitly install underscore 
Use the same (outdated) version that is already in place.
 2020-11-11 16:23:12 +00:00
Jakob Ackermann
222caeb9a8 Merge pull request #196 from overleaf/jpa-drop-duplicate-join 
[RoomManager] drop duplicate joining of entities
 2020-10-19 17:01:40 +02:00
Jakob Ackermann
d6ac8c14e7 [RoomManager] drop duplicate joining of entities 
REF: 0437e1d03f89a058f97a8884e3532a9a58b68b9d
REF: 62be5e29e5232150e7063bc189c5ad8a1189f972
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2020-10-19 15:54:37 +01:00
Jakob Ackermann
b20cdc0122 [misc] bump the dev-env to 3.3.5 -- drop custom unit test timeout 2020-10-19 15:20:57 +01:00
Jakob Ackermann
e8911babe0 Merge pull request #182 from overleaf/dependabot-npm_and_yarn-lodash-4.17.20 
Bump lodash from 4.17.15 to 4.17.20
 2020-10-08 10:42:21 +02:00
Jakob Ackermann
e9c23b6409 Merge pull request #191 from overleaf/dependabot-npm_and_yarn-node-fetch-2.6.1 
Bump node-fetch from 2.6.0 to 2.6.1
 2020-10-08 10:42:08 +02:00
Jakob Ackermann
5b53832bf2 Merge pull request #192 from overleaf/jpa-bump-dev-env-3-3-4-node-10-22-1 
[misc] bump the dev-env to 3.3.4 and bump the node version to 10.22.1
 2020-10-08 10:41:56 +02:00
Jakob Ackermann
e846192db0 [MatrixTests] add a large testing matrix 
Layers/Dimensions:
- users: anonymous, registered, registeredWithOwnedProject
- session setup: noop, joinReadWriteProject, joinReadWriteProjectAndDoc,
   joinOwnProject, joinOwnProjectAndDoc
- invalid requests: noop, joinProjectWithDocId, joinDocWithDocId,
   joinProjectWithProjectId, joinDocWithProjectId,
   joinProjectWithProjectIdThenJoinDocWithDocId
 2020-10-07 16:20:21 +01:00
Jakob Ackermann
4ec73acb88 Merge branches 'jpa-prepare-for-matrix-tests' and 'jpa-setup-session-helper' into jpa-matrix-tests 2020-10-01 12:57:40 +01:00
Jakob Ackermann
2900c60c4a [FixturesManager] add a helper for setting up project and doc together 2020-10-01 12:33:15 +01:00
Jakob Ackermann
78fbd04ef8 [MockWebServer] grant users at least the privileges of anonymous users 2020-10-01 12:28:39 +01:00
Jakob Ackermann
4f860995d8 [MockDocUpdaterServer] return a 404 when a requested doc does not exist 2020-10-01 12:25:41 +01:00
Jakob Ackermann
cc681a94f4 [app] ignore error from writing to disconnected long-polling client 2020-09-22 14:10:23 +01:00
Jakob Ackermann
f3c7619fb5 [misc] revert back to high timeout for unit tests 2020-09-17 15:34:03 +01:00
Jakob Ackermann
ba3b4f7dcd [misc] bump the dev-env to 3.3.4 and bump the node version to 10.22.1 2020-09-17 15:26:45 +01:00
dependabot[bot]
ee06705377 Bump lodash from 4.17.15 to 4.17.20 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.20.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.20)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-09-16 10:38:21 +00:00
dependabot[bot]
10c6e01bd8 Bump node-fetch from 2.6.0 to 2.6.1 
Bumps [node-fetch](https://github.com/bitinn/node-fetch) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/bitinn/node-fetch/releases)
- [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-09-12 15:08:05 +00:00
Jakob Ackermann
a2f5bf463e Merge pull request #190 from overleaf/jpa-socket-io-logger 
[misc] socket.io: use a custom logger
 2020-09-02 10:46:23 +02:00
Jakob Ackermann
c6f2a3b387 Merge pull request #189 from overleaf/jpa-relevel-log 
[misc] re-level expected error log messages
 2020-09-02 10:46:08 +02:00
Jakob Ackermann
72282a07df [misc] socket.io: use a custom logger 
- forward the previously enabled log messages to our logger-module
- stub the previously disabled logger methods
- drop the log-level config for socket.io
 2020-09-01 09:30:11 +01:00
Jakob Ackermann
4960bdd6fe [misc] re-level log: 404 from web -> WARN and emit 'project not found' 
A stale browser tab tried to join a deleted project.
Emitting 'project not found'/'ProjectNotFound' will trigger a page
 reload in the frontend, upon web can render a 404.
See frontend: ConnectionManager.joinProject callback
 2020-08-27 11:51:57 +01:00
Jakob Ackermann
55d938ba14 [misc] add tests for web replying with a 404 for joinProject 2020-08-27 11:51:56 +01:00
Jakob Ackermann
884b340c75 [misc] re-level log: 403 from web goes to WARN and emit 'not authorized' 
Users will get redirected to the login page and will see a 'restricted'
 page after logging in again.
See frontend: ConnectionManager.reportConnectionError
 2020-08-27 11:51:56 +01:00
Jakob Ackermann
d2a2b9d46e [misc] add tests for web replying with a 403 for joinProject 2020-08-27 11:51:56 +01:00
Jakob Ackermann
1ff9c1e71b [misc] add the rpc-method into the log context in Router._handleError 2020-08-27 11:51:55 +01:00
Jakob Ackermann
0647abf433 [misc] drop info-log in WebApiManager for joinProject being rate-limited 
The CodedError is logged at warn-level in Router._handleError.
 2020-08-27 11:51:09 +01:00
Jakob Ackermann
2ce7b36c95 [misc] drop duplicate log line for unauthorized applyOtUpdate calls 
The violation is logged in Router._handleError.
 2020-08-27 10:22:31 +01:00
Jakob Ackermann
dee4749e6d [misc] re-level log: properly silence unauthorized updateClientPosition 2020-08-27 10:11:40 +01:00
Jakob Ackermann
425052ff91 Merge pull request #187 from overleaf/jpa-o-error-tagging 
[misc] migrate to OError tagging/wrapping
 2020-08-25 11:46:28 +02:00
Jakob Ackermann
64e659bf43 Merge pull request #186 from overleaf/jpa-fix-join-project-error-context 
[misc] fix join project error context
 2020-08-25 11:46:18 +02:00
Jakob Ackermann
849a1cf416 Merge pull request #185 from overleaf/jpa-doc-id-in-error-context 
[misc] add/bring back doc_id in error context
 2020-08-25 11:42:41 +02:00
Jakob Ackermann
ee3d3b09ed [misc] wrap redis errors as tagging does not work with them 
ioredis may reuse the error instance for multiple callbacks. E.g. when
 the connection to redis fails, the queue is flushed with the same
 MaxRetriesPerRequestError instance.
 2020-08-24 10:12:20 +01:00
Jakob Ackermann
537e97be73 [misc] OError.tag all the errors in async contexts 
See the docs of OError.tag:
https://github.com/overleaf/o-error#long-stack-traces-with-oerrortag
(currently at 221dd902e7bfa0ee92de1ea5a3cbf3152c3ceeb4)

I am tagging all errors at each async hop. Most of the controller code
 will only ever see already tagged errors -- or new errors created in
 our app code. They should have enough info that we do not need to tag
 them again.
 2020-08-24 10:12:06 +01:00
Jakob Ackermann
8e31cc5c23 [Router] _handleError: joinProject error-context may not have project_id 
The ol_context patch changed the priority of client context and rpc
 context.
This lead to the (possibly missing) project_id of the client context
 overwriting the project_id of the rpc context.
REF: f1d55c0a5437a518e9f4617473caed9ba928e648
 2020-08-21 13:29:28 +01:00
Jakob Ackermann
f935b1881a [Router] leaveDoc: pass the doc_id into the error-context 2020-08-21 12:47:42 +01:00
Jakob Ackermann
fd88819eec [Router] _handleError: ol_context.doc_id does not exist, drop overwrite 2020-08-21 12:47:42 +01:00
Jakob Ackermann
880056d397 [Router] use a new UnexpectedArgumentsError 2020-08-21 12:47:08 +01:00
Jakob Ackermann
50140f785a [WebsocketController] use a new JoinLeaveEpochMismatchError 2020-08-21 12:47:08 +01:00
Jakob Ackermann
0462e3e437 [WebsocketController] use a new NotJoinedError 2020-08-21 12:47:07 +01:00
Jakob Ackermann
4cb8cc4a85 [DocumentUpdaterManager] use a new ClientRequestedMissingOpsError 2020-08-21 12:47:07 +01:00
Jakob Ackermann
8abfdb87ff [DocumentUpdaterManager] use a new DocumentUpdaterRequestFailedError 2020-08-21 12:47:07 +01:00
Jakob Ackermann
02a2382264 [WebApiManager] use a new CorruptedJoinProjectResponseError 2020-08-21 12:47:07 +01:00
Jakob Ackermann
68bc9d0d23 [WebApiManager] use a new WebApiRequestFailedError 2020-08-21 12:47:06 +01:00
Jakob Ackermann
59c4c884a5 [WebsocketController] use the new NotAuthorizedError 2020-08-21 12:47:06 +01:00
Jakob Ackermann
a8c51de510 [AuthorizationManager] use a new NotAuthorizedError 2020-08-21 12:47:06 +01:00
Jakob Ackermann
de518ea4eb [SessionSockets] use a new MissingSessionError 2020-08-21 12:47:05 +01:00
Jakob Ackermann
6828becb46 [DocumentUpdaterManager] use a new NullBytesInOpError 2020-08-21 12:47:05 +01:00
Jakob Ackermann
af50f9b02c [DocumentUpdaterManager] use a new UpdateTooLargeError 2020-08-21 12:47:05 +01:00
Jakob Ackermann
5950b26a42 [SafeJsonParse] migrate to OError and use a new DataTooLargeToParseError 2020-08-21 12:47:05 +01:00
Jakob Ackermann
f82177a46a [Errors] migrate to OError 2020-08-21 12:47:04 +01:00
Jakob Ackermann
c7b9dadd16 [misc] increase the timeout for unit tests -- CI boxes are too slow 2020-08-21 12:45:30 +01:00
Jakob Ackermann
7ce76775ea Merge pull request #181 from overleaf/jpa-forcefully-disconnect-from-shutdown-process 
[misc] forcefully disconnect from shutdown process
 2020-08-17 12:56:18 +02:00
Jakob Ackermann
1ffffbcbc2 Merge pull request #180 from overleaf/jpa-harden-cleanup-take2 
[misc] bump socket.io and socket.io-client
 2020-08-17 12:56:04 +02:00
Jakob Ackermann
67d9543cf3 Merge pull request #179 from overleaf/csh-issue-1123-bump-logger-220 
[misc] bump logger-sharelatex to version 2.2.0
 2020-08-17 12:55:40 +02:00
Jakob Ackermann
3505fae253 Merge pull request #178 from overleaf/jpa-ignore-broken-socket-errors 
[app] ignore errors from accessing disconnected client sockets
 2020-08-17 12:55:29 +02:00
Jakob Ackermann
1550e9f885 Merge pull request #176 from overleaf/jpa-bump-dev-env-3-3-2 
[misc] bump the dev-env to 3.3.2
 2020-08-17 12:55:13 +02:00
Jakob Ackermann
83b8b077fa [misc] bump socket.io and socket.io-client 2020-08-14 11:16:26 +01:00
Jakob Ackermann
ea75b84eef [misc] let the orchestrator handle the process restart 
Note that there is also the `shutdownCleanly` interval which may notice
 that the shutdown has completed. There is some network IO required to
 signal all clients the server disconnect, so we cannot run process.exit
 immediately.
 2020-08-13 13:39:25 +01:00
Jakob Ackermann
ee59056c6e [misc] forcefully disconnect stale clients from shutdown process 2020-08-13 13:39:22 +01:00
Christopher Hoskin
fe41f1b00c [misc] bump logger-sharelatex to version 2.2.0 2020-08-12 16:04:21 +01:00
Jakob Ackermann
a0028646a5 [app] ignore errors from accessing disconnected client sockets 
Technically `EHOSTUNREACH` and `ETIMEDOUT` should go into a
 'disconnected_read' metric. But this would require changes to
 dashboards and a larger diff.
 2020-08-12 11:45:49 +01:00
Brian Gough
831d794bf4 clean up join/leave handling 
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
 2020-08-12 10:54:22 +01:00
Jakob Ackermann
370baa06c3 [misc] bump the dev-env to 3.3.2 2020-08-10 17:10:31 +01:00
Jakob Ackermann
51e56dafcf [misc] bump the dev-env to 3.3.1 2020-08-05 10:46:23 +01:00
Jakob Ackermann
1b52366c61 Merge pull request #174 from overleaf/jpa-bump-dev-env 
[misc] bump the dev-env to 3.3.0
 2020-08-04 12:29:22 +02:00
Jakob Ackermann
f4d9ae1150 [misc] bump the dev-env to 3.3.0 2020-08-04 11:07:19 +01:00
Jakob Ackermann
a91f78634b [misc] bump socket.io and socket.io-client 2020-07-31 16:32:03 +01:00
Shane Kilkelly
3d92858939 Upgrade redis-sharelatex to 1.0.13 2020-07-24 10:42:46 +01:00
Jakob Ackermann
562375d351 [misc] fix express deprecations 2020-07-22 09:45:14 +01:00
Jakob Ackermann
ca4168ce90 [misc] use overleaf/socket.io and overleaf/socket.io-client forks 2020-07-08 15:07:19 +01:00
Jakob Ackermann
9fff03bca5 [misc] optionally expose the hostname in the 'debug.getHostname' rpc 2020-07-07 11:07:28 +01:00
Jakob Ackermann
aa9d6c8dc9 [misc] reland decaff cleanup (#166) 
* [misc] decaff cleanup: RoomManager

* [misc] decaff cleanup: RedisClientManager

* [misc] decaff cleanup: SafeJsonParse

* [misc] decaff cleanup: WebApiManager

* [misc] decaff cleanup: WebsocketController

* [misc] decaff cleanup: WebsocketLoadBalancer

* [misc] decaff cleanup: SessionSockets

* [misc] decaff cleanup: HttpController

* [misc] decaff cleanup: HttpApiController

* [misc] decaff cleanup: HealthCheckManager

* [misc] decaff cleanup: EventLogger

* [misc] decaff cleanup: Errors

o-error will eliminate most of it -- when we migrate over.

* [misc] decaff cleanup: DrainManager

* [misc] decaff cleanup: DocumentUpdaterManager

* [misc] decaff cleanup: DocumentUpdaterController: no-unused-vars

* [misc] decaff cleanup: DocumentUpdaterController: Array.from

* [misc] decaff cleanup: DocumentUpdaterController: implicit return

* [misc] decaff cleanup: DocumentUpdaterController: IIFE

* [misc] decaff cleanup: DocumentUpdaterController: null checks

* [misc] decaff cleanup: DocumentUpdaterController: simpler loops

* [misc] decaff cleanup: DocumentUpdaterController: move module name def

* [misc] decaff cleanup: ConnectedUsersManager: handle-callback-err

* [misc] decaff cleanup: ConnectedUsersManager: implicit returns

* [misc] decaff cleanup: ConnectedUsersManager: null checks

* [misc] decaff cleanup: ChannelManager: no-unused-vars

* [misc] decaff cleanup: ChannelManager: implicit returns

* [misc] decaff cleanup: ChannelManager: other cleanup

- var -> const
- drop variable assignment before return

* [misc] decaff cleanup: AuthorizationManager: handle-callback-err

Note: This requires a change in WebsocketController to provide a dummy
 callback.

* [misc] decaff cleanup: AuthorizationManager: Array.from

* [misc] decaff cleanup: AuthorizationManager: implicit returns

* [misc] decaff cleanup: AuthorizationManager: null checks

* [misc] decaff cleanup: Router: handle-callback-err

* [misc] decaff cleanup: Router: standard/no-callback-literal

* [misc] decaff cleanup: Router: Array.from

* [misc] decaff cleanup: Router: implicit returns

* [misc] decaff cleanup: Router: refactor __guard__ wrapper

* [misc] decaff cleanup: Router: null checks

And a minor bug fix: user.id -> user._id

* [misc] decaff cleanup: Router: move variable declarations to assignments

* [misc] decaff cleanup: app: implicit returns

* [misc] decaff cleanup: app: __guard__

* [misc] decaff cleanup: app: null checks

* [misc] decaff cleanup: app: function definitions

* [misc] decaff cleanup: app: drop unused next argument

* [misc] decaff cleanup: app: var -> const
 2020-07-07 11:06:02 +01:00
Jakob Ackermann
8601084a2e [misc] WebsocketControllerTests: remove debugging code 2020-07-01 11:36:36 +01:00
Jakob Ackermann
a77222470b [misc] WebsocketControllerTests: always pass cb to updateClientPosition 
...and fix the inconsistent async behaviour for logged-in vs anonymous
 users in the app.
 2020-06-29 18:11:33 +01:00
Jakob Ackermann
4d01c02946 [misc] WebsocketControllerTests: explicitly pass a cb to leaveProject 
...and fix the stubbing of `io.sockets.clients`.

We were running the assertions prior to the actual completion of the
 leaveProject. Which in turn shadowed the broken test setup --
 `io.sockets.clients()` was returning the `this.clientsInRoom=[]` from a
 previous context.
 2020-06-29 17:38:22 +01:00
Jakob Ackermann
bea0e9beb7 [misc] RoomManagerTests: explicitly pass a cb to joinProject/joinDoc 2020-06-29 17:12:26 +01:00
Jakob Ackermann
5e191be171 [misc] replace console logging with logger-sharelatex 2020-06-24 11:12:22 +01:00
Jakob Ackermann
fa42166be3 [misc] ignore camelcase warning in settings (key_schema parameter) 2020-06-24 10:52:38 +01:00