Commit graph

627 commits

Author SHA1 Message Date
Brian Gough
534cb1c5ab Merge pull request #5661 from overleaf/bg-fix-status-on-restart
[real-time] skip 60s closing delay on start up

GitOrigin-RevId: 5fbb3b4f2940505c2ab6e5cfe6a757ee2501bf2d
2021-11-09 09:03:34 +00:00
Jakob Ackermann
638fbd5510 Merge pull request #5573 from overleaf/jpa-prefer-regex-literals
[misc] fix eslint violations for prefer-regex-literals

GitOrigin-RevId: 216288af22b9eeb18cddace7ec285e0a8046f94a
2021-10-28 08:03:41 +00:00
Jakob Ackermann
f6795a324a Merge pull request #5572 from overleaf/jpa-no-path-concat
[misc] fix eslint violations for node/no-path-concat

GitOrigin-RevId: 5122826fb9ae23c373e8c5a6802ebb35eb20314f
2021-10-28 08:03:37 +00:00
Jakob Ackermann
669606f797 Merge pull request #5568 from overleaf/jpa-node-no-callback-literal
[misc] fix eslint violations for node/no-callback-literal

GitOrigin-RevId: af5aace52e1476b1b0ee48cc8be2aabbe04efac6
2021-10-28 08:03:33 +00:00
Jakob Ackermann
489df8360e Merge pull request #5523 from overleaf/jpa-no-loss-of-precision
[misc] fix eslint violations for no-loss-of-precision

GitOrigin-RevId: 06ecbeb5f10569933e99a472a92fd3efbe546278
2021-10-28 08:03:30 +00:00
Jakob Ackermann
7f9fd00bda Merge pull request #5367 from overleaf/jpa-node-handle-callback-err
[misc] fix eslint violations for node/handle-callback-err

GitOrigin-RevId: 83a4900e8861010df1917bff49382bd9c93375bd
2021-10-28 08:03:26 +00:00
Jakob Ackermann
5862359ff0 Merge pull request #5518 from overleaf/jpa-no-empty
[misc] fix eslint violations for no-empty

GitOrigin-RevId: 8fc1688ac28ccf29b830ba43cd6091e64b5e3b07
2021-10-27 08:03:14 +00:00
Jakob Ackermann
4cefebec5d Merge pull request #5517 from overleaf/jpa-no-dupe-else-if
[misc] fix eslint violations for no-dupe-else-if

GitOrigin-RevId: 5f72eb1d033322b2d419c1dbad29ea378ee776f5
2021-10-27 08:03:09 +00:00
Jakob Ackermann
0f529df1e4 Merge pull request #5360 from overleaf/jpa-array-callback-return
[misc] fix eslint violations for array-callback-return

GitOrigin-RevId: 4104342239f4da32c541855d6fc7032c4c96004f
2021-10-27 08:03:05 +00:00
Jakob Ackermann
a6f05109a3 Merge pull request #5352 from overleaf/jpa-no-var
[misc] fix eslint violations for `no-var`

GitOrigin-RevId: c52e82f3a8a993b8662cc5aa56e7b95ca3c55832
2021-10-27 08:03:00 +00:00
Brian Gough
51b42381bd Merge pull request #5551 from overleaf/bg-node-upgrade-user-facing
upgrade node to v12.22.7 (user facing)

GitOrigin-RevId: ddc6a29beac27fe1ae5d88830fb901667d91ec89
2021-10-25 08:03:02 +00:00
Allemand
4313cba624 Update app.js (#5452)
change metrics app name

COPYBARA_INTEGRATE_REVIEW=https://github.com/overleaf/overleaf/pull/947 from salleman33:patch-1 052d4aa0a8

Co-authored-by: Allemand <37148072+salleman33@users.noreply.github.com>
GitOrigin-RevId: fe23c5431402f918bce1c933051bebd5416d5d94
2021-10-13 08:04:17 +00:00
John Lees-Miller
c4f1020f2a Merge pull request #5440 from overleaf/revert-5430-import-public-pr-947
Revert "[Imported] Update real-time app.js, (#947)"

GitOrigin-RevId: 9aadf63459b861c4316ead76741a2a9aeea7aad3
2021-10-13 08:04:05 +00:00
Brian Gough
96d9e6578a Merge pull request #5430 from overleaf/import-public-pr-947
[Imported] Update real-time app.js, (#947)

GitOrigin-RevId: 4212705fff58252d02056f7744c97aaeeac065ce
2021-10-13 08:03:52 +00:00
Brian Gough
6cece12369 Merge pull request #5109 from overleaf/bg-fix-handshake-exception
fix handshake exception

GitOrigin-RevId: 917518ccac3f5fddf4029f1b1ec3e3806dde9a29
2021-09-16 08:03:08 +00:00
Brian Gough
618cf99548 Merge pull request #4950 from overleaf/bg-realtime-log-relevelling
realtime log re-levelling

GitOrigin-RevId: 81d86ba648e7fc1436b638b67371b35fd831a62f
2021-09-15 08:03:25 +00:00
June Kelly
7b1044b8a8 Merge pull request #4870 from overleaf/jk-bg-validate-ids
RealTime: Validate IDs
GitOrigin-RevId: 884600125d362c5632faa75dc22d957cdddc101b
2021-09-02 08:03:14 +00:00
Jakob Ackermann
d6616fdfc1 Merge pull request #221 from overleaf/dependabot-npm_and_yarn-glob-parent-5.1.2
Bump glob-parent from 5.1.1 to 5.1.2
2021-07-15 13:28:22 +02:00
Jakob Ackermann
b8fcb265b2 [misc] EventLogger: drop explicit metrics.inc amount
The module has a hard-coded increment of 1, which is the only value the
 prometheus backend supports.
2021-07-13 12:40:46 +01:00
Jakob Ackermann
bd5e8b8f71 [misc] fix chai assertions .equal.true -> .equals(true) 2021-07-13 12:34:56 +01:00
Jakob Ackermann
89928dd19d [misc] upgrade node version to latest v12 LTS version 12.22.3 2021-07-13 12:26:33 +01:00
Jakob Ackermann
9e4624315b [misc] temporary override a few new/changed eslint rules 2021-07-13 12:24:39 +01:00
Jakob Ackermann
7e8e231059 [misc] run format_fix and lint:fix 2021-07-13 12:04:45 +01:00
Jakob Ackermann
d17cda1a7b [misc] upgrade build scripts to version 3.11.0 and cleanup packages
```
npm uninstall prettier-eslint-cli eslint-plugin-standard eslint-plugin-jsx-a11y eslint-plugin-react eslint-config-standard-jsx eslint-config-standard-react babel-eslint
npm dedupe
```
2021-07-13 11:55:16 +01:00
Jakob Ackermann
91fe61b167 [misc] goodbye coffee-script 2021-07-13 10:07:02 +01:00
Jakob Ackermann
e66d1231f2 [misc] run npm dedupe 2021-07-12 17:51:04 +01:00
Jakob Ackermann
a26ae73597 [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:18 +01:00
Jakob Ackermann
7af2c6a5ea [misc] install bunyan as production dependency
```
Error: Cannot find module 'bunyan'
Require stack:
- .../node_modules/@google-cloud/logging-bunyan/build/src/middleware/express.js
- .../node_modules/@google-cloud/logging-bunyan/build/src/index.js
- .../node_modules/logger-sharelatex/logging-manager.js
- .../node_modules/logger-sharelatex/index.js
- .../app.js
```
2021-07-12 17:37:59 +01:00
dependabot[bot]
1160e9e98f Bump glob-parent from 5.1.1 to 5.1.2
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.1 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.1...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-12 00:00:31 +00:00
Brian Gough
219d6832df upgrade to socket.io 0.9.19-overleaf-5
fix for exception in client.transport getter
2021-05-19 14:28:54 +01:00
Brian Gough
09bb3ac1dd Merge pull request #217 from overleaf/bg-add-metric-transport-label
add transport type to real-time metrics
2021-05-18 15:03:47 +01:00
Brian Gough
b78b41ae24 Merge pull request #214 from overleaf/jpa-explicit-dependencies
[misc] add linting for missing explicit dependencies and fix any errors
2021-05-18 15:03:26 +01:00
Brian Gough
c7b0e908ff Merge pull request #213 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1
Bump y18n from 4.0.0 to 4.0.1
2021-05-18 15:02:55 +01:00
Brian Gough
4b32f3d010 Merge pull request #216 from overleaf/dependabot-npm_and_yarn-underscore-1.13.1
Bump underscore from 1.7.0 to 1.13.1
2021-05-18 15:02:28 +01:00
Brian Gough
56aed80a9e Merge pull request #218 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21
Bump lodash from 4.17.20 to 4.17.21
2021-05-18 15:00:35 +01:00
Brian Gough
3044bc4cf5 add transport to get-connected-users metric 2021-05-13 14:51:26 +01:00
dependabot[bot]
8a16c55000 Bump hosted-git-info from 2.8.8 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 20:43:06 +00:00
dependabot[bot]
d5f2f4b7e4 Bump lodash from 4.17.20 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 03:16:58 +00:00
Brian Gough
9622b341b7 add transport type to metrics 2021-05-10 10:16:50 +01:00
dependabot[bot]
19507962db Bump underscore from 1.7.0 to 1.13.1
Bumps [underscore](https://github.com/jashkenas/underscore) from 1.7.0 to 1.13.1.
- [Release notes](https://github.com/jashkenas/underscore/releases)
- [Commits](https://github.com/jashkenas/underscore/compare/1.7.0...1.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-07 05:18:17 +00:00
Jakob Ackermann
a6689f00f6 [misc] add linting for missing explicit dependencies and fix any errors 2021-04-29 15:30:52 +01:00
Brian Gough
a3009d2ef2 Merge pull request #210 from overleaf/bg-real-time-status-file
add blue-green deployment support with load-balancer health-checks
2021-04-01 09:09:57 +01:00
dependabot[bot]
9c50bbffd1 Bump y18n from 4.0.0 to 4.0.1
Bumps [y18n](https://github.com/yargs/y18n) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/yargs/y18n/releases)
- [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yargs/y18n/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-01 03:49:36 +00:00
Thomas Mees
e63b9ec5ea Lower log level of 'no project_id found on client' messages 2021-03-30 12:40:26 +02:00
Brian Gough
a55aa61d71 use .includes instead of .indexOf 2021-03-29 11:13:59 +01:00
Eric Mc Sween
61f43bf5d8 Upgrade metrics module to 3.5.1 2021-03-25 08:10:43 -04:00
Brian Gough
32184330d9 delay closing by 1 minute for deployment flip 2021-03-22 11:26:02 +00:00
Brian Gough
17e3068499 close real-time via a status file 2021-03-22 10:46:25 +00:00
Eric Mc Sween
b27cb4ec12 Upgrade to Node 12 2021-03-18 16:41:31 -04:00
Eric Mc Sween
b9e7cbf73b Add global test setup
Configure chai and SandboxedModule in a central place. Configure
SandboxedModule globals that are needed in Node 12.
2021-03-18 16:41:31 -04:00
Jakob Ackermann
17c3d7729e Merge remote-tracking branch 'origin/master' into jpa-let-mocha-catch-errors 2021-03-02 08:54:25 +00:00
Jakob Ackermann
a26a497743 [misc] let mocha catch errors when running acceptance tests 2021-03-01 15:51:55 +00:00
Jakob Ackermann
b7b13a90d1 [misc] ApplyUpdateTests: process all the pending-updates-lists 2021-03-01 15:26:18 +00:00
Brian Gough
22762f698c Merge pull request #201 from overleaf/bg-add-health-check-comment
add comment about health checks on '/'
2021-02-25 11:41:54 +00:00
Henry Oswald
27e7b77bdb bump default key shards to 10 2021-02-16 10:53:35 +00:00
Henry Oswald
49bed6595d added queueKey to error info when trying to push to redis 2021-02-09 12:42:58 +00:00
Henry Oswald
46b389e8b3 fix off by 1 error in key sharding 2021-02-09 10:48:40 +00:00
Henry Oswald
31e1808dd8 shard the pending-updates-list queue 2021-02-08 16:02:41 +00:00
Henry Oswald
c308e60b6b speed up DocumentUpdaterControllerTests by directly injecting redis into
tests
2021-02-08 11:13:22 +00:00
Jakob Ackermann
c2f8f26da4 [misc] bump the node version to 10.23.1 2021-01-05 18:26:52 +00:00
John Lees-Miller
8fec6723d8 Add correct license 2020-12-03 10:27:38 +00:00
Brian Gough
1ddcdf38ca add comment about health checks on '/' 2020-11-27 14:42:42 +00:00
Jakob Ackermann
8a5858f430 Merge pull request #200 from overleaf/jpa-bump-metrics-module
[misc] bump metrics module to 3.4.1
2020-11-26 13:13:50 +00:00
Jakob Ackermann
746c5aeb80 [misc] bump metrics module to 3.4.1
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
2020-11-25 11:57:22 +00:00
Eric Mc Sween
96126ecf34 Upgrade build-scripts to 3.4.0
This version fixes docker-compose health checks for dependent services. See
https://github.com/overleaf/dev-environment/pull/409 for details.
2020-11-24 08:03:29 -05:00
Jakob Ackermann
15af5c7977 [misc] bump @overleaf/redis-wrapper to version 2.0.0 2020-11-11 16:24:22 +00:00
Jakob Ackermann
dea6968fa9 [misc] explicitly install underscore
Use the same (outdated) version that is already in place.
2020-11-11 16:23:12 +00:00
Jakob Ackermann
222caeb9a8 Merge pull request #196 from overleaf/jpa-drop-duplicate-join
[RoomManager] drop duplicate joining of entities
2020-10-19 17:01:40 +02:00
Jakob Ackermann
d6ac8c14e7 [RoomManager] drop duplicate joining of entities
REF: 0437e1d03f89a058f97a8884e3532a9a58b68b9d
REF: 62be5e29e5232150e7063bc189c5ad8a1189f972
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-10-19 15:54:37 +01:00
Jakob Ackermann
b20cdc0122 [misc] bump the dev-env to 3.3.5 -- drop custom unit test timeout 2020-10-19 15:20:57 +01:00
Jakob Ackermann
e8911babe0 Merge pull request #182 from overleaf/dependabot-npm_and_yarn-lodash-4.17.20
Bump lodash from 4.17.15 to 4.17.20
2020-10-08 10:42:21 +02:00
Jakob Ackermann
e9c23b6409 Merge pull request #191 from overleaf/dependabot-npm_and_yarn-node-fetch-2.6.1
Bump node-fetch from 2.6.0 to 2.6.1
2020-10-08 10:42:08 +02:00
Jakob Ackermann
5b53832bf2 Merge pull request #192 from overleaf/jpa-bump-dev-env-3-3-4-node-10-22-1
[misc] bump the dev-env to 3.3.4 and bump the node version to 10.22.1
2020-10-08 10:41:56 +02:00
Jakob Ackermann
e846192db0 [MatrixTests] add a large testing matrix
Layers/Dimensions:
- users: anonymous, registered, registeredWithOwnedProject
- session setup: noop, joinReadWriteProject, joinReadWriteProjectAndDoc,
   joinOwnProject, joinOwnProjectAndDoc
- invalid requests: noop, joinProjectWithDocId, joinDocWithDocId,
   joinProjectWithProjectId, joinDocWithProjectId,
   joinProjectWithProjectIdThenJoinDocWithDocId
2020-10-07 16:20:21 +01:00
Jakob Ackermann
4ec73acb88 Merge branches 'jpa-prepare-for-matrix-tests' and 'jpa-setup-session-helper' into jpa-matrix-tests 2020-10-01 12:57:40 +01:00
Jakob Ackermann
2900c60c4a [FixturesManager] add a helper for setting up project and doc together 2020-10-01 12:33:15 +01:00
Jakob Ackermann
78fbd04ef8 [MockWebServer] grant users at least the privileges of anonymous users 2020-10-01 12:28:39 +01:00
Jakob Ackermann
4f860995d8 [MockDocUpdaterServer] return a 404 when a requested doc does not exist 2020-10-01 12:25:41 +01:00
Jakob Ackermann
cc681a94f4 [app] ignore error from writing to disconnected long-polling client 2020-09-22 14:10:23 +01:00
Jakob Ackermann
f3c7619fb5 [misc] revert back to high timeout for unit tests 2020-09-17 15:34:03 +01:00
Jakob Ackermann
ba3b4f7dcd [misc] bump the dev-env to 3.3.4 and bump the node version to 10.22.1 2020-09-17 15:26:45 +01:00
dependabot[bot]
ee06705377 Bump lodash from 4.17.15 to 4.17.20
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.20.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.20)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-16 10:38:21 +00:00
dependabot[bot]
10c6e01bd8 Bump node-fetch from 2.6.0 to 2.6.1
Bumps [node-fetch](https://github.com/bitinn/node-fetch) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/bitinn/node-fetch/releases)
- [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-12 15:08:05 +00:00
Jakob Ackermann
a2f5bf463e Merge pull request #190 from overleaf/jpa-socket-io-logger
[misc] socket.io: use a custom logger
2020-09-02 10:46:23 +02:00
Jakob Ackermann
c6f2a3b387 Merge pull request #189 from overleaf/jpa-relevel-log
[misc] re-level expected error log messages
2020-09-02 10:46:08 +02:00
Jakob Ackermann
72282a07df [misc] socket.io: use a custom logger
- forward the previously enabled log messages to our logger-module
- stub the previously disabled logger methods
- drop the log-level config for socket.io
2020-09-01 09:30:11 +01:00
Jakob Ackermann
4960bdd6fe [misc] re-level log: 404 from web -> WARN and emit 'project not found'
A stale browser tab tried to join a deleted project.
Emitting 'project not found'/'ProjectNotFound' will trigger a page
 reload in the frontend, upon web can render a 404.
See frontend: ConnectionManager.joinProject callback
2020-08-27 11:51:57 +01:00
Jakob Ackermann
55d938ba14 [misc] add tests for web replying with a 404 for joinProject 2020-08-27 11:51:56 +01:00
Jakob Ackermann
884b340c75 [misc] re-level log: 403 from web goes to WARN and emit 'not authorized'
Users will get redirected to the login page and will see a 'restricted'
 page after logging in again.
See frontend: ConnectionManager.reportConnectionError
2020-08-27 11:51:56 +01:00
Jakob Ackermann
d2a2b9d46e [misc] add tests for web replying with a 403 for joinProject 2020-08-27 11:51:56 +01:00
Jakob Ackermann
1ff9c1e71b [misc] add the rpc-method into the log context in Router._handleError 2020-08-27 11:51:55 +01:00
Jakob Ackermann
0647abf433 [misc] drop info-log in WebApiManager for joinProject being rate-limited
The CodedError is logged at warn-level in Router._handleError.
2020-08-27 11:51:09 +01:00
Jakob Ackermann
2ce7b36c95 [misc] drop duplicate log line for unauthorized applyOtUpdate calls
The violation is logged in Router._handleError.
2020-08-27 10:22:31 +01:00
Jakob Ackermann
dee4749e6d [misc] re-level log: properly silence unauthorized updateClientPosition 2020-08-27 10:11:40 +01:00
Jakob Ackermann
425052ff91 Merge pull request #187 from overleaf/jpa-o-error-tagging
[misc] migrate to OError tagging/wrapping
2020-08-25 11:46:28 +02:00
Jakob Ackermann
64e659bf43 Merge pull request #186 from overleaf/jpa-fix-join-project-error-context
[misc] fix join project error context
2020-08-25 11:46:18 +02:00
Jakob Ackermann
849a1cf416 Merge pull request #185 from overleaf/jpa-doc-id-in-error-context
[misc] add/bring back doc_id in error context
2020-08-25 11:42:41 +02:00
Jakob Ackermann
ee3d3b09ed [misc] wrap redis errors as tagging does not work with them
ioredis may reuse the error instance for multiple callbacks. E.g. when
 the connection to redis fails, the queue is flushed with the same
 MaxRetriesPerRequestError instance.
2020-08-24 10:12:20 +01:00
Jakob Ackermann
537e97be73 [misc] OError.tag all the errors in async contexts
See the docs of OError.tag:
https://github.com/overleaf/o-error#long-stack-traces-with-oerrortag
(currently at 221dd902e7bfa0ee92de1ea5a3cbf3152c3ceeb4)

I am tagging all errors at each async hop. Most of the controller code
 will only ever see already tagged errors -- or new errors created in
 our app code. They should have enough info that we do not need to tag
 them again.
2020-08-24 10:12:06 +01:00
Jakob Ackermann
8e31cc5c23 [Router] _handleError: joinProject error-context may not have project_id
The ol_context patch changed the priority of client context and rpc
 context.
This lead to the (possibly missing) project_id of the client context
 overwriting the project_id of the rpc context.
REF: f1d55c0a5437a518e9f4617473caed9ba928e648
2020-08-21 13:29:28 +01:00