Commit graph

905 commits

Author SHA1 Message Date
Brian Gough
2cd7b615bf Merge pull request #5159 from overleaf/bg-docupdater-log-relevelling
docupdater log re-levelling

GitOrigin-RevId: c27478d48f1cb1eae022086fe7af9fe55dfacfa0
2021-10-01 08:03:04 +00:00
Brian Gough
8b1665a97c Merge pull request #5143 from overleaf/bg-upgrade-docupdater-dependencies
[docupdater] update package-lock.json

GitOrigin-RevId: 097ded1070e31c77add37a89f9a8967571e59b0f
2021-09-21 08:02:58 +00:00
Brian Gough
865a973426 Merge pull request #5132 from overleaf/bg-fix-size-check
[docupdater] use more accurate doc size check

GitOrigin-RevId: f66d68a7f7fdc127cc31539abdcab65549823d02
2021-09-21 08:02:55 +00:00
Henry Oswald
8197411cdf Merge pull request #4891 from overleaf/ho-web-mongo-4-2
bump mongo to 4.2.15

GitOrigin-RevId: 1ac2057f264bd6dbe042d8f999a4ab1054c9edf5
2021-09-15 08:03:54 +00:00
Eric Mc Sween
9f4541c266 Merge pull request #4761 from overleaf/em-peek-doc
Do not unarchive docs when resyncing project history

GitOrigin-RevId: c7df75789c01e6c85b464a9b94b14654d8568407
2021-08-20 08:03:50 +00:00
Brian Gough
5e2d4d2169 add acceptance tests for peek 2021-08-02 14:13:38 +01:00
Brian Gough
0fd24b5133 peek at docs without fetching from mongo 2021-07-30 16:13:48 +01:00
Jakob Ackermann
3ccd3a228f Merge pull request #178 from overleaf/dependabot-npm_and_yarn-glob-parent-5.1.2
Bump glob-parent from 5.1.1 to 5.1.2
2021-07-16 10:07:28 +02:00
Jakob Ackermann
29cea1743b [misc] DeletingADocumentTests: fix behavior checks
The preloading setup does an initial getDocument request to web-api.
2021-07-13 13:12:34 +01:00
Jakob Ackermann
bf8224e2a0 [misc] ApplyingUpdatesToADocTests: adhere to stricter chai assertions 2021-07-13 12:51:06 +01:00
Jakob Ackermann
1184f8f5b6 [misc] fix chai assertions .equal.false -> .equals(false) 2021-07-13 12:35:32 +01:00
Jakob Ackermann
40aafa27e0 [misc] upgrade node version to latest v12 LTS version 12.22.3 2021-07-13 12:26:30 +01:00
Jakob Ackermann
e8697f7f97 [misc] temporary override a few new/changed eslint rules 2021-07-13 12:24:36 +01:00
Jakob Ackermann
c532376e21 [misc] run format_fix and lint:fix 2021-07-13 12:04:42 +01:00
Jakob Ackermann
2a3f264140 [misc] upgrade build scripts to version 3.11.0 and cleanup packages
```
npm uninstall prettier-eslint-cli eslint-plugin-standard eslint-plugin-jsx-a11y eslint-plugin-react eslint-config-standard-jsx eslint-config-standard-react babel-eslint
npm dedupe
```
2021-07-13 11:55:13 +01:00
Jakob Ackermann
98097b0804 [misc] goodbye coffee-script 2021-07-13 10:07:00 +01:00
Jakob Ackermann
34de603e90 [misc] run npm dedupe 2021-07-12 17:51:01 +01:00
Jakob Ackermann
519dd274bf [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:15 +01:00
Jakob Ackermann
835dbe784b [misc] install bunyan as production dependency
```
Error: Cannot find module 'bunyan'
Require stack:
- .../node_modules/@google-cloud/logging-bunyan/build/src/middleware/express.js
- .../node_modules/@google-cloud/logging-bunyan/build/src/index.js
- .../node_modules/logger-sharelatex/logging-manager.js
- .../node_modules/logger-sharelatex/index.js
- .../app.js
```
2021-07-12 17:35:45 +01:00
dependabot[bot]
0090748379 Bump glob-parent from 5.1.1 to 5.1.2
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.1 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.1...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-11 14:36:06 +00:00
Alf Eaton
674ad4acf0 Rename folder from coffee to js 2021-05-25 16:57:24 +01:00
Alf Eaton
5961e85b10 Fix path in test script 2021-05-25 16:56:57 +01:00
Alf Eaton
24fd9449f2 Run format:fix 2021-05-25 15:52:19 +01:00
Alf Eaton
358786fccd Run format:fix 2021-05-25 15:45:42 +01:00
Alf Eaton
f4db3d8f45 Manual tidying 2021-05-25 14:32:03 +01:00
Alf Eaton
de50e59557 Run format:fix 2021-05-25 14:27:45 +01:00
decaffeinate
03cbad4e9f decaffeinate: Run post-processing cleanups on expire_docops.coffee and 2 other files 2021-05-19 20:25:07 +01:00
decaffeinate
5ad0b6eea2 decaffeinate: Convert expire_docops.coffee and 2 other files to JS 2021-05-19 20:25:07 +01:00
decaffeinate
e1e0d26b1f decaffeinate: Rename expire_docops.coffee and 2 other files from .coffee to .js 2021-05-19 20:25:06 +01:00
Alf Eaton
edacebb53c Remove unused parameter 2021-05-19 20:25:04 +01:00
Jakob Ackermann
dde645e770 Merge pull request #176 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9
Bump hosted-git-info from 2.8.8 to 2.8.9
2021-05-17 10:57:18 +02:00
Jakob Ackermann
e92b9fc754 Merge pull request #174 from overleaf/jpa-explicit-dependencies
[misc] add linting for missing explicit dependencies and fix any errors
2021-05-17 10:57:12 +02:00
Jakob Ackermann
f97117a6ba Merge pull request #175 from overleaf/jpa-double-down-doc-size-limit
[misc] double down on enforcing doc size limits
2021-05-17 10:56:55 +02:00
dependabot[bot]
da81a97b82 Bump hosted-git-info from 2.8.8 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 17:18:40 +00:00
Jakob Ackermann
309ad818f6 [SizeCheckTests] add acceptance tests for size limit checks 2021-05-06 18:21:19 +01:00
Jakob Ackermann
757e1e98c5 [RedisManager] block updating of too large docs in redis 2021-05-06 18:21:19 +01:00
Jakob Ackermann
c707d0b345 [RedisManager] block inserting of too large docs into redis 2021-05-06 18:21:18 +01:00
Jakob Ackermann
a4ae0ea12f [ShareJsUpdateManager] double check doc size before flushing 2021-05-06 18:20:51 +01:00
Jakob Ackermann
33fad08c47 [misc] add linting for missing explicit dependencies and fix any errors 2021-04-29 15:30:48 +01:00
Brian Gough
ed3ddacac6 Merge pull request #172 from overleaf/bg-upgrade-ioredis
upgrade ioredis via @overleaf/redis-wrapper
2021-04-29 14:15:45 +01:00
Brian Gough
01112e10ab upgrade ioredis via @overleaf/redis-wrapper
also update dependencies
2021-04-29 11:49:13 +01:00
Brian Gough
b1a1446054 remove the limit of 300 maxSockets 2021-04-29 10:29:34 +01:00
Shane Kilkelly
8006e122d5 Merge pull request #170 from overleaf/sk-logger-update-2
upgrade logger to 2.2.0
2021-04-14 09:58:14 +01:00
Jakob Ackermann
34fc349646 [benchmarks] add benchmark for multi vs mget/mset 2021-04-13 11:47:10 +01:00
Jakob Ackermann
6e551f9b34 [perf] use MGET/MSET/DEL for manipulating multiple keys in one operation
In some cases we can get rid of MULTI/EXEC operations too.
- putDocInMemory: from 10 down to 2 operations
- removeDocFromMemory: from 14+4 down to 4+4 operations
- updateDoc: from 13 down to 8 operations
2021-04-13 11:47:10 +01:00
Jakob Ackermann
178440395f [perf] switch write sequence for doc contents and doc tracking
Doc contents are added only after the tracking has been setup.
All read paths on the tracking have been checked to gracefully handle
 the case of existing doc_id but missing doc contents.

- getDoc: -1 operation

REF: 0a2b47c660c60b95e360d8f3b3e30b862ceb6d79
2021-04-13 11:46:44 +01:00
Shane Kilkelly
83bbdfa9d1 upgrade logger to 2.2.0 2021-04-13 10:58:05 +01:00
Jakob Ackermann
50b24043b7 [perf] use MGET for fetching multiple keys in one operation
- getDoc: from 13 down to 2 operations
2021-04-09 08:42:35 +01:00
Simon Detheridge
8ba799035b Revert "Update logger-sharelatex to 2.2.0" 2021-04-06 15:29:09 +01:00
Shane Kilkelly
1e4e4b5ec0 Update logger-sharelatex to 2.2.0 2021-04-06 11:47:22 +01:00
dependabot[bot]
acbbd88d8a Bump y18n from 4.0.0 to 4.0.1
Bumps [y18n](https://github.com/yargs/y18n) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/yargs/y18n/releases)
- [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yargs/y18n/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-06 10:40:56 +00:00
Eric Mc Sween
4d70bd664f Reintroduce Node 12 and metrics upgrades
These changes were previously merged, not deployed, and reverted. This
reverts the revert.

This reverts commit a6b8c6c658b33b6eee78b8b99e43308f32211ae2, reversing
changes made to 93c98921372eed4244d22fce800716cb27eca299.
2021-04-01 15:51:00 -04:00
Eric Mc Sween
4dd1b26b2e Revert "Merge pull request #161 from overleaf/em-upgrade-node-12"
This reverts commit d44102751b9436ad89c5b3b05e7abdff51fcc78a, reversing
changes made to 6c2f5b8d053b75c677da2b7ddd04f998d2be6fff.
2021-03-31 12:07:11 -04:00
Eric Mc Sween
b233e6588c Revert "Merge pull request #158 from overleaf/jpa-metrics-module-3-5-1"
This reverts commit 93c98921372eed4244d22fce800716cb27eca299, reversing
changes made to d44102751b9436ad89c5b3b05e7abdff51fcc78a.
2021-03-31 12:06:41 -04:00
Jakob Ackermann
98a32833a9 [misc] bump the version of the metrics module to 3.5.1 2021-03-31 10:08:04 -04:00
Eric Mc Sween
1a2235a219 Upgrade to Node 12 2021-03-16 16:48:45 -04:00
Eric Mc Sween
dc5bfea181 Global test setup
Configure chai and SandboxedModule globally with options used in many
tests.

That required upgrading mocha and SandboxedModule.
2021-03-16 16:48:45 -04:00
Thomas
23738540ed Fix API request errors which could contain API hostname or address (#160)
Wrap errors produced by failing requests to web API, and remove the url/hostname from thrown error messages. (But keep the URL path for info.)
2021-02-24 15:09:19 +01:00
Henry Oswald
20a373d95c stop listening on the pending updates channels 10 times 2021-02-23 08:27:29 +00:00
Henry Oswald
c7e57cd28f add Dispatchers running on old queue while we migrate
revert once migrated
2021-02-15 14:16:45 +00:00
Henry Oswald
854e24bb57 remove unneeded anonymous func 2021-02-15 14:12:28 +00:00
Henry Oswald
0cdeffae6c fix off by 1 error in Doc updater client helper file 2021-02-09 10:50:37 +00:00
Henry Oswald
bcfc7e66fc add missing comma in settings file 2021-02-09 10:32:16 +00:00
Henry Oswald
40de999766 Update config/settings.defaults.js
parseint on dispatcher count

Co-authored-by: John Lees-Miller <jdleesmiller@gmail.com>
2021-02-04 09:30:35 +00:00
Henry Oswald
11c8cfc939 shard the pending-updates-list queue 2021-02-02 16:38:25 +00:00
Eric Mc Sween
98f8d7f51c Set the diff-match-patch timeout to 100ms
This might result in worse diffs, but we don't want to spend a second
blocking the event loop while we figure out nicer diffs when comparing
documents.
2021-01-14 15:11:15 -05:00
Christopher Hoskin
9af14ef513 Merge pull request #153 from overleaf/dependabot-npm_and_yarn-ini-1.3.8
Bump ini from 1.3.5 to 1.3.8
2021-01-06 10:25:17 +00:00
Christopher Hoskin
9cbd8751a8 Update Node to 10.23.1 2021-01-06 09:57:08 +00:00
dependabot[bot]
ab6705481d Bump ini from 1.3.5 to 1.3.8
Bumps [ini](https://github.com/isaacs/ini) from 1.3.5 to 1.3.8.
- [Release notes](https://github.com/isaacs/ini/releases)
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.8)

Signed-off-by: dependabot[bot] <support@github.com>
2020-12-13 07:35:06 +00:00
Eric Mc Sween
de247302b1 Use a centralized diff-match-patch package
We use our own fork of the diff-match-patch npm package, which adds an
optimization for the semantic alignment loop.
2020-12-07 16:15:19 -05:00
Eric Mc Sween
dce5b8759a Decaf cleanup: capitalize class names 2020-12-07 15:30:02 -05:00
Eric Mc Sween
db4b0a6f38 Decaf cleanup: do not throw strings 2020-12-07 15:28:25 -05:00
Eric Mc Sween
8c70e72bfa Decaf cleanup: unused variable 2020-12-07 15:27:41 -05:00
Eric Mc Sween
9f17f3ea0a Decaf cleanup: remove default callback 2020-12-07 15:27:01 -05:00
Eric Mc Sween
b74e7f6feb Decaf cleanup: unnecessary returns 2020-12-07 15:25:52 -05:00
Eric Mc Sween
a91770e979 Decaf cleanup: remove Array.from() 2020-12-07 15:25:20 -05:00
Eric Mc Sween
408db87d77 Merge pull request #150 from overleaf/em-docker-compose-healthchecks
Upgrade build-scripts to 3.4.0
2020-11-26 11:05:49 -05:00
Jakob Ackermann
5e00684dbb [misc] bump metrics module to 3.4.1
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
2020-11-25 11:57:19 +00:00
Eric Mc Sween
ed1e192d2e Upgrade build-scripts to 3.4.0
This version fixes docker-compose health checks for dependent services. See
https://github.com/overleaf/dev-environment/pull/409 for details.
2020-11-24 08:03:29 -05:00
Jakob Ackermann
08ed5f6c9b [misc] bump @overleaf/redis-wrapper to version 2.0.0 2020-11-11 16:20:32 +00:00
Brian Gough
0c7d8ae5ae downgrade to logger 1.9.1
investigating possible regression in 2.2.0
2020-10-16 14:27:36 +01:00
Jakob Ackermann
c749d2043e Merge pull request #145 from overleaf/dependabot-npm_and_yarn-bl-2.2.1
Bump bl from 2.2.0 to 2.2.1
2020-09-30 10:16:27 +02:00
Jakob Ackermann
3b7b14ccea Merge pull request #143 from overleaf/dependabot-npm_and_yarn-lodash-4.17.20
Bump lodash from 4.17.15 to 4.17.20
2020-09-30 10:14:59 +02:00
Jakob Ackermann
89801b592f Merge pull request #146 from overleaf/dependabot-npm_and_yarn-node-fetch-2.6.1
Bump node-fetch from 2.6.0 to 2.6.1
2020-09-30 10:12:31 +02:00
Jakob Ackermann
dee4f6f0d1 Merge pull request #147 from overleaf/jpa-bump-dev-env-3-3-4-node-10-22-1
[misc] bump the dev-env to 3.3.4 and bump the node version to 10.22.1
2020-09-30 10:12:17 +02:00
dependabot[bot]
033cc28e7f Bump bl from 2.2.0 to 2.2.1
Bumps [bl](https://github.com/rvagg/bl) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/rvagg/bl/releases)
- [Commits](https://github.com/rvagg/bl/compare/v2.2.0...v2.2.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-30 08:11:14 +00:00
dependabot[bot]
ffc55a0502 Bump lodash from 4.17.15 to 4.17.20
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.20.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.20)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-30 08:10:39 +00:00
Jakob Ackermann
ccdf555ecb [misc] bump the dev-env to 3.3.4 and bump the node version to 10.22.1 2020-09-17 13:36:27 +01:00
dependabot[bot]
e47c3b747b Bump node-fetch from 2.6.0 to 2.6.1
Bumps [node-fetch](https://github.com/bitinn/node-fetch) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/bitinn/node-fetch/releases)
- [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-12 15:08:02 +00:00
Jakob Ackermann
16ef0d9610 [misc] mongodb: use the new db connector by default
mongojs was enabling it by default as well.
2020-09-10 10:40:05 +01:00
Jakob Ackermann
c337cf1c4f [misc] mongodb: refactor the process of setting up the db construct
Co-Authored-By: John Lees-Miller <jdleesmiller@gmail.com>
2020-09-07 09:49:06 +01:00
Jakob Ackermann
1d57706d44 [misc] migrate acceptance tests to the native mongo driver, drop mongojs 2020-08-28 17:06:25 +01:00
Jakob Ackermann
f80a92ce46 [misc] migrate the app to the native mongo driver
acceptance tests to follow in a separate commit
2020-08-28 15:12:30 +01:00
Christopher Hoskin
a3430cfdae [misc] bump logger-sharelatex to version 2.2.0 2020-08-13 08:57:47 +01:00
Jakob Ackermann
7f19fa6081 [misc] bump the dev-env to 3.3.2 2020-08-10 17:01:12 +01:00
Jakob Ackermann
bd5cb98517 [misc] bump the dev-env to 3.3.1 2020-08-06 12:39:58 +01:00
Shane Kilkelly
f331c89da5 Fix integrity hash on coffee-script dependency 2020-07-24 11:14:20 +01:00
Shane Kilkelly
2ef5f471ab Upgrade to redis-sharelatex 1.0.13 2020-07-24 10:14:48 +01:00
Brian Gough
cf9f3f48dd update buildscript.txt to node 10.21.0 2020-06-03 11:10:45 +01:00
Brian Gough
ca98a604ff update to node 10.21.0 2020-06-03 10:22:56 +01:00