Commit graph

736 commits

Author SHA1 Message Date
Jakob Ackermann
d5478c11ea Merge pull request #21623 from overleaf/jpa-s3-ssec-static-dek-path
[object-persistor] s3SSEC: use static path for DEK inside project folder

GitOrigin-RevId: 0c43ef8964c16d3e7638f17ff47b1c4641e439df
2024-11-08 09:06:51 +00:00
Jakob Ackermann
a67560c26b Merge pull request #21589 from overleaf/jpa-s3-ssec-hkdf
[object-persistor] s3SSEC: add HKDF layer to KEK

GitOrigin-RevId: 1def9e378e1dfd90f449ad392b0db2101584e17f
2024-11-08 09:06:46 +00:00
Jakob Ackermann
bec73ddfae Merge pull request #21543 from overleaf/jpa-s3-seec-kek-rotation
[object-persistor] s3SSEC: add support for (automatic) KEK rotation

GitOrigin-RevId: 315082e894c74e276a8efbc46b41ec7e102f9010
2024-11-08 09:06:42 +00:00
Jakob Ackermann
859901ac0c Merge pull request #21380 from overleaf/jpa-s3-ssec-backend
[object-persistor] add backend for SSE-C with S3 using KEK and DEK

GitOrigin-RevId: 9676f5cd5e08107c8c284b68b8d450a1c05bf1b1
2024-11-08 09:06:37 +00:00
Eric Mc Sween
d95981cb0c Merge pull request #21682 from overleaf/em-revert-ranges-tracker-changes
Revert RangesTracker fix for multiple tracked deletes at same position

GitOrigin-RevId: ee6ade406a86319f5551863e18ed40d508128ecb
2024-11-07 09:05:03 +00:00
Eric Mc Sween
bc1e3dacda Merge pull request #21643 from overleaf/em-tracked-deletes-at-same-position
Improve handling of tracked delete rejections

GitOrigin-RevId: 79308539f5e7af70f6292a0fd6352e882eac84dd
2024-11-07 09:04:45 +00:00
Andrew Rumble
57d12dfb17 Update to node 18.20.4
GitOrigin-RevId: 4b613ea3973cb7944c3d414ea91bcf367dbf50a6
2024-11-07 09:04:31 +00:00
Jakob Ackermann
5bb90dc6cb Merge pull request #21442 from overleaf/jpa-align-mongo-version
[misc] align all the mongodb dependency versions

GitOrigin-RevId: 1194fe57601af98bb61250a285bfc85b4b8179dd
2024-11-01 09:05:23 +00:00
Jakob Ackermann
7b3e39f63f Merge pull request #21375 from overleaf/jpa-if-none-match
[object-persistor] add support for ifNoneMatch=* in sendStream

GitOrigin-RevId: 268f054ac1b6452105b02757cdec32bad00702fd
2024-11-01 09:05:11 +00:00
Andrew Rumble
79d80cc266 Merge pull request #21462 from overleaf/revert-18828-ac-node-20
Revert "Upgrade Node from `18.20.2` to `20.17.0`"

GitOrigin-RevId: cc12c33b4588caa2959817a113547d20d437d6f4
2024-10-30 09:05:07 +00:00
Andrew Rumble
a480df8a89 Merge pull request #18828 from overleaf/ac-node-20
Upgrade Node from `18.20.2` to `20.17.0`

GitOrigin-RevId: 0ba9b0921fc246a97d3da0aad7d48635c0e24ae8
2024-10-30 09:04:38 +00:00
Eric Mc Sween
16d3d59bc1 Merge pull request #20869 from overleaf/em-ranges-tracker-duplicate-ids
Fix handling of duplicate change ids in RangesTracker

GitOrigin-RevId: 1a75d1dc7e7931c459d2d004973f34d931100d33
2024-10-14 11:05:30 +00:00
Jakob Ackermann
ea55a850f9 Merge pull request #20761 from overleaf/jpa-error-instance
[logger] forward an Error instance to Sentry.captureException

GitOrigin-RevId: 3a310994622c85f3791181b87d4d382c34f6fd32
2024-10-14 10:58:16 +00:00
Jakob Ackermann
85bc305df2 Merge pull request #20750 from overleaf/jpa-issue-20743
[logger] sanitize sentry exception context

GitOrigin-RevId: 7f141ad93fc8df807acad9bcf691027efe799150
2024-10-14 10:58:07 +00:00
Antoine Clausse
6c2cf20125 Merge pull request #20552 from overleaf/ac-update-node-fetch-2
Reapply "Upgrade node-fetch to 2.7.0", Fix fetch-utils tests

GitOrigin-RevId: b42a2d2c50ce73f474e39755845e4df065f30b48
2024-10-01 08:05:18 +00:00
Liangjun Song
cfd0b39b96 Merge pull request #20557 from overleaf/ls-bump-express
update express and other dependencies

GitOrigin-RevId: 22107eab0fa95d534ab07360548ae2b2123776db
2024-09-27 08:04:18 +00:00
Jakob Ackermann
baaac44172 Merge pull request #20302 from overleaf/jpa-object-persistor-metrics
[object-persistor] add more metrics to getObjectStream and sendStream

GitOrigin-RevId: 9fe6b9d205de6ad27838f91d92d2b3a3d6c2f129
2024-09-24 08:05:54 +00:00
Jakob Ackermann
8fa676082e Merge pull request #20299 from overleaf/jpa-object-persistor-metrics-dep
[object-persistor] depend on @overleaf/metrics directly

GitOrigin-RevId: eb0c07af8101d44def14154abb552bc77254e074
2024-09-24 08:05:50 +00:00
Jakob Ackermann
3ba3ffe56d Merge pull request #20277 from overleaf/jpa-ro-mirror-chunk
[misc] use Chunk for read-only mirror in frontend

GitOrigin-RevId: b97e520c7a63015179792542310d18f346752e92
2024-09-24 08:05:42 +00:00
Antoine Clausse
2179d62d74 Merge pull request #20549 from overleaf/revert-20165-ac-update-node-fetch
Revert "Upgrade node-fetch to 2.7.0"

GitOrigin-RevId: 928df97c6fbc81f21097aaf9c331b0c70eeac002
2024-09-24 08:04:44 +00:00
Antoine Clausse
da3553d800 Upgrade node-fetch to 2.7.0 (#20165)
* Set `node-fetch` to `^2.7.0`

* Update package-lock.json

```
# root
bin/npm update node-fetch
bin/npm update cross-fetch

# in other path in docker
npm update node-fetch
```

* Update node-fetch patch

* [fetch-utils] Skip the test: destroys the request body if it doesn't get consumed

```
  1) fetch-utils
       fetchJson
         destroys the request body if it doesn't get consumed:
     FetchError: Invalid response body while trying to fetch http://example.com:30001/json/ignore-request: write EPIPE
      at PassThrough.<anonymous> (/overleaf/node_modules/node-fetch/lib/index.js:400:12)
      at PassThrough.emit (node:events:529:35)
      at emitErrorNT (node:internal/streams/destroy:151:8)
      at emitErrorCloseNT (node:internal/streams/destroy:116:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
```

* [fetch-utils] Delete the test: destroys the request body if it doesn't get consumed

* Remove the `setTimeout` in the node-fetch patch

Fixes a test and doesn't break filestore acceptance tests

* Update node-fetch patch again: bring changes from https://github.com/node-fetch/node-fetch/blob/e87b093/src/index.js

* Update node-fetch patch again: bring changes from https://github.com/node-fetch/node-fetch/blob/e87b093/src/index.js

* Update node-fetch patches back to single lines

Per https://github.com/overleaf/internal/pull/20165#discussion_r1739035513

GitOrigin-RevId: 945e5a12e838673b7bc87b588b7aca1bcd9109e2
2024-09-24 08:04:39 +00:00
Eric Mc Sween
250acbf33c Merge pull request #20520 from overleaf/em-ts-import
Use new import JSDoc syntax for Typescript annotations

GitOrigin-RevId: 782456d637fc8f2de6163b5d70fabf06c1d74964
2024-09-23 08:04:47 +00:00
Eric Mc Sween
1061152c31 Merge pull request #20466 from overleaf/em-revert-history-limit
Revert #20421

GitOrigin-RevId: c2ab656214354e3927f0e56ee7c09a8d0a53fd73
2024-09-18 08:05:10 +00:00
Eric Mc Sween
7cd16a84e0 Merge pull request #20421 from overleaf/em-history-limit-tracked-deletes
Do not count tracked deletes towards size limit in history

GitOrigin-RevId: 0185e6df80d8f3433aa489a1c90f5a6499af5ef4
2024-09-18 08:05:05 +00:00
Jakob Ackermann
84be385550 Merge pull request #20331 from overleaf/jpa-build-scripts-sync
[misc] sync build scripts

GitOrigin-RevId: 343fd861b6c7b3d81b21babbd41bbb968f0fd448
2024-09-10 08:05:01 +00:00
Antoine Clausse
9bc1085566 [fetch-utils] Tests updates (#20280)
* [fetch-utils] Check when `aborted` is thrown

* [fetch-utils] Rename `AbortError` to `abortErrorMessage`

* [fetch-utils] Don't test `caughtAbortedErr`

Partially reverts 87604105c4fd66e548d917d6d93f1603977689be

GitOrigin-RevId: fd91f952793d6727e3efcbb58867c95642d04983
2024-09-06 08:05:44 +00:00
andrew rumble
bdc907ecc6 Move overrides to top level
See documentation:
https://docs.npmjs.com/cli/v10/configuring-npm/package-json#overrides

GitOrigin-RevId: 13f7a475335e9ef54419f18fab14f31da32ef250
2024-09-06 08:04:44 +00:00
Antoine Clausse
5afa892981 [fetch-utils] Fix unit tests (#20210)
* [fetch-utils] Fix FetchUtilsTests.js tests

* Cleanup between tests
* Define `AbortError`

* [fetch-utils] Add `expectConnectionCount` showcasing that the connection stays on when the stream is destroyed

* Revert "[fetch-utils] Add `expectConnectionCount` showcasing that the connection stays on when the stream is destroyed"

This reverts commit b10da7b3fc06a7345df8fd70f27fad70a478bbb4.

* [fetch-utils] Fix `supports abort signals` test

* [fetch-utils] Add tests "aborts the request when the request body is destroyed during transfer"

* [fetch-utils] Add extra waiting step before `expectRequestAborted`

* [fetch-utils] Add `while (!req?.destroyed) await wait(10)`

Unfortunately I couldn't find a better event to await.

To try with this to test flakiness
```
for i in {1..100}; do npm run test:unit || break; echo "Run $i completed"; done
```

* [fetch-utils] Replace arbitrary `wait(10)` by `this.server.waitForEvent('request-received')`

* [fetch-utils] Improve tests per PR comments

See https://github.com/overleaf/internal/pull/20210

1. Replace `waitForEvent` with `once`
2. Replace `waitForRequestAborted` by `expectRequestAborted`
3. Add comment in `expectRequestAborted` empty catch block

Non-flakiness rechecked with `for i in {1..100}; do npm run test:unit || break; echo "Run $i completed"; done`

* [fetch-utils] Add small wait before checking `lastReq === undefined`

Per https://github.com/overleaf/internal/pull/20210#discussion_r1743329462

GitOrigin-RevId: 5fe542e0a8e6011307e03237e2f81404d9a0e674
2024-09-05 08:05:57 +00:00
Mathias Jakobsen
496a63cf23 Merge pull request #20058 from overleaf/mj-promisify-tpr
[third-party-references] Promisify third-party-references

GitOrigin-RevId: cfc462abaca95ed10798b3d1756950f2997a5947
2024-09-02 08:04:55 +00:00
Eric Mc Sween
8b4b454b97 Merge pull request #20052 from overleaf/em-tracked-deletes-limit
Include tracked deletes when limiting document size

GitOrigin-RevId: 9d7b2870001fb942eaa92db78993fee1ddb3937c
2024-08-27 08:04:31 +00:00
Jakob Ackermann
1182049d89 Merge pull request #20057 from overleaf/jpa-readonly-tuning
[web] tuning for readonly mirror on client

GitOrigin-RevId: c0ecc8923cdec6c515da17750133632ebc430e8d
2024-08-23 08:04:48 +00:00
Jakob Ackermann
577497b655 Merge pull request #19842 from overleaf/jpa-ro-mirror-on-client
[misc] add readonly mirror of full project content on the client

GitOrigin-RevId: 940bd93bfd587f83ca383d10fc44579b38fc3e88
2024-08-22 08:05:13 +00:00
Mathias Jakobsen
654d96ace6 Merge pull request #19743 from overleaf/mj-restore-project
[web+project-history] Add project version reverting

GitOrigin-RevId: 0f77cec730393187d531c0c6561faaa652bebf29
2024-08-21 08:05:08 +00:00
andrew rumble
56f70b3246 Support v5 of mongodb when recording metrics
GitOrigin-RevId: 6365fa54b0b50dc8527fe75f371c53a43e2c5a72
2024-08-21 08:04:55 +00:00
Eric Mc Sween
97b62e01f8 Merge pull request #19851 from overleaf/em-user-content-logging
Avoid logging user content when errors occur in history

GitOrigin-RevId: 0326065a15dfc30155847100bb5a077efea771b5
2024-08-12 08:04:59 +00:00
Jakob Ackermann
3836323724 Merge pull request #19817 from overleaf/jpa-types
[overleaf-editor-core] stronger type checking via web

GitOrigin-RevId: 427019f40e2905f2e0ec11dc09f5fccdbb1f905b
2024-08-08 08:05:01 +00:00
Jakob Ackermann
e26b6de51b Merge pull request #19740 from overleaf/jpa-linked-file-metadata
[misc] persist linkedFileData in full project history as file metadata

GitOrigin-RevId: f3e8ba947ea34b6796e210a076a248c57188d148
2024-08-06 08:04:44 +00:00
Jakob Ackermann
8cba7935b7 Merge pull request #19689 from overleaf/jpa-editor-core-cjs
[overleaf-editor-core] remove Node.js stdlib import and tweak babel config

GitOrigin-RevId: 2ac13cc17651a3b522606f8df23c7e6915be19a6
2024-08-02 08:05:01 +00:00
Jakob Ackermann
e43491a04f Merge pull request #19547 from overleaf/jpa-move-v2-doc-versions
[overleaf-editor-core] keep v2DocVersions in-sync with fileMap

GitOrigin-RevId: 23491dfe51b71561837c96b8c550f75b0835a176
2024-08-02 08:04:57 +00:00
Alf Eaton
136214f37a Return the Promise from an expressified async function (#19359)
* Return the Promise from an expressified async function

GitOrigin-RevId: ca4c221a92de888e210e995faad97d0ea241e93f
2024-07-15 09:05:29 +00:00
Antoine Clausse
778aa717d9 Merge pull request #19322 from overleaf/revert-19259-ac-update-google-cloud-storage
Revert "Update `@google-cloud/storage` and remove one patch"

GitOrigin-RevId: a99b4cdf69804edb2bad0251d9ed693a9c1d0418
2024-07-15 09:01:58 +00:00
Antoine Clausse
2dae2780c1 Update @google-cloud/storage and remove one patch (#19259)
* Update `@google-cloud/storage` and remove its patches

* Re-add patch on `@google-cloud/storage/retry-request`

Replace mentions of `.pipe()` and run `patch-package @google-cloud/storage/retry-request`

It was very hard to test whether the package could still cause bugs, so I'd rather put the patch back

GitOrigin-RevId: be83b54b6d55849995ac131bfcf3c9acd9b6dd8e
2024-07-15 09:01:40 +00:00
Antoine Clausse
01188589f8 Add some JSDoc types to @overleaf/logger (#19153)
* Add some JSDoc types to `@overleaf/logger`

* Update `logger.error` calls

* Fixup `logger.err` JSDoc

* Update `logger.err` calls

* Fix `args` type

* Remove "Error message" description

* Replace `arguments` by actual arguments of the method

* Fix: "ESLint: Unnecessary '.apply()'.(no-useless-call)"

* Add JSDoc params to `debug` `info` `warn`

* Remove extra `args` param in JSDoc so developers aren't invited to use it

Not sure if this is the best thing to do because it creates a warning in the IDE: "Parameter args is not described in JSDoc"

* Add comment about serialization of `err` `req` `res`

* Allow strings as first param in `debug` `info` `warn`

* Fix syntax for optional parameters in JSDoc

* Add 2 signatures, to avoid "string, string" params

* Fix `@signature` names copy-pastes

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* Revert the double `@param attributes`. It doesn't work

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 086dee8bbf30d577c5e1f844a9df5e518c46aca7
2024-06-28 08:04:25 +00:00
Antoine Clausse
fc34b41b5b [web] Promisify two-factor-authentication (#19004)
* Fixup typos

* Promisify `getPendingUser`

* Promisify `getPendingUserWithExpectations`

* Add promises to `AccessTokenEncryptor`

* Promisify `TwoFactorAuthenticationHandler`

* Promisify/Expressify `TwoFactorAuthenticationController`

* Centralise `unprocessableEntity` error handling into a function

* Fixup: entralise `unprocessableEntity` error handling into a function

Avoid "responding to the query twice"

* Remove unnecessary try/catch

* Fixup: Add `async` to AccessTokenEncryptor promises

* Add tests on `AccessTokenEncryptor.promises`

* Revert "Fixup: entralise `unprocessableEntity` error handling into a function"

This reverts commit 23bd9d73260733062908b98961846733c61171e9.

* Revert "Centralise `unprocessableEntity` error handling into a function"

This reverts commit 197ca3da02412d5224d411b29df1d7b9e5327d01.

GitOrigin-RevId: 1a2864d28e87fd5e48cd3723a3da8047b79a1596
2024-06-26 08:05:06 +00:00
Eric Mc Sween
e73fdfba63 Merge pull request #18906 from overleaf/em-migrate-existing-histories-2
History ranges migration script - second attempt

GitOrigin-RevId: 60a2c04e2a72e76a58e9e179fefc4186a96fde32
2024-06-19 08:05:02 +00:00
Domagoj Kriskovic
7e8e2b0585 Send origin metadata through docupdater and project-history when restoring files (#18721)
* add RestoreFileOrigin in overleaf-editor-core

* support source to be an object

* use sourceOrOrigin as param

* rename to originOrSource so the priority is more clear

* get timestamp from version

* fix test

* include version and min_count in getUpdatesFromHistory

* extractOriginOrSource util function

* fix RestoreManagerTests

GitOrigin-RevId: 0ace05a6ade2794c753a9d0bffb4f858ecc6899a
2024-06-18 08:04:34 +00:00
Jakob Ackermann
e36de5a62d Merge pull request #18856 from overleaf/jpa-server-ce-shutdown
[server-ce] improve shutdown procedure

GitOrigin-RevId: 5a99868d17f597c366e42625cd39f05146dcb682
2024-06-18 08:04:04 +00:00
andrew rumble
ee31d44f03 Explicitly specify latest version of async
GitOrigin-RevId: 9112007de0f5eb4e75604de9f01c2d77b3b8b980
2024-06-17 08:04:00 +00:00
andrew rumble
0e2f0e7f7d Update protobufjs to 7.2.5 everywhere
GitOrigin-RevId: 353d3a6d7bef5527fe0a068775dd6ac8c3557483
2024-06-17 08:03:46 +00:00
Eric Mc Sween
55c342134c Merge pull request #18659 from overleaf/em-crop-comments-tracked-deletes
Crop comments when processing tracked deletes

GitOrigin-RevId: 662c9ed86a8ed4959d1671ce466548487f334f45
2024-06-05 08:03:53 +00:00