Commit graph

725 commits

Author SHA1 Message Date
Eric Mc Sween
16d3d59bc1 Merge pull request #20869 from overleaf/em-ranges-tracker-duplicate-ids
Fix handling of duplicate change ids in RangesTracker

GitOrigin-RevId: 1a75d1dc7e7931c459d2d004973f34d931100d33
2024-10-14 11:05:30 +00:00
Jakob Ackermann
ea55a850f9 Merge pull request #20761 from overleaf/jpa-error-instance
[logger] forward an Error instance to Sentry.captureException

GitOrigin-RevId: 3a310994622c85f3791181b87d4d382c34f6fd32
2024-10-14 10:58:16 +00:00
Jakob Ackermann
85bc305df2 Merge pull request #20750 from overleaf/jpa-issue-20743
[logger] sanitize sentry exception context

GitOrigin-RevId: 7f141ad93fc8df807acad9bcf691027efe799150
2024-10-14 10:58:07 +00:00
Antoine Clausse
6c2cf20125 Merge pull request #20552 from overleaf/ac-update-node-fetch-2
Reapply "Upgrade node-fetch to 2.7.0", Fix fetch-utils tests

GitOrigin-RevId: b42a2d2c50ce73f474e39755845e4df065f30b48
2024-10-01 08:05:18 +00:00
Liangjun Song
cfd0b39b96 Merge pull request #20557 from overleaf/ls-bump-express
update express and other dependencies

GitOrigin-RevId: 22107eab0fa95d534ab07360548ae2b2123776db
2024-09-27 08:04:18 +00:00
Jakob Ackermann
baaac44172 Merge pull request #20302 from overleaf/jpa-object-persistor-metrics
[object-persistor] add more metrics to getObjectStream and sendStream

GitOrigin-RevId: 9fe6b9d205de6ad27838f91d92d2b3a3d6c2f129
2024-09-24 08:05:54 +00:00
Jakob Ackermann
8fa676082e Merge pull request #20299 from overleaf/jpa-object-persistor-metrics-dep
[object-persistor] depend on @overleaf/metrics directly

GitOrigin-RevId: eb0c07af8101d44def14154abb552bc77254e074
2024-09-24 08:05:50 +00:00
Jakob Ackermann
3ba3ffe56d Merge pull request #20277 from overleaf/jpa-ro-mirror-chunk
[misc] use Chunk for read-only mirror in frontend

GitOrigin-RevId: b97e520c7a63015179792542310d18f346752e92
2024-09-24 08:05:42 +00:00
Antoine Clausse
2179d62d74 Merge pull request #20549 from overleaf/revert-20165-ac-update-node-fetch
Revert "Upgrade node-fetch to 2.7.0"

GitOrigin-RevId: 928df97c6fbc81f21097aaf9c331b0c70eeac002
2024-09-24 08:04:44 +00:00
Antoine Clausse
da3553d800 Upgrade node-fetch to 2.7.0 (#20165)
* Set `node-fetch` to `^2.7.0`

* Update package-lock.json

```
# root
bin/npm update node-fetch
bin/npm update cross-fetch

# in other path in docker
npm update node-fetch
```

* Update node-fetch patch

* [fetch-utils] Skip the test: destroys the request body if it doesn't get consumed

```
  1) fetch-utils
       fetchJson
         destroys the request body if it doesn't get consumed:
     FetchError: Invalid response body while trying to fetch http://example.com:30001/json/ignore-request: write EPIPE
      at PassThrough.<anonymous> (/overleaf/node_modules/node-fetch/lib/index.js:400:12)
      at PassThrough.emit (node:events:529:35)
      at emitErrorNT (node:internal/streams/destroy:151:8)
      at emitErrorCloseNT (node:internal/streams/destroy:116:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
```

* [fetch-utils] Delete the test: destroys the request body if it doesn't get consumed

* Remove the `setTimeout` in the node-fetch patch

Fixes a test and doesn't break filestore acceptance tests

* Update node-fetch patch again: bring changes from https://github.com/node-fetch/node-fetch/blob/e87b093/src/index.js

* Update node-fetch patch again: bring changes from https://github.com/node-fetch/node-fetch/blob/e87b093/src/index.js

* Update node-fetch patches back to single lines

Per https://github.com/overleaf/internal/pull/20165#discussion_r1739035513

GitOrigin-RevId: 945e5a12e838673b7bc87b588b7aca1bcd9109e2
2024-09-24 08:04:39 +00:00
Eric Mc Sween
250acbf33c Merge pull request #20520 from overleaf/em-ts-import
Use new import JSDoc syntax for Typescript annotations

GitOrigin-RevId: 782456d637fc8f2de6163b5d70fabf06c1d74964
2024-09-23 08:04:47 +00:00
Eric Mc Sween
1061152c31 Merge pull request #20466 from overleaf/em-revert-history-limit
Revert #20421

GitOrigin-RevId: c2ab656214354e3927f0e56ee7c09a8d0a53fd73
2024-09-18 08:05:10 +00:00
Eric Mc Sween
7cd16a84e0 Merge pull request #20421 from overleaf/em-history-limit-tracked-deletes
Do not count tracked deletes towards size limit in history

GitOrigin-RevId: 0185e6df80d8f3433aa489a1c90f5a6499af5ef4
2024-09-18 08:05:05 +00:00
Jakob Ackermann
84be385550 Merge pull request #20331 from overleaf/jpa-build-scripts-sync
[misc] sync build scripts

GitOrigin-RevId: 343fd861b6c7b3d81b21babbd41bbb968f0fd448
2024-09-10 08:05:01 +00:00
Antoine Clausse
9bc1085566 [fetch-utils] Tests updates (#20280)
* [fetch-utils] Check when `aborted` is thrown

* [fetch-utils] Rename `AbortError` to `abortErrorMessage`

* [fetch-utils] Don't test `caughtAbortedErr`

Partially reverts 87604105c4fd66e548d917d6d93f1603977689be

GitOrigin-RevId: fd91f952793d6727e3efcbb58867c95642d04983
2024-09-06 08:05:44 +00:00
andrew rumble
bdc907ecc6 Move overrides to top level
See documentation:
https://docs.npmjs.com/cli/v10/configuring-npm/package-json#overrides

GitOrigin-RevId: 13f7a475335e9ef54419f18fab14f31da32ef250
2024-09-06 08:04:44 +00:00
Antoine Clausse
5afa892981 [fetch-utils] Fix unit tests (#20210)
* [fetch-utils] Fix FetchUtilsTests.js tests

* Cleanup between tests
* Define `AbortError`

* [fetch-utils] Add `expectConnectionCount` showcasing that the connection stays on when the stream is destroyed

* Revert "[fetch-utils] Add `expectConnectionCount` showcasing that the connection stays on when the stream is destroyed"

This reverts commit b10da7b3fc06a7345df8fd70f27fad70a478bbb4.

* [fetch-utils] Fix `supports abort signals` test

* [fetch-utils] Add tests "aborts the request when the request body is destroyed during transfer"

* [fetch-utils] Add extra waiting step before `expectRequestAborted`

* [fetch-utils] Add `while (!req?.destroyed) await wait(10)`

Unfortunately I couldn't find a better event to await.

To try with this to test flakiness
```
for i in {1..100}; do npm run test:unit || break; echo "Run $i completed"; done
```

* [fetch-utils] Replace arbitrary `wait(10)` by `this.server.waitForEvent('request-received')`

* [fetch-utils] Improve tests per PR comments

See https://github.com/overleaf/internal/pull/20210

1. Replace `waitForEvent` with `once`
2. Replace `waitForRequestAborted` by `expectRequestAborted`
3. Add comment in `expectRequestAborted` empty catch block

Non-flakiness rechecked with `for i in {1..100}; do npm run test:unit || break; echo "Run $i completed"; done`

* [fetch-utils] Add small wait before checking `lastReq === undefined`

Per https://github.com/overleaf/internal/pull/20210#discussion_r1743329462

GitOrigin-RevId: 5fe542e0a8e6011307e03237e2f81404d9a0e674
2024-09-05 08:05:57 +00:00
Mathias Jakobsen
496a63cf23 Merge pull request #20058 from overleaf/mj-promisify-tpr
[third-party-references] Promisify third-party-references

GitOrigin-RevId: cfc462abaca95ed10798b3d1756950f2997a5947
2024-09-02 08:04:55 +00:00
Eric Mc Sween
8b4b454b97 Merge pull request #20052 from overleaf/em-tracked-deletes-limit
Include tracked deletes when limiting document size

GitOrigin-RevId: 9d7b2870001fb942eaa92db78993fee1ddb3937c
2024-08-27 08:04:31 +00:00
Jakob Ackermann
1182049d89 Merge pull request #20057 from overleaf/jpa-readonly-tuning
[web] tuning for readonly mirror on client

GitOrigin-RevId: c0ecc8923cdec6c515da17750133632ebc430e8d
2024-08-23 08:04:48 +00:00
Jakob Ackermann
577497b655 Merge pull request #19842 from overleaf/jpa-ro-mirror-on-client
[misc] add readonly mirror of full project content on the client

GitOrigin-RevId: 940bd93bfd587f83ca383d10fc44579b38fc3e88
2024-08-22 08:05:13 +00:00
Mathias Jakobsen
654d96ace6 Merge pull request #19743 from overleaf/mj-restore-project
[web+project-history] Add project version reverting

GitOrigin-RevId: 0f77cec730393187d531c0c6561faaa652bebf29
2024-08-21 08:05:08 +00:00
andrew rumble
56f70b3246 Support v5 of mongodb when recording metrics
GitOrigin-RevId: 6365fa54b0b50dc8527fe75f371c53a43e2c5a72
2024-08-21 08:04:55 +00:00
Eric Mc Sween
97b62e01f8 Merge pull request #19851 from overleaf/em-user-content-logging
Avoid logging user content when errors occur in history

GitOrigin-RevId: 0326065a15dfc30155847100bb5a077efea771b5
2024-08-12 08:04:59 +00:00
Jakob Ackermann
3836323724 Merge pull request #19817 from overleaf/jpa-types
[overleaf-editor-core] stronger type checking via web

GitOrigin-RevId: 427019f40e2905f2e0ec11dc09f5fccdbb1f905b
2024-08-08 08:05:01 +00:00
Jakob Ackermann
e26b6de51b Merge pull request #19740 from overleaf/jpa-linked-file-metadata
[misc] persist linkedFileData in full project history as file metadata

GitOrigin-RevId: f3e8ba947ea34b6796e210a076a248c57188d148
2024-08-06 08:04:44 +00:00
Jakob Ackermann
8cba7935b7 Merge pull request #19689 from overleaf/jpa-editor-core-cjs
[overleaf-editor-core] remove Node.js stdlib import and tweak babel config

GitOrigin-RevId: 2ac13cc17651a3b522606f8df23c7e6915be19a6
2024-08-02 08:05:01 +00:00
Jakob Ackermann
e43491a04f Merge pull request #19547 from overleaf/jpa-move-v2-doc-versions
[overleaf-editor-core] keep v2DocVersions in-sync with fileMap

GitOrigin-RevId: 23491dfe51b71561837c96b8c550f75b0835a176
2024-08-02 08:04:57 +00:00
Alf Eaton
136214f37a Return the Promise from an expressified async function (#19359)
* Return the Promise from an expressified async function

GitOrigin-RevId: ca4c221a92de888e210e995faad97d0ea241e93f
2024-07-15 09:05:29 +00:00
Antoine Clausse
778aa717d9 Merge pull request #19322 from overleaf/revert-19259-ac-update-google-cloud-storage
Revert "Update `@google-cloud/storage` and remove one patch"

GitOrigin-RevId: a99b4cdf69804edb2bad0251d9ed693a9c1d0418
2024-07-15 09:01:58 +00:00
Antoine Clausse
2dae2780c1 Update @google-cloud/storage and remove one patch (#19259)
* Update `@google-cloud/storage` and remove its patches

* Re-add patch on `@google-cloud/storage/retry-request`

Replace mentions of `.pipe()` and run `patch-package @google-cloud/storage/retry-request`

It was very hard to test whether the package could still cause bugs, so I'd rather put the patch back

GitOrigin-RevId: be83b54b6d55849995ac131bfcf3c9acd9b6dd8e
2024-07-15 09:01:40 +00:00
Antoine Clausse
01188589f8 Add some JSDoc types to @overleaf/logger (#19153)
* Add some JSDoc types to `@overleaf/logger`

* Update `logger.error` calls

* Fixup `logger.err` JSDoc

* Update `logger.err` calls

* Fix `args` type

* Remove "Error message" description

* Replace `arguments` by actual arguments of the method

* Fix: "ESLint: Unnecessary '.apply()'.(no-useless-call)"

* Add JSDoc params to `debug` `info` `warn`

* Remove extra `args` param in JSDoc so developers aren't invited to use it

Not sure if this is the best thing to do because it creates a warning in the IDE: "Parameter args is not described in JSDoc"

* Add comment about serialization of `err` `req` `res`

* Allow strings as first param in `debug` `info` `warn`

* Fix syntax for optional parameters in JSDoc

* Add 2 signatures, to avoid "string, string" params

* Fix `@signature` names copy-pastes

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* Revert the double `@param attributes`. It doesn't work

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 086dee8bbf30d577c5e1f844a9df5e518c46aca7
2024-06-28 08:04:25 +00:00
Antoine Clausse
fc34b41b5b [web] Promisify two-factor-authentication (#19004)
* Fixup typos

* Promisify `getPendingUser`

* Promisify `getPendingUserWithExpectations`

* Add promises to `AccessTokenEncryptor`

* Promisify `TwoFactorAuthenticationHandler`

* Promisify/Expressify `TwoFactorAuthenticationController`

* Centralise `unprocessableEntity` error handling into a function

* Fixup: entralise `unprocessableEntity` error handling into a function

Avoid "responding to the query twice"

* Remove unnecessary try/catch

* Fixup: Add `async` to AccessTokenEncryptor promises

* Add tests on `AccessTokenEncryptor.promises`

* Revert "Fixup: entralise `unprocessableEntity` error handling into a function"

This reverts commit 23bd9d73260733062908b98961846733c61171e9.

* Revert "Centralise `unprocessableEntity` error handling into a function"

This reverts commit 197ca3da02412d5224d411b29df1d7b9e5327d01.

GitOrigin-RevId: 1a2864d28e87fd5e48cd3723a3da8047b79a1596
2024-06-26 08:05:06 +00:00
Eric Mc Sween
e73fdfba63 Merge pull request #18906 from overleaf/em-migrate-existing-histories-2
History ranges migration script - second attempt

GitOrigin-RevId: 60a2c04e2a72e76a58e9e179fefc4186a96fde32
2024-06-19 08:05:02 +00:00
Domagoj Kriskovic
7e8e2b0585 Send origin metadata through docupdater and project-history when restoring files (#18721)
* add RestoreFileOrigin in overleaf-editor-core

* support source to be an object

* use sourceOrOrigin as param

* rename to originOrSource so the priority is more clear

* get timestamp from version

* fix test

* include version and min_count in getUpdatesFromHistory

* extractOriginOrSource util function

* fix RestoreManagerTests

GitOrigin-RevId: 0ace05a6ade2794c753a9d0bffb4f858ecc6899a
2024-06-18 08:04:34 +00:00
Jakob Ackermann
e36de5a62d Merge pull request #18856 from overleaf/jpa-server-ce-shutdown
[server-ce] improve shutdown procedure

GitOrigin-RevId: 5a99868d17f597c366e42625cd39f05146dcb682
2024-06-18 08:04:04 +00:00
andrew rumble
ee31d44f03 Explicitly specify latest version of async
GitOrigin-RevId: 9112007de0f5eb4e75604de9f01c2d77b3b8b980
2024-06-17 08:04:00 +00:00
andrew rumble
0e2f0e7f7d Update protobufjs to 7.2.5 everywhere
GitOrigin-RevId: 353d3a6d7bef5527fe0a068775dd6ac8c3557483
2024-06-17 08:03:46 +00:00
Eric Mc Sween
55c342134c Merge pull request #18659 from overleaf/em-crop-comments-tracked-deletes
Crop comments when processing tracked deletes

GitOrigin-RevId: 662c9ed86a8ed4959d1671ce466548487f334f45
2024-06-05 08:03:53 +00:00
Mathias Jakobsen
f14fdc7b7c Merge pull request #17958 from overleaf/mj-web-revert-file
[web] Initial version of file reverting

GitOrigin-RevId: ef500b2c3ddf21b0acb6650c2526edd10ab2cc18
2024-05-29 08:04:16 +00:00
Eric Mc Sween
3a1560894a Merge pull request #18516 from overleaf/em-web-resolved-comment-ids
Reintroduce resolved comment ids in getDocument()

GitOrigin-RevId: 591589efc643c815c40df440d1297158901f7a79
2024-05-28 08:04:01 +00:00
Eric Mc Sween
ea8f8d0651 Merge pull request #18506 from overleaf/em-revert-web-resolved-comment-ids
Revert #18398 - return resolved commentIds with getDocument()

GitOrigin-RevId: c5d2dd91d02d88029eb2702d73ac37cb8dbca32c
2024-05-27 10:24:25 +00:00
Eric Mc Sween
83e1452991 Merge pull request #18398 from overleaf/em-web-resolved-comment-ids
Return resolved comment ids with getDocument()

GitOrigin-RevId: 30234f72d52b90b313821290b6c63aa6cc6cb243
2024-05-27 10:24:13 +00:00
Andrew Rumble
71187a51ba Merge pull request #18289 from overleaf/ac-ar-eslint-return-await
Add ESLint rule @typescript-eslint/return-await to backend services

GitOrigin-RevId: 75e3e32597827fcc852e69d479515fc72e8f45e4
2024-05-27 10:22:49 +00:00
Eric Mc Sween
5d0190da0f Merge pull request #18342 from overleaf/em-tracking-props-none
Separate handling of "none" tracking type in operations

GitOrigin-RevId: b07ee5320ac1a9c63e3b0059aade1a1564819943
2024-05-17 08:04:46 +00:00
Mathias Jakobsen
655f9b50ae Merge pull request #18309 from overleaf/mj-add-comment-resolved
[overleaf-editor-core] Omit resolved: false for comments serialisation

GitOrigin-RevId: 909f20efd8f3c3e50d40e09366951d317a4c31bf
2024-05-16 08:04:46 +00:00
Brian Gough
6680bedf6c Merge pull request #16728 from overleaf/bg-clsi-timeout-fetchutils-connect-timeout
add connect timeout to fetch-utils using custom agents

GitOrigin-RevId: 877dbc1311197461cf1e9bfee74196bd5d3e42dc
2024-05-16 08:04:41 +00:00
Mathias Jakobsen
478a6157cc Merge pull request #18305 from overleaf/mj-add-comment-invert
[overleaf-editor-core] Make AddCommentOperation inversion restore state

GitOrigin-RevId: 5a47a2463e3650ba7a6b0a0f53c895fa6ec1b54c
2024-05-16 08:04:37 +00:00
Eric Mc Sween
e87df4a791 Merge pull request #18167 from overleaf/em-simpler-typescript-backend
Simpler Typescript setup in the backend

GitOrigin-RevId: 3a90f26a5ac8029d14203faa98008375ce933e7d
2024-05-07 08:04:00 +00:00
Eric Mc Sween
b75ba32774 Merge pull request #18114 from overleaf/em-resync-comments
Handle comments when resyncing history

GitOrigin-RevId: cd848fbd91f017a0a66e46df5c009bd16498d154
2024-05-06 08:04:25 +00:00