Commit graph

1050 commits

Author SHA1 Message Date
Liangjun Song
cfd0b39b96 Merge pull request #20557 from overleaf/ls-bump-express
update express and other dependencies

GitOrigin-RevId: 22107eab0fa95d534ab07360548ae2b2123776db
2024-09-27 08:04:18 +00:00
Jakob Ackermann
e3feafb7b0 Merge pull request #20332 from overleaf/jpa-small-runner
[misc] CI improvements

GitOrigin-RevId: 819d4d357fcb130f133337668644eedcf5f4a688
2024-09-26 08:04:38 +00:00
andrew rumble
bb12dd33ee Capture more logging information for output.zip errors
GitOrigin-RevId: a2f15d05100dfdba06778a1a795c8eabefd2666d
2024-09-24 08:05:33 +00:00
andrew rumble
d4911ea246 Stop waiting for finalize
Finalize promise will only resolve when the archive is closed. We want
to stream as soon as we have the data so this does not suit us. We want
to log errors that are thrown due to finalize, these should be
propogated by archiver to the response already.

GitOrigin-RevId: 4f9d727a00ead1be3caee62e1e0adba069a545c7
2024-09-24 08:05:28 +00:00
andrew rumble
d4824b9c5f Get full path only when adding file to archive
This allows us to:
* Get files that are in subdirectories (not known to happen but might as
well support)
* Removes need to sanitise missing_files output

GitOrigin-RevId: 4b27b7f1b506578a6d713ea23979aa9370dfdcb0
2024-09-24 08:05:24 +00:00
andrew rumble
60a75d12b0 Rename files to reflect refactoring elsewhere
GitOrigin-RevId: 335f876913b7868cf69dcd55dd156261edd98945
2024-09-24 08:05:19 +00:00
andrew rumble
487d9125a2 Improve stream error safety
GitOrigin-RevId: de4c512a62d304b3eeb2a1521aac173fa93d8411
2024-09-24 08:05:15 +00:00
andrew rumble
1409e32010 Move logging into ArchiveManager
GitOrigin-RevId: 71a28a0215c5f1a6975c9e2fcdcd476513df1cbc
2024-09-24 08:05:10 +00:00
andrew rumble
c387e60a28 Remove unnecessary symlink check
GitOrigin-RevId: 08d7295403a258818276b9fbd7666a20fbc2e00f
2024-09-24 08:05:06 +00:00
andrew rumble
3fa35ef03a Share ignore logic between web and clsi
GitOrigin-RevId: c4ae306413efb71989d2db410c3859cb69b3d5de
2024-09-24 08:05:01 +00:00
andrew rumble
d627a0bee9 Remove files param from output.zip endpoint
GitOrigin-RevId: 300cd870578237ad8e61be4e9e41991f884aa754
2024-09-24 08:04:57 +00:00
andrew rumble
c764566148 Allow all files to be in zip (in same directory)
GitOrigin-RevId: 14645a0c3db88faf00e2718b9574b5892ac3efcb
2024-09-24 08:04:53 +00:00
Antoine Clausse
2179d62d74 Merge pull request #20549 from overleaf/revert-20165-ac-update-node-fetch
Revert "Upgrade node-fetch to 2.7.0"

GitOrigin-RevId: 928df97c6fbc81f21097aaf9c331b0c70eeac002
2024-09-24 08:04:44 +00:00
Antoine Clausse
da3553d800 Upgrade node-fetch to 2.7.0 (#20165)
* Set `node-fetch` to `^2.7.0`

* Update package-lock.json

```
# root
bin/npm update node-fetch
bin/npm update cross-fetch

# in other path in docker
npm update node-fetch
```

* Update node-fetch patch

* [fetch-utils] Skip the test: destroys the request body if it doesn't get consumed

```
  1) fetch-utils
       fetchJson
         destroys the request body if it doesn't get consumed:
     FetchError: Invalid response body while trying to fetch http://example.com:30001/json/ignore-request: write EPIPE
      at PassThrough.<anonymous> (/overleaf/node_modules/node-fetch/lib/index.js:400:12)
      at PassThrough.emit (node:events:529:35)
      at emitErrorNT (node:internal/streams/destroy:151:8)
      at emitErrorCloseNT (node:internal/streams/destroy:116:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
```

* [fetch-utils] Delete the test: destroys the request body if it doesn't get consumed

* Remove the `setTimeout` in the node-fetch patch

Fixes a test and doesn't break filestore acceptance tests

* Update node-fetch patch again: bring changes from https://github.com/node-fetch/node-fetch/blob/e87b093/src/index.js

* Update node-fetch patch again: bring changes from https://github.com/node-fetch/node-fetch/blob/e87b093/src/index.js

* Update node-fetch patches back to single lines

Per https://github.com/overleaf/internal/pull/20165#discussion_r1739035513

GitOrigin-RevId: 945e5a12e838673b7bc87b588b7aca1bcd9109e2
2024-09-24 08:04:39 +00:00
Alf Eaton
2976fd0b44 Fix Dockerfile FromAsCasing warnings (#20388)
GitOrigin-RevId: 6d6be3ba17eec695907dde25da2dbe98818fae10
2024-09-23 08:04:52 +00:00
Jakob Ackermann
84be385550 Merge pull request #20331 from overleaf/jpa-build-scripts-sync
[misc] sync build scripts

GitOrigin-RevId: 343fd861b6c7b3d81b21babbd41bbb968f0fd448
2024-09-10 08:05:01 +00:00
Liangjun Song
2133dde8bf remove dry run (#19820)
GitOrigin-RevId: b92e08da6654cdd37314f7c52a6946cc7ec8983a
2024-08-08 08:04:17 +00:00
Liangjun Song
5d472e9b38 limit the number of concurrent compile requests in clsi (#19717)
GitOrigin-RevId: 17909a4dd0717ea4a75288f734ddef19c7d6592e
2024-08-06 08:04:59 +00:00
Liangjun Song
e2d87f4821 Revert "Revert "log clsi concurrency (#19651)" (#19655)" (#19656)
This reverts commit 3719f468a6336ff4bf5126f3e5d79dcde172b7ea.

GitOrigin-RevId: 6f4809f3c607c62892fc1e110eeaa2a1e7a124a4
2024-07-31 08:04:28 +00:00
Liangjun Song
39112dafb7 Revert "log clsi concurrency (#19651)" (#19655)
This reverts commit 504f6f9cc3d4b7386dbcde86f76c1cf3a13e944f.

GitOrigin-RevId: 3719f468a6336ff4bf5126f3e5d79dcde172b7ea
2024-07-30 08:05:52 +00:00
Liangjun Song
01b7896717 log clsi concurrency (#19651)
GitOrigin-RevId: 504f6f9cc3d4b7386dbcde86f76c1cf3a13e944f
2024-07-30 08:05:26 +00:00
Brian Gough
1a64d37fde Merge pull request #19540 from overleaf:bg-fix-nginx-clsi-config-in-dev-env
synchonise clsi nginx.conf in dev env with production nginx proxy.conf

GitOrigin-RevId: 986c32d7d7d7dbd5245555c3db9b779298465762
2024-07-23 08:04:40 +00:00
Jakob Ackermann
864ac883cd Merge pull request #19284 from overleaf/jpa-align-pipelines
[misc] align CI pipelines

GitOrigin-RevId: e2a74cbbff15986d5ffc8f5befa3697d406b8cdd
2024-07-15 09:05:16 +00:00
Jakob Ackermann
cbe7a2f5a9 Merge pull request #19282 from overleaf/jpa-filestore-sharding
[filestore] add sharding to test_acceptance in CI

GitOrigin-RevId: dd16a0370878d76a934d277838083e08ae0ad8c2
2024-07-15 09:05:11 +00:00
Jakob Ackermann
9f68bc5660 Merge pull request #19296 from overleaf/jpa-issue-19290-3
[clsi] atomic writing of LaTeXMk output

GitOrigin-RevId: d81c497370587b98fc7ad282035cd59b0ae09ec8
2024-07-15 09:01:04 +00:00
Jakob Ackermann
51a24601ec Merge pull request #19293 from overleaf/jpa-issue-19290-2
[clsi] fix parsing of the requested file in symlink validation

GitOrigin-RevId: 86cfe8d62bb99ed6844faee0ff4af507e571e04d
2024-07-15 09:00:59 +00:00
andrew rumble
ee31d44f03 Explicitly specify latest version of async
GitOrigin-RevId: 9112007de0f5eb4e75604de9f01c2d77b3b8b980
2024-06-17 08:04:00 +00:00
Alf Eaton
e3fcbb4e79 Unpin async dependency
GitOrigin-RevId: 2fd885030ebda5cd4e1e8faad0ba5f57926e38a3
2024-06-17 08:03:51 +00:00
Jakob Ackermann
a486f7e481 Merge pull request #18537 from overleaf/jpa-cleanup-synctex
[clsi] cleanup unused SYNCTEX_BIN_HOST_PATH

GitOrigin-RevId: 062aed9dceb1388fab0e59770aded5a440eaa3bd
2024-05-29 08:03:57 +00:00
Andrew Rumble
80ede301fa Merge pull request #18474 from overleaf/ar-return-build-id-from-clsi-after-compile
[clsi] Return buildId after compiles

GitOrigin-RevId: 872048f4fea8f5a00b817e29bd26a444d179a45f
2024-05-27 10:24:06 +00:00
Andrew Rumble
71187a51ba Merge pull request #18289 from overleaf/ac-ar-eslint-return-await
Add ESLint rule @typescript-eslint/return-await to backend services

GitOrigin-RevId: 75e3e32597827fcc852e69d479515fc72e8f45e4
2024-05-27 10:22:49 +00:00
Christopher Hoskin
3342d672c2 Merge pull request #18397 from overleaf/em-revert-download-all-link
Revert "Merge pull request #18190 from overleaf/ar-add-download-all-l…

GitOrigin-RevId: 681eb2734636d76558e682dc85083bfcaa6b7d2d
2024-05-17 08:05:10 +00:00
Brian Gough
c1a4e4a873 Merge pull request #16775 from overleaf/bg-clsi-timeout-use-fetchutils-connect-timeout
Update UrlFetcher to use custom agent with connect timeout

GitOrigin-RevId: c92dc27a0e682b0b79859b3521bf64cbf8485345
2024-05-16 08:05:18 +00:00
Andrew Rumble
105d67bd04 Merge pull request #18190 from overleaf/ar-add-download-all-link
[web] Add download all link for output files

GitOrigin-RevId: a275f17b6acf3130250d91462ba01588b93b62f5
2024-05-16 08:05:04 +00:00
Eric Mc Sween
e87df4a791 Merge pull request #18167 from overleaf/em-simpler-typescript-backend
Simpler Typescript setup in the backend

GitOrigin-RevId: 3a90f26a5ac8029d14203faa98008375ce933e7d
2024-05-07 08:04:00 +00:00
Andrew Rumble
92f62f91c1 Merge pull request #18148 from overleaf/ar-add-output-zip-endpoint-to-clsi
[clsi] Add endpoints to get zip of output files

GitOrigin-RevId: a1a935e8170ab5a8d40baa6d96f8e42fe22c2e8c
2024-05-02 08:03:44 +00:00
Jakob Ackermann
a540754f6e Merge pull request #18116 from overleaf/jpa-bulk-replace-localhost
[misc] bulk replace localhost with 127.0.0.1

GitOrigin-RevId: d238f3635302e8ff5500d611108c4d1bef216726
2024-04-26 08:04:39 +00:00
roo hutton
2469c93da4 Merge pull request #17870 from overleaf/bg-node-upgrade-18-20-2
upgrade node to 18.20.2

GitOrigin-RevId: e4ac30c740ced3885353f1472cf690eb5ec0ece1
2024-04-24 08:05:16 +00:00
roo hutton
d75024a7a1 Merge pull request #17974 from overleaf/rh-express-4192
Upgrade express to 4.19.2

GitOrigin-RevId: 383877c39670093033d802af0fae54ec578ba327
2024-04-23 08:05:01 +00:00
Jakob Ackermann
9351c03c3b Merge pull request #17874 from overleaf/jpa-ol-project-id-in-env
[clsi] expose Overleaf project id to LaTeX compilation via env var

GitOrigin-RevId: cf8be6f8d83351a587d9ac7940f61a046a14035e
2024-04-12 08:04:56 +00:00
Jakob Ackermann
a6fb983afc Merge pull request #17871 from overleaf/jpa-bind-v4-v6
[misc] align the host for binding and requesting in tests

GitOrigin-RevId: 916521c56928329ff2cbf2817f3a0a27aeaf8c3d
2024-04-12 08:04:51 +00:00
Brian Gough
a9d1ddcb22 Merge pull request #17748 from overleaf/bg-node-security-upgrade-2024-04-05
upgrade node version to 18.20.1

GitOrigin-RevId: 28f8f7c7f63285f7b7355e65713d9ed2dbda8a0e
2024-04-08 08:04:41 +00:00
Jimmy Domagala-Tang
683ea90f64 Merge pull request #17299 from overleaf/jdt-clsi-allow-getrandom
Allow getrandom calls in CLSI

GitOrigin-RevId: 3fcb9239422e25456ef5211248aad4bcd2ec786f
2024-04-05 08:04:09 +00:00
Alf Eaton
6cc2db3cdd Merge pull request #17525 from overleaf/ae-upgrade-prettier
Upgrade Prettier to v3

GitOrigin-RevId: 6f1338f196408f3edb4892d5220ad3665ff1a5bc
2024-03-26 09:04:05 +00:00
Jakob Ackermann
51994579cd Merge pull request #17472 from overleaf/jpa-clsi-debug
[clsi] disable load shedding in dev-env

GitOrigin-RevId: 8ebd1d6d8f45947690d3c4480ccd734e02f1d8be
2024-03-12 09:03:50 +00:00
Jakob Ackermann
39189978ac Merge pull request #17170 from overleaf/jpa-node-18-19-1
[misc] upgrade Node.js to 18.19.1

GitOrigin-RevId: 1bd2008682b585c3fccc65f74c90ba7fc9edc023
2024-02-20 09:04:55 +00:00
Christopher Hoskin
e9fe823128 Merge pull request #17081 from overleaf/csh-build_scripts-redis-acceptance
Include REDIS env vars in acceptance tests only when redis a dependency

GitOrigin-RevId: 42282d33b8beb7999a598bddbcb3d1ecb37ea814
2024-02-14 09:06:09 +00:00
Jakob Ackermann
6d98d6a7c0 Merge pull request #16923 from overleaf/jpa-server-pro-fs-rebranding
[server-pro] remove ShareLaTeX branding from filesystem

GitOrigin-RevId: c1d2c63bdc888d81e03bb1721e44b6ed6f64524b
2024-02-12 09:03:39 +00:00
Jakob Ackermann
63520c7076 Merge pull request #16859 from overleaf/jpa-sharelatex-cleanup
[misc] ShareLaTeX cleanup - high impact

GitOrigin-RevId: 6dcce9b0f15e30f7afcf6d69c3df36a369f38120
2024-02-09 09:04:11 +00:00
ilkin-overleaf
640ea9c916 Merge pull request #16806 from overleaf/ii-compile-dir-readonly
[clsi] Use readOnly bind-mounts for synctex and wordcount containers

GitOrigin-RevId: 8de094350604544dba4571aa849ad41a4a4d88d8
2024-02-05 09:03:58 +00:00