github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
22238 commits 3 branches 7 tags 206 MiB
Commit graph

1009 commits

Author SHA1 Message Date
Eric Mc Sween
3235119302 Merge pull request #7228 from overleaf/em-node-16 
Upgrade to Node 16

GitOrigin-RevId: 3db1ae57ffb02f8a2b9012ffbb3efecfc01d2b04
 2022-04-05 12:20:52 +00:00
Eric Mc Sween
7119141524 Merge pull request #7304 from overleaf/em-split-queues 
Split the config for web and analytics queues

GitOrigin-RevId: fa3fc86c874e653649aefad030607105d514a52b
 2022-04-05 12:19:32 +00:00
Christopher Schenk
d6d664312c Ignore stderr output when running commands 
Fixes #1006
 2022-04-01 16:03:19 +02:00
Jakob Ackermann
b25ff076b8 Merge pull request #6962 from overleaf/jpa-clsi-tweak-health-check 
[clsi] tweak health check

GitOrigin-RevId: 2caad90d55bb441c7008e3084b34d683b072dfbd
 2022-03-03 09:03:19 +00:00
Jakob Ackermann
a3e85f5357 Merge pull request #6959 from overleaf/jpa-clsi-zero-bucket 
[clsi] add zero bucket and collect histogram data in timing as well

GitOrigin-RevId: 6390f1c213e4850ceb629dae1e6406f6077759aa
 2022-03-03 09:03:16 +00:00
Jakob Ackermann
264b107e39 Merge pull request #6927 from overleaf/jpa-split-clsi-metrics 
[misc] split clsi metrics into multiple categories

GitOrigin-RevId: 964ab57b344b92383a2937beff50139bae273a0e
 2022-03-02 09:02:52 +00:00
Jakob Ackermann
958216d578 Merge pull request #6926 from overleaf/jpa-clsi-bump-logger-metrics 
[clsi] upgrade logger and metrics module

GitOrigin-RevId: 85c346b5eed683672a77d86d6a434dc313b7824b
 2022-03-02 09:02:50 +00:00
Miguel Serrano
e9fe9f5397 Merge pull request #6806 from overleaf/msm-remove-docker-cli 
[clsi] remove `docker-ce-cli`

GitOrigin-RevId: bb49621091aafd43e9e6f157c5fd3b09ca9d07a8
 2022-02-23 09:03:31 +00:00
Simon Detheridge
fe1cb010b4 Merge pull request #5718 from overleaf/spd-built-images 
Support prebuilt images and add overrides for using mapped folders

GitOrigin-RevId: b1a1f7fe61ead1c08f3b8eef35ef90926ca42f36
 2022-02-16 11:31:30 +00:00
Jakob Ackermann
ae0c347f27 Merge pull request #6542 from overleaf/jpa-em-code-sharing-workspaces 
[misc] npm workspaces

GitOrigin-RevId: 87aa72db6637fb238d7cd35b0a48ac3ed58ab3eb
 2022-02-16 11:31:07 +00:00
Jakob Ackermann
2ed696b3e0 Merge pull request #6517 from overleaf/jpa-node-fetch-2-6-7 
[misc] upgrade node-fetch from 2.6.x to 2.6.7

GitOrigin-RevId: 5a43e82e23d78fd6e68a38655dab4665a401a9db
 2022-02-01 09:03:24 +00:00
Jakob Ackermann
8789296f5a Merge pull request #6481 from overleaf/jpa-node-14-18-3 
[misc] upgrade node version from 14.18.1 to 14.18.3

GitOrigin-RevId: 976542fcc0271e323695c59a987dcf1614ee633e
 2022-01-28 09:03:53 +00:00
Jakob Ackermann
409d8cd1a8 Merge pull request #6470 from overleaf/jpa-o-error-3-4 
[misc] upgrade o-error to v3.4.0 in all the places

GitOrigin-RevId: 7b7ce1211cbe5e41754ca2ae112d237befdcf40e
 2022-01-28 09:03:45 +00:00
Eric Mc Sween
f3f0be5c56 Merge pull request #6211 from overleaf/em-code-sharing 
Change directory layout in service containers

GitOrigin-RevId: 09ff19db2a123cbf7691d51e9ce9be6eee264287
 2022-01-07 09:03:22 +00:00
Jakob Ackermann
729cbf3fce Merge pull request #6224 from overleaf/jpa-cache-install-deps 
[misc] add docker image caching for install_deps, pre-fetch node image

GitOrigin-RevId: b52e52351c62b780538e0668b99fc1c28723addc
 2022-01-06 09:03:07 +00:00
Jakob Ackermann
4b308553be Merge pull request #6120 from overleaf/jpa-same-linting-packages 
[misc] move the linting setup to the root of the monorepo

GitOrigin-RevId: 1633e2a58598add0b727738cd3bfba0ab7bae781
 2021-12-17 09:03:06 +00:00
Jakob Ackermann
2261e761f4 Merge pull request #5705 from overleaf/jpa-node-14 
[misc] upgrade node version from 12 to 14

GitOrigin-RevId: 15c465c7bb5e42b0c71e2dedb09cf4fe9f1d8ec6
 2021-11-19 09:02:56 +00:00
Christopher Hoskin
d8f4780679 Merge pull request #5839 from overleaf/csh-issue-5677-clsi-env 
Add CLSI env var to indicate running in CLSI

GitOrigin-RevId: 9d55b001a32ecf7fa792023312c31bd9e6b64646
 2021-11-19 09:02:52 +00:00
Jakob Ackermann
9c9beef305 Merge pull request #5737 from overleaf/jpa-clsi-cleanup-in-memory 
[clsi] separate clearing cache and clearing output cache

GitOrigin-RevId: 9cd903eaf8d291d4821ba8c9ac6c1a9f8d3d9632
 2021-11-16 09:02:36 +00:00
Jakob Ackermann
03b4e02c0f Merge pull request #5381 from overleaf/jpa-sqlite-cleanup 
[misc] cleanup some more references following the removal of sqlite

GitOrigin-RevId: db5db9c0e2ba2557b11d64f65fdb5db1dcd2ec74
 2021-10-28 08:03:54 +00:00
Jakob Ackermann
638fbd5510 Merge pull request #5573 from overleaf/jpa-prefer-regex-literals 
[misc] fix eslint violations for prefer-regex-literals

GitOrigin-RevId: 216288af22b9eeb18cddace7ec285e0a8046f94a
 2021-10-28 08:03:41 +00:00
Jakob Ackermann
f6795a324a Merge pull request #5572 from overleaf/jpa-no-path-concat 
[misc] fix eslint violations for node/no-path-concat

GitOrigin-RevId: 5122826fb9ae23c373e8c5a6802ebb35eb20314f
 2021-10-28 08:03:37 +00:00
Jakob Ackermann
669606f797 Merge pull request #5568 from overleaf/jpa-node-no-callback-literal 
[misc] fix eslint violations for node/no-callback-literal

GitOrigin-RevId: af5aace52e1476b1b0ee48cc8be2aabbe04efac6
 2021-10-28 08:03:33 +00:00
Jakob Ackermann
489df8360e Merge pull request #5523 from overleaf/jpa-no-loss-of-precision 
[misc] fix eslint violations for no-loss-of-precision

GitOrigin-RevId: 06ecbeb5f10569933e99a472a92fd3efbe546278
 2021-10-28 08:03:30 +00:00
Jakob Ackermann
7f9fd00bda Merge pull request #5367 from overleaf/jpa-node-handle-callback-err 
[misc] fix eslint violations for node/handle-callback-err

GitOrigin-RevId: 83a4900e8861010df1917bff49382bd9c93375bd
 2021-10-28 08:03:26 +00:00
Jakob Ackermann
5862359ff0 Merge pull request #5518 from overleaf/jpa-no-empty 
[misc] fix eslint violations for no-empty

GitOrigin-RevId: 8fc1688ac28ccf29b830ba43cd6091e64b5e3b07
 2021-10-27 08:03:14 +00:00
Jakob Ackermann
4cefebec5d Merge pull request #5517 from overleaf/jpa-no-dupe-else-if 
[misc] fix eslint violations for no-dupe-else-if

GitOrigin-RevId: 5f72eb1d033322b2d419c1dbad29ea378ee776f5
 2021-10-27 08:03:09 +00:00
Jakob Ackermann
0f529df1e4 Merge pull request #5360 from overleaf/jpa-array-callback-return 
[misc] fix eslint violations for array-callback-return

GitOrigin-RevId: 4104342239f4da32c541855d6fc7032c4c96004f
 2021-10-27 08:03:05 +00:00
Jakob Ackermann
a6f05109a3 Merge pull request #5352 from overleaf/jpa-no-var 
[misc] fix eslint violations for `no-var`

GitOrigin-RevId: c52e82f3a8a993b8662cc5aa56e7b95ca3c55832
 2021-10-27 08:03:00 +00:00
Brian Gough
43c381e4d5 Merge pull request #5552 from overleaf/bg-node-upgrade-other 
upgrade node to v12.22.7 (other services)

GitOrigin-RevId: 889cf9a77173119fec8f200eeef9ddeefca0720b
 2021-10-26 08:03:13 +00:00
Jakob Ackermann
358e8b7424 Merge pull request #5349 from overleaf/jpa-no-depreacted-api 
[misc] fix eslint violations for node/no-depreacted-api

GitOrigin-RevId: 0f7d64984da9e789c4ab95381db34afb89fa1a94
 2021-10-21 08:03:18 +00:00
Jakob Ackermann
94c208311c Merge pull request #5351 from overleaf/jpa-clsi-drop-sqlite 
[clsi] goodbye sqlite db

GitOrigin-RevId: e7f16920c44e74a425b92884b48a60272dfb015b
 2021-10-07 08:03:50 +00:00
Jakob Ackermann
44d0a8d162 Merge pull request #4650 from overleaf/jpa-in-memory-last-access-time 
[perf] store the last access time of a project in memory

GitOrigin-RevId: 16e2bee28f58eced18f4c3ec5571ea9d10805cbb
 2021-10-07 08:03:21 +00:00
Jakob Ackermann
02918e7483 Merge pull request #4649 from overleaf/jpa-fs-based-caching 
[perf] UrlCache: pure fs based cache state for downloads

GitOrigin-RevId: d19afc396324d4c3318b31620c8ad0c04e0544ce
 2021-10-07 08:03:18 +00:00
Brian Gough
cc1b73336a Merge pull request #4944 from overleaf/bg-use-file-line-errors 
use file line errors in log output

GitOrigin-RevId: 6732b19552fe15431a70fbefbc572253c389c64e
 2021-09-22 08:03:25 +00:00
Brian Gough
dc576f3b6f Merge pull request #4943 from overleaf/bg-increase-max-print-line 
allow setting texlive max_print_line variable

GitOrigin-RevId: 7588fed6aa5868a6ed6b6121cbd6f9c008c2aa0f
 2021-09-22 08:03:21 +00:00
Eric Mc Sween
9ee92daea3 Merge pull request #4893 from overleaf/em-synctex 
Use the synctex distributed with TeX Live

GitOrigin-RevId: 5a133f21f48fd1e217ab463e8cb2a5cdec8be1af
 2021-09-07 16:26:17 +00:00
Eric Mc Sween
2feea2592d Merge pull request #4887 from overleaf/em-decaf 
Decaf cleanup for the CLSI CompileManager

GitOrigin-RevId: 06bba5c8af8808d0fa04187b10c8f31e08cd8754
 2021-08-31 08:03:32 +00:00
Jakob Ackermann
1dc9f428fa Merge pull request #4890 from overleaf/jpa-clsi-maint 
[clsi] put server into maintenance mode when reaching zero capacity

GitOrigin-RevId: d1af260a3bfba7852519b3bc369a6747e79f3b18
 2021-08-27 08:03:39 +00:00
Jakob Ackermann
b5e1ceef17 Merge pull request #4865 from overleaf/jpa-ho-load-shed-mvp 
[clsi] very basic mvp for shedding load

GitOrigin-RevId: 4b82934b395f8b9358e463c03cadda6a16c20b4f
 2021-08-27 08:03:21 +00:00
Jakob Ackermann
5122558469 Merge pull request #4842 from overleaf/jpa-drop-unused-packages 
[misc] drop unused debug packages

GitOrigin-RevId: 83b1d96ee5a62c14c4a2dc51cefd179306a75a84
 2021-08-27 08:03:17 +00:00
Jakob Ackermann
56a3b0dcde Merge pull request #4819 from overleaf/jpa-pdf-caching-new-event-loop 
[clsi] pdf-caching: move cpu intensive work onto a new event loop

GitOrigin-RevId: 4cb5cd4528fa1c5df6a8e91f9caa38cb64d94463
 2021-08-25 08:03:38 +00:00
Jakob Ackermann
5d2549fe52 [misc] add/update CI pipelines for monorepo 
GitOrigin-RevId: 8946b8be124e6977d2cafc6c3bf065412d765583
 2021-08-05 13:48:05 +00:00
Jakob Ackermann
5e6401a1e0 Merge pull request #264 from overleaf/jpa-align-with-dev-env 
[misc] make build scripts happy again
 2021-07-27 10:46:16 +02:00
Jakob Ackermann
abcc532751 Merge pull request #262 from overleaf/jpa-faster-install 
[perf] trim down install_deps.sh -- install docker cli only
 2021-07-27 10:46:05 +02:00
Jakob Ackermann
76e749777d [misc] make build scripts happy again 
- move pdf.js tests into test/unit/js
- add env override to build script config file
- update build scripts
 2021-07-26 12:20:04 +01:00
Brian Gough
262793c04f add option for apparmor profile 2021-07-21 14:53:35 +01:00
Jakob Ackermann
d1c06d1878 [perf] trim down install_deps.sh -- install docker cli only 2021-07-20 14:14:00 +01:00
Jakob Ackermann
bc401ac44e [misc] upgrade node version to latest v12 LTS version 12.22.3 2021-07-13 12:26:36 +01:00
Jakob Ackermann
0e5f5afe79 [misc] temporary override a few new/changed eslint rules 2021-07-13 12:24:43 +01:00
Jakob Ackermann
f285e503b4 [misc] run format_fix and lint:fix 2021-07-13 12:04:48 +01:00
Jakob Ackermann
631873fc83 [misc] upgrade build scripts to version 3.11.0 and cleanup packages 
```
npm uninstall prettier-eslint-cli eslint-plugin-standard eslint-plugin-jsx-a11y eslint-plugin-react eslint-config-standard-jsx eslint-config-standard-react babel-eslint
npm dedupe
```
 2021-07-13 11:55:19 +01:00
Jakob Ackermann
72a19ad895 [misc] goodbye coffee-script 2021-07-13 10:07:04 +01:00
Jakob Ackermann
4b066f615f [misc] run npm dedupe 2021-07-12 17:51:07 +01:00
Jakob Ackermann
64551f0198 [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:21 +01:00
Jakob Ackermann
ec49c4c9a6 [misc] install bunyan as production dependency 
```
Error: Cannot find module 'bunyan'
Require stack:
- .../node_modules/@google-cloud/logging-bunyan/build/src/middleware/express.js
- .../node_modules/@google-cloud/logging-bunyan/build/src/index.js
- .../node_modules/logger-sharelatex/logging-manager.js
- .../node_modules/logger-sharelatex/index.js
- .../app.js
```
 2021-07-12 17:35:51 +01:00
Jakob Ackermann
516525126b [UrlFetcher] do not override domain for clsi-perf requests 2021-07-02 09:17:29 +01:00
Jakob Ackermann
efa0c54ca5 [misc] add timings for the sync stage and output stage 2021-06-30 11:45:34 +01:00
Jakob Ackermann
743bfced64 [misc] ContentCacheManager: apply review feedback 
- count stages
- lower bound is 1s

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
 2021-06-23 14:42:39 +01:00
Jakob Ackermann
b09e52510f [misc] bail out from pdf caching processing after 10s or earlier 
...for fast compiles.
 2021-06-23 14:20:04 +01:00
Jakob Ackermann
c3dba7f74d Merge pull request #253 from overleaf/jpa-debug-slow-pdf-caching 
[misc] ContentCacheMetrics: log slow pdf caching performance
 2021-06-23 12:39:13 +02:00
Jakob Ackermann
97693b49c2 [ContentCacheMetrics] add new metric for absolute time spent in PDF.js 2021-06-23 11:28:31 +01:00
Jakob Ackermann
11a44ff07c [misc] ContentCacheMetrics: apply review feedback: ignore fast compiles 
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
 2021-06-23 11:27:19 +01:00
Jakob Ackermann
a4389fb761 [misc] ContentCacheMetrics: log slow pdf caching performance 2021-06-23 09:25:16 +01:00
Jakob Ackermann
ffaff1bd72 [CompileController] emit status=failure for an empty output.pdf file 2021-06-22 12:15:12 +01:00
Jakob Ackermann
fb3966ef35 [misc] CompileController: simplify composing of outputFiles 2021-06-22 12:15:12 +01:00
Jakob Ackermann
f32f5e58a8 Merge pull request #248 from overleaf/jpa-add-missing-dep 
[misc] add missing dependency entry for send
 2021-06-15 10:27:35 +02:00
Jakob Ackermann
a76a9a3ef8 [misc] add missing dependency entry for send 2021-06-15 09:24:23 +01:00
Jakob Ackermann
8caa19afed Merge pull request #247 from overleaf/jpa-track-db-times 
[misc] track delays of using sqlite in a few places
 2021-06-15 10:11:46 +02:00
Jakob Ackermann
7659f462da Merge pull request #246 from overleaf/dependabot-npm_and_yarn-normalize-url-4.5.1 
Bump normalize-url from 4.5.0 to 4.5.1
 2021-06-15 10:10:59 +02:00
Jakob Ackermann
f7bfc6bf5f Merge pull request #245 from overleaf/dependabot-npm_and_yarn-glob-parent-5.1.2 
Bump glob-parent from 5.1.0 to 5.1.2
 2021-06-15 10:10:53 +02:00
Jakob Ackermann
2fbc3d9127 Merge pull request #223 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
Bump hosted-git-info from 2.8.5 to 2.8.9
 2021-06-15 10:10:45 +02:00
Jakob Ackermann
83e5405839 Merge pull request #222 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
Bump lodash from 4.17.20 to 4.17.21
 2021-06-15 10:10:38 +02:00
Jakob Ackermann
8da9a5b7c8 Merge pull request #220 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1 
Bump y18n from 4.0.0 to 4.0.1
 2021-06-15 10:10:20 +02:00
Jakob Ackermann
878cea8087 Merge pull request #221 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-06-15 10:09:43 +02:00
Jakob Ackermann
8cc4517d4c [misc] move import to avoid git conflict 2021-06-15 09:08:32 +01:00
Jakob Ackermann
8f1ea7f0d1 [misc] track delays of using sqlite for url caching details 2021-06-14 13:03:02 +01:00
Jakob Ackermann
d40065fd8e [misc] track delay of using sqlite for last access time of a project 2021-06-14 12:41:16 +01:00
dependabot[bot]
138119a063 Bump normalize-url from 4.5.0 to 4.5.1 
Bumps [normalize-url](https://github.com/sindresorhus/normalize-url) from 4.5.0 to 4.5.1.
- [Release notes](https://github.com/sindresorhus/normalize-url/releases)
- [Commits](https://github.com/sindresorhus/normalize-url/commits)

---
updated-dependencies:
- dependency-name: normalize-url
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-12 07:23:12 +00:00
dependabot[bot]
0c06d0daa6 Bump glob-parent from 5.1.0 to 5.1.2 
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.0 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.0...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-11 07:58:49 +00:00
Jakob Ackermann
7380b54900 [ProjectPersistenceManager] check all user content dirs for full disk 2021-06-01 16:13:48 +01:00
Jakob Ackermann
1127bf640e [misc] wait for refreshing of expiry timeout before triggering cleanup 2021-06-01 15:51:09 +01:00
Jakob Ackermann
294088fb27 [ContentCacheManager] use PDF.js Xref table instead of stream detection (#242) 
* make the content cache manager tests configurable

* extend stream content in unit tests

* [ContentCacheManagerTests] prepare for full object caching

* filesystem stream for pdfjs

* working??

* cleaning up

* handle overflow

* [misc] install pdfjs-dist

* [misc] move pdfjs code into app/lib/ and scripts/, also use CamelCase

* [misc] abstract the file loading and parsing of xRef tables into helper

* [misc] pdfjsTests: add snapshot based tests for the Xref table parser

* [misc] FSStream: throw proper error and drop commented code

* [misc] FSStream: integrate throwing of MissingDataException into getter

* [misc] pdfjs: fix eslint errors

* [misc] pdfjs: run format_fix

* [misc] pdfjs: allocate very small non empty dummy buffers explicitly

* [misc] install @overleaf/o-error

* [ContentCacheManager] use PDF.js Xref table instead of stream detection

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>

* [pdfjs] parseXrefTable: handle empty PDF files gracefully

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
 2021-05-31 09:20:25 +01:00
Jakob Ackermann
d56a490121 [misc] AllowedImageNamesTests: add missing done callbacks 2021-05-25 10:24:49 +01:00
dependabot[bot]
4d578bf52d Bump lodash from 4.17.20 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-21 09:35:58 +00:00
Jakob Ackermann
f820d32221 Merge pull request #240 from overleaf/bg-jpa-hash-tracker 
[ContentCacheManager] track hashes and expire unused ones
 2021-05-21 11:35:05 +02:00
Jakob Ackermann
fdcfbcd6ea Merge pull request #238 from overleaf/jpa-drop-broken-timings 
[LatexRunner] do not emit empty cpu/sys timings
 2021-05-21 11:34:54 +02:00
Alf Eaton
6507d668ae Update references to .coffee files 2021-05-19 18:57:35 +01:00
Alf Eaton
1b1c4aa4a0 Fix formatting 2021-05-19 18:47:51 +01:00
Alf Eaton
f31abe7cb3 Tidy and format 2021-05-19 18:36:27 +01:00
decaffeinate
85920b3120 decaffeinate: Run post-processing cleanups on settings.test.coffee 2021-05-19 16:50:46 +01:00
decaffeinate
8c984a77c4 decaffeinate: Convert settings.test.coffee to JS 2021-05-19 16:50:45 +01:00
decaffeinate
5af8f57dd0 decaffeinate: Rename settings.test.coffee from .coffee to .js 2021-05-19 16:50:45 +01:00
Jakob Ackermann
b53a105681 [LatexRunner] do not emit empty cpu/sys timings 2021-05-19 11:17:08 +01:00
Jakob Ackermann
f361cdfaca [ContentCacheManager] deeply integrate the HashFileTracker 
- Update the tracker contents as we hash ranges
- Let the tracker be authoritative for already written ranges
  This is saving a stat call per newly written range.
 2021-05-18 18:15:58 +01:00
Jakob Ackermann
7aeeb5a5a9 [ContentCacheManager] finish tracking of ranges across builds 2021-05-18 18:06:15 +01:00
Jakob Ackermann
011a228727 [misc] install p-limit and nodemon 2021-05-18 17:32:21 +01:00
Brian Gough
d493238eaf wip expire old hash files 2021-05-18 16:25:24 +01:00
Jakob Ackermann
567d02881d [misc] fix unit tests following the merge of atomic writes 2021-05-18 11:09:30 +01:00
Jakob Ackermann
23dd93ae50 Merge pull request #236 from overleaf/jpa-skip-duplicate-writes 
[ContentCacheManager] skip writing of duplicate streams
 2021-05-18 11:44:23 +02:00
Jakob Ackermann
e787106eed Merge pull request #235 from overleaf/jpa-atomic-writes 
[ContentCacheManager] write streams to disk atomically
 2021-05-18 11:44:16 +02:00
Jakob Ackermann
9b8763aed4 Merge pull request #234 from overleaf/jpa-stream-detection-across-chunks 
[ContentCacheManager] add support for stream detection across chunks
 2021-05-18 11:44:09 +02:00
Jakob Ackermann
74e45c60f4 Merge pull request #233 from overleaf/jpa-drop-sync-syscall 
[perf] drop useless synchronous syscall on hot path for writing docs
 2021-05-18 11:44:00 +02:00
Jakob Ackermann
8a3ec2bfda Merge pull request #232 from overleaf/bg-upgrade-fs-extra 
upgrade fs-extra
 2021-05-18 11:43:50 +02:00
Jakob Ackermann
7cf019ab78 Merge pull request #231 from overleaf/bg-use-fse-copy 
use fs.copyFile for performance
 2021-05-18 11:43:43 +02:00
Jakob Ackermann
b254e4285b Merge pull request #229 from overleaf/bg-log-expiry-timeout 
log the expiry timeout value when disk space is low
 2021-05-18 11:43:15 +02:00
Jakob Ackermann
59f18b667c Merge pull request #228 from overleaf/bg-add-content-id-validation 
add validation for express content routes
 2021-05-18 11:43:08 +02:00
Jakob Ackermann
bc1ed82c6c [ContentCacheManager] skip writing of duplicate streams 2021-05-18 10:22:05 +01:00
Jakob Ackermann
224ae0c254 [ContentCacheManager] add support for stream detection across chunks 
Retain a small part (6 or 9 bytes) of each chunk in memory for providing
 the next iteration with enough context for finding the start/end marker
 of a range.
 2021-05-17 17:36:05 +01:00
Jakob Ackermann
6b9c8bced6 [ContentCacheManager] write streams to disk atomically 
Use an intermediate file for writing to disk, then rename to the target.
 2021-05-17 14:22:37 +01:00
Brian Gough
8c80ddd27a remove unnecessary console.log from hash benchmark 2021-05-17 14:14:00 +01:00
Brian Gough
939ad3af31 include UV_THREADPOOL_SIZE in benchmark logs 2021-05-17 14:14:00 +01:00
Brian Gough
cbeba4af00 run the hash benchmark 10 times 2021-05-17 14:14:00 +01:00
Brian Gough
7b6434ef81 add benchmark script for hashing 2021-05-17 14:14:00 +01:00
Brian Gough
371de76a4a use fs.copyFile instead of fse.copy in UrlCache module 2021-05-17 10:54:11 +01:00
Jakob Ackermann
18e943742e [perf] drop useless synchronous syscall on hot path for writing docs 2021-05-17 09:25:29 +01:00
Brian Gough
24ef5786a0 upgrade fs-extra 2021-05-14 15:58:11 +01:00
Brian Gough
44e0742aa3 use fse.copy for performance 
it uses the native fs.copyFile method
 2021-05-14 15:49:20 +01:00
Brian Gough
ff2175e727 add validation for express :content_id parameter 2021-05-13 15:48:17 +01:00
Brian Gough
013d38552a log the expiry timeout value when disk is low 2021-05-13 15:13:29 +01:00
Jakob Ackermann
b456ea726d [misc] merge pdf caching into main (#226) 
* wip generate directory for hash content

* cleanup, remove console logging

* add content caching module

* Return PDF stream ranges with compile response

* Return the PDF file size in the compile response

* PDF range endpoint

* [misc] WIP: pdf caching: preserve the m-time on static content files

* [misc] WIP: pdf caching: improve browser caching, emit caching headers

* [misc] WIP: pdf caching: do not emit very small chunks <1kB

* [misc] keep up with moving output files into a separate directory

* [OutputCacheManager] add global feature flag for enabling pdf caching

* [misc] add contentId into the URL for protecting PDF stream contents

* [misc] support PDF stream caching for anonymous users

* [misc] add per-request feature flag for enabling PDF stream caching

* [misc] enable pdf caching in CI and emit metrics at the end of run

* [misc] expose compile stats and timings to the frontend

* [misc] log an error in case saving output files fails

* [misc] add metrics for pdf bandwidth and pdf caching performance

* [misc] add a dark mode to the pdf caching for computing ranges only

* [misc] move pdf caching metrics into ContentCacheMetrics

* [misc] add a config option for the min chunk size of pdf ranges

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
 2021-05-13 14:07:54 +01:00
dependabot[bot]
e66ff7a2ca Bump hosted-git-info from 2.8.5 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 14:32:37 +00:00
Jakob Ackermann
cfa8127c1b [misc] add linting for missing explicit dependencies and fix any errors 2021-04-29 15:30:54 +01:00
dependabot[bot]
ab57729f98 Bump y18n from 4.0.0 to 4.0.1 
Bumps [y18n](https://github.com/yargs/y18n) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/yargs/y18n/releases)
- [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yargs/y18n/commits)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-03-31 17:52:26 +00:00
Jakob Ackermann
b3ed820444 Merge pull request #219 from overleaf/jpa-read-image-name-for-synctex 
[misc] consume and validate a custom imageName for synctex requests
 2021-03-31 10:33:53 +02:00
Jakob Ackermann
c1fe1cf5b7 Merge pull request #218 from overleaf/jpa-metrics-module-3-5-1 
[misc] bump the version of the metrics module to 3.5.1
 2021-03-31 10:33:41 +02:00
Jakob Ackermann
1a41fa55ad Merge pull request #217 from overleaf/sk-qpdf-newline 
Add flag to qpdf, to preserve PDF/A compliance
 2021-03-31 10:33:16 +02:00
Jakob Ackermann
6cb4246eb1 [misc] consume and validate a custom imageName for synctex requests 2021-03-30 13:57:40 +01:00
Alf Eaton
a271d5ce10 Merge pull request #212 from overleaf/ae-chown-app-folders 
Ensure that app folders exist before running chown in entrypoint.sh
 2021-03-23 09:43:31 +00:00
Alf Eaton
db0cd53ed4 Merge pull request #213 from overleaf/ae-readme 
Update README
 2021-03-23 09:41:44 +00:00
Jakob Ackermann
b563deef14 [misc] bump the version of the metrics module to 3.5.1 2021-03-22 12:59:43 +00:00
Eric Mc Sween
7a6b86185b Merge pull request #215 from overleaf/ho-clsi-cookie-post 
support post and get for getting status of clsi project
 2021-03-22 07:56:09 -04:00
Christopher Hoskin
6ca016bb05 Explain the situation with permissions on Linux 2021-03-19 14:32:54 +00:00
Alf Eaton
268a144e43 Merge remote-tracking branch 'origin/master' into ae-readme 2021-03-19 12:06:03 +00:00
Alf Eaton
7e3edcee2c Add instructions for Linux 2021-03-19 12:05:22 +00:00
Alf Eaton
d4b4cb399a Revert "Make TEXLIVE_IMAGE_USER instruction macOS only" 
This reverts commit ab6fe228cadd3329885fdc60aea867d601579759.
 2021-03-19 12:00:47 +00:00
Shane Kilkelly
072d0a9dab Add flag to qpdf, to preserve PDF/A compliance 2021-03-19 11:35:59 +00:00
Christopher Hoskin
34b572de9e Merge branch 'ae-readme' of github.com:overleaf/clsi into ae-readme 2021-03-19 10:35:25 +00:00
Christopher Hoskin
638e72295b Add link to \openout primitive docs 2021-03-19 10:34:52 +00:00
Alf Eaton
984766a9fb Make TEXLIVE_IMAGE_USER instruction macOS only 2021-03-18 09:55:31 +00:00
Christopher Hoskin
b8d0389a53 README typos 2021-03-16 15:29:53 +00:00
Christopher Hoskin
0a89b6537c Fill in missing text for environment variables 2021-03-16 12:30:06 +00:00
Christopher Hoskin
17a83bc169 Expand list of environment variables 2021-03-16 12:00:48 +00:00
Henry Oswald
7cd36e139a support post and get for getting status of clsi project 2021-03-16 10:24:59 +00:00
Eric Mc Sween
3eca505319 Upgrade to Node 12 2021-03-12 16:27:33 -05:00
Eric Mc Sween
9f2d219102 Add a global test setup file 
Set up generally useful stuff:

* chai.should()
* logger stubs
* globals in SandboxedModule, including Buffer and process, which are
  now required in Node 12
 2021-03-12 16:27:33 -05:00
Alf Eaton
f38cef6e56 Change settings file .coffee to .js 2021-03-11 11:50:47 +00:00
Alf Eaton
9c596fb17d Update README.md 2021-03-10 22:38:34 +00:00
Alf Eaton
e7be75a4b7 Update README.md 2021-03-10 22:36:33 +00:00
Alf Eaton
18dbb12434 Add /app/output 2021-02-18 15:33:16 +00:00