Commit graph

870 commits

Author SHA1 Message Date
Jakob Ackermann
b5e1ceef17 Merge pull request #4865 from overleaf/jpa-ho-load-shed-mvp
[clsi] very basic mvp for shedding load

GitOrigin-RevId: 4b82934b395f8b9358e463c03cadda6a16c20b4f
2021-08-27 08:03:21 +00:00
Jakob Ackermann
5122558469 Merge pull request #4842 from overleaf/jpa-drop-unused-packages
[misc] drop unused debug packages

GitOrigin-RevId: 83b1d96ee5a62c14c4a2dc51cefd179306a75a84
2021-08-27 08:03:17 +00:00
Jakob Ackermann
56a3b0dcde Merge pull request #4819 from overleaf/jpa-pdf-caching-new-event-loop
[clsi] pdf-caching: move cpu intensive work onto a new event loop

GitOrigin-RevId: 4cb5cd4528fa1c5df6a8e91f9caa38cb64d94463
2021-08-25 08:03:38 +00:00
Jakob Ackermann
5d2549fe52 [misc] add/update CI pipelines for monorepo
GitOrigin-RevId: 8946b8be124e6977d2cafc6c3bf065412d765583
2021-08-05 13:48:05 +00:00
Jakob Ackermann
5e6401a1e0 Merge pull request #264 from overleaf/jpa-align-with-dev-env
[misc] make build scripts happy again
2021-07-27 10:46:16 +02:00
Jakob Ackermann
abcc532751 Merge pull request #262 from overleaf/jpa-faster-install
[perf] trim down install_deps.sh -- install docker cli only
2021-07-27 10:46:05 +02:00
Jakob Ackermann
76e749777d [misc] make build scripts happy again
- move pdf.js tests into test/unit/js
- add env override to build script config file
- update build scripts
2021-07-26 12:20:04 +01:00
Brian Gough
262793c04f add option for apparmor profile 2021-07-21 14:53:35 +01:00
Jakob Ackermann
d1c06d1878 [perf] trim down install_deps.sh -- install docker cli only 2021-07-20 14:14:00 +01:00
Jakob Ackermann
bc401ac44e [misc] upgrade node version to latest v12 LTS version 12.22.3 2021-07-13 12:26:36 +01:00
Jakob Ackermann
0e5f5afe79 [misc] temporary override a few new/changed eslint rules 2021-07-13 12:24:43 +01:00
Jakob Ackermann
f285e503b4 [misc] run format_fix and lint:fix 2021-07-13 12:04:48 +01:00
Jakob Ackermann
631873fc83 [misc] upgrade build scripts to version 3.11.0 and cleanup packages
```
npm uninstall prettier-eslint-cli eslint-plugin-standard eslint-plugin-jsx-a11y eslint-plugin-react eslint-config-standard-jsx eslint-config-standard-react babel-eslint
npm dedupe
```
2021-07-13 11:55:19 +01:00
Jakob Ackermann
72a19ad895 [misc] goodbye coffee-script 2021-07-13 10:07:04 +01:00
Jakob Ackermann
4b066f615f [misc] run npm dedupe 2021-07-12 17:51:07 +01:00
Jakob Ackermann
64551f0198 [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:21 +01:00
Jakob Ackermann
ec49c4c9a6 [misc] install bunyan as production dependency
```
Error: Cannot find module 'bunyan'
Require stack:
- .../node_modules/@google-cloud/logging-bunyan/build/src/middleware/express.js
- .../node_modules/@google-cloud/logging-bunyan/build/src/index.js
- .../node_modules/logger-sharelatex/logging-manager.js
- .../node_modules/logger-sharelatex/index.js
- .../app.js
```
2021-07-12 17:35:51 +01:00
Jakob Ackermann
516525126b [UrlFetcher] do not override domain for clsi-perf requests 2021-07-02 09:17:29 +01:00
Jakob Ackermann
efa0c54ca5 [misc] add timings for the sync stage and output stage 2021-06-30 11:45:34 +01:00
Jakob Ackermann
743bfced64 [misc] ContentCacheManager: apply review feedback
- count stages
- lower bound is 1s

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2021-06-23 14:42:39 +01:00
Jakob Ackermann
b09e52510f [misc] bail out from pdf caching processing after 10s or earlier
...for fast compiles.
2021-06-23 14:20:04 +01:00
Jakob Ackermann
c3dba7f74d Merge pull request #253 from overleaf/jpa-debug-slow-pdf-caching
[misc] ContentCacheMetrics: log slow pdf caching performance
2021-06-23 12:39:13 +02:00
Jakob Ackermann
97693b49c2 [ContentCacheMetrics] add new metric for absolute time spent in PDF.js 2021-06-23 11:28:31 +01:00
Jakob Ackermann
11a44ff07c [misc] ContentCacheMetrics: apply review feedback: ignore fast compiles
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2021-06-23 11:27:19 +01:00
Jakob Ackermann
a4389fb761 [misc] ContentCacheMetrics: log slow pdf caching performance 2021-06-23 09:25:16 +01:00
Jakob Ackermann
ffaff1bd72 [CompileController] emit status=failure for an empty output.pdf file 2021-06-22 12:15:12 +01:00
Jakob Ackermann
fb3966ef35 [misc] CompileController: simplify composing of outputFiles 2021-06-22 12:15:12 +01:00
Jakob Ackermann
f32f5e58a8 Merge pull request #248 from overleaf/jpa-add-missing-dep
[misc] add missing dependency entry for send
2021-06-15 10:27:35 +02:00
Jakob Ackermann
a76a9a3ef8 [misc] add missing dependency entry for send 2021-06-15 09:24:23 +01:00
Jakob Ackermann
8caa19afed Merge pull request #247 from overleaf/jpa-track-db-times
[misc] track delays of using sqlite in a few places
2021-06-15 10:11:46 +02:00
Jakob Ackermann
7659f462da Merge pull request #246 from overleaf/dependabot-npm_and_yarn-normalize-url-4.5.1
Bump normalize-url from 4.5.0 to 4.5.1
2021-06-15 10:10:59 +02:00
Jakob Ackermann
f7bfc6bf5f Merge pull request #245 from overleaf/dependabot-npm_and_yarn-glob-parent-5.1.2
Bump glob-parent from 5.1.0 to 5.1.2
2021-06-15 10:10:53 +02:00
Jakob Ackermann
2fbc3d9127 Merge pull request #223 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9
Bump hosted-git-info from 2.8.5 to 2.8.9
2021-06-15 10:10:45 +02:00
Jakob Ackermann
83e5405839 Merge pull request #222 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21
Bump lodash from 4.17.20 to 4.17.21
2021-06-15 10:10:38 +02:00
Jakob Ackermann
8da9a5b7c8 Merge pull request #220 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1
Bump y18n from 4.0.0 to 4.0.1
2021-06-15 10:10:20 +02:00
Jakob Ackermann
878cea8087 Merge pull request #221 from overleaf/jpa-explicit-dependencies
[misc] add linting for missing explicit dependencies and fix any errors
2021-06-15 10:09:43 +02:00
Jakob Ackermann
8cc4517d4c [misc] move import to avoid git conflict 2021-06-15 09:08:32 +01:00
Jakob Ackermann
8f1ea7f0d1 [misc] track delays of using sqlite for url caching details 2021-06-14 13:03:02 +01:00
Jakob Ackermann
d40065fd8e [misc] track delay of using sqlite for last access time of a project 2021-06-14 12:41:16 +01:00
dependabot[bot]
138119a063 Bump normalize-url from 4.5.0 to 4.5.1
Bumps [normalize-url](https://github.com/sindresorhus/normalize-url) from 4.5.0 to 4.5.1.
- [Release notes](https://github.com/sindresorhus/normalize-url/releases)
- [Commits](https://github.com/sindresorhus/normalize-url/commits)

---
updated-dependencies:
- dependency-name: normalize-url
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-12 07:23:12 +00:00
dependabot[bot]
0c06d0daa6 Bump glob-parent from 5.1.0 to 5.1.2
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.0 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.0...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-11 07:58:49 +00:00
Jakob Ackermann
7380b54900 [ProjectPersistenceManager] check all user content dirs for full disk 2021-06-01 16:13:48 +01:00
Jakob Ackermann
1127bf640e [misc] wait for refreshing of expiry timeout before triggering cleanup 2021-06-01 15:51:09 +01:00
Jakob Ackermann
294088fb27 [ContentCacheManager] use PDF.js Xref table instead of stream detection (#242)
* make the content cache manager tests configurable

* extend stream content in unit tests

* [ContentCacheManagerTests] prepare for full object caching

* filesystem stream for pdfjs

* working??

* cleaning up

* handle overflow

* [misc] install pdfjs-dist

* [misc] move pdfjs code into app/lib/ and scripts/, also use CamelCase

* [misc] abstract the file loading and parsing of xRef tables into helper

* [misc] pdfjsTests: add snapshot based tests for the Xref table parser

* [misc] FSStream: throw proper error and drop commented code

* [misc] FSStream: integrate throwing of MissingDataException into getter

* [misc] pdfjs: fix eslint errors

* [misc] pdfjs: run format_fix

* [misc] pdfjs: allocate very small non empty dummy buffers explicitly

* [misc] install @overleaf/o-error

* [ContentCacheManager] use PDF.js Xref table instead of stream detection

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>

* [pdfjs] parseXrefTable: handle empty PDF files gracefully

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
2021-05-31 09:20:25 +01:00
Jakob Ackermann
d56a490121 [misc] AllowedImageNamesTests: add missing done callbacks 2021-05-25 10:24:49 +01:00
dependabot[bot]
4d578bf52d Bump lodash from 4.17.20 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-21 09:35:58 +00:00
Jakob Ackermann
f820d32221 Merge pull request #240 from overleaf/bg-jpa-hash-tracker
[ContentCacheManager] track hashes and expire unused ones
2021-05-21 11:35:05 +02:00
Jakob Ackermann
fdcfbcd6ea Merge pull request #238 from overleaf/jpa-drop-broken-timings
[LatexRunner] do not emit empty cpu/sys timings
2021-05-21 11:34:54 +02:00
Alf Eaton
6507d668ae Update references to .coffee files 2021-05-19 18:57:35 +01:00
Alf Eaton
1b1c4aa4a0 Fix formatting 2021-05-19 18:47:51 +01:00