github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
7143 commits 3 branches 7 tags 206 MiB
Commit graph

5201 commits

Author SHA1 Message Date
Jakob Ackermann
efa0c54ca5 [misc] add timings for the sync stage and output stage 2021-06-30 11:45:34 +01:00
Simon Detheridge
4c11b69167 Revert "Merge pull request #107 from overleaf/sk-upgrade-jetty-2106" 
This reverts commit f13d8a990a927f5983442ffce00ff009cdc0e8ab, reversing
changes made to cb3ba256b7b27e424616d7bc67b73385fef55be8.
 2021-06-28 17:01:43 +01:00
Simon Detheridge
a239b70bfc Revert change to make Jetty idle timeout configurable 2021-06-28 17:00:52 +01:00
Shane Kilkelly
899e781b33 Set sqlite soft_heap_limit via pragma 2021-06-25 12:03:57 +01:00
Shane Kilkelly
048571a3f6 Add sqlite to the base image 2021-06-25 11:06:03 +01:00
Simon Detheridge
9b8b13877a Make idleTimeout an integer in the config template 2021-06-24 20:49:33 +01:00
Eric Mc Sween
6a5892f996 Fix typo in config 2021-06-24 14:56:31 -04:00
Simon Detheridge
d816a0f02d Merge pull request #121 from overleaf/spd-idletimeout 
Add support for configuring idle timeout and bind IP via settings
 2021-06-24 17:43:17 +01:00
Simon Detheridge
d12b564ec3 Use ServerConnector to specify bind IP and idle timeout 2021-06-24 17:20:43 +01:00
Simon Detheridge
cafa94fcba Add support for bindIp and idleTimeout to config object 2021-06-24 17:00:04 +01:00
Simon Detheridge
28d98f1a25 Add settings for idle timeout and bind IP to config template 2021-06-24 16:59:11 +01:00
Simon Detheridge
928e6fed42 Use timestamp as version number for compiler 2021-06-24 16:12:31 +01:00
dependabot[bot]
9c2fd4b331 Bump commons-io from 2.6 to 2.10.0 
Bumps commons-io from 2.6 to 2.10.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-24 09:05:16 +00:00
dependabot[bot]
81e095283b Bump guava from 24.0-jre to 30.1.1-jre 
Bumps [guava](https://github.com/google/guava) from 24.0-jre to 30.1.1-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-24 09:04:45 +00:00
dependabot[bot]
0beee9be53 Bump commons-compress from 1.15 to 1.20 
Bumps commons-compress from 1.15 to 1.20.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-compress
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-24 09:04:42 +00:00
dependabot[bot]
88b46a43dd Bump junit from 4.12 to 4.13.2 
Bumps [junit](https://github.com/junit-team/junit4) from 4.12 to 4.13.2.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.12.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.12...r4.13.2)

---
updated-dependencies:
- dependency-name: junit:junit
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-24 09:04:32 +00:00
Shane Kilkelly
8d5fa9f1bc Add a config file for dependabot 
This allows us to set the separator used by dependabot
when setting the branch name. Otherwise, we end up with
branches that break our cloud build workflows
 2021-06-24 08:48:06 +01:00
Simon Detheridge
72233ec334 Add parameters to enable debug agent to startup script 2021-06-23 15:05:21 +01:00
Simon Detheridge
9c8ff2f022 Add cloud debug agent to Docker image 2021-06-23 14:58:49 +01:00
Simon Detheridge
a469bc6732 Add parameters to enable profile agent to startup script 2021-06-23 14:58:25 +01:00
Jakob Ackermann
743bfced64 [misc] ContentCacheManager: apply review feedback 
- count stages
- lower bound is 1s

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
 2021-06-23 14:42:39 +01:00
Simon Detheridge
c46f460527 Add cloud profiler to Docker image 2021-06-23 14:22:04 +01:00
Jakob Ackermann
b09e52510f [misc] bail out from pdf caching processing after 10s or earlier 
...for fast compiles.
 2021-06-23 14:20:04 +01:00
Simon Detheridge
ad8fafc5b4 Add http endpoint to print out native memory diagnostics 2021-06-23 12:43:30 +01:00
Simon Detheridge
f51b8139e9 Allow overriding JVM args with an environment variable 2021-06-23 12:43:00 +01:00
Jakob Ackermann
c3dba7f74d Merge pull request #253 from overleaf/jpa-debug-slow-pdf-caching 
[misc] ContentCacheMetrics: log slow pdf caching performance
 2021-06-23 12:39:13 +02:00
Jakob Ackermann
97693b49c2 [ContentCacheMetrics] add new metric for absolute time spent in PDF.js 2021-06-23 11:28:31 +01:00
Jakob Ackermann
11a44ff07c [misc] ContentCacheMetrics: apply review feedback: ignore fast compiles 
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
 2021-06-23 11:27:19 +01:00
Jakob Ackermann
a4389fb761 [misc] ContentCacheMetrics: log slow pdf caching performance 2021-06-23 09:25:16 +01:00
Simon Detheridge
1eb22a7d6d Export prometheus metrics directly from handler 2021-06-22 16:40:18 +01:00
Simon Detheridge
63ca055637 Use ServletHolder lifecycle more carefully 
Fixes metrics with Jetty > 9.4.20
 2021-06-22 15:41:24 +01:00
Simon Detheridge
70f4e5801d Merge pull request #107 from overleaf/sk-upgrade-jetty-2106 
Upgrade jetty and jetty-servlet
 2021-06-22 13:15:03 +01:00
Simon Detheridge
eb9cf6f256 Export prometheus metrics 2021-06-22 13:06:40 +01:00
Jakob Ackermann
ffaff1bd72 [CompileController] emit status=failure for an empty output.pdf file 2021-06-22 12:15:12 +01:00
Jakob Ackermann
fb3966ef35 [misc] CompileController: simplify composing of outputFiles 2021-06-22 12:15:12 +01:00
Shane Kilkelly
c29795f043 Upgrade jetty-servlet to 9.4.38.v20210224 2021-06-22 11:33:14 +01:00
dependabot[bot]
c6897d3b42 Bump jetty-server from 9.4.8.v20171121 to 9.4.38.v20210224 
Bumps [jetty-server](https://github.com/eclipse/jetty.project) from 9.4.8.v20171121 to 9.4.38.v20210224.
- [Release notes](https://github.com/eclipse/jetty.project/releases)
- [Commits](https://github.com/eclipse/jetty.project/compare/jetty-9.4.8.v20171121...jetty-9.4.38.v20210224)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-22 10:36:28 +01:00
Simon Detheridge
7a934b8eba Update mockito 2021-06-21 11:22:44 +01:00
Simon Detheridge
6009393013 Add some core tools for debugging processes 2021-06-21 11:21:58 +01:00
Simon Detheridge
bce5af615f Add gc logging to the Git Bridge 2021-06-21 11:21:58 +01:00
Simon Detheridge
9ef4d1ee5c Update Java to 11 2021-06-21 11:21:53 +01:00
Shane Kilkelly
594009aeab Reduce memory limit to 50 percent 2021-06-17 14:24:07 +01:00
Shane Kilkelly
ed3507977b Switch to UseContainerSupport and MaxRAMPercentage 2021-06-16 17:23:38 +01:00
Shane Kilkelly
8a68c3aef0 Merge pull request #102 from overleaf/sk-add-sqlite-prod 
Add sqlite3 package to the production image
 2021-06-16 10:36:33 +01:00
Shane Kilkelly
cbad8bcfb6 Add sqlite3 package to the production image 2021-06-16 09:46:53 +01:00
Shane Kilkelly
d824cb8451 Add heap flags to production entry script 2021-06-15 14:48:45 +01:00
Jakob Ackermann
f32f5e58a8 Merge pull request #248 from overleaf/jpa-add-missing-dep 
[misc] add missing dependency entry for send
 2021-06-15 10:27:35 +02:00
Jakob Ackermann
a76a9a3ef8 [misc] add missing dependency entry for send 2021-06-15 09:24:23 +01:00
Jakob Ackermann
8caa19afed Merge pull request #247 from overleaf/jpa-track-db-times 
[misc] track delays of using sqlite in a few places
 2021-06-15 10:11:46 +02:00
Jakob Ackermann
7659f462da Merge pull request #246 from overleaf/dependabot-npm_and_yarn-normalize-url-4.5.1 
Bump normalize-url from 4.5.0 to 4.5.1
 2021-06-15 10:10:59 +02:00
Jakob Ackermann
f7bfc6bf5f Merge pull request #245 from overleaf/dependabot-npm_and_yarn-glob-parent-5.1.2 
Bump glob-parent from 5.1.0 to 5.1.2
 2021-06-15 10:10:53 +02:00
Jakob Ackermann
2fbc3d9127 Merge pull request #223 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
Bump hosted-git-info from 2.8.5 to 2.8.9
 2021-06-15 10:10:45 +02:00
Jakob Ackermann
83e5405839 Merge pull request #222 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
Bump lodash from 4.17.20 to 4.17.21
 2021-06-15 10:10:38 +02:00
Jakob Ackermann
8da9a5b7c8 Merge pull request #220 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1 
Bump y18n from 4.0.0 to 4.0.1
 2021-06-15 10:10:20 +02:00
Jakob Ackermann
878cea8087 Merge pull request #221 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-06-15 10:09:43 +02:00
Jakob Ackermann
8cc4517d4c [misc] move import to avoid git conflict 2021-06-15 09:08:32 +01:00
Jakob Ackermann
8f1ea7f0d1 [misc] track delays of using sqlite for url caching details 2021-06-14 13:03:02 +01:00
Jakob Ackermann
d40065fd8e [misc] track delay of using sqlite for last access time of a project 2021-06-14 12:41:16 +01:00
dependabot[bot]
138119a063 Bump normalize-url from 4.5.0 to 4.5.1 
Bumps [normalize-url](https://github.com/sindresorhus/normalize-url) from 4.5.0 to 4.5.1.
- [Release notes](https://github.com/sindresorhus/normalize-url/releases)
- [Commits](https://github.com/sindresorhus/normalize-url/commits)

---
updated-dependencies:
- dependency-name: normalize-url
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-12 07:23:12 +00:00
dependabot[bot]
1160e9e98f Bump glob-parent from 5.1.1 to 5.1.2 
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.1 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.1...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-12 00:00:31 +00:00
dependabot[bot]
0090748379 Bump glob-parent from 5.1.1 to 5.1.2 
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.1 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.1...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-11 14:36:06 +00:00
dependabot[bot]
0c06d0daa6 Bump glob-parent from 5.1.0 to 5.1.2 
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.0 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.0...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-11 07:58:49 +00:00
dependabot[bot]
302576b386 Bump glob-parent from 5.1.1 to 5.1.2 
Bumps [glob-parent](https://github.com/gulpjs/glob-parent) from 5.1.1 to 5.1.2.
- [Release notes](https://github.com/gulpjs/glob-parent/releases)
- [Changelog](https://github.com/gulpjs/glob-parent/blob/main/CHANGELOG.md)
- [Commits](https://github.com/gulpjs/glob-parent/compare/v5.1.1...v5.1.2)

---
updated-dependencies:
- dependency-name: glob-parent
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-06-10 09:11:02 +00:00
Jakob Ackermann
d69c29e4dc [misc] apply review feedback on naming functions and parameters 
Co-Authored-By: Simon Detheridge <s@sd.ai>
 2021-06-08 19:29:10 +01:00
Jakob Ackermann
4996f8bbcd [DocArchiveManager] (un-)archive docs in batches and let db filter docs 
Also drop the broken 404 logic after switching to db-side filtering.
 2021-06-08 12:59:13 +01:00
Jakob Ackermann
7380b54900 [ProjectPersistenceManager] check all user content dirs for full disk 2021-06-01 16:13:48 +01:00
Jakob Ackermann
1127bf640e [misc] wait for refreshing of expiry timeout before triggering cleanup 2021-06-01 15:51:09 +01:00
Jakob Ackermann
294088fb27 [ContentCacheManager] use PDF.js Xref table instead of stream detection (#242) 
* make the content cache manager tests configurable

* extend stream content in unit tests

* [ContentCacheManagerTests] prepare for full object caching

* filesystem stream for pdfjs

* working??

* cleaning up

* handle overflow

* [misc] install pdfjs-dist

* [misc] move pdfjs code into app/lib/ and scripts/, also use CamelCase

* [misc] abstract the file loading and parsing of xRef tables into helper

* [misc] pdfjsTests: add snapshot based tests for the Xref table parser

* [misc] FSStream: throw proper error and drop commented code

* [misc] FSStream: integrate throwing of MissingDataException into getter

* [misc] pdfjs: fix eslint errors

* [misc] pdfjs: run format_fix

* [misc] pdfjs: allocate very small non empty dummy buffers explicitly

* [misc] install @overleaf/o-error

* [ContentCacheManager] use PDF.js Xref table instead of stream detection

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>

* [pdfjs] parseXrefTable: handle empty PDF files gracefully

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
 2021-05-31 09:20:25 +01:00
Alf Eaton
2764b9c606 Merge pull request #112 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
build(deps): bump hosted-git-info from 2.8.5 to 2.8.9
 2021-05-28 10:25:11 +01:00
Alf Eaton
3aeaafaee4 Merge pull request #111 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
build(deps): bump lodash from 4.17.20 to 4.17.21
 2021-05-28 10:25:01 +01:00
Alf Eaton
4f57b6ee37 Merge pull request #110 from overleaf/dependabot-npm_and_yarn-underscore-1.13.1 
build(deps): bump underscore from 1.9.2 to 1.13.1
 2021-05-28 10:24:30 +01:00
Alf Eaton
93b4bc8a4a Merge pull request #108 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-05-28 10:24:14 +01:00
Alf Eaton
0ea35ab52b Merge pull request #107 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1 
build(deps): bump y18n from 4.0.0 to 4.0.1
 2021-05-28 10:23:30 +01:00
nate stemen
cee6c0aad9 Merge pull request #105 from overleaf/dependabot-npm_and_yarn-underscore-1.13.1 
Bump underscore from 1.10.2 to 1.13.1
 2021-05-27 11:53:50 -04:00
nate stemen
081201f796 Merge pull request #103 from overleaf/jpa-retry-archive-deletion 
[DocArchiveManager] retry the deletion of doc archives three times
 2021-05-27 11:45:53 -04:00
nate stemen
3bbc9156cd Merge pull request #108 from overleaf/ae-decaffeinate-config 
Finish decaffeinating settings.default.js
 2021-05-27 11:45:13 -04:00
nate stemen
64e201283b Merge pull request #107 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
Bump hosted-git-info from 2.8.5 to 2.8.9
 2021-05-27 11:45:00 -04:00
nate stemen
fcf0e026c3 Merge pull request #106 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
Bump lodash from 4.17.19 to 4.17.21
 2021-05-27 11:44:52 -04:00
dependabot[bot]
91c19e1f88 Bump underscore from 1.10.2 to 1.13.1 
Bumps [underscore](https://github.com/jashkenas/underscore) from 1.10.2 to 1.13.1.
- [Release notes](https://github.com/jashkenas/underscore/releases)
- [Commits](https://github.com/jashkenas/underscore/compare/1.10.2...1.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-27 15:43:37 +00:00
nate stemen
d0d9cec7e2 Merge pull request #102 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-05-27 11:43:04 -04:00
nate stemen
20f70b120a Merge pull request #97 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1 
Bump y18n from 4.0.0 to 4.0.1
 2021-05-27 11:42:43 -04:00
Alf Eaton
f716134b53 Delete CoffeeScript scripts 2021-05-26 15:56:08 +01:00
Alf Eaton
de3e7a82c3 Delete prepopulate.coffee 2021-05-26 13:21:14 +01:00
Alf Eaton
674ad4acf0 Rename folder from coffee to js 2021-05-25 16:57:24 +01:00
Alf Eaton
5961e85b10 Fix path in test script 2021-05-25 16:56:57 +01:00
Alf Eaton
0649773c15 Manual tidying 2021-05-25 15:54:50 +01:00
Alf Eaton
b5b2bb3d22 Run format:fix 2021-05-25 15:53:36 +01:00
Alf Eaton
24fd9449f2 Run format:fix 2021-05-25 15:52:19 +01:00
Alf Eaton
358786fccd Run format:fix 2021-05-25 15:45:42 +01:00
Alf Eaton
f4db3d8f45 Manual tidying 2021-05-25 14:32:03 +01:00
Alf Eaton
de50e59557 Run format:fix 2021-05-25 14:27:45 +01:00
Jakob Ackermann
d56a490121 [misc] AllowedImageNamesTests: add missing done callbacks 2021-05-25 10:24:49 +01:00
dependabot[bot]
4d578bf52d Bump lodash from 4.17.20 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-21 09:35:58 +00:00
Jakob Ackermann
f820d32221 Merge pull request #240 from overleaf/bg-jpa-hash-tracker 
[ContentCacheManager] track hashes and expire unused ones
 2021-05-21 11:35:05 +02:00
Jakob Ackermann
fdcfbcd6ea Merge pull request #238 from overleaf/jpa-drop-broken-timings 
[LatexRunner] do not emit empty cpu/sys timings
 2021-05-21 11:34:54 +02:00
Shane Kilkelly
82ab89fab3 Remove exception from signature 2021-05-21 09:57:28 +01:00
Shane Kilkelly
2efc7f22a7 Remove old username/password settings 
These were once used to secure interactions with the web api, back when
we allowed anonymous access to git repositories. This feature was
dropped in the migration to Overleaf v2, and we use OAuth on those
interactions anyway, so these settings are not actually used for
anything, and keeping these around is just confusing.
 2021-05-20 15:59:26 +01:00
Shane Kilkelly
c043988027 Remove default user/pass in config template 2021-05-20 10:05:27 +01:00
Shane Kilkelly
ae08604417 Fix variable name in config template 2021-05-20 09:50:05 +01:00
decaffeinate
03cbad4e9f decaffeinate: Run post-processing cleanups on expire_docops.coffee and 2 other files 2021-05-19 20:25:07 +01:00
decaffeinate
5ad0b6eea2 decaffeinate: Convert expire_docops.coffee and 2 other files to JS 2021-05-19 20:25:07 +01:00
decaffeinate
e1e0d26b1f decaffeinate: Rename expire_docops.coffee and 2 other files from .coffee to .js 2021-05-19 20:25:06 +01:00
Alf Eaton
edacebb53c Remove unused parameter 2021-05-19 20:25:04 +01:00
Alf Eaton
acbf8b5898 Tidy and format 2021-05-19 20:22:02 +01:00
Alf Eaton
6507d668ae Update references to .coffee files 2021-05-19 18:57:35 +01:00
Alf Eaton
1b1c4aa4a0 Fix formatting 2021-05-19 18:47:51 +01:00
Alf Eaton
f31abe7cb3 Tidy and format 2021-05-19 18:36:27 +01:00
decaffeinate
0b62828041 decaffeinate: Run post-processing cleanups on settings.defaults.coffee 2021-05-19 17:24:23 +01:00
decaffeinate
3acf183240 decaffeinate: Convert settings.defaults.coffee to JS 2021-05-19 17:24:23 +01:00
decaffeinate
8786542ad6 decaffeinate: Rename settings.defaults.coffee from .coffee to .js 2021-05-19 17:24:22 +01:00
decaffeinate
85920b3120 decaffeinate: Run post-processing cleanups on settings.test.coffee 2021-05-19 16:50:46 +01:00
decaffeinate
8c984a77c4 decaffeinate: Convert settings.test.coffee to JS 2021-05-19 16:50:45 +01:00
decaffeinate
5af8f57dd0 decaffeinate: Rename settings.test.coffee from .coffee to .js 2021-05-19 16:50:45 +01:00
Brian Gough
219d6832df upgrade to socket.io 0.9.19-overleaf-5 
fix for exception in client.transport getter
 2021-05-19 14:28:54 +01:00
Jakob Ackermann
b53a105681 [LatexRunner] do not emit empty cpu/sys timings 2021-05-19 11:17:08 +01:00
Jakob Ackermann
f361cdfaca [ContentCacheManager] deeply integrate the HashFileTracker 
- Update the tracker contents as we hash ranges
- Let the tracker be authoritative for already written ranges
  This is saving a stat call per newly written range.
 2021-05-18 18:15:58 +01:00
Jakob Ackermann
7aeeb5a5a9 [ContentCacheManager] finish tracking of ranges across builds 2021-05-18 18:06:15 +01:00
Jakob Ackermann
011a228727 [misc] install p-limit and nodemon 2021-05-18 17:32:21 +01:00
Brian Gough
d493238eaf wip expire old hash files 2021-05-18 16:25:24 +01:00
Shane Kilkelly
bbc062f942 default user/pass for development 2021-05-18 16:08:59 +01:00
nate stemen
1f10b45ce7 Merge pull request #147 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
Bump lodash from 4.17.20 to 4.17.21
 2021-05-18 10:20:48 -04:00
nate stemen
948b4929c7 Merge pull request #146 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-05-18 10:20:27 -04:00
Brian Gough
09bb3ac1dd Merge pull request #217 from overleaf/bg-add-metric-transport-label 
add transport type to real-time metrics
 2021-05-18 15:03:47 +01:00
Brian Gough
b78b41ae24 Merge pull request #214 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-05-18 15:03:26 +01:00
Brian Gough
c7b0e908ff Merge pull request #213 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1 
Bump y18n from 4.0.0 to 4.0.1
 2021-05-18 15:02:55 +01:00
Brian Gough
4b32f3d010 Merge pull request #216 from overleaf/dependabot-npm_and_yarn-underscore-1.13.1 
Bump underscore from 1.7.0 to 1.13.1
 2021-05-18 15:02:28 +01:00
Brian Gough
56aed80a9e Merge pull request #218 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
Bump lodash from 4.17.20 to 4.17.21
 2021-05-18 15:00:35 +01:00
nate stemen
b6a81c18eb Merge pull request #89 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
Bump hosted-git-info from 2.8.8 to 2.8.9
 2021-05-18 07:31:25 -04:00
nate stemen
7f4bce262d Merge pull request #88 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
Bump lodash from 4.17.20 to 4.17.21
 2021-05-18 07:31:19 -04:00
Jakob Ackermann
567d02881d [misc] fix unit tests following the merge of atomic writes 2021-05-18 11:09:30 +01:00
Jakob Ackermann
23dd93ae50 Merge pull request #236 from overleaf/jpa-skip-duplicate-writes 
[ContentCacheManager] skip writing of duplicate streams
 2021-05-18 11:44:23 +02:00
Jakob Ackermann
e787106eed Merge pull request #235 from overleaf/jpa-atomic-writes 
[ContentCacheManager] write streams to disk atomically
 2021-05-18 11:44:16 +02:00
Jakob Ackermann
9b8763aed4 Merge pull request #234 from overleaf/jpa-stream-detection-across-chunks 
[ContentCacheManager] add support for stream detection across chunks
 2021-05-18 11:44:09 +02:00
Jakob Ackermann
74e45c60f4 Merge pull request #233 from overleaf/jpa-drop-sync-syscall 
[perf] drop useless synchronous syscall on hot path for writing docs
 2021-05-18 11:44:00 +02:00
Jakob Ackermann
8a3ec2bfda Merge pull request #232 from overleaf/bg-upgrade-fs-extra 
upgrade fs-extra
 2021-05-18 11:43:50 +02:00
Jakob Ackermann
7cf019ab78 Merge pull request #231 from overleaf/bg-use-fse-copy 
use fs.copyFile for performance
 2021-05-18 11:43:43 +02:00
Jakob Ackermann
b254e4285b Merge pull request #229 from overleaf/bg-log-expiry-timeout 
log the expiry timeout value when disk space is low
 2021-05-18 11:43:15 +02:00
Jakob Ackermann
59f18b667c Merge pull request #228 from overleaf/bg-add-content-id-validation 
add validation for express content routes
 2021-05-18 11:43:08 +02:00
Jakob Ackermann
bc1ed82c6c [ContentCacheManager] skip writing of duplicate streams 2021-05-18 10:22:05 +01:00
nate stemen
0592a6be74 Merge pull request #65 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
build(deps): bump hosted-git-info from 2.8.5 to 2.8.9
 2021-05-17 17:07:51 -04:00
nate stemen
ee62d864e5 Merge pull request #64 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
build(deps): bump lodash from 4.17.19 to 4.17.21
 2021-05-17 17:07:44 -04:00
nate stemen
fcaaef09b8 Merge pull request #63 from overleaf/dependabot-npm_and_yarn-underscore-1.13.1 
build(deps): bump underscore from 1.9.2 to 1.13.1
 2021-05-17 17:07:37 -04:00
nate stemen
bc99218596 Merge pull request #61 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-05-17 17:07:30 -04:00
Jakob Ackermann
224ae0c254 [ContentCacheManager] add support for stream detection across chunks 
Retain a small part (6 or 9 bytes) of each chunk in memory for providing
 the next iteration with enough context for finding the start/end marker
 of a range.
 2021-05-17 17:36:05 +01:00
nate stemen
290c0d90c8 Merge pull request #44 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
Bump hosted-git-info from 2.8.8 to 2.8.9
 2021-05-17 11:02:06 -04:00
nate stemen
1de6c67910 Merge pull request #43 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
Bump lodash from 4.17.20 to 4.17.21
 2021-05-17 11:01:50 -04:00
nate stemen
b076823e7a Merge pull request #42 from overleaf/dependabot-npm_and_yarn-underscore-1.13.1 
Bump underscore from 1.9.2 to 1.13.1
 2021-05-17 11:01:41 -04:00
nate stemen
c33d6e4188 Merge pull request #40 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-05-17 11:01:30 -04:00
Jakob Ackermann
6b9c8bced6 [ContentCacheManager] write streams to disk atomically 
Use an intermediate file for writing to disk, then rename to the target.
 2021-05-17 14:22:37 +01:00
Brian Gough
8c80ddd27a remove unnecessary console.log from hash benchmark 2021-05-17 14:14:00 +01:00
Brian Gough
939ad3af31 include UV_THREADPOOL_SIZE in benchmark logs 2021-05-17 14:14:00 +01:00
Brian Gough
cbeba4af00 run the hash benchmark 10 times 2021-05-17 14:14:00 +01:00
Brian Gough
7b6434ef81 add benchmark script for hashing 2021-05-17 14:14:00 +01:00
nate stemen
0b5b8f56cb Merge pull request #73 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
Bump hosted-git-info from 2.8.8 to 2.8.9
 2021-05-17 08:10:57 -04:00
nate stemen
a7c31132fe Merge pull request #72 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21 
Bump lodash from 4.17.20 to 4.17.21
 2021-05-17 08:10:50 -04:00
nate stemen
007141f545 Merge pull request #71 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-05-17 08:10:43 -04:00
nate stemen
d1c2dd6cf1 Merge pull request #70 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1 
Bump y18n from 4.0.0 to 4.0.1
 2021-05-17 08:10:32 -04:00
nate stemen
227b9e9ed8 Merge pull request #66 from overleaf/dependabot-npm_and_yarn-ini-1.3.8 
Bump ini from 1.3.5 to 1.3.8
 2021-05-17 08:10:17 -04:00
nate stemen
9b234e5946 Merge pull request #65 from overleaf/dependabot-npm_and_yarn-dot-prop-4.2.1 
Bump dot-prop from 4.2.0 to 4.2.1
 2021-05-17 08:10:08 -04:00
Brian Gough
371de76a4a use fs.copyFile instead of fse.copy in UrlCache module 2021-05-17 10:54:11 +01:00
Jakob Ackermann
dde645e770 Merge pull request #176 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9 
Bump hosted-git-info from 2.8.8 to 2.8.9
 2021-05-17 10:57:18 +02:00
Jakob Ackermann
e92b9fc754 Merge pull request #174 from overleaf/jpa-explicit-dependencies 
[misc] add linting for missing explicit dependencies and fix any errors
 2021-05-17 10:57:12 +02:00
Jakob Ackermann
f97117a6ba Merge pull request #175 from overleaf/jpa-double-down-doc-size-limit 
[misc] double down on enforcing doc size limits
 2021-05-17 10:56:55 +02:00
Jakob Ackermann
18e943742e [perf] drop useless synchronous syscall on hot path for writing docs 2021-05-17 09:25:29 +01:00
Brian Gough
24ef5786a0 upgrade fs-extra 2021-05-14 15:58:11 +01:00
Brian Gough
44e0742aa3 use fse.copy for performance 
it uses the native fs.copyFile method
 2021-05-14 15:49:20 +01:00
Brian Gough
ff2175e727 add validation for express :content_id parameter 2021-05-13 15:48:17 +01:00
Brian Gough
013d38552a log the expiry timeout value when disk is low 2021-05-13 15:13:29 +01:00
Brian Gough
3044bc4cf5 add transport to get-connected-users metric 2021-05-13 14:51:26 +01:00
Jakob Ackermann
b456ea726d [misc] merge pdf caching into main (#226) 
* wip generate directory for hash content

* cleanup, remove console logging

* add content caching module

* Return PDF stream ranges with compile response

* Return the PDF file size in the compile response

* PDF range endpoint

* [misc] WIP: pdf caching: preserve the m-time on static content files

* [misc] WIP: pdf caching: improve browser caching, emit caching headers

* [misc] WIP: pdf caching: do not emit very small chunks <1kB

* [misc] keep up with moving output files into a separate directory

* [OutputCacheManager] add global feature flag for enabling pdf caching

* [misc] add contentId into the URL for protecting PDF stream contents

* [misc] support PDF stream caching for anonymous users

* [misc] add per-request feature flag for enabling PDF stream caching

* [misc] enable pdf caching in CI and emit metrics at the end of run

* [misc] expose compile stats and timings to the frontend

* [misc] log an error in case saving output files fails

* [misc] add metrics for pdf bandwidth and pdf caching performance

* [misc] add a dark mode to the pdf caching for computing ranges only

* [misc] move pdf caching metrics into ContentCacheMetrics

* [misc] add a config option for the min chunk size of pdf ranges

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
 2021-05-13 14:07:54 +01:00
Christopher Hoskin
725a9e0669 Use CMD as in other apps 2021-05-12 12:32:14 +01:00
Christopher Hoskin
0827795fc7 Consistently use GIT_BRIDGE 2021-05-12 08:42:45 +01:00
dependabot[bot]
8a16c55000 Bump hosted-git-info from 2.8.8 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 20:43:06 +00:00
dependabot[bot]
da81a97b82 Bump hosted-git-info from 2.8.8 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 17:18:40 +00:00
dependabot[bot]
e66ff7a2ca Bump hosted-git-info from 2.8.5 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 14:32:37 +00:00
dependabot[bot]
eb8a3118d8 Bump hosted-git-info from 2.8.5 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 13:04:24 +00:00
dependabot[bot]
bc423fa0cd build(deps): bump hosted-git-info from 2.8.5 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 12:49:25 +00:00
dependabot[bot]
218f0fd4d8 build(deps): bump hosted-git-info from 2.8.5 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 12:33:05 +00:00
dependabot[bot]
ce32ca9238 Bump hosted-git-info from 2.8.8 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 12:30:36 +00:00
dependabot[bot]
326c7da1d8 Bump hosted-git-info from 2.8.8 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 12:19:56 +00:00
dependabot[bot]
54bbb3dbbd Bump hosted-git-info from 2.8.8 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 10:38:18 +00:00
dependabot[bot]
fc48f8dd98 Bump hosted-git-info from 2.8.8 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 08:45:50 +00:00
dependabot[bot]
847ff87ce3 Bump lodash from 4.17.19 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.19 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.19...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 04:33:05 +00:00
dependabot[bot]
c0fc4c329e build(deps): bump lodash from 4.17.19 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.19 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.19...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 04:20:08 +00:00
dependabot[bot]
78c0b90eb7 Bump lodash from 4.17.20 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 03:28:23 +00:00
dependabot[bot]
39bc2b7685 build(deps): bump lodash from 4.17.20 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 03:21:10 +00:00
dependabot[bot]
5ab599e163 Bump lodash from 4.17.20 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 03:18:32 +00:00
dependabot[bot]
d5f2f4b7e4 Bump lodash from 4.17.20 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 03:16:58 +00:00
dependabot[bot]
39249d0b7e Bump lodash from 4.17.20 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 00:17:10 +00:00
dependabot[bot]
785a69bdb4 Bump lodash from 4.17.20 to 4.17.21 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-10 17:09:32 +00:00
Brian Gough
9622b341b7 add transport type to metrics 2021-05-10 10:16:50 +01:00
Shane Kilkelly
b47f21a44e Configure from env vars at run time 
In both local developent, and production.  Also, vendor the `envsubst`
binary from https://github.com/a8m/envsubst, as it supports default
values, which the gnu envsubst (from gettext-base) does not.
 2021-05-07 09:47:49 +01:00
dependabot[bot]
387caf66b8 Bump underscore from 1.9.2 to 1.13.1 
Bumps [underscore](https://github.com/jashkenas/underscore) from 1.9.2 to 1.13.1.
- [Release notes](https://github.com/jashkenas/underscore/releases)
- [Commits](https://github.com/jashkenas/underscore/compare/1.9.2...1.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-07 05:22:05 +00:00
dependabot[bot]
7d8f439b4c build(deps): bump underscore from 1.9.2 to 1.13.1 
Bumps [underscore](https://github.com/jashkenas/underscore) from 1.9.2 to 1.13.1.
- [Release notes](https://github.com/jashkenas/underscore/releases)
- [Commits](https://github.com/jashkenas/underscore/compare/1.9.2...1.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-07 05:21:44 +00:00
dependabot[bot]
19507962db Bump underscore from 1.7.0 to 1.13.1 
Bumps [underscore](https://github.com/jashkenas/underscore) from 1.7.0 to 1.13.1.
- [Release notes](https://github.com/jashkenas/underscore/releases)
- [Commits](https://github.com/jashkenas/underscore/compare/1.7.0...1.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-07 05:18:17 +00:00
dependabot[bot]
dd88d03254 Bump underscore from 1.9.2 to 1.13.1 
Bumps [underscore](https://github.com/jashkenas/underscore) from 1.9.2 to 1.13.1.
- [Release notes](https://github.com/jashkenas/underscore/releases)
- [Commits](https://github.com/jashkenas/underscore/compare/1.9.2...1.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-07 05:10:51 +00:00
dependabot[bot]
c30fe0364a build(deps): bump underscore from 1.9.2 to 1.13.1 
Bumps [underscore](https://github.com/jashkenas/underscore) from 1.9.2 to 1.13.1.
- [Release notes](https://github.com/jashkenas/underscore/releases)
- [Commits](https://github.com/jashkenas/underscore/compare/1.9.2...1.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-07 05:08:44 +00:00
Jakob Ackermann
309ad818f6 [SizeCheckTests] add acceptance tests for size limit checks 2021-05-06 18:21:19 +01:00
Jakob Ackermann
757e1e98c5 [RedisManager] block updating of too large docs in redis 2021-05-06 18:21:19 +01:00
Jakob Ackermann
c707d0b345 [RedisManager] block inserting of too large docs into redis 2021-05-06 18:21:18 +01:00