Commit graph

968 commits

Author SHA1 Message Date
Jakob Ackermann
fdcfbcd6ea Merge pull request #238 from overleaf/jpa-drop-broken-timings
[LatexRunner] do not emit empty cpu/sys timings
2021-05-21 11:34:54 +02:00
Alf Eaton
6507d668ae Update references to .coffee files 2021-05-19 18:57:35 +01:00
Alf Eaton
1b1c4aa4a0 Fix formatting 2021-05-19 18:47:51 +01:00
Alf Eaton
f31abe7cb3 Tidy and format 2021-05-19 18:36:27 +01:00
decaffeinate
85920b3120 decaffeinate: Run post-processing cleanups on settings.test.coffee 2021-05-19 16:50:46 +01:00
decaffeinate
8c984a77c4 decaffeinate: Convert settings.test.coffee to JS 2021-05-19 16:50:45 +01:00
decaffeinate
5af8f57dd0 decaffeinate: Rename settings.test.coffee from .coffee to .js 2021-05-19 16:50:45 +01:00
Jakob Ackermann
b53a105681 [LatexRunner] do not emit empty cpu/sys timings 2021-05-19 11:17:08 +01:00
Jakob Ackermann
f361cdfaca [ContentCacheManager] deeply integrate the HashFileTracker
- Update the tracker contents as we hash ranges
- Let the tracker be authoritative for already written ranges
  This is saving a stat call per newly written range.
2021-05-18 18:15:58 +01:00
Jakob Ackermann
7aeeb5a5a9 [ContentCacheManager] finish tracking of ranges across builds 2021-05-18 18:06:15 +01:00
Jakob Ackermann
011a228727 [misc] install p-limit and nodemon 2021-05-18 17:32:21 +01:00
Brian Gough
d493238eaf wip expire old hash files 2021-05-18 16:25:24 +01:00
Jakob Ackermann
31a29403a1 Merge pull request #237 from overleaf/jpa-fix-unit-tests
[misc] fix unit tests following the merge of atomic writes
2021-05-18 12:17:56 +02:00
Jakob Ackermann
567d02881d [misc] fix unit tests following the merge of atomic writes 2021-05-18 11:09:30 +01:00
Jakob Ackermann
23dd93ae50 Merge pull request #236 from overleaf/jpa-skip-duplicate-writes
[ContentCacheManager] skip writing of duplicate streams
2021-05-18 11:44:23 +02:00
Jakob Ackermann
e787106eed Merge pull request #235 from overleaf/jpa-atomic-writes
[ContentCacheManager] write streams to disk atomically
2021-05-18 11:44:16 +02:00
Jakob Ackermann
9b8763aed4 Merge pull request #234 from overleaf/jpa-stream-detection-across-chunks
[ContentCacheManager] add support for stream detection across chunks
2021-05-18 11:44:09 +02:00
Jakob Ackermann
74e45c60f4 Merge pull request #233 from overleaf/jpa-drop-sync-syscall
[perf] drop useless synchronous syscall on hot path for writing docs
2021-05-18 11:44:00 +02:00
Jakob Ackermann
8a3ec2bfda Merge pull request #232 from overleaf/bg-upgrade-fs-extra
upgrade fs-extra
2021-05-18 11:43:50 +02:00
Jakob Ackermann
7cf019ab78 Merge pull request #231 from overleaf/bg-use-fse-copy
use fs.copyFile for performance
2021-05-18 11:43:43 +02:00
Jakob Ackermann
b254e4285b Merge pull request #229 from overleaf/bg-log-expiry-timeout
log the expiry timeout value when disk space is low
2021-05-18 11:43:15 +02:00
Jakob Ackermann
59f18b667c Merge pull request #228 from overleaf/bg-add-content-id-validation
add validation for express content routes
2021-05-18 11:43:08 +02:00
Jakob Ackermann
a93c2c3942 Merge pull request #224 from overleaf/bg-add-hash-benchmark-script
add hash benchmark script
2021-05-18 11:42:59 +02:00
Jakob Ackermann
bc1ed82c6c [ContentCacheManager] skip writing of duplicate streams 2021-05-18 10:22:05 +01:00
Jakob Ackermann
224ae0c254 [ContentCacheManager] add support for stream detection across chunks
Retain a small part (6 or 9 bytes) of each chunk in memory for providing
 the next iteration with enough context for finding the start/end marker
 of a range.
2021-05-17 17:36:05 +01:00
Jakob Ackermann
6b9c8bced6 [ContentCacheManager] write streams to disk atomically
Use an intermediate file for writing to disk, then rename to the target.
2021-05-17 14:22:37 +01:00
Brian Gough
8c80ddd27a remove unnecessary console.log from hash benchmark 2021-05-17 14:14:00 +01:00
Brian Gough
939ad3af31 include UV_THREADPOOL_SIZE in benchmark logs 2021-05-17 14:14:00 +01:00
Brian Gough
cbeba4af00 run the hash benchmark 10 times 2021-05-17 14:14:00 +01:00
Brian Gough
7b6434ef81 add benchmark script for hashing 2021-05-17 14:14:00 +01:00
Brian Gough
371de76a4a use fs.copyFile instead of fse.copy in UrlCache module 2021-05-17 10:54:11 +01:00
Jakob Ackermann
18e943742e [perf] drop useless synchronous syscall on hot path for writing docs 2021-05-17 09:25:29 +01:00
Brian Gough
24ef5786a0 upgrade fs-extra 2021-05-14 15:58:11 +01:00
Brian Gough
44e0742aa3 use fse.copy for performance
it uses the native fs.copyFile method
2021-05-14 15:49:20 +01:00
Brian Gough
ff2175e727 add validation for express :content_id parameter 2021-05-13 15:48:17 +01:00
Brian Gough
013d38552a log the expiry timeout value when disk is low 2021-05-13 15:13:29 +01:00
Jakob Ackermann
b456ea726d [misc] merge pdf caching into main (#226)
* wip generate directory for hash content

* cleanup, remove console logging

* add content caching module

* Return PDF stream ranges with compile response

* Return the PDF file size in the compile response

* PDF range endpoint

* [misc] WIP: pdf caching: preserve the m-time on static content files

* [misc] WIP: pdf caching: improve browser caching, emit caching headers

* [misc] WIP: pdf caching: do not emit very small chunks <1kB

* [misc] keep up with moving output files into a separate directory

* [OutputCacheManager] add global feature flag for enabling pdf caching

* [misc] add contentId into the URL for protecting PDF stream contents

* [misc] support PDF stream caching for anonymous users

* [misc] add per-request feature flag for enabling PDF stream caching

* [misc] enable pdf caching in CI and emit metrics at the end of run

* [misc] expose compile stats and timings to the frontend

* [misc] log an error in case saving output files fails

* [misc] add metrics for pdf bandwidth and pdf caching performance

* [misc] add a dark mode to the pdf caching for computing ranges only

* [misc] move pdf caching metrics into ContentCacheMetrics

* [misc] add a config option for the min chunk size of pdf ranges

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
2021-05-13 14:07:54 +01:00
dependabot[bot]
e66ff7a2ca Bump hosted-git-info from 2.8.5 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 14:32:37 +00:00
Jakob Ackermann
cfa8127c1b [misc] add linting for missing explicit dependencies and fix any errors 2021-04-29 15:30:54 +01:00
dependabot[bot]
ab57729f98 Bump y18n from 4.0.0 to 4.0.1
Bumps [y18n](https://github.com/yargs/y18n) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/yargs/y18n/releases)
- [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yargs/y18n/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-31 17:52:26 +00:00
Jakob Ackermann
b3ed820444 Merge pull request #219 from overleaf/jpa-read-image-name-for-synctex
[misc] consume and validate a custom imageName for synctex requests
2021-03-31 10:33:53 +02:00
Jakob Ackermann
c1fe1cf5b7 Merge pull request #218 from overleaf/jpa-metrics-module-3-5-1
[misc] bump the version of the metrics module to 3.5.1
2021-03-31 10:33:41 +02:00
Jakob Ackermann
1a41fa55ad Merge pull request #217 from overleaf/sk-qpdf-newline
Add flag to qpdf, to preserve PDF/A compliance
2021-03-31 10:33:16 +02:00
Jakob Ackermann
6cb4246eb1 [misc] consume and validate a custom imageName for synctex requests 2021-03-30 13:57:40 +01:00
Alf Eaton
a271d5ce10 Merge pull request #212 from overleaf/ae-chown-app-folders
Ensure that app folders exist before running chown in entrypoint.sh
2021-03-23 09:43:31 +00:00
Alf Eaton
db0cd53ed4 Merge pull request #213 from overleaf/ae-readme
Update README
2021-03-23 09:41:44 +00:00
Jakob Ackermann
b563deef14 [misc] bump the version of the metrics module to 3.5.1 2021-03-22 12:59:43 +00:00
Eric Mc Sween
7a6b86185b Merge pull request #215 from overleaf/ho-clsi-cookie-post
support post and get for getting status of clsi project
2021-03-22 07:56:09 -04:00
Christopher Hoskin
6ca016bb05 Explain the situation with permissions on Linux 2021-03-19 14:32:54 +00:00
Alf Eaton
268a144e43 Merge remote-tracking branch 'origin/master' into ae-readme 2021-03-19 12:06:03 +00:00