github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
968 commits 3 branches 7 tags 206 MiB
Commit graph

862 commits

Author SHA1 Message Date
Brian Gough
d493238eaf wip expire old hash files 2021-05-18 16:25:24 +01:00
Jakob Ackermann
567d02881d [misc] fix unit tests following the merge of atomic writes 2021-05-18 11:09:30 +01:00
Jakob Ackermann
23dd93ae50 Merge pull request #236 from overleaf/jpa-skip-duplicate-writes 
[ContentCacheManager] skip writing of duplicate streams
 2021-05-18 11:44:23 +02:00
Jakob Ackermann
e787106eed Merge pull request #235 from overleaf/jpa-atomic-writes 
[ContentCacheManager] write streams to disk atomically
 2021-05-18 11:44:16 +02:00
Jakob Ackermann
9b8763aed4 Merge pull request #234 from overleaf/jpa-stream-detection-across-chunks 
[ContentCacheManager] add support for stream detection across chunks
 2021-05-18 11:44:09 +02:00
Jakob Ackermann
74e45c60f4 Merge pull request #233 from overleaf/jpa-drop-sync-syscall 
[perf] drop useless synchronous syscall on hot path for writing docs
 2021-05-18 11:44:00 +02:00
Jakob Ackermann
8a3ec2bfda Merge pull request #232 from overleaf/bg-upgrade-fs-extra 
upgrade fs-extra
 2021-05-18 11:43:50 +02:00
Jakob Ackermann
7cf019ab78 Merge pull request #231 from overleaf/bg-use-fse-copy 
use fs.copyFile for performance
 2021-05-18 11:43:43 +02:00
Jakob Ackermann
b254e4285b Merge pull request #229 from overleaf/bg-log-expiry-timeout 
log the expiry timeout value when disk space is low
 2021-05-18 11:43:15 +02:00
Jakob Ackermann
59f18b667c Merge pull request #228 from overleaf/bg-add-content-id-validation 
add validation for express content routes
 2021-05-18 11:43:08 +02:00
Jakob Ackermann
bc1ed82c6c [ContentCacheManager] skip writing of duplicate streams 2021-05-18 10:22:05 +01:00
Jakob Ackermann
224ae0c254 [ContentCacheManager] add support for stream detection across chunks 
Retain a small part (6 or 9 bytes) of each chunk in memory for providing
 the next iteration with enough context for finding the start/end marker
 of a range.
 2021-05-17 17:36:05 +01:00
Jakob Ackermann
6b9c8bced6 [ContentCacheManager] write streams to disk atomically 
Use an intermediate file for writing to disk, then rename to the target.
 2021-05-17 14:22:37 +01:00
Brian Gough
8c80ddd27a remove unnecessary console.log from hash benchmark 2021-05-17 14:14:00 +01:00
Brian Gough
939ad3af31 include UV_THREADPOOL_SIZE in benchmark logs 2021-05-17 14:14:00 +01:00
Brian Gough
cbeba4af00 run the hash benchmark 10 times 2021-05-17 14:14:00 +01:00
Brian Gough
7b6434ef81 add benchmark script for hashing 2021-05-17 14:14:00 +01:00
Brian Gough
371de76a4a use fs.copyFile instead of fse.copy in UrlCache module 2021-05-17 10:54:11 +01:00
Jakob Ackermann
18e943742e [perf] drop useless synchronous syscall on hot path for writing docs 2021-05-17 09:25:29 +01:00
Brian Gough
24ef5786a0 upgrade fs-extra 2021-05-14 15:58:11 +01:00
Brian Gough
44e0742aa3 use fse.copy for performance 
it uses the native fs.copyFile method
 2021-05-14 15:49:20 +01:00
Brian Gough
ff2175e727 add validation for express :content_id parameter 2021-05-13 15:48:17 +01:00
Brian Gough
013d38552a log the expiry timeout value when disk is low 2021-05-13 15:13:29 +01:00
Jakob Ackermann
b456ea726d [misc] merge pdf caching into main (#226) 
* wip generate directory for hash content

* cleanup, remove console logging

* add content caching module

* Return PDF stream ranges with compile response

* Return the PDF file size in the compile response

* PDF range endpoint

* [misc] WIP: pdf caching: preserve the m-time on static content files

* [misc] WIP: pdf caching: improve browser caching, emit caching headers

* [misc] WIP: pdf caching: do not emit very small chunks <1kB

* [misc] keep up with moving output files into a separate directory

* [OutputCacheManager] add global feature flag for enabling pdf caching

* [misc] add contentId into the URL for protecting PDF stream contents

* [misc] support PDF stream caching for anonymous users

* [misc] add per-request feature flag for enabling PDF stream caching

* [misc] enable pdf caching in CI and emit metrics at the end of run

* [misc] expose compile stats and timings to the frontend

* [misc] log an error in case saving output files fails

* [misc] add metrics for pdf bandwidth and pdf caching performance

* [misc] add a dark mode to the pdf caching for computing ranges only

* [misc] move pdf caching metrics into ContentCacheMetrics

* [misc] add a config option for the min chunk size of pdf ranges

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
 2021-05-13 14:07:54 +01:00
dependabot[bot]
e66ff7a2ca Bump hosted-git-info from 2.8.5 to 2.8.9 
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-11 14:32:37 +00:00
Jakob Ackermann
cfa8127c1b [misc] add linting for missing explicit dependencies and fix any errors 2021-04-29 15:30:54 +01:00
dependabot[bot]
ab57729f98 Bump y18n from 4.0.0 to 4.0.1 
Bumps [y18n](https://github.com/yargs/y18n) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/yargs/y18n/releases)
- [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yargs/y18n/commits)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-03-31 17:52:26 +00:00
Jakob Ackermann
b3ed820444 Merge pull request #219 from overleaf/jpa-read-image-name-for-synctex 
[misc] consume and validate a custom imageName for synctex requests
 2021-03-31 10:33:53 +02:00
Jakob Ackermann
c1fe1cf5b7 Merge pull request #218 from overleaf/jpa-metrics-module-3-5-1 
[misc] bump the version of the metrics module to 3.5.1
 2021-03-31 10:33:41 +02:00
Jakob Ackermann
1a41fa55ad Merge pull request #217 from overleaf/sk-qpdf-newline 
Add flag to qpdf, to preserve PDF/A compliance
 2021-03-31 10:33:16 +02:00
Jakob Ackermann
6cb4246eb1 [misc] consume and validate a custom imageName for synctex requests 2021-03-30 13:57:40 +01:00
Alf Eaton
a271d5ce10 Merge pull request #212 from overleaf/ae-chown-app-folders 
Ensure that app folders exist before running chown in entrypoint.sh
 2021-03-23 09:43:31 +00:00
Alf Eaton
db0cd53ed4 Merge pull request #213 from overleaf/ae-readme 
Update README
 2021-03-23 09:41:44 +00:00
Jakob Ackermann
b563deef14 [misc] bump the version of the metrics module to 3.5.1 2021-03-22 12:59:43 +00:00
Eric Mc Sween
7a6b86185b Merge pull request #215 from overleaf/ho-clsi-cookie-post 
support post and get for getting status of clsi project
 2021-03-22 07:56:09 -04:00
Christopher Hoskin
6ca016bb05 Explain the situation with permissions on Linux 2021-03-19 14:32:54 +00:00
Alf Eaton
268a144e43 Merge remote-tracking branch 'origin/master' into ae-readme 2021-03-19 12:06:03 +00:00
Alf Eaton
7e3edcee2c Add instructions for Linux 2021-03-19 12:05:22 +00:00
Alf Eaton
d4b4cb399a Revert "Make TEXLIVE_IMAGE_USER instruction macOS only" 
This reverts commit ab6fe228cadd3329885fdc60aea867d601579759.
 2021-03-19 12:00:47 +00:00
Shane Kilkelly
072d0a9dab Add flag to qpdf, to preserve PDF/A compliance 2021-03-19 11:35:59 +00:00
Christopher Hoskin
34b572de9e Merge branch 'ae-readme' of github.com:overleaf/clsi into ae-readme 2021-03-19 10:35:25 +00:00
Christopher Hoskin
638e72295b Add link to \openout primitive docs 2021-03-19 10:34:52 +00:00
Alf Eaton
984766a9fb Make TEXLIVE_IMAGE_USER instruction macOS only 2021-03-18 09:55:31 +00:00
Christopher Hoskin
b8d0389a53 README typos 2021-03-16 15:29:53 +00:00
Christopher Hoskin
0a89b6537c Fill in missing text for environment variables 2021-03-16 12:30:06 +00:00
Christopher Hoskin
17a83bc169 Expand list of environment variables 2021-03-16 12:00:48 +00:00
Henry Oswald
7cd36e139a support post and get for getting status of clsi project 2021-03-16 10:24:59 +00:00
Eric Mc Sween
3eca505319 Upgrade to Node 12 2021-03-12 16:27:33 -05:00
Eric Mc Sween
9f2d219102 Add a global test setup file 
Set up generally useful stuff:

* chai.should()
* logger stubs
* globals in SandboxedModule, including Buffer and process, which are
  now required in Node 12
 2021-03-12 16:27:33 -05:00
Alf Eaton
f38cef6e56 Change settings file .coffee to .js 2021-03-11 11:50:47 +00:00
Alf Eaton
9c596fb17d Update README.md 2021-03-10 22:38:34 +00:00
Alf Eaton
e7be75a4b7 Update README.md 2021-03-10 22:36:33 +00:00
Alf Eaton
18dbb12434 Add /app/output 2021-02-18 15:33:16 +00:00
Alf Eaton
498ca80cd3 Ensure that app folders exist before running chown 2021-02-18 15:09:48 +00:00
Brian Gough
bad5fd20a8 Merge pull request #208 from overleaf/bg-fix-use-output-dir 
fix for exception in "decaff cleanup ResourceStateManager"
 2021-02-01 11:24:19 +00:00
Brian Gough
d0d36bf9d8 Merge pull request #206 from overleaf/bg-add-missing-unit-test 
add unit test for non-existent state file
 2021-02-01 11:23:56 +00:00
Brian Gough
f0b4f1238b provide a /oops-internal endpoint for testing uncaughtExceptions 2021-01-26 16:35:39 +00:00
Brian Gough
fc11574698 add uncaughtException handler 2021-01-26 14:08:29 +00:00
Brian Gough
865e68051e include fallback for missing state file 2021-01-26 13:59:28 +00:00
Brian Gough
8946f87ca6 Merge branch 'bg-add-missing-unit-test' into bg-fix-use-output-dir 2021-01-26 11:09:00 +00:00
Brian Gough
a8e47da9e9 Revert "Merge pull request #205 from overleaf/revert-200-bg-decaff-cleanup" 
This reverts commit 76d8d3181b9464d1e1bbc713a2729ca269d9c047, reversing
changes made to 31a8dc3a98d73c2707d633712f0ef7207013e78b.
 2021-01-26 11:04:33 +00:00
Brian Gough
a86e521ac0 add unit test for non-existent state file 2021-01-26 11:03:18 +00:00
Brian Gough
9dc55729ac Revert "decaff cleanup ResourceStateManager" 2021-01-25 15:26:53 +00:00
Brian Gough
f41f3aba8c Merge pull request #204 from overleaf/bg-use-output-dir 
use separate output dir for generated files
 2021-01-25 14:10:09 +00:00
Brian Gough
90c0d931ad Merge pull request #201 from overleaf/bg-more-cleanup 
decaff cleanup of OutputFileManager
 2021-01-25 14:09:46 +00:00
Brian Gough
c67c8a6c80 Merge pull request #200 from overleaf/bg-decaff-cleanup 
decaff cleanup ResourceStateManager
 2021-01-25 14:09:24 +00:00
Brian Gough
e52871e07b Merge pull request #199 from overleaf/dependabot-npm_and_yarn-ini-1.3.8 
Bump ini from 1.3.5 to 1.3.8
 2021-01-25 14:09:02 +00:00
Brian Gough
6fa081522d add a warning for requests without build id 2021-01-22 11:05:52 +00:00
Brian Gough
b5346658b0 clear output directory when clearing project 2021-01-22 11:05:52 +00:00
Brian Gough
565cd53eb5 add git ignore for output directory 2021-01-22 11:05:52 +00:00
Brian Gough
692dbc8d6b add output directory 2021-01-22 11:05:52 +00:00
Brian Gough
bdbfe70086 rename staticServer to staticCompileServer 2021-01-22 11:05:52 +00:00
Jakob Ackermann
96b8d001f7 Merge pull request #202 from overleaf/jpa-fix-output-pdf-check 
[misc] CompileController: exact match for output.pdf
 2021-01-13 09:35:31 +00:00
Jakob Ackermann
cac18ac4b9 [misc] bump the node version to 10.23.1 2021-01-05 18:32:01 +00:00
Jakob Ackermann
49b764a308 [misc] CompileController: exact match for output.pdf 
The previous regex could mistake user provided pdf files, like
 `fake_output.pdf`, as the final output file.
The frontend expects to find a `output.pdf` file on success.
 2020-12-28 13:16:31 +00:00
Brian Gough
e183fffa4b format fix 2020-12-18 16:15:06 +00:00
Brian Gough
dafe69b5e6 remove unused module var 2020-12-18 16:15:06 +00:00
Brian Gough
36b646bb84 remove comments 2020-12-18 16:15:06 +00:00
Brian Gough
f20a0d1a69 remove unused var 2020-12-18 16:15:03 +00:00
Brian Gough
df1caa14b8 remove unused require 2020-12-18 16:14:36 +00:00
Brian Gough
ca98ee5cff use a Set instead of an Object 2020-12-18 16:14:09 +00:00
Brian Gough
43b0429c28 clean up relative path checking 2020-12-18 16:14:05 +00:00
Brian Gough
73e09ff99f remove unnecessary requires 2020-12-18 15:28:06 +00:00
Brian Gough
cee93a0cd9 clean up unnecessary var 2020-12-18 15:28:06 +00:00
Brian Gough
4169f7fc73 use once for callback 2020-12-18 15:28:06 +00:00
Brian Gough
e84d6305f3 remove unnecessary return 2020-12-18 15:16:07 +00:00
Brian Gough
ae064e8997 lint fix 2020-12-18 15:11:29 +00:00
Brian Gough
323890cedb remove Array.from 2020-12-18 15:10:53 +00:00
Brian Gough
c044db4897 remove guard helper 2020-12-18 15:09:48 +00:00
Brian Gough
0bae53c7f3 remove unnecessary returns 2020-12-18 15:02:40 +00:00
Brian Gough
eaec57cd01 simplify null check 2020-12-18 15:01:25 +00:00
Brian Gough
411d69e36b remove unnecessary callback code 2020-12-18 14:59:48 +00:00
Brian Gough
11ef3c27ed use Set instead of object 2020-12-18 14:56:53 +00:00
Brian Gough
64ea22d259 remove unnecessary null checks 2020-12-17 12:05:17 +00:00
Brian Gough
299d7cc5fe remove unnecessary returns 2020-12-17 12:01:14 +00:00
Brian Gough
5a539b7f90 remove guard function 2020-12-17 11:59:15 +00:00
Brian Gough
4d8e4d54e5 remove Array.from 2020-12-17 11:49:02 +00:00
dependabot[bot]
f5594e023f Bump ini from 1.3.5 to 1.3.8 
Bumps [ini](https://github.com/isaacs/ini) from 1.3.5 to 1.3.8.
- [Release notes](https://github.com/isaacs/ini/releases)
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.8)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-12-12 11:02:53 +00:00
Jakob Ackermann
5d5241156c [misc] install settings-sharelatex from npm 2020-12-02 12:26:42 +00:00
Jakob Ackermann
1b73c6dad2 [misc] work around missing stubs 2020-11-25 13:33:54 +00:00