Brian Gough
371de76a4a
use fs.copyFile instead of fse.copy in UrlCache module
2021-05-17 10:54:11 +01:00
Jakob Ackermann
18e943742e
[perf] drop useless synchronous syscall on hot path for writing docs
2021-05-17 09:25:29 +01:00
Brian Gough
24ef5786a0
upgrade fs-extra
2021-05-14 15:58:11 +01:00
Brian Gough
44e0742aa3
use fse.copy for performance
...
it uses the native fs.copyFile method
2021-05-14 15:49:20 +01:00
Brian Gough
ff2175e727
add validation for express :content_id parameter
2021-05-13 15:48:17 +01:00
Brian Gough
013d38552a
log the expiry timeout value when disk is low
2021-05-13 15:13:29 +01:00
Jakob Ackermann
b456ea726d
[misc] merge pdf caching into main ( #226 )
...
* wip generate directory for hash content
* cleanup, remove console logging
* add content caching module
* Return PDF stream ranges with compile response
* Return the PDF file size in the compile response
* PDF range endpoint
* [misc] WIP: pdf caching: preserve the m-time on static content files
* [misc] WIP: pdf caching: improve browser caching, emit caching headers
* [misc] WIP: pdf caching: do not emit very small chunks <1kB
* [misc] keep up with moving output files into a separate directory
* [OutputCacheManager] add global feature flag for enabling pdf caching
* [misc] add contentId into the URL for protecting PDF stream contents
* [misc] support PDF stream caching for anonymous users
* [misc] add per-request feature flag for enabling PDF stream caching
* [misc] enable pdf caching in CI and emit metrics at the end of run
* [misc] expose compile stats and timings to the frontend
* [misc] log an error in case saving output files fails
* [misc] add metrics for pdf bandwidth and pdf caching performance
* [misc] add a dark mode to the pdf caching for computing ranges only
* [misc] move pdf caching metrics into ContentCacheMetrics
* [misc] add a config option for the min chunk size of pdf ranges
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
2021-05-13 14:07:54 +01:00
dependabot[bot]
e66ff7a2ca
Bump hosted-git-info from 2.8.5 to 2.8.9
...
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info ) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases )
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md )
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9 )
Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 14:32:37 +00:00
Jakob Ackermann
cfa8127c1b
[misc] add linting for missing explicit dependencies and fix any errors
2021-04-29 15:30:54 +01:00
dependabot[bot]
ab57729f98
Bump y18n from 4.0.0 to 4.0.1
...
Bumps [y18n](https://github.com/yargs/y18n ) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/yargs/y18n/releases )
- [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md )
- [Commits](https://github.com/yargs/y18n/commits )
Signed-off-by: dependabot[bot] <support@github.com>
2021-03-31 17:52:26 +00:00
Jakob Ackermann
b3ed820444
Merge pull request #219 from overleaf/jpa-read-image-name-for-synctex
...
[misc] consume and validate a custom imageName for synctex requests
2021-03-31 10:33:53 +02:00
Jakob Ackermann
c1fe1cf5b7
Merge pull request #218 from overleaf/jpa-metrics-module-3-5-1
...
[misc] bump the version of the metrics module to 3.5.1
2021-03-31 10:33:41 +02:00
Jakob Ackermann
1a41fa55ad
Merge pull request #217 from overleaf/sk-qpdf-newline
...
Add flag to qpdf, to preserve PDF/A compliance
2021-03-31 10:33:16 +02:00
Jakob Ackermann
6cb4246eb1
[misc] consume and validate a custom imageName for synctex requests
2021-03-30 13:57:40 +01:00
Alf Eaton
a271d5ce10
Merge pull request #212 from overleaf/ae-chown-app-folders
...
Ensure that app folders exist before running chown in entrypoint.sh
2021-03-23 09:43:31 +00:00
Alf Eaton
db0cd53ed4
Merge pull request #213 from overleaf/ae-readme
...
Update README
2021-03-23 09:41:44 +00:00
Jakob Ackermann
b563deef14
[misc] bump the version of the metrics module to 3.5.1
2021-03-22 12:59:43 +00:00
Eric Mc Sween
7a6b86185b
Merge pull request #215 from overleaf/ho-clsi-cookie-post
...
support post and get for getting status of clsi project
2021-03-22 07:56:09 -04:00
Christopher Hoskin
6ca016bb05
Explain the situation with permissions on Linux
2021-03-19 14:32:54 +00:00
Alf Eaton
268a144e43
Merge remote-tracking branch 'origin/master' into ae-readme
2021-03-19 12:06:03 +00:00
Alf Eaton
7e3edcee2c
Add instructions for Linux
2021-03-19 12:05:22 +00:00
Alf Eaton
d4b4cb399a
Revert "Make TEXLIVE_IMAGE_USER instruction macOS only"
...
This reverts commit ab6fe228cadd3329885fdc60aea867d601579759.
2021-03-19 12:00:47 +00:00
Shane Kilkelly
072d0a9dab
Add flag to qpdf, to preserve PDF/A compliance
2021-03-19 11:35:59 +00:00
Christopher Hoskin
34b572de9e
Merge branch 'ae-readme' of github.com:overleaf/clsi into ae-readme
2021-03-19 10:35:25 +00:00
Christopher Hoskin
638e72295b
Add link to \openout primitive docs
2021-03-19 10:34:52 +00:00
Alf Eaton
984766a9fb
Make TEXLIVE_IMAGE_USER instruction macOS only
2021-03-18 09:55:31 +00:00
Christopher Hoskin
b8d0389a53
README typos
2021-03-16 15:29:53 +00:00
Christopher Hoskin
0a89b6537c
Fill in missing text for environment variables
2021-03-16 12:30:06 +00:00
Christopher Hoskin
17a83bc169
Expand list of environment variables
2021-03-16 12:00:48 +00:00
Henry Oswald
7cd36e139a
support post and get for getting status of clsi project
2021-03-16 10:24:59 +00:00
Eric Mc Sween
3eca505319
Upgrade to Node 12
2021-03-12 16:27:33 -05:00
Eric Mc Sween
9f2d219102
Add a global test setup file
...
Set up generally useful stuff:
* chai.should()
* logger stubs
* globals in SandboxedModule, including Buffer and process, which are
now required in Node 12
2021-03-12 16:27:33 -05:00
Alf Eaton
f38cef6e56
Change settings file .coffee to .js
2021-03-11 11:50:47 +00:00
Alf Eaton
9c596fb17d
Update README.md
2021-03-10 22:38:34 +00:00
Alf Eaton
e7be75a4b7
Update README.md
2021-03-10 22:36:33 +00:00
Alf Eaton
18dbb12434
Add /app/output
2021-02-18 15:33:16 +00:00
Alf Eaton
498ca80cd3
Ensure that app folders exist before running chown
2021-02-18 15:09:48 +00:00
Brian Gough
bad5fd20a8
Merge pull request #208 from overleaf/bg-fix-use-output-dir
...
fix for exception in "decaff cleanup ResourceStateManager"
2021-02-01 11:24:19 +00:00
Brian Gough
d0d36bf9d8
Merge pull request #206 from overleaf/bg-add-missing-unit-test
...
add unit test for non-existent state file
2021-02-01 11:23:56 +00:00
Brian Gough
f0b4f1238b
provide a /oops-internal endpoint for testing uncaughtExceptions
2021-01-26 16:35:39 +00:00
Brian Gough
fc11574698
add uncaughtException handler
2021-01-26 14:08:29 +00:00
Brian Gough
865e68051e
include fallback for missing state file
2021-01-26 13:59:28 +00:00
Brian Gough
8946f87ca6
Merge branch 'bg-add-missing-unit-test' into bg-fix-use-output-dir
2021-01-26 11:09:00 +00:00
Brian Gough
a8e47da9e9
Revert "Merge pull request #205 from overleaf/revert-200-bg-decaff-cleanup"
...
This reverts commit 76d8d3181b9464d1e1bbc713a2729ca269d9c047, reversing
changes made to 31a8dc3a98d73c2707d633712f0ef7207013e78b.
2021-01-26 11:04:33 +00:00
Brian Gough
a86e521ac0
add unit test for non-existent state file
2021-01-26 11:03:18 +00:00
Brian Gough
9dc55729ac
Revert "decaff cleanup ResourceStateManager"
2021-01-25 15:26:53 +00:00
Brian Gough
f41f3aba8c
Merge pull request #204 from overleaf/bg-use-output-dir
...
use separate output dir for generated files
2021-01-25 14:10:09 +00:00
Brian Gough
90c0d931ad
Merge pull request #201 from overleaf/bg-more-cleanup
...
decaff cleanup of OutputFileManager
2021-01-25 14:09:46 +00:00
Brian Gough
c67c8a6c80
Merge pull request #200 from overleaf/bg-decaff-cleanup
...
decaff cleanup ResourceStateManager
2021-01-25 14:09:24 +00:00
Brian Gough
e52871e07b
Merge pull request #199 from overleaf/dependabot-npm_and_yarn-ini-1.3.8
...
Bump ini from 1.3.5 to 1.3.8
2021-01-25 14:09:02 +00:00
Brian Gough
6fa081522d
add a warning for requests without build id
2021-01-22 11:05:52 +00:00
Brian Gough
b5346658b0
clear output directory when clearing project
2021-01-22 11:05:52 +00:00
Brian Gough
565cd53eb5
add git ignore for output directory
2021-01-22 11:05:52 +00:00
Brian Gough
692dbc8d6b
add output directory
2021-01-22 11:05:52 +00:00
Brian Gough
bdbfe70086
rename staticServer to staticCompileServer
2021-01-22 11:05:52 +00:00
Jakob Ackermann
96b8d001f7
Merge pull request #202 from overleaf/jpa-fix-output-pdf-check
...
[misc] CompileController: exact match for output.pdf
2021-01-13 09:35:31 +00:00
Jakob Ackermann
cac18ac4b9
[misc] bump the node version to 10.23.1
2021-01-05 18:32:01 +00:00
Jakob Ackermann
49b764a308
[misc] CompileController: exact match for output.pdf
...
The previous regex could mistake user provided pdf files, like
`fake_output.pdf`, as the final output file.
The frontend expects to find a `output.pdf` file on success.
2020-12-28 13:16:31 +00:00
Brian Gough
e183fffa4b
format fix
2020-12-18 16:15:06 +00:00
Brian Gough
dafe69b5e6
remove unused module var
2020-12-18 16:15:06 +00:00
Brian Gough
36b646bb84
remove comments
2020-12-18 16:15:06 +00:00
Brian Gough
f20a0d1a69
remove unused var
2020-12-18 16:15:03 +00:00
Brian Gough
df1caa14b8
remove unused require
2020-12-18 16:14:36 +00:00
Brian Gough
ca98ee5cff
use a Set instead of an Object
2020-12-18 16:14:09 +00:00
Brian Gough
43b0429c28
clean up relative path checking
2020-12-18 16:14:05 +00:00
Brian Gough
73e09ff99f
remove unnecessary requires
2020-12-18 15:28:06 +00:00
Brian Gough
cee93a0cd9
clean up unnecessary var
2020-12-18 15:28:06 +00:00
Brian Gough
4169f7fc73
use once for callback
2020-12-18 15:28:06 +00:00
Brian Gough
e84d6305f3
remove unnecessary return
2020-12-18 15:16:07 +00:00
Brian Gough
ae064e8997
lint fix
2020-12-18 15:11:29 +00:00
Brian Gough
323890cedb
remove Array.from
2020-12-18 15:10:53 +00:00
Brian Gough
c044db4897
remove guard helper
2020-12-18 15:09:48 +00:00
Brian Gough
0bae53c7f3
remove unnecessary returns
2020-12-18 15:02:40 +00:00
Brian Gough
eaec57cd01
simplify null check
2020-12-18 15:01:25 +00:00
Brian Gough
411d69e36b
remove unnecessary callback code
2020-12-18 14:59:48 +00:00
Brian Gough
11ef3c27ed
use Set instead of object
2020-12-18 14:56:53 +00:00
Brian Gough
64ea22d259
remove unnecessary null checks
2020-12-17 12:05:17 +00:00
Brian Gough
299d7cc5fe
remove unnecessary returns
2020-12-17 12:01:14 +00:00
Brian Gough
5a539b7f90
remove guard function
2020-12-17 11:59:15 +00:00
Brian Gough
4d8e4d54e5
remove Array.from
2020-12-17 11:49:02 +00:00
dependabot[bot]
f5594e023f
Bump ini from 1.3.5 to 1.3.8
...
Bumps [ini](https://github.com/isaacs/ini ) from 1.3.5 to 1.3.8.
- [Release notes](https://github.com/isaacs/ini/releases )
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.8 )
Signed-off-by: dependabot[bot] <support@github.com>
2020-12-12 11:02:53 +00:00
Jakob Ackermann
5d5241156c
[misc] install settings-sharelatex from npm
2020-12-02 12:26:42 +00:00
Jakob Ackermann
1b73c6dad2
[misc] work around missing stubs
2020-11-25 13:33:54 +00:00
Jakob Ackermann
7b7cd8cc8c
[misc] bump metrics module to 3.4.1
...
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
2020-11-25 11:57:25 +00:00
Eric Mc Sween
c7fa34a6a8
Upgrade build-scripts to 3.4.0
...
This version fixes docker-compose health checks for dependent services. See
https://github.com/overleaf/dev-environment/pull/409 for details.
2020-11-24 08:03:28 -05:00
Shane Kilkelly
fcc52cb1a0
No more blank line
2020-10-27 16:17:22 +00:00
Shane Kilkelly
aaa0681c39
In example request, show example using curl
...
This clarifies the exact way to construct this request. It seems that
some users have taken the existing documentation to mean that the
request should be sent from the browser, to the web server. This example
clarifies how to send this compile request directly to the clsi server.
2020-10-27 15:53:10 +00:00
Eric Mc Sween
20a0f3bcd1
Merge pull request #193 from overleaf/dependabot-npm_and_yarn-node-fetch-2.6.1
...
Bump node-fetch from 2.6.0 to 2.6.1
2020-09-16 12:48:21 -04:00
Eric Mc Sween
40f38b09f0
Merge pull request #191 from overleaf/dependabot-npm_and_yarn-bl-4.0.3
...
Bump bl from 4.0.1 to 4.0.3
2020-09-16 12:48:15 -04:00
Eric Mc Sween
6c73eb5e34
Merge pull request #190 from overleaf/dependabot-npm_and_yarn-lodash-4.17.20
...
Bump lodash from 4.17.15 to 4.17.20
2020-09-16 12:48:07 -04:00
Eric Mc Sween
5b11dc5a77
Bump Node version to 10.22.1
2020-09-16 12:24:42 -04:00
dependabot[bot]
f85e2dec5c
Bump node-fetch from 2.6.0 to 2.6.1
...
Bumps [node-fetch](https://github.com/bitinn/node-fetch ) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/bitinn/node-fetch/releases )
- [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md )
- [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1 )
Signed-off-by: dependabot[bot] <support@github.com>
2020-09-12 17:04:49 +00:00
Eric Mc Sween
14e1e02a68
Test anonymous volumes are removed with containers
2020-09-04 11:34:08 -04:00
Eric Mc Sween
1c13f6fe94
Mount /home/tex in an anonymous volume
...
When we mount the container's root filesystem as read-only, mount
an anonymous volume in /home/tex so that it's writable. Our TeX Live
images have cached content in /home/tex. This content will automatically
get copied by Docker into this anonymous volume.
2020-09-04 08:49:56 -04:00
Eric Mc Sween
67f4a6eeeb
Decaf cleanup: normalize functions
...
Use function keyword for declarations and arrow functions for callbacks.
2020-09-03 15:58:16 -04:00
Eric Mc Sween
5cd889038e
Use _.once() instead of ad hoc implementation
2020-09-03 15:52:09 -04:00
Eric Mc Sween
a853950a99
Fix container monitor cleanup function
...
The intent here is clearly to clear both the timeout and the interval.
2020-09-03 15:50:45 -04:00
Eric Mc Sween
30a44edded
Decaf cleanup: simplify variable declarations
2020-09-03 15:50:12 -04:00
Eric Mc Sween
a282bccd48
Do not instantiate errors at module load time
...
This prevents the right stack trace from being captured.
2020-09-03 15:34:19 -04:00
Eric Mc Sween
4905e7db2c
Decaf cleanup: unused vars
2020-09-03 15:30:07 -04:00