github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
897 commits 3 branches 7 tags 206 MiB
Commit graph

803 commits

Author SHA1 Message Date
Eric Mc Sween
14e1e02a68 Test anonymous volumes are removed with containers 2020-09-04 11:34:08 -04:00
Eric Mc Sween
1c13f6fe94 Mount /home/tex in an anonymous volume 
When we mount the container's root filesystem as read-only, mount
an anonymous volume in /home/tex so that it's writable. Our TeX Live
images have cached content in /home/tex. This content will automatically
get copied by Docker into this anonymous volume.
 2020-09-04 08:49:56 -04:00
Eric Mc Sween
67f4a6eeeb Decaf cleanup: normalize functions 
Use function keyword for declarations and arrow functions for callbacks.
 2020-09-03 15:58:16 -04:00
Eric Mc Sween
5cd889038e Use _.once() instead of ad hoc implementation 2020-09-03 15:52:09 -04:00
Eric Mc Sween
a853950a99 Fix container monitor cleanup function 
The intent here is clearly to clear both the timeout and the interval.
 2020-09-03 15:50:45 -04:00
Eric Mc Sween
30a44edded Decaf cleanup: simplify variable declarations 2020-09-03 15:50:12 -04:00
Eric Mc Sween
a282bccd48 Do not instantiate errors at module load time 
This prevents the right stack trace from being captured.
 2020-09-03 15:34:19 -04:00
Eric Mc Sween
4905e7db2c Decaf cleanup: unused vars 2020-09-03 15:30:07 -04:00
Eric Mc Sween
44bf38d6db Decaf cleanup: convert async function to sync 
The examineOldContainer() function doesn't need to use callbacks since
it only does synchronous work.
 2020-09-03 15:13:04 -04:00
Eric Mc Sween
2584586ba2 Decaf cleanup: camel-case variables 2020-09-03 15:13:04 -04:00
Eric Mc Sween
c52d7d8f02 Decaf cleanup: simplify null checks 2020-09-03 15:13:04 -04:00
Eric Mc Sween
ee4c08868c Decaf cleanup: remove default callbacks 2020-09-03 15:13:04 -04:00
Eric Mc Sween
32f0bbe266 Decaf cleanup: remove IIFEs 2020-09-03 15:13:04 -04:00
Eric Mc Sween
f650da8675 Decaf cleanup: remove __guard__ 2020-09-02 17:06:35 -04:00
Eric Mc Sween
99648341e2 Decaf cleanup: remove unnecessary returns 2020-09-02 16:58:41 -04:00
Eric Mc Sween
08be54a43e Decaf cleanup: remove unnecessary Array.from() 2020-09-02 16:45:49 -04:00
dependabot[bot]
6b93bbfe09 Bump bl from 4.0.1 to 4.0.3 
Bumps [bl](https://github.com/rvagg/bl) from 4.0.1 to 4.0.3.
- [Release notes](https://github.com/rvagg/bl/releases)
- [Commits](https://github.com/rvagg/bl/compare/v4.0.1...v4.0.3)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-09-02 16:50:44 +00:00
dependabot[bot]
7caf7f4201 Bump lodash from 4.17.15 to 4.17.20 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.20.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.20)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-08-14 05:11:31 +00:00
Christopher Hoskin
b1b2a1c134 [misc] bump logger-sharelatex to version 2.2.0 2020-08-12 15:11:21 +01:00
Jakob Ackermann
1d1b9ebebc [misc] bump the dev-env to 3.3.2 2020-08-10 17:01:11 +01:00
Jakob Ackermann
5846ebb367 Merge pull request #182 from overleaf/msm-fix-npe-community-edition 
Fixed NPE when Settings.clsi is defined but Settings.clsi.docker is not
 2020-07-15 11:01:08 +02:00
Jakob Ackermann
0f9d0d9790 Merge pull request #183 from overleaf/jpa-clsi-allowed-image-names 
[misc] RequestParser: restrict imageName to an allow list and add tests
 2020-07-15 10:58:36 +02:00
Jakob Ackermann
4489a080fd Merge pull request #186 from overleaf/jpa-import-132 
[LocalCommandRunner] run: block a double call of the callback
 2020-07-15 10:57:46 +02:00
Jakob Ackermann
c3a3ed341b Merge pull request #132 from das7pad/hotfix-double-call 
[LocalCommandRunner] run: block a double call of the callback
 2020-07-03 12:58:25 +02:00
Jakob Ackermann
cd87d3018e [misc] fix formatting 2020-07-03 11:47:53 +01:00
Jakob Ackermann
a5c3bad7f1 [ExampleDocumentTests] drop out in case of an error during compilation 
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2020-07-03 11:38:12 +01:00
Jakob Ackermann
f703b2ca41 [misc] move the image check prior to the base image override 2020-07-01 10:01:25 +01:00
Jakob Ackermann
aa02df7b81 [misc] apply review feedback 
- move setting into clsi.docker namespace
- rename the variable for images to allowedImages / ALLOWED_IMAGES
- add an additional check for the image name into the DockerRunner

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
 2020-06-30 12:01:21 +01:00
Jakob Ackermann
c857371fed [misc] wordcount: restrict image to an allow list and add tests 2020-06-26 13:28:12 +01:00
Jakob Ackermann
8846efe7ce [misc] RequestParser: restrict imageName to an allow list and add tests 2020-06-26 13:28:09 +01:00
Miguel Serrano
f8ca906393 Fixed NPE when Settings.clsi is defined but Settings.clsi.docker is not 2020-06-25 12:31:10 +02:00
Brian Gough
2b342c6e53 Merge pull request #181 from overleaf/bg-fix-503-response 
handle EPIPE errors in CompileController
 2020-06-22 09:30:35 +01:00
Brian Gough
22480f1a52 handle EPIPE errors in CompileController 2020-06-18 09:54:18 +01:00
Brian Gough
bdebef91eb Merge branch 'master' into bg-add-compile-groups 2020-06-17 11:58:26 +01:00
Brian Gough
f6fddf2944 Merge pull request #179 from overleaf/bg-fix-synctex-error 
fix synctex error
 2020-06-16 08:57:54 +01:00
Brian Gough
2eb4327256 fix format 2020-06-16 08:45:53 +01:00
Brian Gough
f41b4daca2 Merge branch 'master' into bg-use-lodash 2020-06-16 08:35:17 +01:00
Brian Gough
634fad8c6e Merge pull request #176 from overleaf/ta-epipe-retry-revert 
Remove Retries in EPIPE Errors
 2020-06-16 08:33:30 +01:00
Brian Gough
85f4f348dd add default settings to remove wordcount and synctex containers 2020-06-15 15:49:38 +01:00
Brian Gough
2ce03f0554 add initial compileGroup support 2020-06-15 15:28:53 +01:00
Brian Gough
0914908676 downgrade NotFoundError log-level 2020-06-15 11:06:54 +01:00
Brian Gough
262ea01911 add acceptance test for synctex when project/file does not exist 2020-06-15 10:55:01 +01:00
Brian Gough
ede70b6f99 migrate from underscore to lodash 2020-06-15 09:52:21 +01:00
Brian Gough
f99023320d use json parsing in request 2020-06-12 15:15:51 +01:00
Brian Gough
5368630754 check output file exists before running synctex 2020-06-12 15:15:27 +01:00
Brian Gough
8fadf9b241 Merge pull request #175 from overleaf/bg-503-on-unavailable 
send 503 unavailable response on EPIPE
 2020-06-12 09:29:04 +01:00
Brian Gough
69b7ecfd2e Merge pull request #177 from overleaf/bg-add-docker-setting 
add missing setting for optimiseInDocker
 2020-06-12 09:28:31 +01:00
Tim Alby
d3ff214b88 partially revert "[DockerRunner] fix metric incrementing and error logging" 
This reverts commits:
- 2b2fcca39ce8dee0fdc0c342aa0d6c822592bcec
- 9e82ab0890c5cc8c7fb95362c3f7edbcaad0cf29
- e3da458b376871c3ce72d6984d14bf1ee668b04b
 2020-06-11 12:51:26 +02:00
Brian Gough
f239fc3b5d send 503 unavailable response on EPIPE 2020-06-11 11:12:02 +01:00
Brian Gough
6c98c7af4a error on missing profile 2020-06-10 11:42:07 +01:00
Brian Gough
54d1cf7eeb add missing setting for optimiseInDocker 2020-06-09 11:22:28 +01:00
Brian Gough
8464ce33a1 Merge pull request #173 from overleaf/bg-openout-any 
add setting TEXLIVE_OPENOUT_ANY
 2020-06-08 09:03:05 +01:00
Brian Gough
3d6b911295 add setting TEXLIVE_OPENOUT_ANY 2020-06-04 11:47:22 +01:00
Brian Gough
b64d68bc9f update buildscript.txt to node 10.21.0 2020-06-03 11:11:51 +01:00
Brian Gough
8b8acfed4f update to node 10.21.0 2020-06-03 10:22:31 +01:00
Brian Gough
ce1f2e2bd4 fix broken unit test 2020-06-02 11:12:57 +01:00
Brian Gough
aa44bae222 fix eslint errors 2020-06-02 09:51:34 +01:00
Brian Gough
0de2933812 fix unreachable code lint error 2020-06-02 09:28:04 +01:00
Brian Gough
a3d27b63f1 fix formatting with make format_fix 2020-06-02 09:18:38 +01:00
Brian Gough
d9265b779f Merge pull request #170 from overleaf/jpa-import-141 
[DockerRunner] destroyOldContainers: fix a race confition
 2020-06-02 09:04:59 +01:00
Brian Gough
59cd14e160 Merge pull request #169 from overleaf/bg-record-latexmk-output 
record latexmk output
 2020-06-02 09:03:43 +01:00
Jakob Ackermann
563a973388 [DockerRunner] destroyOldContainers: normalize the container name 
The docker api returns each name with a `/` prefix.

In order to not interfere with pending compiles, the deletion process
 has to acquire an internal lock on the container. The LockManager uses
 the plain container name without the slash: `project-xxx`.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2020-05-29 11:28:26 +01:00
Brian Gough
6d5dfb7758 clean up log file deletion and add unit test 2020-05-20 14:12:08 +01:00
Brian Gough
a684619bce add unit tests 2020-05-20 11:52:53 +01:00
Brian Gough
63770bf390 clean up the stdout/stderr recording 2020-05-20 11:45:29 +01:00
Henry Oswald
5c3da905ba Merge pull request #162 from overleaf/ta-jpa-epipe-retry 
[DockerRunner] retry container inspect on EPIPE
 2020-05-19 11:15:25 +01:00
Jakob Ackermann
36e81cbe15 [misc] apply review feedback 2020-05-19 10:30:59 +01:00
Jakob Ackermann
2a3c2dd3d5 [misc] simplify the smoke test and process shutdown 2020-05-19 10:30:59 +01:00
Henry Oswald
2547725f35 Merge pull request #164 from overleaf/bg-fix-buffer-deprecations 
fix deprecated usage of Buffer constructor
 2020-05-19 10:26:56 +01:00
Henry Oswald
e9c6f3b074 Merge pull request #163 from overleaf/bg-use-encoding-on-process-output 
set encoding when reading from streams
 2020-05-19 10:26:26 +01:00
Henry Oswald
457b7d6657 Merge pull request #165 from overleaf/ho-retry-url-downloads 
add pipeUrlToFileWithRetry for file downloads
 2020-05-19 10:25:19 +01:00
Henry Oswald
3925839c7f add refreshExpiryTimeout function 
on clsi all data lives inside of / dir
dynamically reduce size of EXPIRY_TIMEOUT if disk starts to get full
 2020-05-18 15:17:19 +01:00
Brian Gough
3db513cfc9 record latexmk output 2020-05-15 16:08:10 +01:00
Henry Oswald
77e8ba74a7 add pipeUrlToFileWithRetry function to retry file downloads 3 times 2020-05-14 13:24:58 +01:00
Brian Gough
4947abe88b fix deprecated usage of Buffer constructor 2020-05-07 10:42:05 +01:00
Brian Gough
8e86f02c43 set encoding when reading from streams 
using .toString() works most of the time but can lead to utf8 characters being
broken across chunk boundaries.

https://nodejs.org/api/stream.html#stream_readable_setencoding_encoding
 2020-05-07 10:30:14 +01:00
Henry Oswald
e1c90ec234 Merge branch 'master' into bg-support-other-runtimes 
port coffee changes to js
 2020-04-23 13:59:05 +01:00
Henry Oswald
a5887fad50 Merge pull request #158 from overleaf/jpa-entry-point-cleanup 
[misc] entry point cleanup
 2020-04-23 13:52:04 +01:00
Henry Oswald
ea1ff96e0e Merge pull request #155 from overleaf/ho-limit-lifespan 
limit clsi lifespan via health checks and PROCESS_LIFE_SPAN_LIMIT_MS
 2020-04-23 13:51:09 +01:00
Henry Oswald
adc73df53d cleanup the shutdown code a bit 2020-04-23 11:32:33 +01:00
Jakob Ackermann
cf8533bee1 [LocalCommandRunner] run: block a double call of the callback 
The subprocess event handler fires the "error" and "close" event in case
 of a failure.
Both events would call the given callback, resulting in double
 processing of the subprocess result downstream.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2020-04-16 15:55:55 +02:00
Henry Oswald
acbfc465dc add variance into shutdown time to avoid stampeed 2020-04-14 13:36:11 +01:00
Jakob Ackermann
cb9e9321f0 [DockerRunner] fix metric incrementing and error logging 
- do not log on first EPIPE
- inc 'container-inspect-epipe-error' on permanent error only

Co-Authored-By: Tim Alby <timothee.alby@gmail.com>
 2020-04-10 14:44:57 +02:00
Tim Alby
ecaa7035f5 add metrics for EPIPE errors 
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
 2020-04-10 12:28:48 +02:00
Tim Alby
e3b1472c7f retry once on EPIPE errors 
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
 2020-04-10 12:28:11 +02:00
Tim Alby
66447211d0 fix arguments order 
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
 2020-04-10 12:27:15 +02:00
Jakob Ackermann
6a615d50d3 Merge remote-tracking branch 'origin/master' into jpa-entry-point-cleanup 2020-04-03 14:05:36 +02:00
Jakob Ackermann
c700a0ab9d Merge pull request #159 from overleaf/jpa-fix-docker-modem-error-scheme 
[misc] keep up with the error signature of dockerode/docker-modem
 2020-04-03 14:04:06 +02:00
Jakob Ackermann
bb9e8bb6f5 Merge pull request #157 from overleaf/jpa-download-failed-metric 
[misc] add a metric for failing downloads
 2020-04-03 14:03:16 +02:00
Jakob Ackermann
3b10f80adb Merge pull request #156 from overleaf/jpa-logger-1-9-1 
[misc] bump logger-sharelatex to 1.9.1
 2020-04-03 14:03:04 +02:00
Jakob Ackermann
101c4ea4dc [misc] use a directory in /tmp for temporary data 2020-04-03 12:36:32 +02:00
Jakob Ackermann
b9deec1095 [misc] narrow down write access/ownership for the run-time user 2020-04-03 12:36:32 +02:00
Jakob Ackermann
3309adf2ad [misc] move the sqlite database into a db/ directory 2020-04-03 12:36:31 +02:00
Jakob Ackermann
5b1656b884 [misc] drop debug output and log docker version on stderr 2020-04-03 12:36:31 +02:00
Jakob Ackermann
6777432569 [misc] add a metric for failing downloads 2020-04-03 12:35:40 +02:00
Jakob Ackermann
6b02921409 [misc] bump the build-scripts to version 2.1.0 
This will put acceptance and unit tests in own namespaces so that they
 can run and be teared down individually.
 2020-04-03 12:18:09 +02:00
Henry Oswald
87188e0830 Merge branch 'master' of github.com:overleaf/clsi into ho-limit-lifespan 2020-03-31 09:52:37 +01:00
Jakob Ackermann
4a19f022fc [misc] keep up with the error signature of dockerode/docker-modem 
https://github.com/apocas/docker-modem/blob/v2.1.1/lib/modem.js#L296
 2020-03-27 10:39:45 +01:00
Jakob Ackermann
ac5feb4a77 [misc] bump logger-sharelatex to 1.9.1 2020-03-23 16:18:07 +01:00
Henry Oswald
3e07af3f1b limit clsi lifespan via health checks and PROCESS_LIFE_SPAN_LIMIT_MS 2020-03-20 13:37:58 +00:00
Miguel Serrano
77cc3b3cc6 Merge pull request #154 from overleaf/msm-dependency-update 
Dependency update
 2020-03-19 16:12:51 +01:00
mserranom
8641d0ef0f updated sandboxed-module, chai and metrics-sharelatex 2020-03-16 17:14:04 +01:00
mserranom
d5bd1790a8 updated sequelize 2020-03-16 16:31:02 +01:00
mserranom
7e7d234350 updated mocha and sinon, fixed tests 2020-03-12 10:35:11 +01:00
mserranom
cf6c8ab496 removed mkdirp dependency and replaced with fs.mkdir 2020-03-12 10:22:08 +01:00
mserranom
35f69e4f8b updated mkdirp 2020-03-11 19:39:08 +01:00
mserranom
a0b9d74a25 updated dockerode, heapdump, lyns and fs-extra 2020-03-11 19:25:50 +01:00
mserranom
9883c0e3ff updated minor/patch dependencies 2020-03-11 12:20:48 +01:00
Henry Oswald
052b5624d3 copy synctex over to /app/bin/synctex-mount in entrypoint 2020-03-11 11:06:46 +00:00
mserranom
a18ddffce9 npm audit fix 2020-03-11 11:58:42 +01:00
mserranom
44ec451b11 updated build-scripts 2020-03-11 11:55:44 +01:00
Henry Oswald
01c46e9e94 remove ./bin/install_texlive_gce.sh which shouldn't be needed 
we shouldn't have needed this for a while, I think it is a cause of
startup delay, however this should have stopped other missing texlive
images in the past which is strange
 2020-03-11 10:06:55 +00:00
Simon Detheridge
c2a940e1fe Merge pull request #150 from overleaf/jpa-dependencies-cleanup 
[misc] dependencies cleanup
 2020-02-25 17:47:02 +00:00
mserranom
41533d8888 moved decaffeinated files to js folder 2020-02-25 17:15:31 +01:00
decaffeinate
252f4c704b decaffeinate: Run post-processing cleanups on loadTest.coffee 2020-02-25 17:03:58 +01:00
decaffeinate
3901fe9cd2 decaffeinate: Convert loadTest.coffee to JS 2020-02-25 17:03:57 +01:00
decaffeinate
e20da38e09 decaffeinate: Rename loadTest.coffee from .coffee to .js 2020-02-25 17:03:57 +01:00
decaffeinate
2057a565ec decaffeinate: Run post-processing cleanups on SmokeTests.coffee 2020-02-25 17:03:39 +01:00
decaffeinate
b6991f5ff9 decaffeinate: Convert SmokeTests.coffee to JS 2020-02-25 17:03:37 +01:00
decaffeinate
92ed86bb33 decaffeinate: Rename SmokeTests.coffee from .coffee to .js 2020-02-25 17:03:36 +01:00
Jakob Ackermann
6155c8a097 [misc] cleanup unused dependency on mongo and redis 2020-02-25 14:45:09 +01:00
Jakob Ackermann
a05c048b3b [misc] rename npm-shrinkwrap.json to package-lock.json 2020-02-25 14:45:09 +01:00
mserranom
fa37ed865a added container monitor cleanup to fix hanging tests 2020-02-20 17:24:28 +01:00
mserranom
4c0fd2b6f4 fixed test paths 2020-02-19 12:38:54 +01:00
mserranom
a5c7051fc9 prettier: convert individual decaffeinated files to Prettier format 2020-02-19 12:16:18 +01:00
mserranom
d23250a4bb decaffeinate: convert individual files to js 2020-02-19 12:16:17 +01:00
mserranom
66ce5847a3 decaffeinate: rename individual coffee files to js files 2020-02-19 12:16:15 +01:00
mserranom
778f2a8420 prettier: convert test/acceptance decaffeinated files to Prettier format 2020-02-19 12:16:14 +01:00
mserranom
c5a6496b71 decaffeinate: rename test/acceptance/coffee to test/acceptance/js 2020-02-19 12:16:11 +01:00
decaffeinate
d1da978e39 decaffeinate: Run post-processing cleanups on BrokenLatexFileTests.coffee and 9 other files 2020-02-19 12:16:07 +01:00
decaffeinate
3d3861cb24 decaffeinate: Convert BrokenLatexFileTests.coffee and 9 other files to JS 2020-02-19 12:16:00 +01:00
decaffeinate
035786b204 decaffeinate: Rename BrokenLatexFileTests.coffee and 9 other files from .coffee to .js 2020-02-19 12:15:54 +01:00
mserranom
a62d8186e9 prettier: convert test/unit decaffeinated files to Prettier format 2020-02-19 12:15:37 +01:00
mserranom
89360bfe77 decaffeinate: rename test/unit/coffee to test/unit/js 2020-02-19 12:15:30 +01:00
decaffeinate
4c79f22389 decaffeinate: Run post-processing cleanups on CompileControllerTests.coffee and 17 other files 2020-02-19 12:15:25 +01:00
decaffeinate
37b4e96de4 decaffeinate: Convert CompileControllerTests.coffee and 17 other files to JS 2020-02-19 12:15:08 +01:00
decaffeinate
e57097afb4 decaffeinate: Rename CompileControllerTests.coffee and 17 other files from .coffee to .js 2020-02-19 12:14:56 +01:00
mserranom
8729acd48c prettier: convert app/js decaffeinated files to Prettier format 2020-02-19 12:14:37 +01:00
mserranom
3af6bdd588 decaffeinate: rename app/coffee dir to app/js 2020-02-19 12:14:34 +01:00
decaffeinate
beb6100e25 decaffeinate: Run post-processing cleanups on CommandRunner.coffee and 25 other files 2020-02-19 12:14:28 +01:00
decaffeinate
4eab36e3d5 decaffeinate: Convert CommandRunner.coffee and 25 other files to JS 2020-02-19 12:14:14 +01:00
decaffeinate
2d07bab23d decaffeinate: Rename CommandRunner.coffee and 25 other files from .coffee to .js 2020-02-19 12:14:01 +01:00
mserranom
66ffcb626f decaffeinate: add eslint and prettier packages 2020-02-19 12:13:44 +01:00
mserranom
2d4ea9febf decaffeinate: update .gitignore 2020-02-19 12:11:31 +01:00
mserranom
564707b1d1 decaffeinate: update build scripts to es 2020-02-19 12:10:00 +01:00
mserranom
e37c261bb0 removed unneeded default function arg preventing from decaffeination 2020-02-19 12:06:09 +01:00
Brian Gough
5a28cbad29 remove unused .travis.yml file 2020-02-12 12:37:00 +00:00
Jakob Ackermann
b064fede63 [misc] update the build scripts to 1.3.5 2020-02-11 12:27:59 +01:00
Brian Gough
659cd44377 support other runtimes 2020-02-10 16:10:18 +00:00
Jakob Ackermann
0ab94f96c1 [misc] use node:10.19.0 as base image 
Also adjust the node version in the other build-script files.
 2020-02-07 14:49:42 +01:00