Commit graph

779 commits

Author SHA1 Message Date
Brian Gough
cee93a0cd9 clean up unnecessary var 2020-12-18 15:28:06 +00:00
Brian Gough
4169f7fc73 use once for callback 2020-12-18 15:28:06 +00:00
Brian Gough
e84d6305f3 remove unnecessary return 2020-12-18 15:16:07 +00:00
Brian Gough
ae064e8997 lint fix 2020-12-18 15:11:29 +00:00
Brian Gough
323890cedb remove Array.from 2020-12-18 15:10:53 +00:00
Brian Gough
c044db4897 remove guard helper 2020-12-18 15:09:48 +00:00
Brian Gough
0bae53c7f3 remove unnecessary returns 2020-12-18 15:02:40 +00:00
Brian Gough
eaec57cd01 simplify null check 2020-12-18 15:01:25 +00:00
Brian Gough
411d69e36b remove unnecessary callback code 2020-12-18 14:59:48 +00:00
Brian Gough
11ef3c27ed use Set instead of object 2020-12-18 14:56:53 +00:00
Brian Gough
64ea22d259 remove unnecessary null checks 2020-12-17 12:05:17 +00:00
Brian Gough
299d7cc5fe remove unnecessary returns 2020-12-17 12:01:14 +00:00
Brian Gough
5a539b7f90 remove guard function 2020-12-17 11:59:15 +00:00
Brian Gough
4d8e4d54e5 remove Array.from 2020-12-17 11:49:02 +00:00
dependabot[bot]
f5594e023f Bump ini from 1.3.5 to 1.3.8
Bumps [ini](https://github.com/isaacs/ini) from 1.3.5 to 1.3.8.
- [Release notes](https://github.com/isaacs/ini/releases)
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.8)

Signed-off-by: dependabot[bot] <support@github.com>
2020-12-12 11:02:53 +00:00
Jakob Ackermann
5d5241156c [misc] install settings-sharelatex from npm 2020-12-02 12:26:42 +00:00
Jakob Ackermann
1b73c6dad2 [misc] work around missing stubs 2020-11-25 13:33:54 +00:00
Jakob Ackermann
7b7cd8cc8c [misc] bump metrics module to 3.4.1
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
2020-11-25 11:57:25 +00:00
Eric Mc Sween
c7fa34a6a8 Upgrade build-scripts to 3.4.0
This version fixes docker-compose health checks for dependent services. See
https://github.com/overleaf/dev-environment/pull/409 for details.
2020-11-24 08:03:28 -05:00
Shane Kilkelly
fcc52cb1a0 No more blank line 2020-10-27 16:17:22 +00:00
Shane Kilkelly
aaa0681c39 In example request, show example using curl
This clarifies the exact way to construct this request. It seems that
some users have taken the existing documentation to mean that the
request should be sent from the browser, to the web server. This example
clarifies how to send this compile request directly to the clsi server.
2020-10-27 15:53:10 +00:00
Eric Mc Sween
20a0f3bcd1 Merge pull request #193 from overleaf/dependabot-npm_and_yarn-node-fetch-2.6.1
Bump node-fetch from 2.6.0 to 2.6.1
2020-09-16 12:48:21 -04:00
Eric Mc Sween
40f38b09f0 Merge pull request #191 from overleaf/dependabot-npm_and_yarn-bl-4.0.3
Bump bl from 4.0.1 to 4.0.3
2020-09-16 12:48:15 -04:00
Eric Mc Sween
6c73eb5e34 Merge pull request #190 from overleaf/dependabot-npm_and_yarn-lodash-4.17.20
Bump lodash from 4.17.15 to 4.17.20
2020-09-16 12:48:07 -04:00
Eric Mc Sween
5b11dc5a77 Bump Node version to 10.22.1 2020-09-16 12:24:42 -04:00
dependabot[bot]
f85e2dec5c Bump node-fetch from 2.6.0 to 2.6.1
Bumps [node-fetch](https://github.com/bitinn/node-fetch) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/bitinn/node-fetch/releases)
- [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-12 17:04:49 +00:00
Eric Mc Sween
14e1e02a68 Test anonymous volumes are removed with containers 2020-09-04 11:34:08 -04:00
Eric Mc Sween
1c13f6fe94 Mount /home/tex in an anonymous volume
When we mount the container's root filesystem as read-only, mount
an anonymous volume in /home/tex so that it's writable. Our TeX Live
images have cached content in /home/tex. This content will automatically
get copied by Docker into this anonymous volume.
2020-09-04 08:49:56 -04:00
Eric Mc Sween
67f4a6eeeb Decaf cleanup: normalize functions
Use function keyword for declarations and arrow functions for callbacks.
2020-09-03 15:58:16 -04:00
Eric Mc Sween
5cd889038e Use _.once() instead of ad hoc implementation 2020-09-03 15:52:09 -04:00
Eric Mc Sween
a853950a99 Fix container monitor cleanup function
The intent here is clearly to clear both the timeout and the interval.
2020-09-03 15:50:45 -04:00
Eric Mc Sween
30a44edded Decaf cleanup: simplify variable declarations 2020-09-03 15:50:12 -04:00
Eric Mc Sween
a282bccd48 Do not instantiate errors at module load time
This prevents the right stack trace from being captured.
2020-09-03 15:34:19 -04:00
Eric Mc Sween
4905e7db2c Decaf cleanup: unused vars 2020-09-03 15:30:07 -04:00
Eric Mc Sween
44bf38d6db Decaf cleanup: convert async function to sync
The examineOldContainer() function doesn't need to use callbacks since
it only does synchronous work.
2020-09-03 15:13:04 -04:00
Eric Mc Sween
2584586ba2 Decaf cleanup: camel-case variables 2020-09-03 15:13:04 -04:00
Eric Mc Sween
c52d7d8f02 Decaf cleanup: simplify null checks 2020-09-03 15:13:04 -04:00
Eric Mc Sween
ee4c08868c Decaf cleanup: remove default callbacks 2020-09-03 15:13:04 -04:00
Eric Mc Sween
32f0bbe266 Decaf cleanup: remove IIFEs 2020-09-03 15:13:04 -04:00
Eric Mc Sween
f650da8675 Decaf cleanup: remove __guard__ 2020-09-02 17:06:35 -04:00
Eric Mc Sween
99648341e2 Decaf cleanup: remove unnecessary returns 2020-09-02 16:58:41 -04:00
Eric Mc Sween
08be54a43e Decaf cleanup: remove unnecessary Array.from() 2020-09-02 16:45:49 -04:00
dependabot[bot]
6b93bbfe09 Bump bl from 4.0.1 to 4.0.3
Bumps [bl](https://github.com/rvagg/bl) from 4.0.1 to 4.0.3.
- [Release notes](https://github.com/rvagg/bl/releases)
- [Commits](https://github.com/rvagg/bl/compare/v4.0.1...v4.0.3)

Signed-off-by: dependabot[bot] <support@github.com>
2020-09-02 16:50:44 +00:00
dependabot[bot]
7caf7f4201 Bump lodash from 4.17.15 to 4.17.20
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.20.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.20)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-14 05:11:31 +00:00
Christopher Hoskin
b1b2a1c134 [misc] bump logger-sharelatex to version 2.2.0 2020-08-12 15:11:21 +01:00
Jakob Ackermann
1d1b9ebebc [misc] bump the dev-env to 3.3.2 2020-08-10 17:01:11 +01:00
Jakob Ackermann
5846ebb367 Merge pull request #182 from overleaf/msm-fix-npe-community-edition
Fixed NPE when Settings.clsi is defined but Settings.clsi.docker is not
2020-07-15 11:01:08 +02:00
Jakob Ackermann
0f9d0d9790 Merge pull request #183 from overleaf/jpa-clsi-allowed-image-names
[misc] RequestParser: restrict imageName to an allow list and add tests
2020-07-15 10:58:36 +02:00
Jakob Ackermann
4489a080fd Merge pull request #186 from overleaf/jpa-import-132
[LocalCommandRunner] run: block a double call of the callback
2020-07-15 10:57:46 +02:00
Jakob Ackermann
c3a3ed341b Merge pull request #132 from das7pad/hotfix-double-call
[LocalCommandRunner] run: block a double call of the callback
2020-07-03 12:58:25 +02:00
Jakob Ackermann
cd87d3018e [misc] fix formatting 2020-07-03 11:47:53 +01:00
Jakob Ackermann
a5c3bad7f1 [ExampleDocumentTests] drop out in case of an error during compilation
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-07-03 11:38:12 +01:00
Jakob Ackermann
f703b2ca41 [misc] move the image check prior to the base image override 2020-07-01 10:01:25 +01:00
Jakob Ackermann
aa02df7b81 [misc] apply review feedback
- move setting into clsi.docker namespace
- rename the variable for images to allowedImages / ALLOWED_IMAGES
- add an additional check for the image name into the DockerRunner

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2020-06-30 12:01:21 +01:00
Jakob Ackermann
c857371fed [misc] wordcount: restrict image to an allow list and add tests 2020-06-26 13:28:12 +01:00
Jakob Ackermann
8846efe7ce [misc] RequestParser: restrict imageName to an allow list and add tests 2020-06-26 13:28:09 +01:00
Miguel Serrano
f8ca906393 Fixed NPE when Settings.clsi is defined but Settings.clsi.docker is not 2020-06-25 12:31:10 +02:00
Brian Gough
2b342c6e53 Merge pull request #181 from overleaf/bg-fix-503-response
handle EPIPE errors in CompileController
2020-06-22 09:30:35 +01:00
Brian Gough
22480f1a52 handle EPIPE errors in CompileController 2020-06-18 09:54:18 +01:00
Brian Gough
bdebef91eb Merge branch 'master' into bg-add-compile-groups 2020-06-17 11:58:26 +01:00
Brian Gough
f6fddf2944 Merge pull request #179 from overleaf/bg-fix-synctex-error
fix synctex error
2020-06-16 08:57:54 +01:00
Brian Gough
2eb4327256 fix format 2020-06-16 08:45:53 +01:00
Brian Gough
f41b4daca2 Merge branch 'master' into bg-use-lodash 2020-06-16 08:35:17 +01:00
Brian Gough
634fad8c6e Merge pull request #176 from overleaf/ta-epipe-retry-revert
Remove Retries in EPIPE Errors
2020-06-16 08:33:30 +01:00
Brian Gough
85f4f348dd add default settings to remove wordcount and synctex containers 2020-06-15 15:49:38 +01:00
Brian Gough
2ce03f0554 add initial compileGroup support 2020-06-15 15:28:53 +01:00
Brian Gough
0914908676 downgrade NotFoundError log-level 2020-06-15 11:06:54 +01:00
Brian Gough
262ea01911 add acceptance test for synctex when project/file does not exist 2020-06-15 10:55:01 +01:00
Brian Gough
ede70b6f99 migrate from underscore to lodash 2020-06-15 09:52:21 +01:00
Brian Gough
f99023320d use json parsing in request 2020-06-12 15:15:51 +01:00
Brian Gough
5368630754 check output file exists before running synctex 2020-06-12 15:15:27 +01:00
Brian Gough
8fadf9b241 Merge pull request #175 from overleaf/bg-503-on-unavailable
send 503 unavailable response on EPIPE
2020-06-12 09:29:04 +01:00
Brian Gough
69b7ecfd2e Merge pull request #177 from overleaf/bg-add-docker-setting
add missing setting for optimiseInDocker
2020-06-12 09:28:31 +01:00
Tim Alby
d3ff214b88 partially revert "[DockerRunner] fix metric incrementing and error logging"
This reverts commits:
- 2b2fcca39ce8dee0fdc0c342aa0d6c822592bcec
- 9e82ab0890c5cc8c7fb95362c3f7edbcaad0cf29
- e3da458b376871c3ce72d6984d14bf1ee668b04b
2020-06-11 12:51:26 +02:00
Brian Gough
f239fc3b5d send 503 unavailable response on EPIPE 2020-06-11 11:12:02 +01:00
Brian Gough
6c98c7af4a error on missing profile 2020-06-10 11:42:07 +01:00
Brian Gough
54d1cf7eeb add missing setting for optimiseInDocker 2020-06-09 11:22:28 +01:00
Brian Gough
8464ce33a1 Merge pull request #173 from overleaf/bg-openout-any
add setting TEXLIVE_OPENOUT_ANY
2020-06-08 09:03:05 +01:00
Brian Gough
3d6b911295 add setting TEXLIVE_OPENOUT_ANY 2020-06-04 11:47:22 +01:00
Brian Gough
b64d68bc9f update buildscript.txt to node 10.21.0 2020-06-03 11:11:51 +01:00
Brian Gough
8b8acfed4f update to node 10.21.0 2020-06-03 10:22:31 +01:00
Brian Gough
ce1f2e2bd4 fix broken unit test 2020-06-02 11:12:57 +01:00
Brian Gough
aa44bae222 fix eslint errors 2020-06-02 09:51:34 +01:00
Brian Gough
0de2933812 fix unreachable code lint error 2020-06-02 09:28:04 +01:00
Brian Gough
a3d27b63f1 fix formatting with make format_fix 2020-06-02 09:18:38 +01:00
Brian Gough
d9265b779f Merge pull request #170 from overleaf/jpa-import-141
[DockerRunner] destroyOldContainers: fix a race confition
2020-06-02 09:04:59 +01:00
Brian Gough
59cd14e160 Merge pull request #169 from overleaf/bg-record-latexmk-output
record latexmk output
2020-06-02 09:03:43 +01:00
Jakob Ackermann
563a973388 [DockerRunner] destroyOldContainers: normalize the container name
The docker api returns each name with a `/` prefix.

In order to not interfere with pending compiles, the deletion process
 has to acquire an internal lock on the container. The LockManager uses
 the plain container name without the slash: `project-xxx`.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-05-29 11:28:26 +01:00
Brian Gough
6d5dfb7758 clean up log file deletion and add unit test 2020-05-20 14:12:08 +01:00
Brian Gough
a684619bce add unit tests 2020-05-20 11:52:53 +01:00
Brian Gough
63770bf390 clean up the stdout/stderr recording 2020-05-20 11:45:29 +01:00
Henry Oswald
5c3da905ba Merge pull request #162 from overleaf/ta-jpa-epipe-retry
[DockerRunner] retry container inspect on EPIPE
2020-05-19 11:15:25 +01:00
Jakob Ackermann
36e81cbe15 [misc] apply review feedback 2020-05-19 10:30:59 +01:00
Jakob Ackermann
2a3c2dd3d5 [misc] simplify the smoke test and process shutdown 2020-05-19 10:30:59 +01:00
Henry Oswald
2547725f35 Merge pull request #164 from overleaf/bg-fix-buffer-deprecations
fix deprecated usage of Buffer constructor
2020-05-19 10:26:56 +01:00
Henry Oswald
e9c6f3b074 Merge pull request #163 from overleaf/bg-use-encoding-on-process-output
set encoding when reading from streams
2020-05-19 10:26:26 +01:00
Henry Oswald
457b7d6657 Merge pull request #165 from overleaf/ho-retry-url-downloads
add pipeUrlToFileWithRetry for file downloads
2020-05-19 10:25:19 +01:00
Henry Oswald
3925839c7f add refreshExpiryTimeout function
on clsi all data lives inside of / dir
dynamically reduce size of EXPIRY_TIMEOUT if disk starts to get full
2020-05-18 15:17:19 +01:00
Brian Gough
3db513cfc9 record latexmk output 2020-05-15 16:08:10 +01:00
Henry Oswald
77e8ba74a7 add pipeUrlToFileWithRetry function to retry file downloads 3 times 2020-05-14 13:24:58 +01:00