Brian Gough
6fa081522d
add a warning for requests without build id
2021-01-22 11:05:52 +00:00
Brian Gough
b5346658b0
clear output directory when clearing project
2021-01-22 11:05:52 +00:00
Brian Gough
565cd53eb5
add git ignore for output directory
2021-01-22 11:05:52 +00:00
Brian Gough
692dbc8d6b
add output directory
2021-01-22 11:05:52 +00:00
Brian Gough
bdbfe70086
rename staticServer to staticCompileServer
2021-01-22 11:05:52 +00:00
Jakob Ackermann
96b8d001f7
Merge pull request #202 from overleaf/jpa-fix-output-pdf-check
...
[misc] CompileController: exact match for output.pdf
2021-01-13 09:35:31 +00:00
Jakob Ackermann
cac18ac4b9
[misc] bump the node version to 10.23.1
2021-01-05 18:32:01 +00:00
Jakob Ackermann
49b764a308
[misc] CompileController: exact match for output.pdf
...
The previous regex could mistake user provided pdf files, like
`fake_output.pdf`, as the final output file.
The frontend expects to find a `output.pdf` file on success.
2020-12-28 13:16:31 +00:00
Brian Gough
e183fffa4b
format fix
2020-12-18 16:15:06 +00:00
Brian Gough
dafe69b5e6
remove unused module var
2020-12-18 16:15:06 +00:00
Brian Gough
36b646bb84
remove comments
2020-12-18 16:15:06 +00:00
Brian Gough
f20a0d1a69
remove unused var
2020-12-18 16:15:03 +00:00
Brian Gough
df1caa14b8
remove unused require
2020-12-18 16:14:36 +00:00
Brian Gough
ca98ee5cff
use a Set instead of an Object
2020-12-18 16:14:09 +00:00
Brian Gough
43b0429c28
clean up relative path checking
2020-12-18 16:14:05 +00:00
Brian Gough
73e09ff99f
remove unnecessary requires
2020-12-18 15:28:06 +00:00
Brian Gough
cee93a0cd9
clean up unnecessary var
2020-12-18 15:28:06 +00:00
Brian Gough
4169f7fc73
use once for callback
2020-12-18 15:28:06 +00:00
Brian Gough
e84d6305f3
remove unnecessary return
2020-12-18 15:16:07 +00:00
Brian Gough
ae064e8997
lint fix
2020-12-18 15:11:29 +00:00
Brian Gough
323890cedb
remove Array.from
2020-12-18 15:10:53 +00:00
Brian Gough
c044db4897
remove guard helper
2020-12-18 15:09:48 +00:00
Brian Gough
0bae53c7f3
remove unnecessary returns
2020-12-18 15:02:40 +00:00
Brian Gough
eaec57cd01
simplify null check
2020-12-18 15:01:25 +00:00
Brian Gough
411d69e36b
remove unnecessary callback code
2020-12-18 14:59:48 +00:00
Brian Gough
11ef3c27ed
use Set instead of object
2020-12-18 14:56:53 +00:00
Brian Gough
64ea22d259
remove unnecessary null checks
2020-12-17 12:05:17 +00:00
Brian Gough
299d7cc5fe
remove unnecessary returns
2020-12-17 12:01:14 +00:00
Brian Gough
5a539b7f90
remove guard function
2020-12-17 11:59:15 +00:00
Brian Gough
4d8e4d54e5
remove Array.from
2020-12-17 11:49:02 +00:00
dependabot[bot]
f5594e023f
Bump ini from 1.3.5 to 1.3.8
...
Bumps [ini](https://github.com/isaacs/ini ) from 1.3.5 to 1.3.8.
- [Release notes](https://github.com/isaacs/ini/releases )
- [Commits](https://github.com/isaacs/ini/compare/v1.3.5...v1.3.8 )
Signed-off-by: dependabot[bot] <support@github.com>
2020-12-12 11:02:53 +00:00
Jakob Ackermann
5d5241156c
[misc] install settings-sharelatex from npm
2020-12-02 12:26:42 +00:00
Jakob Ackermann
1b73c6dad2
[misc] work around missing stubs
2020-11-25 13:33:54 +00:00
Jakob Ackermann
7b7cd8cc8c
[misc] bump metrics module to 3.4.1
...
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
2020-11-25 11:57:25 +00:00
Eric Mc Sween
c7fa34a6a8
Upgrade build-scripts to 3.4.0
...
This version fixes docker-compose health checks for dependent services. See
https://github.com/overleaf/dev-environment/pull/409 for details.
2020-11-24 08:03:28 -05:00
Shane Kilkelly
fcc52cb1a0
No more blank line
2020-10-27 16:17:22 +00:00
Shane Kilkelly
aaa0681c39
In example request, show example using curl
...
This clarifies the exact way to construct this request. It seems that
some users have taken the existing documentation to mean that the
request should be sent from the browser, to the web server. This example
clarifies how to send this compile request directly to the clsi server.
2020-10-27 15:53:10 +00:00
Eric Mc Sween
20a0f3bcd1
Merge pull request #193 from overleaf/dependabot-npm_and_yarn-node-fetch-2.6.1
...
Bump node-fetch from 2.6.0 to 2.6.1
2020-09-16 12:48:21 -04:00
Eric Mc Sween
40f38b09f0
Merge pull request #191 from overleaf/dependabot-npm_and_yarn-bl-4.0.3
...
Bump bl from 4.0.1 to 4.0.3
2020-09-16 12:48:15 -04:00
Eric Mc Sween
6c73eb5e34
Merge pull request #190 from overleaf/dependabot-npm_and_yarn-lodash-4.17.20
...
Bump lodash from 4.17.15 to 4.17.20
2020-09-16 12:48:07 -04:00
Eric Mc Sween
5b11dc5a77
Bump Node version to 10.22.1
2020-09-16 12:24:42 -04:00
dependabot[bot]
f85e2dec5c
Bump node-fetch from 2.6.0 to 2.6.1
...
Bumps [node-fetch](https://github.com/bitinn/node-fetch ) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/bitinn/node-fetch/releases )
- [Changelog](https://github.com/node-fetch/node-fetch/blob/master/docs/CHANGELOG.md )
- [Commits](https://github.com/bitinn/node-fetch/compare/v2.6.0...v2.6.1 )
Signed-off-by: dependabot[bot] <support@github.com>
2020-09-12 17:04:49 +00:00
Eric Mc Sween
14e1e02a68
Test anonymous volumes are removed with containers
2020-09-04 11:34:08 -04:00
Eric Mc Sween
1c13f6fe94
Mount /home/tex in an anonymous volume
...
When we mount the container's root filesystem as read-only, mount
an anonymous volume in /home/tex so that it's writable. Our TeX Live
images have cached content in /home/tex. This content will automatically
get copied by Docker into this anonymous volume.
2020-09-04 08:49:56 -04:00
Eric Mc Sween
67f4a6eeeb
Decaf cleanup: normalize functions
...
Use function keyword for declarations and arrow functions for callbacks.
2020-09-03 15:58:16 -04:00
Eric Mc Sween
5cd889038e
Use _.once() instead of ad hoc implementation
2020-09-03 15:52:09 -04:00
Eric Mc Sween
a853950a99
Fix container monitor cleanup function
...
The intent here is clearly to clear both the timeout and the interval.
2020-09-03 15:50:45 -04:00
Eric Mc Sween
30a44edded
Decaf cleanup: simplify variable declarations
2020-09-03 15:50:12 -04:00
Eric Mc Sween
a282bccd48
Do not instantiate errors at module load time
...
This prevents the right stack trace from being captured.
2020-09-03 15:34:19 -04:00
Eric Mc Sween
4905e7db2c
Decaf cleanup: unused vars
2020-09-03 15:30:07 -04:00
Eric Mc Sween
44bf38d6db
Decaf cleanup: convert async function to sync
...
The examineOldContainer() function doesn't need to use callbacks since
it only does synchronous work.
2020-09-03 15:13:04 -04:00
Eric Mc Sween
2584586ba2
Decaf cleanup: camel-case variables
2020-09-03 15:13:04 -04:00
Eric Mc Sween
c52d7d8f02
Decaf cleanup: simplify null checks
2020-09-03 15:13:04 -04:00
Eric Mc Sween
ee4c08868c
Decaf cleanup: remove default callbacks
2020-09-03 15:13:04 -04:00
Eric Mc Sween
32f0bbe266
Decaf cleanup: remove IIFEs
2020-09-03 15:13:04 -04:00
Eric Mc Sween
f650da8675
Decaf cleanup: remove __guard__
2020-09-02 17:06:35 -04:00
Eric Mc Sween
99648341e2
Decaf cleanup: remove unnecessary returns
2020-09-02 16:58:41 -04:00
Eric Mc Sween
08be54a43e
Decaf cleanup: remove unnecessary Array.from()
2020-09-02 16:45:49 -04:00
dependabot[bot]
6b93bbfe09
Bump bl from 4.0.1 to 4.0.3
...
Bumps [bl](https://github.com/rvagg/bl ) from 4.0.1 to 4.0.3.
- [Release notes](https://github.com/rvagg/bl/releases )
- [Commits](https://github.com/rvagg/bl/compare/v4.0.1...v4.0.3 )
Signed-off-by: dependabot[bot] <support@github.com>
2020-09-02 16:50:44 +00:00
dependabot[bot]
7caf7f4201
Bump lodash from 4.17.15 to 4.17.20
...
Bumps [lodash](https://github.com/lodash/lodash ) from 4.17.15 to 4.17.20.
- [Release notes](https://github.com/lodash/lodash/releases )
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.20 )
Signed-off-by: dependabot[bot] <support@github.com>
2020-08-14 05:11:31 +00:00
Christopher Hoskin
b1b2a1c134
[misc] bump logger-sharelatex to version 2.2.0
2020-08-12 15:11:21 +01:00
Jakob Ackermann
1d1b9ebebc
[misc] bump the dev-env to 3.3.2
2020-08-10 17:01:11 +01:00
Jakob Ackermann
5846ebb367
Merge pull request #182 from overleaf/msm-fix-npe-community-edition
...
Fixed NPE when Settings.clsi is defined but Settings.clsi.docker is not
2020-07-15 11:01:08 +02:00
Jakob Ackermann
0f9d0d9790
Merge pull request #183 from overleaf/jpa-clsi-allowed-image-names
...
[misc] RequestParser: restrict imageName to an allow list and add tests
2020-07-15 10:58:36 +02:00
Jakob Ackermann
4489a080fd
Merge pull request #186 from overleaf/jpa-import-132
...
[LocalCommandRunner] run: block a double call of the callback
2020-07-15 10:57:46 +02:00
Jakob Ackermann
c3a3ed341b
Merge pull request #132 from das7pad/hotfix-double-call
...
[LocalCommandRunner] run: block a double call of the callback
2020-07-03 12:58:25 +02:00
Jakob Ackermann
cd87d3018e
[misc] fix formatting
2020-07-03 11:47:53 +01:00
Jakob Ackermann
a5c3bad7f1
[ExampleDocumentTests] drop out in case of an error during compilation
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-07-03 11:38:12 +01:00
Jakob Ackermann
f703b2ca41
[misc] move the image check prior to the base image override
2020-07-01 10:01:25 +01:00
Jakob Ackermann
aa02df7b81
[misc] apply review feedback
...
- move setting into clsi.docker namespace
- rename the variable for images to allowedImages / ALLOWED_IMAGES
- add an additional check for the image name into the DockerRunner
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
2020-06-30 12:01:21 +01:00
Jakob Ackermann
c857371fed
[misc] wordcount: restrict image to an allow list and add tests
2020-06-26 13:28:12 +01:00
Jakob Ackermann
8846efe7ce
[misc] RequestParser: restrict imageName to an allow list and add tests
2020-06-26 13:28:09 +01:00
Miguel Serrano
f8ca906393
Fixed NPE when Settings.clsi is defined but Settings.clsi.docker is not
2020-06-25 12:31:10 +02:00
Brian Gough
2b342c6e53
Merge pull request #181 from overleaf/bg-fix-503-response
...
handle EPIPE errors in CompileController
2020-06-22 09:30:35 +01:00
Brian Gough
22480f1a52
handle EPIPE errors in CompileController
2020-06-18 09:54:18 +01:00
Brian Gough
bdebef91eb
Merge branch 'master' into bg-add-compile-groups
2020-06-17 11:58:26 +01:00
Brian Gough
f6fddf2944
Merge pull request #179 from overleaf/bg-fix-synctex-error
...
fix synctex error
2020-06-16 08:57:54 +01:00
Brian Gough
2eb4327256
fix format
2020-06-16 08:45:53 +01:00
Brian Gough
f41b4daca2
Merge branch 'master' into bg-use-lodash
2020-06-16 08:35:17 +01:00
Brian Gough
634fad8c6e
Merge pull request #176 from overleaf/ta-epipe-retry-revert
...
Remove Retries in EPIPE Errors
2020-06-16 08:33:30 +01:00
Brian Gough
85f4f348dd
add default settings to remove wordcount and synctex containers
2020-06-15 15:49:38 +01:00
Brian Gough
2ce03f0554
add initial compileGroup support
2020-06-15 15:28:53 +01:00
Brian Gough
0914908676
downgrade NotFoundError log-level
2020-06-15 11:06:54 +01:00
Brian Gough
262ea01911
add acceptance test for synctex when project/file does not exist
2020-06-15 10:55:01 +01:00
Brian Gough
ede70b6f99
migrate from underscore to lodash
2020-06-15 09:52:21 +01:00
Brian Gough
f99023320d
use json parsing in request
2020-06-12 15:15:51 +01:00
Brian Gough
5368630754
check output file exists before running synctex
2020-06-12 15:15:27 +01:00
Brian Gough
8fadf9b241
Merge pull request #175 from overleaf/bg-503-on-unavailable
...
send 503 unavailable response on EPIPE
2020-06-12 09:29:04 +01:00
Brian Gough
69b7ecfd2e
Merge pull request #177 from overleaf/bg-add-docker-setting
...
add missing setting for optimiseInDocker
2020-06-12 09:28:31 +01:00
Tim Alby
d3ff214b88
partially revert "[DockerRunner] fix metric incrementing and error logging"
...
This reverts commits:
- 2b2fcca39ce8dee0fdc0c342aa0d6c822592bcec
- 9e82ab0890c5cc8c7fb95362c3f7edbcaad0cf29
- e3da458b376871c3ce72d6984d14bf1ee668b04b
2020-06-11 12:51:26 +02:00
Brian Gough
f239fc3b5d
send 503 unavailable response on EPIPE
2020-06-11 11:12:02 +01:00
Brian Gough
6c98c7af4a
error on missing profile
2020-06-10 11:42:07 +01:00
Brian Gough
54d1cf7eeb
add missing setting for optimiseInDocker
2020-06-09 11:22:28 +01:00
Brian Gough
8464ce33a1
Merge pull request #173 from overleaf/bg-openout-any
...
add setting TEXLIVE_OPENOUT_ANY
2020-06-08 09:03:05 +01:00
Brian Gough
3d6b911295
add setting TEXLIVE_OPENOUT_ANY
2020-06-04 11:47:22 +01:00
Brian Gough
b64d68bc9f
update buildscript.txt to node 10.21.0
2020-06-03 11:11:51 +01:00
Brian Gough
8b8acfed4f
update to node 10.21.0
2020-06-03 10:22:31 +01:00
Brian Gough
ce1f2e2bd4
fix broken unit test
2020-06-02 11:12:57 +01:00
Brian Gough
aa44bae222
fix eslint errors
2020-06-02 09:51:34 +01:00
Brian Gough
0de2933812
fix unreachable code lint error
2020-06-02 09:28:04 +01:00
Brian Gough
a3d27b63f1
fix formatting with make format_fix
2020-06-02 09:18:38 +01:00
Brian Gough
d9265b779f
Merge pull request #170 from overleaf/jpa-import-141
...
[DockerRunner] destroyOldContainers: fix a race confition
2020-06-02 09:04:59 +01:00
Brian Gough
59cd14e160
Merge pull request #169 from overleaf/bg-record-latexmk-output
...
record latexmk output
2020-06-02 09:03:43 +01:00
Jakob Ackermann
563a973388
[DockerRunner] destroyOldContainers: normalize the container name
...
The docker api returns each name with a `/` prefix.
In order to not interfere with pending compiles, the deletion process
has to acquire an internal lock on the container. The LockManager uses
the plain container name without the slash: `project-xxx`.
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-05-29 11:28:26 +01:00
Brian Gough
6d5dfb7758
clean up log file deletion and add unit test
2020-05-20 14:12:08 +01:00
Brian Gough
a684619bce
add unit tests
2020-05-20 11:52:53 +01:00
Brian Gough
63770bf390
clean up the stdout/stderr recording
2020-05-20 11:45:29 +01:00
Henry Oswald
5c3da905ba
Merge pull request #162 from overleaf/ta-jpa-epipe-retry
...
[DockerRunner] retry container inspect on EPIPE
2020-05-19 11:15:25 +01:00
Jakob Ackermann
36e81cbe15
[misc] apply review feedback
2020-05-19 10:30:59 +01:00
Jakob Ackermann
2a3c2dd3d5
[misc] simplify the smoke test and process shutdown
2020-05-19 10:30:59 +01:00
Henry Oswald
2547725f35
Merge pull request #164 from overleaf/bg-fix-buffer-deprecations
...
fix deprecated usage of Buffer constructor
2020-05-19 10:26:56 +01:00
Henry Oswald
e9c6f3b074
Merge pull request #163 from overleaf/bg-use-encoding-on-process-output
...
set encoding when reading from streams
2020-05-19 10:26:26 +01:00
Henry Oswald
457b7d6657
Merge pull request #165 from overleaf/ho-retry-url-downloads
...
add pipeUrlToFileWithRetry for file downloads
2020-05-19 10:25:19 +01:00
Henry Oswald
3925839c7f
add refreshExpiryTimeout function
...
on clsi all data lives inside of / dir
dynamically reduce size of EXPIRY_TIMEOUT if disk starts to get full
2020-05-18 15:17:19 +01:00
Brian Gough
3db513cfc9
record latexmk output
2020-05-15 16:08:10 +01:00
Henry Oswald
77e8ba74a7
add pipeUrlToFileWithRetry function to retry file downloads 3 times
2020-05-14 13:24:58 +01:00
Brian Gough
4947abe88b
fix deprecated usage of Buffer constructor
2020-05-07 10:42:05 +01:00
Brian Gough
8e86f02c43
set encoding when reading from streams
...
using .toString() works most of the time but can lead to utf8 characters being
broken across chunk boundaries.
https://nodejs.org/api/stream.html#stream_readable_setencoding_encoding
2020-05-07 10:30:14 +01:00
Henry Oswald
e1c90ec234
Merge branch 'master' into bg-support-other-runtimes
...
port coffee changes to js
2020-04-23 13:59:05 +01:00
Henry Oswald
a5887fad50
Merge pull request #158 from overleaf/jpa-entry-point-cleanup
...
[misc] entry point cleanup
2020-04-23 13:52:04 +01:00
Henry Oswald
ea1ff96e0e
Merge pull request #155 from overleaf/ho-limit-lifespan
...
limit clsi lifespan via health checks and PROCESS_LIFE_SPAN_LIMIT_MS
2020-04-23 13:51:09 +01:00
Henry Oswald
adc73df53d
cleanup the shutdown code a bit
2020-04-23 11:32:33 +01:00
Jakob Ackermann
cf8533bee1
[LocalCommandRunner] run: block a double call of the callback
...
The subprocess event handler fires the "error" and "close" event in case
of a failure.
Both events would call the given callback, resulting in double
processing of the subprocess result downstream.
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-04-16 15:55:55 +02:00
Henry Oswald
acbfc465dc
add variance into shutdown time to avoid stampeed
2020-04-14 13:36:11 +01:00
Jakob Ackermann
cb9e9321f0
[DockerRunner] fix metric incrementing and error logging
...
- do not log on first EPIPE
- inc 'container-inspect-epipe-error' on permanent error only
Co-Authored-By: Tim Alby <timothee.alby@gmail.com>
2020-04-10 14:44:57 +02:00
Tim Alby
ecaa7035f5
add metrics for EPIPE errors
...
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
2020-04-10 12:28:48 +02:00
Tim Alby
e3b1472c7f
retry once on EPIPE errors
...
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
2020-04-10 12:28:11 +02:00
Tim Alby
66447211d0
fix arguments order
...
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
2020-04-10 12:27:15 +02:00
Jakob Ackermann
6a615d50d3
Merge remote-tracking branch 'origin/master' into jpa-entry-point-cleanup
2020-04-03 14:05:36 +02:00
Jakob Ackermann
c700a0ab9d
Merge pull request #159 from overleaf/jpa-fix-docker-modem-error-scheme
...
[misc] keep up with the error signature of dockerode/docker-modem
2020-04-03 14:04:06 +02:00
Jakob Ackermann
bb9e8bb6f5
Merge pull request #157 from overleaf/jpa-download-failed-metric
...
[misc] add a metric for failing downloads
2020-04-03 14:03:16 +02:00
Jakob Ackermann
3b10f80adb
Merge pull request #156 from overleaf/jpa-logger-1-9-1
...
[misc] bump logger-sharelatex to 1.9.1
2020-04-03 14:03:04 +02:00
Jakob Ackermann
101c4ea4dc
[misc] use a directory in /tmp for temporary data
2020-04-03 12:36:32 +02:00
Jakob Ackermann
b9deec1095
[misc] narrow down write access/ownership for the run-time user
2020-04-03 12:36:32 +02:00
Jakob Ackermann
3309adf2ad
[misc] move the sqlite database into a db/ directory
2020-04-03 12:36:31 +02:00
Jakob Ackermann
5b1656b884
[misc] drop debug output and log docker version on stderr
2020-04-03 12:36:31 +02:00
Jakob Ackermann
6777432569
[misc] add a metric for failing downloads
2020-04-03 12:35:40 +02:00
Jakob Ackermann
6b02921409
[misc] bump the build-scripts to version 2.1.0
...
This will put acceptance and unit tests in own namespaces so that they
can run and be teared down individually.
2020-04-03 12:18:09 +02:00
Henry Oswald
87188e0830
Merge branch 'master' of github.com:overleaf/clsi into ho-limit-lifespan
2020-03-31 09:52:37 +01:00
Jakob Ackermann
4a19f022fc
[misc] keep up with the error signature of dockerode/docker-modem
...
https://github.com/apocas/docker-modem/blob/v2.1.1/lib/modem.js#L296
2020-03-27 10:39:45 +01:00
Jakob Ackermann
ac5feb4a77
[misc] bump logger-sharelatex to 1.9.1
2020-03-23 16:18:07 +01:00
Henry Oswald
3e07af3f1b
limit clsi lifespan via health checks and PROCESS_LIFE_SPAN_LIMIT_MS
2020-03-20 13:37:58 +00:00
Miguel Serrano
77cc3b3cc6
Merge pull request #154 from overleaf/msm-dependency-update
...
Dependency update
2020-03-19 16:12:51 +01:00
mserranom
8641d0ef0f
updated sandboxed-module, chai and metrics-sharelatex
2020-03-16 17:14:04 +01:00
mserranom
d5bd1790a8
updated sequelize
2020-03-16 16:31:02 +01:00
mserranom
7e7d234350
updated mocha and sinon, fixed tests
2020-03-12 10:35:11 +01:00
mserranom
cf6c8ab496
removed mkdirp dependency and replaced with fs.mkdir
2020-03-12 10:22:08 +01:00
mserranom
35f69e4f8b
updated mkdirp
2020-03-11 19:39:08 +01:00
mserranom
a0b9d74a25
updated dockerode, heapdump, lyns and fs-extra
2020-03-11 19:25:50 +01:00
mserranom
9883c0e3ff
updated minor/patch dependencies
2020-03-11 12:20:48 +01:00