Commit graph

408 commits

Author SHA1 Message Date
Alasdair Smith
87935f30c5 Update docker-runner-sharelatex config 2017-08-24 13:34:24 +01:00
Brian Gough
43f424295e Merge pull request #66 from sharelatex/bg-compile-from-redis
Write files incrementally
2017-08-23 15:35:56 +01:00
Brian Gough
26f85ba75f read resource files safely
put a limit on the amount of data read
2017-08-18 11:17:01 +01:00
Brian Gough
a84c884fc9 splice state management into ResourceStateManager 2017-08-18 10:22:17 +01:00
Brian Gough
96b801b093 fix whitespace 2017-08-18 09:41:59 +01:00
Brian Gough
d614af2860 finish unit test for incremental update 2017-08-18 09:41:43 +01:00
Brian Gough
d3da4e1d3f ResourceWriter unit tests (wip) 2017-08-17 16:59:37 +01:00
Brian Gough
c7a9487216 test syncType in RequestParser 2017-08-17 15:57:05 +01:00
Brian Gough
b901884248 avoid adding draft mode more than once 2017-08-17 15:03:37 +01:00
Brian Gough
8415ea2f71 store the resource list in a file 2017-08-17 14:53:35 +01:00
Brian Gough
c0ed71f65c fix unit tests 2017-08-09 15:22:44 +01:00
Brian Gough
2950c01130 add comment about syncType/syncState 2017-08-09 15:22:38 +01:00
Brian Gough
4ebc7e5e4a clean up the state file if no state passed in 2017-08-08 16:29:57 +01:00
Henry Oswald
1e34f6371e use grunt to make compiles and cache dirs 2017-08-07 16:21:37 +01:00
Henry Oswald
c3511e91ef Revert "Keep compiles and cache directories" 2017-08-07 15:29:18 +01:00
Brian Gough
567a89350b fix broken unit tests 2017-08-07 15:00:16 +01:00
Brian Gough
3a930a636e fix incremental request 2017-08-07 14:32:28 +01:00
Brian Gough
fe5ba5b619 restrict syncType values to full/incremental 2017-08-07 10:19:56 +01:00
Brian Gough
44ae011703 added files out of sync error object 2017-08-03 15:56:59 +01:00
Brian Gough
d4aad06c1f use syncType and syncState for clsi state options 2017-08-03 12:00:32 +01:00
Brian Gough
da86a094a8 write files incrementally 2017-08-02 13:46:10 +01:00
Hayden Faulds
931abb7c81 keep cache directory 2017-07-27 15:54:20 +01:00
Hayden Faulds
d382f2929f keep compiles directory 2017-07-27 14:02:24 +01:00
Henry Oswald
2aeec8a12f Revert "change"
This reverts commit 104ce81ebdf41d88acd7fb6f2abf99fbc4eb91df.
2017-07-23 22:45:04 +01:00
Henry Oswald
19c4d020b5 change 2017-07-23 22:42:07 +01:00
Brian Gough
d43357e8c8 stub out unwanted dependency in unit tests 2017-07-18 11:30:22 +01:00
Brian Gough
fd5b557dde fix exception for empty content in TikzManager 2017-07-18 11:29:59 +01:00
Brian Gough
4a4f3bb24c update acceptance test images for texlive 2017 2017-07-13 13:15:51 +01:00
Brian Gough
0c7a89dfa5 update acceptance tests settings to 2017 image 2017-07-12 16:59:33 +01:00
Shane Kilkelly
577cf7ef4c Merge branch 'sk-reduce-kill-project-errors' 2017-06-27 10:03:51 +01:00
Shane Kilkelly
a55debb79f Send a 404 if the project files have gone away when running synctex.
This is semantically nicer than the 500 response which used to be
produced in these circumstances.
2017-06-23 14:46:40 +01:00
Shane Kilkelly
4b14de05c9 Killing an already stopped project is not an error
Log a warning instead and continue.
2017-06-20 09:18:15 +01:00
Shane Kilkelly
65eaf0ad10 Mock out logger in tests 2017-06-20 08:25:50 +01:00
Brian Gough
f2746b7d5b delete intermediate xdv files from xelatex 2017-06-15 15:37:45 +01:00
Brian Gough
8d8f441ff8 Merge pull request #58 from sharelatex/bg-check-dir-before-synctex
check file exists before running synctex
2017-05-31 10:16:06 +01:00
Brian Gough
550979991f check directory exists and bail out on error 2017-05-31 10:06:27 +01:00
Brian Gough
42fa852b76 check file exists before running synctex 2017-05-24 10:09:43 +01:00
Brian Gough
7d93e77102 don't report compile timeouts to sentry
just log them instead
2017-05-24 09:42:05 +01:00
Brian Gough
86f29a4cfb add setting to avoid optimisations outside docker 2017-04-10 16:12:03 +01:00
Brian Gough
f7c3565281 use pdfinfo on output to ensure pdfs are optimised
needed to check that qpdf runs correctly inside the docker container
2017-04-07 11:11:27 +01:00
Brian Gough
7e58bfd752 check if file is optimised before running qpdf 2017-04-04 16:50:06 +01:00
Shane Kilkelly
222fc4b99c Add a .nvmrc file 2017-03-27 14:47:48 +01:00
Brian Gough
4e34fa2fa6 Merge pull request #53 from sharelatex/bg-sanitise-paths
additional check for valid rootResource
2017-03-21 13:39:27 +00:00
Brian Gough
dd35c5d88b check for \tikzexternalize directly
instead of \usepackage{tikz} and \usepackage{pgf}
2017-03-21 11:36:08 +00:00
Brian Gough
1273a05ad4 fix path match 2017-03-21 11:30:32 +00:00
Brian Gough
54bdc8fed0 create separate function for path checking 2017-03-21 11:29:37 +00:00
Brian Gough
3bd919b3eb support for tikz externalize
make copy of main file as output.tex for tikz externalize
2017-03-20 10:55:28 +00:00
Brian Gough
4806da00a4 additional check for valid rootResource 2017-03-20 10:03:48 +00:00
Brian Gough
a602629406 update xelatex acceptance test pdf 2017-03-08 11:49:21 +00:00
Brian Gough
d09cbfaa8d improve debugging of failed acceptance tests
use the example name in the output filename
2017-03-08 11:49:12 +00:00
Brian Gough
d123f8eb64 include otf extension in fontawesome test 2017-03-08 11:25:25 +00:00
Shane Kilkelly
b32c6b1f39 Upgrade logger 2017-03-06 14:56:32 +00:00
Shane Kilkelly
533804c55b Don't compile acceptance test files during test run 2017-03-06 14:43:14 +00:00
Brian Gough
12b7a372e4 allow latexmk to pass through options
this avoids problems in the latest version of latexmk where the
$pdflatex variable has been replaced by $xelatex and $lualatex when
running with -xelatex or -lualatex
2017-03-02 16:43:35 +00:00
Henry Oswald
40a2684801 remove tcp code, moved to agent load balancer 2017-02-23 11:09:18 +00:00
Brian Gough
1a4fca08eb fix acceptance test config file for latex prefix
latex command prefix was in wrong scope
2017-02-21 09:37:05 +00:00
Brian Gough
61fb90c6aa add fontawesome acceptance test for xelatex 2017-02-21 09:37:05 +00:00
Henry Oswald
6aea5199c8 Merge pull request #49 from sharelatex/ho-one-cpu-size
if host has 1 cpu (staging) then set availableWorkingCpus to 1
2017-02-20 15:20:04 +00:00
Henry Oswald
f9c687d5f8 Update app.coffee 2017-02-20 15:19:04 +00:00
Henry Oswald
a7623bf0d2 if host has 1 cpu (staging) then set availableWorkingCpus to 1 2017-02-20 15:16:52 +00:00
Brian Gough
1a7d8477a3 update acceptance tests for reversion to dvipdf 2017-02-13 13:42:44 +00:00
Brian Gough
29d36399a2 fix #! in test script 2017-02-09 15:38:25 +00:00
Brian Gough
83dd366eb4 show debug info for acceptance tests 2017-02-09 14:17:38 +00:00
Brian Gough
11e4d01e1e update acceptance test output for fontawesome 2017-02-07 11:51:21 +00:00
Brian Gough
42d20cede0 added fontawesome acceptance test 2017-02-07 10:00:41 +00:00
Brian Gough
58bb705555 fix lualatex require 2017-02-07 08:59:45 +00:00
Brian Gough
83a06b1fe1 add luatex85 package to tikz feynman test 2017-02-07 08:49:19 +00:00
Brian Gough
91fa3364f7 added acceptance test for tikz-feynman 2017-02-07 08:12:47 +00:00
Brian Gough
8b22b6c650 update knitr_utf acceptance test output
needs to include table of contents from multiple latexmk runs
2017-02-06 16:27:47 +00:00
Brian Gough
3acf15de5b add knitr utf8 acceptance test 2017-02-03 15:38:06 +00:00
Brian Gough
491528f5a5 add acceptance test for hebrew 2017-02-03 15:16:47 +00:00
Brian Gough
3434f365bb update acceptance test image for lualatex
small pixel-level change in output
2017-01-31 16:04:59 +00:00
Brian Gough
c031ec7a2e increase debugging in acceptance tests 2017-01-31 10:47:49 +00:00
Brian Gough
c83102c665 add debugging to acceptance tests 2017-01-31 10:40:05 +00:00
Brian Gough
a1503a8791 update feymp test image
minor pixel change in position of labels in texlive 2016
2017-01-31 10:21:00 +00:00
Brian Gough
2c26c99509 update asymptote pdf to a4 size for texlive 2016 2017-01-31 09:53:36 +00:00
Brian Gough
b04713b234 try output.pdf generated with texlive 2016 2017-01-30 15:37:26 +00:00
Brian Gough
00d8bc78c3 update latex_compiler test pdf 2017-01-27 12:32:14 +00:00
Brian Gough
354a88e67a specify papersize explicitly in latex test 2017-01-27 12:21:57 +00:00
Brian Gough
a802620d11 log acceptance test server output to file 2017-01-26 12:20:41 +00:00
Brian Gough
07e536677c try running user as jenkins 2017-01-26 12:06:38 +00:00
Brian Gough
5a6f54e6b2 update image for docker tests 2017-01-25 14:12:19 +00:00
Brian Gough
b4afaf9ea8 use local docker image for clsi test 2017-01-25 14:09:44 +00:00
Brian Gough
9461b28707 run tests outside container, add settings file 2017-01-25 14:08:39 +00:00
Brian Gough
0a3bd5fb07 upgrade to latest sqlite3 2017-01-24 16:06:32 +00:00
Brian Gough
0f92ef104a added docker script for acceptance tests 2017-01-24 12:18:30 +00:00
Brian Gough
08dc3bd744 fix acceptance tests 2017-01-24 11:07:54 +00:00
Brian Gough
8add9ff3ea fix exception in error log 2016-10-14 10:23:13 +01:00
Shane Kilkelly
34f47178fa Increase memory limit to 64mb 2016-09-28 11:02:58 +01:00
Shane Kilkelly
2bedd08ff9 Add test for new ulimit options 2016-09-23 15:34:29 +01:00
Shane Kilkelly
4efdddbbac Add CHKTEX_ULIMIT_OPTIONS 2016-09-23 15:32:37 +01:00
James Allen
638359e052 Don't delete knitr cache files 2016-09-22 14:14:29 +01:00
James Allen
4c04a5df3f Sanitize resource path along with rootResourcePath 2016-09-21 15:09:01 +01:00
Brian Gough
56429178df only run chktex on .tex files, not .Rtex files
the .tex files produced from knitr have macros which confuse chktex
2016-09-12 16:29:36 +01:00
Brian Gough
e23512f0e0 handle failed compile due to validation error 2016-08-24 15:46:47 +01:00
Brian Gough
481e842f33 fix commandRunner error to match dockerRunner 2016-08-24 15:45:26 +01:00
Brian Gough
1af44b32b3 remove chktex error
too many false positives from 'unable to execute latex command'
2016-08-22 15:11:39 +01:00
Brian Gough
a26971ba2e update tests 2016-08-11 10:31:37 +01:00
Brian Gough
ed5acc9f93 capture texcount error output 2016-08-11 10:26:08 +01:00
Brian Gough
400590b3ad read wordcount output asynchronously 2016-08-11 09:32:53 +01:00
Brian Gough
872dbc5215 add missing argument parameter to wordcount call 2016-08-11 09:29:03 +01:00
Brian Gough
c5917d4a75 use a command wrapper for synctex
instead of an alternative child_process object
2016-08-04 16:08:14 +01:00
Brian Gough
08a446f03d change logging message to be different from LatexRunner 2016-08-04 16:07:36 +01:00
Brian Gough
5c1e903b14 Merge pull request #44 from sharelatex/add-chktex-support
Add chktex support
2016-08-02 14:55:38 +01:00
Brian Gough
f0c7b455a5 provide setting to override child_process.execFile for synctex 2016-07-29 14:54:24 +01:00
Brian Gough
ffea361d12 provide validation mode where compilation always exits after chktex 2016-07-27 16:54:27 +01:00
Brian Gough
d6d6e18b89 run chktex when request has check:true 2016-07-26 16:22:38 +01:00
Brian Gough
3135ff2842 add support for passing additional environment parameters to command runner
includes an example of passing environment variables to chktex
2016-07-26 12:30:29 +01:00
Brian Gough
e4da985753 Merge pull request #43 from sharelatex/stop-compile
add support for stopping compile
2016-07-18 11:16:53 +01:00
Brian Gough
bd1e1de504 remove dead code 2016-07-18 11:05:45 +01:00
Brian Gough
9b58621fa8 add support for stopping compile 2016-07-14 16:43:52 +01:00
WaeCo
92461742d0 Set default project_cache_length_ms to 1 day
`project_cache_length_ms` was only `60*60*24 = 1.5 min` which is a little bit short. Default of one day seams more reasonable.
2016-07-13 13:26:32 -07:00
Brian Gough
dd4e1e17b5 keep one extra build until per-page pdf serving is enabled 2016-06-29 16:31:16 +01:00
Brian Gough
03188d2a57 only keep a single cached output directory in per-user containers 2016-06-28 09:28:40 +01:00
Shane Kilkelly
638ac52e40 Move the latexmk timing command into a configurable latexmkCommandPrefix.
By default, no timing information will be taken.
On Linux with GNU user land, this value should be configured to `["/usr/bin/time", "-v"]`.
On Mac, gnu-time should be installed and configured to `["/usr/local/bin/gtime", "-v"]`.
2016-06-17 14:38:08 +01:00
Brian Gough
a6f6392e40 add route to serve files from top level of per user containers 2016-06-15 16:12:19 +01:00
Brian Gough
fb12d04282 Merge branch 'per-user-containers-part-2'
Conflicts:
	app/coffee/CompileController.coffee
2016-06-13 09:33:41 +01:00
Brian Gough
feeed9faaf Merge pull request #39 from sharelatex/per-user-containers-part-1
Per user containers part 1
2016-06-09 15:17:35 +01:00
Henry Oswald
989c178546 use process id so link process to smoke test 2016-06-07 14:47:51 +01:00
Henry Oswald
f0dc5d6f86 add random string to smoke tests to avoid collision 2016-06-07 14:39:01 +01:00
Brian Gough
d3395ef763 log user_id when clearing project 2016-06-02 15:32:33 +01:00
Brian Gough
5367bc22e5 add per-user routes for clearing cache and extend expiry methods
this adds separate functionality for clearing the cache (assets and
database) and the project compile directory for a specific user
2016-06-02 15:32:33 +01:00
Brian Gough
df641549c4 add per-user routes and methods 2016-06-02 15:32:31 +01:00
Brian Gough
1462e17f0c put the build id in the output file urls
the url attribute will now give the preferred location for accessing
the output file, without the url having to be constructed by the web
client
2016-06-02 15:30:50 +01:00
Brian Gough
bd22a6e02d put the build id in the output file urls
the url attribute will now give the preferred location for accessing
the output file, without the url having to be constructed by the web
client
2016-06-02 15:29:56 +01:00
Henry Oswald
fdc8d83646 log out error on synctex 2016-05-27 16:18:18 +01:00
Henry Oswald
41ca3a4222 log out errors more clearly 2016-05-27 14:45:39 +01:00
Henry Oswald
5ed6ef571f added logger.info to test setup 2016-05-24 14:12:02 +01:00
Henry Oswald
448a30750b log out which command logger is used 2016-05-24 14:08:39 +01:00
Henry Oswald
ea8134c433 log out if the command running is being used 2016-05-23 15:45:39 +01:00
Henry Oswald
98a07e07ed parallelFileDownloads defaults to 1, sql can't take it 2016-05-23 14:31:27 +01:00
Henry Oswald
d140964c5f add error handler on CommandRunner 2016-05-23 14:13:55 +01:00
Brian Gough
bd55e18d6b be ready to serve files from per-user containers 2016-05-20 10:23:07 +01:00
Brian Gough
340fef38a9 fix error in log for expiry timeout 2016-05-20 10:23:07 +01:00
Henry Oswald
786b9cdb84 Merge branch 'master' of https://github.com/sharelatex/clsi-sharelatex 2016-05-19 16:57:19 +01:00
Henry Oswald
9c873f99ba log out EXPIRY_TIMEOUT 2016-05-19 16:57:14 +01:00
Henry Oswald
9ae37d1814 make cached assets ttl set via config 2016-05-19 16:51:50 +01:00
Brian Gough
07401c5a45 avoid clobbering the existing port variable 2016-05-19 16:38:18 +01:00
Brian Gough
e286aede4e return the file path in the output file list for easy lookup 2016-05-19 16:38:18 +01:00
Brian Gough
2494327c12 make the build id a secure random token
we allow existing build ids to work for backwards compatibility
this can be removed after some time
2016-05-13 10:11:35 +01:00
Brian Gough
8a2665911d allow direct path to output file /project/project_id/build/build_id/output/*
this avoids use of the query string ?build=... and so we can match the
url directly with the nginx location directive
2016-05-13 10:10:48 +01:00
Brian Gough
d5c1fb9997 fix logic excluding smoke test in metric 2016-05-10 10:10:01 +01:00
Brian Gough
00acfbdf86 fix tagname for graphite 2016-05-10 09:41:39 +01:00
Brian Gough
7779f65869 record system time 2016-05-10 09:12:13 +01:00
Brian Gough
d2e4253e1e preserve existing metric name 2016-05-10 09:12:00 +01:00
Brian Gough
3eebca4c01 add timing information from /usr/bin/time 2016-05-09 16:00:24 +01:00
Brian Gough
ebcf9a75ae add a metric for the TeXLive image used on each compile 2016-05-09 15:36:11 +01:00
Henry Oswald
7ab1654ebc move back to 2.5 days cache for moment 2016-04-21 17:40:09 +01:00
Henry Oswald
e024d3f503 added project status endpoint
used for getting the server a project is on
2016-04-20 15:38:05 +01:00
Henry Oswald
d416f75c2c increased clsi cache to 3.5 days 2016-04-13 09:29:57 +01:00