Henry Oswald
e60157094b
improve logging for 2 compile backends
2018-06-20 11:07:34 +01:00
Henry Oswald
b5581c4d40
Merge branch 'master' into ho-clsi-double-backend
2018-06-19 14:55:18 +01:00
Michael Mazour
0246631443
Add public API endpoints to reach CLSIs
...
- `/api/clsi/compile/:submission_id`
- `/api/clsi/compile/:submission_id/build/:build_id/output/:file`
Also per review:
- DRY up ClsiManager.sendRequestOnce and ClsiManager.sendExternalRequest
- Include submission_id in a log message
- Don't include timeout in limits when getting file
2018-06-19 12:04:02 +01:00
Henry Oswald
5f8985811d
mvp to send requests to 2 backends
2018-06-14 17:41:30 +01:00
Henry Oswald
75c9489c06
PDF Downloads work, not tested on large pdf's yet
2018-06-08 17:17:00 +01:00
James Allen
d9a7b1248a
Re-order compile rate limits to do project/user check before everyone check
2018-04-17 14:43:54 +01:00
Hayden Faulds
eb198cd8c3
make getSafePathname a method in CompileController rather than Project
2018-02-16 13:10:48 +00:00
Hayden Faulds
304a6bc332
use ProjectGetter rather than Project directly
2018-02-16 08:55:09 +00:00
Brian Gough
56cb901b41
filter missing doc_ids from root doc_ids list
2017-10-17 15:49:02 +01:00
Brian Gough
f70ab03bd8
show an error if the main file cannot be found
...
also improve logic for detecting the main file, if there's only one file
it must be the main file.
2017-10-17 15:47:50 +01:00
Brian Gough
4ce639db70
Merge pull request #58 from sharelatex/bg-rate-limit-autocompile
...
fix two bugs in auto compile limit logic
2017-10-13 08:18:31 +01:00
Brian Gough
fc3ecddd0e
fix two bugs in auto compile limit logic
...
1. the compileGroup is "standard" not default
2. was not excluding normal compiles from metrics
2017-10-12 16:18:14 +01:00
Brian Gough
18370076a2
Merge pull request #45 from sharelatex/bg-auto-doc-flush
...
allow docupdater to flush docs in background (connects to overleaf/sharelatex#190 )
2017-10-12 14:48:16 +01:00
Brian Gough
45ed090326
Merge pull request #37 from sharelatex/bg-rate-limit-autocompile
...
rate limit autocompile (connects to #18 )
2017-10-12 09:25:59 +01:00
Brian Gough
5b0d3d1429
simplify rate-limit checking code
2017-10-09 16:31:01 +01:00
Brian Gough
9eec245358
allow docupdater to flush docs in background
2017-10-09 14:17:28 +01:00
Brian Gough
d14723f24a
add rate limits for autocompiles
...
global rate limit for all users and a lower rate limit for free users
2017-10-03 16:16:21 +01:00
Brian Gough
bd005d7bb6
fix double callback in precompile resources check
2017-10-02 10:14:52 +01:00
Brian Gough
0f855689a7
show error for compile in progress
2017-09-26 08:07:35 +01:00
Brian Gough
9f9c15f6f5
Merge pull request #599 from sharelatex/bg-reset-project-state
...
clear docupdater project state in deleteAuxFiles
2017-09-15 09:09:29 +01:00
Brian Gough
51eb94a493
handle incremental compile without root doc
2017-09-13 10:10:44 +01:00
Brian Gough
2b4c8bd846
clear docupdater project state in deleteAuxFiles
2017-09-08 15:57:29 +01:00
Brian Gough
e8435e3eae
make condition clearer for incremental compile
2017-09-08 13:39:24 +01:00
Brian Gough
03a5ff2e43
skip incremental compile after docupdater error
2017-09-07 15:06:09 +01:00
Brian Gough
d9557fcbf5
include the options in the project state hash
2017-09-01 16:36:51 +01:00
Brian Gough
90ff58b820
compute project state hash from sorted docs/files
2017-08-16 10:49:29 +01:00
Brian Gough
739445336f
remove unused code and fix flushing
2017-08-15 16:00:00 +01:00
Brian Gough
e2048e1ed5
use incrementalCompilesEnabled as option name
2017-08-09 16:25:57 +01:00
Brian Gough
ddecd26718
flush documents to mongo on incremental compiles
2017-08-09 15:47:44 +01:00
Brian Gough
7eb1c01994
add metrics for incremental compiles
2017-08-09 11:41:09 +01:00
Brian Gough
97b129cbe3
enable incremental compilation for beta users
2017-08-09 10:57:24 +01:00
Brian Gough
8aa77cec5e
provide fallback to normal compile method
2017-08-08 16:48:47 +01:00
Brian Gough
203e42fa4c
clean up options handling
2017-08-08 16:48:37 +01:00
Brian Gough
31e71854a4
fix unit tests
2017-08-08 11:38:31 +01:00
Brian Gough
849e905efb
simplify incremental request to docupdater
...
if project state hasn't changed, get the docs from the docupdater -- we
check/set the hash and return the docs in a single request. Otherwise
do a full request from mongo.
2017-08-07 14:45:04 +01:00
Brian Gough
38c879faf2
improve comment about ClsiStateManager hash
2017-08-03 12:15:27 +01:00
Brian Gough
6d331e8ffd
use projectStateUnchanged instead of stateOk
2017-08-03 12:10:23 +01:00
Brian Gough
f44b844d74
refer to project state as projectStateHash
2017-08-03 12:08:11 +01:00
Brian Gough
5c02255e07
use syncType and syncState for clsi state options
2017-08-03 11:44:10 +01:00
Brian Gough
fb29ac3031
clean up logging
2017-08-03 11:36:59 +01:00
Brian Gough
0a859d3b33
clean up state manager
2017-08-03 11:36:23 +01:00
Brian Gough
a4117487e9
switch from mongoose to mongojs in ClsiManager
...
for efficiency
2017-08-03 11:35:42 +01:00
Brian Gough
17b1075dc9
add rootFolder to attributes in Clsi request
2017-08-03 10:20:57 +01:00
Brian Gough
4d4cf4f693
project state can be stored per project
...
there is no need to store it per project+user because it reflects the
state of the project itself
2017-08-02 16:25:47 +01:00
Brian Gough
cf780fd8bb
start making requests incremental
2017-08-02 13:53:32 +01:00
Brian Gough
d66382382f
split request to mongo into separate method
2017-07-28 15:17:10 +01:00
Brian Gough
3e28eca26f
move docupdater flush to point of use
2017-07-28 15:11:07 +01:00
James Allen
01548a0216
Don't try to set the server id if it isn't returned in the response
2017-05-19 10:23:01 +01:00
James Allen
f708a0cd81
Check for blank string in clsi cookie caching
...
ioredis returns "" for a blank entry, the previous driver returned null,
so also check for a blank string
2017-05-18 11:09:33 +01:00
James Allen
5aa8b71832
Allow writing of clsi cookie cache to redis cluster secondary
2017-05-15 15:35:01 +01:00
