170 commits

Author	SHA1	Message	Date
Henry Oswald	5bb3acc1eb	fix metrics sending to clsi	2018-07-31 12:56:35 +01:00
Henry Oswald	b04c7d2ffe	send clsi response codes to graphite	2018-07-30 18:16:52 +01:00
Henry Oswald	d484c36c6d	improve clsi logging	2018-07-26 16:04:14 +01:00
Henry Oswald	c1a8d8b31a	Merge branch 'master' into ho-clsi-double-backend	2018-06-25 13:43:40 +01:00
Shane Kilkelly	6058f3ef9b	Refactor the LinkedFiles/Agent system, and track build_id for output files	2018-06-22 13:09:18 +01:00
Shane Kilkelly	d93eb448e3	Move Clsi logic to ClsiManager	2018-06-22 13:09:18 +01:00
Henry Oswald	4cb78a5797	move start time inside each clsi request as they are run in series now	2018-06-20 14:15:42 +01:00
Henry Oswald	e60157094b	improve logging for 2 compile backends	2018-06-20 11:07:34 +01:00
Henry Oswald	b5581c4d40	Merge branch 'master' into ho-clsi-double-backend	2018-06-19 14:55:18 +01:00
Michael Mazour	0246631443	Add public API endpoints to reach CLSIs ... - `/api/clsi/compile/:submission_id` - `/api/clsi/compile/:submission_id/build/:build_id/output/:file` Also per review: - DRY up ClsiManager.sendRequestOnce and ClsiManager.sendExternalRequest - Include submission_id in a log message - Don't include timeout in limits when getting file	2018-06-19 12:04:02 +01:00
Henry Oswald	5f8985811d	mvp to send requests to 2 backends	2018-06-14 17:41:30 +01:00
Henry Oswald	75c9489c06	PDF Downloads work, not tested on large pdf's yet	2018-06-08 17:17:00 +01:00
James Allen	d9a7b1248a	Re-order compile rate limits to do project/user check before everyone check	2018-04-17 14:43:54 +01:00
Hayden Faulds	eb198cd8c3	make getSafePathname a method in CompileController rather than Project	2018-02-16 13:10:48 +00:00
Hayden Faulds	304a6bc332	use ProjectGetter rather than Project directly	2018-02-16 08:55:09 +00:00
Brian Gough	56cb901b41	filter missing doc_ids from root doc_ids list	2017-10-17 15:49:02 +01:00
Brian Gough	f70ab03bd8	show an error if the main file cannot be found ... also improve logic for detecting the main file, if there's only one file it must be the main file.	2017-10-17 15:47:50 +01:00
Brian Gough	4ce639db70	Merge pull request #58 from sharelatex/bg-rate-limit-autocompile ... fix two bugs in auto compile limit logic	2017-10-13 08:18:31 +01:00
Brian Gough	fc3ecddd0e	fix two bugs in auto compile limit logic ... 1. the compileGroup is "standard" not default 2. was not excluding normal compiles from metrics	2017-10-12 16:18:14 +01:00
Brian Gough	18370076a2	Merge pull request #45 from sharelatex/bg-auto-doc-flush ... allow docupdater to flush docs in background (connects to overleaf/sharelatex#190)	2017-10-12 14:48:16 +01:00
Brian Gough	45ed090326	Merge pull request #37 from sharelatex/bg-rate-limit-autocompile ... rate limit autocompile (connects to #18)	2017-10-12 09:25:59 +01:00
Brian Gough	5b0d3d1429	simplify rate-limit checking code	2017-10-09 16:31:01 +01:00
Brian Gough	9eec245358	allow docupdater to flush docs in background	2017-10-09 14:17:28 +01:00
Brian Gough	d14723f24a	add rate limits for autocompiles ... global rate limit for all users and a lower rate limit for free users	2017-10-03 16:16:21 +01:00
Brian Gough	bd005d7bb6	fix double callback in precompile resources check	2017-10-02 10:14:52 +01:00
Brian Gough	0f855689a7	show error for compile in progress	2017-09-26 08:07:35 +01:00
Brian Gough	9f9c15f6f5	Merge pull request #599 from sharelatex/bg-reset-project-state ... clear docupdater project state in deleteAuxFiles	2017-09-15 09:09:29 +01:00
Brian Gough	51eb94a493	handle incremental compile without root doc	2017-09-13 10:10:44 +01:00
Brian Gough	2b4c8bd846	clear docupdater project state in deleteAuxFiles	2017-09-08 15:57:29 +01:00
Brian Gough	e8435e3eae	make condition clearer for incremental compile	2017-09-08 13:39:24 +01:00
Brian Gough	03a5ff2e43	skip incremental compile after docupdater error	2017-09-07 15:06:09 +01:00
Brian Gough	d9557fcbf5	include the options in the project state hash	2017-09-01 16:36:51 +01:00
Brian Gough	90ff58b820	compute project state hash from sorted docs/files	2017-08-16 10:49:29 +01:00
Brian Gough	739445336f	remove unused code and fix flushing	2017-08-15 16:00:00 +01:00
Brian Gough	e2048e1ed5	use incrementalCompilesEnabled as option name	2017-08-09 16:25:57 +01:00
Brian Gough	ddecd26718	flush documents to mongo on incremental compiles	2017-08-09 15:47:44 +01:00
Brian Gough	7eb1c01994	add metrics for incremental compiles	2017-08-09 11:41:09 +01:00
Brian Gough	97b129cbe3	enable incremental compilation for beta users	2017-08-09 10:57:24 +01:00
Brian Gough	8aa77cec5e	provide fallback to normal compile method	2017-08-08 16:48:47 +01:00
Brian Gough	203e42fa4c	clean up options handling	2017-08-08 16:48:37 +01:00
Brian Gough	31e71854a4	fix unit tests	2017-08-08 11:38:31 +01:00
Brian Gough	849e905efb	simplify incremental request to docupdater ... if project state hasn't changed, get the docs from the docupdater -- we check/set the hash and return the docs in a single request. Otherwise do a full request from mongo.	2017-08-07 14:45:04 +01:00
Brian Gough	38c879faf2	improve comment about ClsiStateManager hash	2017-08-03 12:15:27 +01:00
Brian Gough	6d331e8ffd	use projectStateUnchanged instead of stateOk	2017-08-03 12:10:23 +01:00
Brian Gough	f44b844d74	refer to project state as projectStateHash	2017-08-03 12:08:11 +01:00
Brian Gough	5c02255e07	use syncType and syncState for clsi state options	2017-08-03 11:44:10 +01:00
Brian Gough	fb29ac3031	clean up logging	2017-08-03 11:36:59 +01:00
Brian Gough	0a859d3b33	clean up state manager	2017-08-03 11:36:23 +01:00
Brian Gough	a4117487e9	switch from mongoose to mongojs in ClsiManager ... for efficiency	2017-08-03 11:35:42 +01:00
Brian Gough	17b1075dc9	add rootFolder to attributes in Clsi request	2017-08-03 10:20:57 +01:00