559 commits

Author	SHA1	Message	Date
Brian Gough	33fadf51c1	fix getDocTimestamps for multiple docs	2019-09-30 13:50:25 +01:00
Brian Gough	89a90399fe	Merge pull request #89 from overleaf/bg-queue-deletes ... queue deletes for deferred processing	2019-09-30 11:14:08 +01:00
John Lees-Miller	c5a9105c33	Update config	2019-09-28 11:07:33 +01:00
Brian Gough	260923f291	keep flushQueuedProjects in the foreground	2019-09-27 10:46:24 +01:00
Brian Gough	7561e05660	check timestamps array length	2019-09-27 10:39:56 +01:00
Brian Gough	0f0682df43	allow flush to complete in acceptance test	2019-09-26 16:00:11 +01:00
Brian Gough	b7f3b848af	remove unused dryRun option ... Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>	2019-09-26 15:50:55 +01:00
Brian Gough	3bc176259b	fix log line	2019-09-26 15:46:54 +01:00
Brian Gough	8cdc8c410a	fix error logging	2019-09-26 15:46:45 +01:00
Brian Gough	fc62abfcfa	run flush of queued projects in the background	2019-09-26 15:46:14 +01:00
Brian Gough	ba35c73cb6	add comment about ZPOPMIN	2019-09-26 15:18:10 +01:00
Brian Gough	a709a0adaa	for simplicity keep the cutoff time the same while flushing the queue	2019-09-26 15:05:38 +01:00
Brian Gough	eae4b352ca	remove unnecessary check	2019-09-26 14:59:03 +01:00
Brian Gough	9f358ead9f	add an acceptance test for flush with queue processing	2019-09-26 10:55:05 +01:00
Brian Gough	b49621b3e9	add comments	2019-09-26 10:14:49 +01:00
Brian Gough	83dd43b809	add metric for queue length	2019-09-25 17:04:36 +01:00
Brian Gough	f6b2ac7360	queue deletes for deferred processing	2019-09-25 16:42:49 +01:00
Brian Gough	c32101f0fb	upgrade redis-sharelatex to 1.0.10	2019-09-19 16:22:43 +01:00
Brian Gough	912a3a7753	remove redis server-side hashing for performance ... we still compute the document hash in node, and check it on retrieval but we don't check the hash at the point of writing it in redis which was previously done with a redis Lua script.	2019-09-09 15:27:58 +01:00
Henry Oswald	aa15a76059	added log lines for all project flushing	2019-08-30 07:38:53 +01:00
Henry Oswald	0ae838dd2d	add logger into project flusher	2019-08-29 20:36:00 +01:00
Brian Gough	e75f2cc325	add unit tests for skipping history flush	2019-08-15 11:01:45 +01:00
Brian Gough	7493462154	add acceptance test for realtime shutdown	2019-08-15 10:38:07 +01:00
Brian Gough	a76e0dca88	skip history flush when project is cleared by realtime shutdown ... history is flushed by a background cron job anyway	2019-08-15 09:51:16 +01:00
Simon Detheridge	f49f5cbf05	Merge pull request #79 from overleaf/ho-mongo-connection-string ... use MONGO_CONNECTION_STRING in settings.defaults.coffee if set	2019-08-08 14:16:49 +01:00
Simon Detheridge	6721b904a7	Merge pull request #82 from overleaf/bg-mongo-health-check ... add a combined health check for mongo and redis	2019-08-08 14:16:38 +01:00
Simon Detheridge	06444d2cc4	Improve/fix serializers for update logging (#80) ... * Improve/fix serializers for update logging	2019-08-08 14:10:54 +01:00
Brian Gough	40f6494b19	add a combined health check for mongo and redis	2019-08-07 16:25:23 +01:00
Henry Oswald	aa1c74a979	use MONGO_CONNECTION_STRING in settings.defaults.coffee if set	2019-08-07 15:44:57 +01:00
Simon Detheridge	df9ca8b272	Add serializer to print only length of large fields in production	2019-07-31 16:42:28 +01:00
Brian Gough	618880f99d	remove unnecessary check for doc_id	2019-07-24 16:57:43 +01:00
Brian Gough	c9ccf62d71	support per-doc pubsub channels	2019-07-22 12:20:06 +01:00
Brian Gough	97487a077e	fix cluster/sentinel connection with real-time	2019-07-10 09:42:05 +01:00
Henry Oswald	06ad0f7acd	Merge pull request #75 from overleaf/ho-pubsub-connection ... Remove real time redis connection and consolidate on pubsub	2019-07-08 13:58:41 +01:00
Henry Oswald	374585f47d	Merge pull request #72 from overleaf/sk-dep-upgrades-2019-06 ... Update logger, metrics, and redis	2019-07-08 13:58:28 +01:00
Henry Oswald	3b3b2da0f5	add pubsub redis connection and remove real time redis connection	2019-07-04 13:34:31 +01:00
Henry Oswald	eed8e215d7	bump redis-sharelatex (and io redis) to 1.0.9	2019-07-02 16:58:30 +01:00
Shane Kilkelly	e8a8c446ed	Increase timeout on test?	2019-06-19 11:34:38 +01:00
Shane Kilkelly	080b482e51	Update logger, metrics, and redis ... Also fix acceptance tests, broken by a change in the redis driver behaviour. It now returns promises from most operations, which confuses mocha.	2019-06-19 10:25:54 +01:00
Henry Oswald	fdef197271	Merge branch 'master' into ho-detailed-flush-status	2019-06-13 14:33:22 +01:00
Henry Oswald	d9a737f97c	return failed and succesfully flushed projects when flushing everything	2019-06-13 14:21:38 +01:00
Brian Gough	e8dd1aae9c	Merge pull request #70 from overleaf/bg-metric-for-invalid-hash ... add metric for invalid hash and other sharejs errors	2019-06-12 13:50:34 +01:00
Brian Gough	d50b93df2f	add metric for invalid hash and other sharejs errors	2019-06-11 16:48:06 +01:00
Brian Gough	984b2d38e6	add unit tests	2019-06-11 16:33:14 +01:00
Brian Gough	966478cac4	fix hash check to use 'v' field instead of version	2019-06-11 14:11:46 +01:00
Brian Gough	e95059f98e	handle non-urgent flushes in background	2019-06-03 10:01:10 +01:00
Brian Gough	0bbfa7de27	Merge branch 'master' into bg-downgrade-delete-component-error	2019-05-08 09:07:02 +01:00
Henry Oswald	d169cca425	Merge branch 'master' into ho-flush-all-projects	2019-05-07 17:02:14 +01:00
Brian Gough	27a8248196	convert "Delete component" errors into warnings	2019-05-07 16:55:17 +01:00
Tim Alby	8b40da701e	update README ... - remove build status badge - change app name - update copyright notice - update links	2019-05-07 17:03:09 +02:00