383 commits

Author	SHA1	Message	Date
Brian Gough	af93193d6e	remove new_project_history and use project_history instead	2020-04-21 14:43:48 +01:00
Brian Gough	ac68f59487	Merge branch 'master' into bg-use-separate-redis-for-project-history	2020-04-16 15:48:21 +01:00
Brian Gough	beb3691795	add metrics for redis get/update	2020-04-06 10:00:48 +01:00
Brian Gough	2b72ec49a1	add comments for redis metrics	2020-04-02 11:33:52 +01:00
Brian Gough	21824d49da	Merge branch 'bg-add-queue-metrics' of github.com:overleaf/document-updater into bg-add-queue-metrics	2020-04-01 16:04:52 +01:00
Brian Gough	3a8c362fba	add doclines set/del metric	2020-04-01 15:59:25 +01:00
Brian Gough	00b11bda96	use separate loop for pendingUpdates metric	2020-04-01 14:50:55 +01:00
Jakob Ackermann	17c2add0cf	[misc] track redis pub/sub payload sizes on publish	2020-03-30 11:31:43 +02:00
Brian Gough	1a0550364d	add metric for getdoc bytes	2020-03-25 14:27:41 +00:00
Brian Gough	891fcc696b	add metric for pending updates queue	2020-03-25 14:27:41 +00:00
Brian Gough	e293d86c14	add metric for project history queue	2020-03-25 12:15:16 +00:00
Eric Mc Sween	ff32104fe6	Merge pull request #123 from overleaf/em-doc-hard-delete ... Add ignore_flush_errors option to the doc delete endpoint	2020-03-10 10:11:00 -04:00
Eric Mc Sween	d9caced0d6	Change skip_flush option to ignore_flush_errors in delete doc ... Instead of skipping the flush, we'll still try to flush and proceed with the doc deletion, even when the flush fails.	2020-03-10 09:40:49 -04:00
Eric Mc Sween	9b70eb75b3	Rename flush param to skip_flush in delete doc ... Also move it to the query string instead of the body.	2020-03-09 16:27:32 -04:00
Eric Mc Sween	c09bc0e868	Add a "flush: false" option to the doc delete endpoint ... This will delete the document from Redis without flushing to web, docstore or history. To be used when something is broken.	2020-03-07 08:59:15 -05:00
nate stemen	ffd8d0745d	use empty object for ranges if it doesn't exist	2020-03-06 13:49:30 -05:00
Brian Gough	0419039d4d	Merge branch 'master' into bg-use-separate-redis-for-project-history	2020-02-21 14:13:33 +00:00
Brian Gough	338d3609f5	add comment about null byte check	2020-01-30 15:17:13 +00:00
Brian Gough	544ae05212	added note about rollback	2020-01-23 16:22:26 +00:00
Brian Gough	626e19ed1a	add logging of migration phase at startup	2020-01-23 15:46:54 +00:00
Brian Gough	d5a2b96df9	add note about deleting the migration key entries	2020-01-23 14:36:59 +00:00
Brian Gough	7036803acf	add missing argument to metrics.inc ... also track retries rather than attempts (which is always 1 for a successful request)	2020-01-14 15:00:21 +00:00
Brian Gough	3caa0e7c05	add failure/retry metrics for web-api requests	2020-01-14 13:53:50 +00:00
Brian Gough	a638ef4251	add comment about locking in redis migration	2020-01-13 15:56:28 +00:00
Brian Gough	27044c2d02	allow migration phase to be modified at runtime for testing	2020-01-06 16:46:35 +00:00
Brian Gough	8ae95ebf60	fix rclient check in migration metrics	2020-01-06 16:45:36 +00:00
Brian Gough	97cbf46160	add metrics for migration	2019-12-16 11:46:35 +00:00
Brian Gough	a2e63d009e	fix migration phase check	2019-12-16 09:55:26 +00:00
Brian Gough	d0c5eb5698	support migration of project history keys to separate redis instance	2019-12-13 16:38:41 +00:00
Brian Gough	ad19fee667	add setting so that double flush is the default ... can be disabled to stop flushing to track-changes	2019-11-25 13:36:25 +00:00
Brian Gough	4f6583bbf2	fix getDocVersion and add tests	2019-11-25 13:28:36 +00:00
Brian Gough	68e12f4d2d	add metrics for queue operations	2019-11-25 10:51:10 +00:00
Brian Gough	8b73bb9f13	Merge branch 'master' into bg-filter-track-changes-updates	2019-11-22 10:41:33 +00:00
Brian Gough	b7055eecee	add metrics for history flushes	2019-11-22 09:14:32 +00:00
Brian Gough	65cf4cf7c7	make flush to track-changes failsafe	2019-11-21 14:58:35 +00:00
Brian Gough	dcd7649bad	filter track-changes updates for projects using project-history	2019-11-19 10:02:56 +00:00
Brian Gough	d82b180b76	avoid project history queues building up with deferred flush	2019-10-03 04:05:24 +01:00
Brian Gough	c1454bc4ac	Merge pull request #92 from overleaf/bg-flush-queue-prod-fixes ... add continuous background flush	2019-10-02 13:11:00 +01:00
Brian Gough	0c14b7d2f8	add comment about background flush limit	2019-10-01 15:06:01 +01:00
Brian Gough	2845b23b70	add smoothing of delete spikes	2019-10-01 15:01:53 +01:00
Brian Gough	2c22a60052	add random jitter to cutoff time	2019-10-01 15:01:20 +01:00
Brian Gough	a32495d2b4	make background flush more adaptive	2019-10-01 14:09:41 +01:00
Brian Gough	73b4262186	add continuous background flush	2019-09-30 16:05:53 +01:00
Brian Gough	33fadf51c1	fix getDocTimestamps for multiple docs	2019-09-30 13:50:25 +01:00
Brian Gough	260923f291	keep flushQueuedProjects in the foreground	2019-09-27 10:46:24 +01:00
Brian Gough	7561e05660	check timestamps array length	2019-09-27 10:39:56 +01:00
Brian Gough	b7f3b848af	remove unused dryRun option ... Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>	2019-09-26 15:50:55 +01:00
Brian Gough	3bc176259b	fix log line	2019-09-26 15:46:54 +01:00
Brian Gough	8cdc8c410a	fix error logging	2019-09-26 15:46:45 +01:00
Brian Gough	fc62abfcfa	run flush of queued projects in the background	2019-09-26 15:46:14 +01:00