104 commits

Author	SHA1	Message	Date
Brian Gough	c1454bc4ac	Merge pull request #92 from overleaf/bg-flush-queue-prod-fixes ... add continuous background flush	2019-10-02 13:11:00 +01:00
Brian Gough	2845b23b70	add smoothing of delete spikes	2019-10-01 15:01:53 +01:00
Brian Gough	33fadf51c1	fix getDocTimestamps for multiple docs	2019-09-30 13:50:25 +01:00
Brian Gough	ba35c73cb6	add comment about ZPOPMIN	2019-09-26 15:18:10 +01:00
Brian Gough	b49621b3e9	add comments	2019-09-26 10:14:49 +01:00
Brian Gough	83dd43b809	add metric for queue length	2019-09-25 17:04:36 +01:00
Brian Gough	f6b2ac7360	queue deletes for deferred processing	2019-09-25 16:42:49 +01:00
Brian Gough	912a3a7753	remove redis server-side hashing for performance ... we still compute the document hash in node, and check it on retrieval but we don't check the hash at the point of writing it in redis which was previously done with a redis Lua script.	2019-09-09 15:27:58 +01:00
Tim Alby	c1c23e4bee	record last author id on document flush ... This is a multi-steps process: * get a update's `user_id` from the metadata * store the `user_id` (`lastUpdatedBy`) and current date (`lastUpdatedAt`) for the document in Redis on every updates * fetch `lastUpdatedAt` and `lastUpdatedBy` from Redis on document flush * send the data to web to be persisted in Mongo	2019-05-02 11:10:02 +01:00
James Allen	3d76f4b9bf	Record a snapshot to mongo when a doc's comments/changes get collapsed	2019-04-11 13:27:46 +01:00
Hayden Faulds	af92ca70a1	coerce projectHistoryId to integer after reading from Redis	2018-04-23 15:19:06 +01:00
Hayden Faulds	1b63141e49	Revert "Revert "Add projectHistoryId to updates""	2018-04-23 12:08:04 +01:00
Hayden Faulds	3cc20d6311	Revert "Add projectHistoryId to updates"	2018-04-23 11:41:29 +01:00
Hayden Faulds	25c0be4eaa	add projectHistryId to project structure updates	2018-04-13 15:22:05 +01:00
Hayden Faulds	fb1852a593	cache projectHistoryId with doc in Redis	2018-04-13 15:22:05 +01:00
Hayden Faulds	4a5731440a	Merge pull request #21 from sharelatex/hof-history-sync ... resync project history endpoint	2018-03-14 10:40:59 +00:00
Hayden Faulds	1226f96fc3	extract ProjectHistoryRedisManager	2018-03-09 15:36:38 +00:00
Hayden Faulds	fc6ef6ea7a	improve logging	2018-03-09 12:21:48 +00:00
Hayden Faulds	088ab6e7ba	rename queueResyncDocContents -> queueResyncDocContent	2018-03-09 12:21:37 +00:00
Hayden Faulds	64d1fe13bc	additional logging for project history resync	2018-03-07 16:24:22 +00:00
Hayden Faulds	d46217be52	queue doc content sync updates	2018-03-02 15:02:27 +00:00
Hayden Faulds	52d6b710bd	add ability to resync project structure	2018-03-02 15:01:53 +00:00
James Allen	939eaa2d4b	Don't allow a document to be loaded without a pathname	2018-03-02 10:02:49 +00:00
James Allen	ccf6cb1a18	Provide hosts as environment settings, add npm run start script and update coffeescript	2017-12-29 08:13:16 +00:00
Hayden Faulds	a3420b1236	version entity additions	2017-11-10 14:54:56 +00:00
Hayden Faulds	7e86afe55e	version file renames	2017-11-08 09:20:38 +00:00
Hayden Faulds	6d571e6d23	version document renames	2017-11-08 09:20:37 +00:00
Hayden Faulds	b8052e7612	allow settings.redis.project_history to be undefined	2017-10-25 13:56:38 +01:00
Hayden Faulds	5b7be93c79	Merge pull request #2 from sharelatex/hof-doc-pathname ... Add metadata to updates for history	2017-10-25 13:13:00 +01:00
Brian Gough	20655b6eeb	remove unused requires of async	2017-10-23 15:59:32 +01:00
Hayden Faulds	d003aef31c	conditionally enqueue history updates for project	2017-10-23 15:32:25 +01:00
Hayden Faulds	cbdace7386	store pathname in Redis cache	2017-10-23 14:23:21 +01:00
Brian Gough	2bbbf3c005	add unflushed time to doc in redis	2017-10-06 12:34:28 +01:00
Brian Gough	f34c12fdc7	add endpoint to clear project state	2017-09-08 15:50:26 +01:00
Brian Gough	47bebf9638	no need to support optional keys.projectState	2017-08-09 16:47:09 +01:00
Brian Gough	9f3ec72f81	switch to single get/set method for getProjectDocs ... if project state hasn't changed, return the docs. Otherwise set the hash and return a 409 Conflict response.	2017-08-07 14:43:28 +01:00
Brian Gough	f5f516a910	delete clsi state when deleting doc (if used)	2017-08-03 15:04:12 +01:00
Brian Gough	0109585092	provide endpoint for current project docs in redis	2017-08-02 13:42:10 +01:00
Brian Gough	59ace9d5c5	avoid long lines in timeout check	2017-06-26 16:36:05 +01:00
Brian Gough	963e513057	add a timeout for getPreviousDocOps ... it uses several redis operations and this makes it prone to subsequent timeouts if getDoc succeeds but is slow	2017-06-23 15:50:21 +01:00
James Allen	7d22e4422c	Merge pull request #49 from sharelatex/bg-add-redis-timeout ... add redis timeout	2017-06-13 14:46:32 +01:00
James Allen	a3d7260619	Ensure expires is called after the key is created	2017-06-08 16:43:06 +01:00
Brian Gough	1ef258c878	add a timeout on the redis getDoc request	2017-06-07 11:34:42 +01:00
Brian Gough	12e8eaa9b6	fix bug in doclines check	2017-05-31 16:08:33 +01:00
Brian Gough	bed64d26c8	check if doc is missing from DocsIn set	2017-05-31 15:33:59 +01:00
Brian Gough	a3947c587f	fix bug, jsonOps is array not string	2017-05-22 15:20:28 +01:00
Brian Gough	54ad45c870	add null byte check for ranges	2017-05-22 15:00:05 +01:00
Brian Gough	a13f055d62	add null byte check for ops	2017-05-22 15:00:00 +01:00
Brian Gough	b15d2ef796	Revert "fail safely if doc cannot be loaded"	2017-05-15 16:18:40 +01:00
Brian Gough	2ee40d0748	ensure document is in redis before consuming ops	2017-05-12 14:22:18 +01:00