400 commits

Author	SHA1	Message	Date
Brian Gough	9d931a3fa7	fix broken unit tests	2017-08-18 12:08:39 +01:00
Brian Gough	230c93766d	fix broken RedisManager unit test	2017-08-18 11:59:31 +01:00
Brian Gough	8d408de875	Merge pull request #56 from sharelatex/bg-compile-from-redis ... provide endpoint for current project docs in redis	2017-08-18 11:33:27 +01:00
Joe Green	0e8ce29414	compile acceptance tests	2017-08-14 15:33:30 +01:00
Joe Green	b1a7f779d1	add acceptance tests	2017-08-14 15:28:04 +01:00
Brian Gough	8372911a1b	return doclines as array from getProjectDocs ... for compatibility with getDoc	2017-08-11 16:56:30 +01:00
Brian Gough	5971a19084	added acceptance tests	2017-08-11 16:55:31 +01:00
Brian Gough	f6be68eb08	added missing unit tests file	2017-08-11 16:49:16 +01:00
Joe Green	1146253c0a	Create Jenkinsfile	2017-08-11 11:06:36 +01:00
Brian Gough	f696ccb0d9	add unit tests for getProjectDocs	2017-08-10 16:41:28 +01:00
Brian Gough	1eb80936f2	fix log line	2017-08-10 14:57:27 +01:00
Brian Gough	47bebf9638	no need to support optional keys.projectState	2017-08-09 16:47:09 +01:00
Brian Gough	bd6b5b2c59	use v instead of rev in getProjectDocs	2017-08-09 15:30:53 +01:00
Brian Gough	ca7d9dce85	fix unit tests	2017-08-08 09:40:39 +01:00
Brian Gough	9f3ec72f81	switch to single get/set method for getProjectDocs ... if project state hasn't changed, return the docs. Otherwise set the hash and return a 409 Conflict response.	2017-08-07 14:43:28 +01:00
Brian Gough	f5f516a910	delete clsi state when deleting doc (if used)	2017-08-03 15:04:12 +01:00
Brian Gough	860537146b	avoid logging doclines in getProjectDocs	2017-08-03 15:04:12 +01:00
Brian Gough	80be5adc0d	change filter to exclude in getProjectDocs	2017-08-03 15:04:06 +01:00
Brian Gough	54e02cd895	update docupdater endpoint to /project/id/docs	2017-08-03 14:41:34 +01:00
Brian Gough	0109585092	provide endpoint for current project docs in redis	2017-08-02 13:42:10 +01:00
Shane Kilkelly	c3465cf3d4	Reduce retry delay to 10ms, and refactor	2017-07-18 15:28:18 +01:00
Shane Kilkelly	dc8d08c030	Update acceptance tests to reflect retry logic	2017-07-17 11:13:21 +01:00
Shane Kilkelly	0d46c3e2b3	WIP: auto-retry web requests	2017-07-17 10:08:21 +01:00
Brian Gough	c21cc013f3	use null instead of err in callbacks on success	2017-07-12 10:48:32 +01:00
Brian Gough	d2730c8d6b	unit tests for locking timeouts	2017-07-12 10:48:32 +01:00
Brian Gough	bb0dc4b4d5	release the lock if it took too long to acquire it	2017-07-12 10:48:32 +01:00
Brian Gough	54c0fc1180	put a 5 second timeout in getting redis lock	2017-07-11 14:28:58 +01:00
Brian Gough	ffea31290c	return total time from the profile .end() method	2017-07-11 14:28:58 +01:00
Brian Gough	41c2bd7fcd	Merge pull request #53 from sharelatex/bg-timeout-on-slow-prev-ops ... add a timeout for getPreviousDocOps	2017-06-27 13:26:02 +01:00
Brian Gough	59ace9d5c5	avoid long lines in timeout check	2017-06-26 16:36:05 +01:00
Brian Gough	8e1e14c9ca	update to redis-sharelatex 1.0.3 ... adds keepalive support	2017-06-26 13:49:44 +01:00
Brian Gough	963e513057	add a timeout for getPreviousDocOps ... it uses several redis operations and this makes it prone to subsequent timeouts if getDoc succeeds but is slow	2017-06-23 15:50:21 +01:00
James Allen	7b6966899b	Add script to expire existing DocOps lists	2017-06-14 12:09:34 +01:00
James Allen	54f128289f	Merge pull request #50 from sharelatex/bg-metric-for-unlock-errors ... add metric for unlock errors	2017-06-13 14:46:40 +01:00
James Allen	7d22e4422c	Merge pull request #49 from sharelatex/bg-add-redis-timeout ... add redis timeout	2017-06-13 14:46:32 +01:00
James Allen	30f57a7bfc	Merge pull request #51 from sharelatex/ja-expires-ordering ... Ensure expires is called after the key is created	2017-06-13 14:05:22 +01:00
James Allen	770b53f6af	Add missing mocks to speed up tests and clean up output	2017-06-08 16:43:27 +01:00
James Allen	a3d7260619	Ensure expires is called after the key is created	2017-06-08 16:43:06 +01:00
Brian Gough	2e5d57cd99	don't increase rate limit when tasks are failing	2017-06-07 11:47:28 +01:00
Brian Gough	1ef258c878	add a timeout on the redis getDoc request	2017-06-07 11:34:42 +01:00
Brian Gough	673f4228ce	add metric for unlock errors ... sentry does not record them reliably, due to rate limiting	2017-06-05 16:29:58 +01:00
Brian Gough	729216c9b1	add unit tests for DocsIn check	2017-05-31 16:08:45 +01:00
Brian Gough	12e8eaa9b6	fix bug in doclines check	2017-05-31 16:08:33 +01:00
Brian Gough	bed64d26c8	check if doc is missing from DocsIn set	2017-05-31 15:33:59 +01:00
Brian Gough	fa089dc28d	avoid graphite drift in active worker count	2017-05-30 14:37:01 +01:00
Brian Gough	891ffda3bf	put a rate limit on worker calls	2017-05-26 15:08:35 +01:00
Brian Gough	c43f97e8e3	add metric for active workers	2017-05-24 12:12:15 +01:00
Brian Gough	a3947c587f	fix bug, jsonOps is array not string	2017-05-22 15:20:28 +01:00
Brian Gough	54ad45c870	add null byte check for ranges	2017-05-22 15:00:05 +01:00
Brian Gough	a13f055d62	add null byte check for ops	2017-05-22 15:00:00 +01:00
Brian Gough	78828ebbc5	fine grained logging for cpu usage in applyUpdate	2017-05-19 16:00:16 +01:00
Brian Gough	a550078071	replace verbose logging by slow query log	2017-05-18 15:04:12 +01:00
Brian Gough	95596061d4	clarify calculation of process.hrtime	2017-05-18 15:02:54 +01:00
Brian Gough	e00f4dde8e	log profilers start and end times for reference	2017-05-18 15:02:44 +01:00
Brian Gough	e11f64f83a	increase profiling cutoff to 1s	2017-05-18 15:02:08 +01:00
Brian Gough	b6efb051e1	add profiling	2017-05-18 11:17:08 +01:00
Brian Gough	387ecac6cb	log for each get/try/release lock	2017-05-18 11:17:08 +01:00
Brian Gough	dff6e2c3da	bug fix in backoff	2017-05-16 16:31:28 +01:00
Brian Gough	c7d8fbbb8a	exponential backoff for lock up to maximum time	2017-05-16 15:22:02 +01:00
Brian Gough	b15d2ef796	Revert "fail safely if doc cannot be loaded"	2017-05-15 16:18:40 +01:00
Paulo Jorge Reis	2c3f0aa6b4	Merge pull request #34 from sharelatex/pr-bulk-actions ... Add methods to bulk accept changes.	2017-05-15 15:14:33 +01:00
Paulo Reis	ef7fa59257	Update ranges tracker.	2017-05-15 11:11:14 +01:00
Paulo Reis	05d7d1b8c6	Merge branch 'master' into pr-bulk-actions	2017-05-15 10:35:16 +01:00
Paulo Reis	36d16a76e0	Unit test accept changes in the ranges manager.	2017-05-12 14:42:40 +01:00
Brian Gough	2ee40d0748	ensure document is in redis before consuming ops	2017-05-12 14:22:18 +01:00
Paulo Reis	d53266ea2d	Update web API stub.	2017-05-11 12:00:34 +01:00
Paulo Reis	5677c7ad9a	Update range tracker.	2017-05-11 11:31:19 +01:00
Paulo Reis	f690ef078a	Update tests; add new ones for multiple changes.	2017-05-09 16:16:25 +01:00
Brian Gough	36407ac726	rename HistoryManager pushUncompressedHistoryOps	2017-05-09 12:02:27 +01:00
Brian Gough	7ce6285e3d	increase flush threshold to 100 ops	2017-05-09 10:44:26 +01:00
Brian Gough	fdf5e8e0b8	get history ops length directly from redis update	2017-05-09 10:34:31 +01:00
Brian Gough	2d158b03d7	rename pushUncompressedHistoryOps	2017-05-09 09:32:56 +01:00
Paulo Reis	144804e516	Update tests with new methods.	2017-05-08 16:34:17 +01:00
Paulo Reis	d7ac57d054	Use new bulk method.	2017-05-08 16:08:42 +01:00
Paulo Reis	70341e4e80	Add bulk method from range tracker.	2017-05-08 16:08:34 +01:00
Brian Gough	e2f70aca1a	fix tests for redis cluster	2017-05-08 16:02:49 +01:00
Brian Gough	79d8fced49	make history update more atomic	2017-05-08 15:58:29 +01:00
Paulo Reis	e96e6d6a29	Remove unused method.	2017-05-05 15:17:19 +01:00
Paulo Reis	9b8a2c3030	Consolidate methods to accept changes.	2017-05-05 15:12:06 +01:00
James Allen	87a5e30a0b	Bump redis-sharelatex version	2017-05-05 13:37:58 +01:00
James Allen	7456238a71	Bump redis-sharelatex version	2017-05-04 15:42:10 +01:00
Paulo Reis	d530ee9501	Add methods to bulk accept changes.	2017-05-04 15:32:54 +01:00
James Allen	5f93640077	Add scripts for testing cluster failover scenarios	2017-05-04 11:14:17 +01:00
James Allen	4104ca4889	Add in separate redis config for the lock and fix a few web -> realtime	2017-05-03 16:27:32 +01:00
James Allen	5c3661d6d9	Merge remote-tracking branch 'origin/bg-redis-cluster-refactor' into ja-redis-cluster-refactor	2017-05-02 15:39:54 +01:00
James Allen	ed493d8ad3	Rename web -> realtime for consistency with realtime	2017-05-02 15:38:33 +01:00
Brian Gough	64aef0b55a	fix acceptance test to work with redis cluster too	2017-04-27 10:42:43 +01:00
Brian Gough	dc77bc207d	change acceptance test to use redis history client	2017-04-24 16:31:23 +01:00
James Allen	c5449ae282	Split out redis config for real-time and track-changes into separate cluster-compatible configs	2017-04-13 17:00:42 +01:00
James Allen	f21208e841	Use new redis-sharelatex instead of RedisBackend for cluster abstraction	2017-04-12 14:53:03 +01:00
Brian Gough	5583764d05	fix callback prototype in getLock	2017-03-30 15:31:47 +01:00
Brian Gough	c11618b475	improve unlock error handling	2017-03-30 15:31:34 +01:00
Brian Gough	b2b4bc44df	upgrade logger-sharelatex to v1.5.6	2017-03-30 14:16:35 +01:00
Brian Gough	527c02a00b	log op versions pushed to track changes	2017-03-30 11:20:50 +01:00
Shane Kilkelly	64658d0034	Add a .nvmrc file	2017-03-27 14:50:09 +01:00
James Allen	0245bfd031	Support an incoming undo flag for history restores	2017-03-21 11:20:38 +00:00
James Allen	a3a5524778	Fix broken unit test from merge	2017-03-20 10:37:02 +00:00
James Allen	199079a5b7	Merge branch 'master' into ja-cut-and-paste-comments	2017-03-20 10:29:01 +00:00
James Allen	9c5299ec7c	Update RangeTracker to support upserting and moving comments for cut and paste	2017-03-16 15:49:41 +00:00
James Allen	5499a67d78	Add in a consistency check after applying updates that ranges still match	2017-03-15 14:12:06 +00:00