514 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Shane Kilkelly	c3465cf3d4	Reduce retry delay to 10ms, and refactor	2017-07-18 15:28:18 +01:00
Shane Kilkelly	dc8d08c030	Update acceptance tests to reflect retry logic	2017-07-17 11:13:21 +01:00
Shane Kilkelly	0d46c3e2b3	WIP: auto-retry web requests	2017-07-17 10:08:21 +01:00
Brian Gough	653f16e3e6	Merge pull request #54 from sharelatex/bg-timeout-on-slow-lock ... add timeout on slow lock	2017-07-12 13:43:34 +01:00
Brian Gough	c21cc013f3	use null instead of err in callbacks on success	2017-07-12 10:48:32 +01:00
Brian Gough	d2730c8d6b	unit tests for locking timeouts	2017-07-12 10:48:32 +01:00
Brian Gough	bb0dc4b4d5	release the lock if it took too long to acquire it	2017-07-12 10:48:32 +01:00
Brian Gough	54c0fc1180	put a 5 second timeout in getting redis lock	2017-07-11 14:28:58 +01:00
Brian Gough	ffea31290c	return total time from the profile .end() method	2017-07-11 14:28:58 +01:00
Brian Gough	41c2bd7fcd	Merge pull request #53 from sharelatex/bg-timeout-on-slow-prev-ops ... add a timeout for getPreviousDocOps	2017-06-27 13:26:02 +01:00
Brian Gough	59ace9d5c5	avoid long lines in timeout check	2017-06-26 16:36:05 +01:00
Brian Gough	8e1e14c9ca	update to redis-sharelatex 1.0.3 ... adds keepalive support	2017-06-26 13:49:44 +01:00
Brian Gough	963e513057	add a timeout for getPreviousDocOps ... it uses several redis operations and this makes it prone to subsequent timeouts if getDoc succeeds but is slow	2017-06-23 15:50:21 +01:00
James Allen	9f31402fb4	Merge pull request #52 from sharelatex/ja-expire-docops ... Add script to expire existing DocOps lists	2017-06-14 15:31:29 +01:00
James Allen	7b6966899b	Add script to expire existing DocOps lists	2017-06-14 12:09:34 +01:00
James Allen	54f128289f	Merge pull request #50 from sharelatex/bg-metric-for-unlock-errors ... add metric for unlock errors	2017-06-13 14:46:40 +01:00
James Allen	7d22e4422c	Merge pull request #49 from sharelatex/bg-add-redis-timeout ... add redis timeout	2017-06-13 14:46:32 +01:00
James Allen	30f57a7bfc	Merge pull request #51 from sharelatex/ja-expires-ordering ... Ensure expires is called after the key is created	2017-06-13 14:05:22 +01:00
James Allen	770b53f6af	Add missing mocks to speed up tests and clean up output	2017-06-08 16:43:27 +01:00
James Allen	a3d7260619	Ensure expires is called after the key is created	2017-06-08 16:43:06 +01:00
Brian Gough	2e5d57cd99	don't increase rate limit when tasks are failing	2017-06-07 11:47:28 +01:00
Brian Gough	1ef258c878	add a timeout on the redis getDoc request	2017-06-07 11:34:42 +01:00
Brian Gough	673f4228ce	add metric for unlock errors ... sentry does not record them reliably, due to rate limiting	2017-06-05 16:29:58 +01:00
Brian Gough	7ecb4128de	Merge pull request #48 from sharelatex/bg-check-doc-in-project-set ... check if doc is missing from DocsIn set	2017-06-01 09:23:29 +01:00
Brian Gough	729216c9b1	add unit tests for DocsIn check	2017-05-31 16:08:45 +01:00
Brian Gough	12e8eaa9b6	fix bug in doclines check	2017-05-31 16:08:33 +01:00
Brian Gough	bed64d26c8	check if doc is missing from DocsIn set	2017-05-31 15:33:59 +01:00
Brian Gough	fa089dc28d	avoid graphite drift in active worker count	2017-05-30 14:37:01 +01:00
Brian Gough	9a8c0fda10	Merge pull request #47 from sharelatex/bg-limit-worker-calls ... limit worker calls	2017-05-30 13:58:54 +01:00
Brian Gough	891ffda3bf	put a rate limit on worker calls	2017-05-26 15:08:35 +01:00
Brian Gough	95046b1f91	Merge pull request #45 from sharelatex/bg-record-active-updates-metric ... add metric for active workers	2017-05-24 14:44:57 +01:00
Brian Gough	c43f97e8e3	add metric for active workers	2017-05-24 12:12:15 +01:00
Brian Gough	7497c066ee	Merge pull request #44 from sharelatex/bg-add-null-byte-checks ... add null byte checks	2017-05-22 15:25:02 +01:00
Brian Gough	a3947c587f	fix bug, jsonOps is array not string	2017-05-22 15:20:28 +01:00
Brian Gough	54ad45c870	add null byte check for ranges	2017-05-22 15:00:05 +01:00
Brian Gough	a13f055d62	add null byte check for ops	2017-05-22 15:00:00 +01:00
Brian Gough	6e26d89090	Merge pull request #43 from sharelatex/bg-log-apply-update ... fine grained logging for cpu usage in applyUpdate	2017-05-19 16:23:47 +01:00
Brian Gough	78828ebbc5	fine grained logging for cpu usage in applyUpdate	2017-05-19 16:00:16 +01:00
Brian Gough	5f45c5de04	Merge pull request #42 from sharelatex/bg-increase-logging ... increase docupdater logging	2017-05-19 10:05:56 +01:00
Brian Gough	a550078071	replace verbose logging by slow query log	2017-05-18 15:04:12 +01:00
Brian Gough	95596061d4	clarify calculation of process.hrtime	2017-05-18 15:02:54 +01:00
Brian Gough	e00f4dde8e	log profilers start and end times for reference	2017-05-18 15:02:44 +01:00
Brian Gough	e11f64f83a	increase profiling cutoff to 1s	2017-05-18 15:02:08 +01:00
Brian Gough	b6efb051e1	add profiling	2017-05-18 11:17:08 +01:00
Brian Gough	387ecac6cb	log for each get/try/release lock	2017-05-18 11:17:08 +01:00
Brian Gough	4622362e42	Merge pull request #39 from sharelatex/bg-lock-exponential-backoff ... exponential backoff for lock up to maximum time	2017-05-16 17:01:43 +01:00
Brian Gough	dff6e2c3da	bug fix in backoff	2017-05-16 16:31:28 +01:00
Brian Gough	c7d8fbbb8a	exponential backoff for lock up to maximum time	2017-05-16 15:22:02 +01:00
Brian Gough	0e0f4c34f0	Merge pull request #41 from sharelatex/revert-38-bg-load-doc-safely-in-lock ... Revert "fail safely if doc cannot be loaded"	2017-05-15 16:19:10 +01:00
Brian Gough	b15d2ef796	Revert "fail safely if doc cannot be loaded"	2017-05-15 16:18:40 +01:00