94 commits

Author	SHA1	Message	Date
James Allen	2d82d56f93	Process updates one at a time and write into redis after each one	2016-09-09 11:01:14 +01:00
James Allen	c85fd7ef41	Merge branch 'master' into ja-consistency-refactor	2016-09-07 11:08:04 +01:00
Brian Gough	9bc7594226	clean up redis query	2016-09-05 10:31:54 +01:00
James Allen	8779f3f686	Only write DocOps atomically with version and lines, after all docs are applied	2016-08-23 16:00:46 +01:00
James Allen	87f3e5e809	Remove old migration code	2016-08-23 11:03:37 +01:00
Shane Kilkelly	39cc9cda6d	Remove docker-related config. test:acceptance:docker now depends on acceptance-test-runner-sharelatex.	2016-07-18 11:09:04 +01:00
Shane Kilkelly	4c0dc5a0ef	Increase timeout for test.	2016-07-08 16:05:36 +01:00
Shane Kilkelly	0a5f95f22d	Move docker-entrypoint.sh into the test/acceptance/scripts directory	2016-07-08 15:52:02 +01:00
Shane Kilkelly	32d06b805c	run only the acceptance tests inside container, rather than compile and test.	2016-07-08 15:15:41 +01:00
Shane Kilkelly	b16b34d7c6	only execute app inside container, instead of compile and execute.	2016-07-08 14:50:30 +01:00
Shane Kilkelly	6d9aecae4c	capture exit code of the test run.	2016-07-08 14:31:43 +01:00
Shane Kilkelly	879482a955	grunt scripts to run the docker acceptance tests	2016-07-08 12:09:01 +01:00
Shane Kilkelly	bca8f6c376	working docker image which can run the acceptance tests	2016-07-08 10:51:22 +01:00
Shane Kilkelly	c38d903f92	Basic docker file.	2016-07-07 16:07:14 +01:00
James Allen	1c62a1c5a1	Don't propagate error on secondary timeout	2016-07-07 09:46:15 +01:00
James Allen	59883023ca	Ignore different order of results from smembers	2016-07-06 14:24:27 +01:00
James Allen	89f90c1b04	Timeout secondary requests if they take longer than 200ms	2016-07-06 12:25:36 +01:00
James Allen	508a95c19b	Use the main redis instance for locks	2016-07-06 11:50:02 +01:00
James Allen	dfd45bd23c	Add timers to time how long each redis request takes	2016-07-04 11:14:23 +01:00
James Allen	fde334acc4	Create script that will ensure data is migrate between redis backends	2016-06-29 12:57:56 +01:00
James Allen	dbe03e2757	Track metrics of when backends match or disagree	2016-06-28 16:49:15 +01:00
James Allen	e04c946ecd	Fix unit tests	2016-06-23 18:04:26 +01:00
James Allen	da89ff7172	Add in external health check rather than internal	2016-06-23 18:00:03 +01:00
James Allen	414ab5d6a9	Refactor TCP redis monitoring to be more explicit	2016-06-23 15:59:25 +01:00
James Allen	8ef03c3d2f	Add in application layer monitoring of the health of each cluster node	2016-06-23 15:38:51 +01:00
James Allen	27a74d6b71	Improve consistency of stress test script	2016-06-21 14:31:20 +01:00
James Allen	392beac57b	Improve error reporting and retry ops in stress test script	2016-06-20 15:01:39 +01:00
James Allen	7915c2ab6c	Tweak stress test to work on server	2016-06-20 13:56:46 +01:00
James Allen	e5cf856ddf	Add in stress test script	2016-06-17 12:17:43 +01:00
James Allen	bc00aab7b1	Explicitly separate calls to web and docupdater redis instances	2016-06-17 12:17:22 +01:00
James Allen	9ff3026807	Merge branch 'master' into ja_redis_cluster	2016-06-09 10:00:58 +01:00
James Allen	48a92b28e5	Don't run redis commands in parallel for easier consistency reasoning	2016-06-09 09:54:13 +01:00
James Allen	c823e06912	Don't run redis commands in parallel for easier consistency reasoning	2016-06-08 16:42:09 +01:00
James Allen	b4936f62af	Check that return values from different redis backends match	2016-06-08 12:18:37 +01:00
James Allen	1db6f8f159	Pull out rclient into RedisBackend that supports sending requests to multiple rclients	2016-06-07 17:58:18 +01:00
James Allen	e38e883081	Clean up and consolidate RedisManager tests for easy refactoring later	2016-06-01 12:22:47 +01:00
James Allen	8004e022fe	Fix async version and fix affected unit tests	2016-06-01 11:49:24 +01:00
James Allen	05b09a447e	Ensure that all multi call keys will hash to the same node in cluster	2016-06-01 11:40:12 +01:00
James Allen	f707783aba	Remove old methods of triggering doc updates ... Remove the old pub/sub listener which is no longer used. Also remove the DocsWithPendingUpdates set, which used to track docs waiting to be updated. This was necessary incase messages were missed on the pub/sub channel, so we knew which docs still had pending updates. However, now we use the BLPOP queue, these updates just sit in the queue until a consumer comes back to continue consuming them.	2016-06-01 11:28:23 +01:00
James Allen	b2e58266ae	Remove unused total doc counting redis commands	2016-06-01 11:03:39 +01:00
James Allen	32ce44f9c6	Remove unused gzipping logic	2016-06-01 10:53:50 +01:00
James Allen	1d7cbd8a78	Fix unit tests	2016-05-31 13:32:15 +01:00
James Allen	11667e8250	Return semantic error if the requested ops are not available when getting a document	2016-05-31 13:24:19 +01:00
Shane Kilkelly	048fd19418	Add one to size of line. ... To account for newline characters in the original document	2016-05-12 09:26:50 +01:00
Shane Kilkelly	82d5a7fafd	check total size of lines, rather than content-length	2016-05-11 15:55:21 +01:00
Shane Kilkelly	210a61112b	reject payloads larger than 2MB for setDoc	2016-04-29 15:08:21 +01:00
James Allen	945c728db2	Use signed locks so only the locking party can remove their lock	2016-04-13 11:59:56 +01:00
James Allen	6c79ab4321	Don't let HTTP calls take longer than 5 seconds since we're inside a 30 second lock	2016-04-12 17:10:39 +01:00
James Allen	3c7c318ea0	Clean up and speed up unit tests by making sure requires are mocked	2016-01-20 17:36:06 +00:00
James Allen	af5d01e440	Flush track changes when unloading data from redis	2016-01-20 15:05:31 +00:00