112 commits

Author	SHA1	Message	Date
James Allen	47b19818ff	Add in new comment op type	2016-12-12 17:53:43 +00:00
James Allen	e3fee1a1d1	Rename 'track changes entries' -> 'ranges'	2016-12-08 12:31:43 +00:00
James Allen	3ea2e07993	Add tests for fetching and flushing track changes entries to persistence layer	2016-12-02 12:01:23 +00:00
James Allen	418405e8b9	Return track changes entries in HTTP request	2016-12-02 11:37:27 +00:00
James Allen	4fadd75ef3	Track changes based on flag on op, not global setting	2016-12-02 11:04:21 +00:00
James Allen	889f5fdf9f	Fix ShareJsDB tests	2016-12-01 17:14:40 +00:00
James Allen	9ee913be39	Update PersistenceManagerTests	2016-12-01 16:40:15 +00:00
James Allen	d36f1a86f1	Merge branch 'master' into ja-track-changes ... Conflicts: app/coffee/PersistenceManager.coffee	2016-12-01 16:06:13 +00:00
James Allen	efab68e6b2	Add in guards on bad data from web API	2016-11-29 17:13:16 +00:00
James Allen	2852043a6d	Don't store doc version in Mongo directly, instead use docstore	2016-11-29 17:06:23 +00:00
James Allen	e739e86c48	Get basic ChangeTracker hooked up. WIP	2016-11-28 10:14:42 +00:00
James Allen	5ce15c4d60	Move check of zero length op array up a level	2016-09-12 10:37:41 +01:00
James Allen	993aab7a78	Don't try to redis rpush with no arguments	2016-09-12 10:37:32 +01:00
James Allen	6ca48523d9	mapSeries -> eachSeries	2016-09-09 15:28:27 +01:00
James Allen	2d82d56f93	Process updates one at a time and write into redis after each one	2016-09-09 11:01:14 +01:00
James Allen	c85fd7ef41	Merge branch 'master' into ja-consistency-refactor	2016-09-07 11:08:04 +01:00
Brian Gough	9bc7594226	clean up redis query	2016-09-05 10:31:54 +01:00
James Allen	8779f3f686	Only write DocOps atomically with version and lines, after all docs are applied	2016-08-23 16:00:46 +01:00
James Allen	87f3e5e809	Remove old migration code	2016-08-23 11:03:37 +01:00
James Allen	a1bac9719a	Update secondary timeout for multi commands too	2016-07-12 11:45:10 +01:00
James Allen	d0c54f1be6	Increase secondary timeout to 600ms	2016-07-12 11:11:55 +01:00
James Allen	4f34cb7363	Only set keys in migration if they exist	2016-07-08 11:45:36 +01:00
James Allen	8b090c0a60	Don't call sadd with blank args	2016-07-07 16:41:38 +01:00
James Allen	6bc78ccf77	Add in some null checks on comparing backend results	2016-07-07 11:24:33 +01:00
James Allen	1c62a1c5a1	Don't propagate error on secondary timeout	2016-07-07 09:46:15 +01:00
James Allen	edf1e1ab49	Initialize primaryResult to an array, not null	2016-07-06 15:02:05 +01:00
James Allen	59883023ca	Ignore different order of results from smembers	2016-07-06 14:24:27 +01:00
James Allen	89f90c1b04	Timeout secondary requests if they take longer than 200ms	2016-07-06 12:25:36 +01:00
James Allen	508a95c19b	Use the main redis instance for locks	2016-07-06 11:50:02 +01:00
James Allen	e912ccc562	Wait for both backends to return so that they are always in sync	2016-07-05 16:07:47 +01:00
James Allen	dfd45bd23c	Add timers to time how long each redis request takes	2016-07-04 11:14:23 +01:00
James Allen	149351aa49	Remove debugging log lines	2016-06-29 21:18:13 +01:00
James Allen	fde334acc4	Create script that will ensure data is migrate between redis backends	2016-06-29 12:57:56 +01:00
James Allen	dbe03e2757	Track metrics of when backends match or disagree	2016-06-28 16:49:15 +01:00
James Allen	da89ff7172	Add in external health check rather than internal	2016-06-23 18:00:03 +01:00
James Allen	414ab5d6a9	Refactor TCP redis monitoring to be more explicit	2016-06-23 15:59:25 +01:00
James Allen	8ef03c3d2f	Add in application layer monitoring of the health of each cluster node	2016-06-23 15:38:51 +01:00
James Allen	39d920682a	Only pass through keys to redis that are explicitly set in the config	2016-06-17 14:11:22 +01:00
James Allen	40b7da8edb	Support endpoints and masterName in redis config	2016-06-17 14:05:54 +01:00
James Allen	bc00aab7b1	Explicitly separate calls to web and docupdater redis instances	2016-06-17 12:17:22 +01:00
James Allen	9ff3026807	Merge branch 'master' into ja_redis_cluster	2016-06-09 10:00:58 +01:00
James Allen	48a92b28e5	Don't run redis commands in parallel for easier consistency reasoning	2016-06-09 09:54:13 +01:00
James Allen	c823e06912	Don't run redis commands in parallel for easier consistency reasoning	2016-06-08 16:42:09 +01:00
James Allen	c6605ed5f0	Fix misapplied arguments to multi commands	2016-06-08 16:41:58 +01:00
James Allen	b4936f62af	Check that return values from different redis backends match	2016-06-08 12:18:37 +01:00
James Allen	ef43e2b325	Configure backend database as redis cluster	2016-06-07 18:38:32 +01:00
James Allen	1db6f8f159	Pull out rclient into RedisBackend that supports sending requests to multiple rclients	2016-06-07 17:58:18 +01:00
James Allen	ea8f4a6d7a	Remove unused getDocOpsLength method	2016-06-01 11:58:15 +01:00
James Allen	05b09a447e	Ensure that all multi call keys will hash to the same node in cluster	2016-06-01 11:40:12 +01:00
James Allen	f707783aba	Remove old methods of triggering doc updates ... Remove the old pub/sub listener which is no longer used. Also remove the DocsWithPendingUpdates set, which used to track docs waiting to be updated. This was necessary incase messages were missed on the pub/sub channel, so we knew which docs still had pending updates. However, now we use the BLPOP queue, these updates just sit in the queue until a consumer comes back to continue consuming them.	2016-06-01 11:28:23 +01:00