350 commits

Author	SHA1	Message	Date
Brian Gough	78828ebbc5	fine grained logging for cpu usage in applyUpdate	2017-05-19 16:00:16 +01:00
Brian Gough	a550078071	replace verbose logging by slow query log	2017-05-18 15:04:12 +01:00
Brian Gough	95596061d4	clarify calculation of process.hrtime	2017-05-18 15:02:54 +01:00
Brian Gough	e00f4dde8e	log profilers start and end times for reference	2017-05-18 15:02:44 +01:00
Brian Gough	e11f64f83a	increase profiling cutoff to 1s	2017-05-18 15:02:08 +01:00
Brian Gough	b6efb051e1	add profiling	2017-05-18 11:17:08 +01:00
Brian Gough	387ecac6cb	log for each get/try/release lock	2017-05-18 11:17:08 +01:00
Brian Gough	dff6e2c3da	bug fix in backoff	2017-05-16 16:31:28 +01:00
Brian Gough	c7d8fbbb8a	exponential backoff for lock up to maximum time	2017-05-16 15:22:02 +01:00
Brian Gough	b15d2ef796	Revert "fail safely if doc cannot be loaded"	2017-05-15 16:18:40 +01:00
Paulo Jorge Reis	2c3f0aa6b4	Merge pull request #34 from sharelatex/pr-bulk-actions ... Add methods to bulk accept changes.	2017-05-15 15:14:33 +01:00
Paulo Reis	ef7fa59257	Update ranges tracker.	2017-05-15 11:11:14 +01:00
Paulo Reis	05d7d1b8c6	Merge branch 'master' into pr-bulk-actions	2017-05-15 10:35:16 +01:00
Paulo Reis	36d16a76e0	Unit test accept changes in the ranges manager.	2017-05-12 14:42:40 +01:00
Brian Gough	2ee40d0748	ensure document is in redis before consuming ops	2017-05-12 14:22:18 +01:00
Paulo Reis	d53266ea2d	Update web API stub.	2017-05-11 12:00:34 +01:00
Paulo Reis	5677c7ad9a	Update range tracker.	2017-05-11 11:31:19 +01:00
Paulo Reis	f690ef078a	Update tests; add new ones for multiple changes.	2017-05-09 16:16:25 +01:00
Brian Gough	36407ac726	rename HistoryManager pushUncompressedHistoryOps	2017-05-09 12:02:27 +01:00
Brian Gough	7ce6285e3d	increase flush threshold to 100 ops	2017-05-09 10:44:26 +01:00
Brian Gough	fdf5e8e0b8	get history ops length directly from redis update	2017-05-09 10:34:31 +01:00
Brian Gough	2d158b03d7	rename pushUncompressedHistoryOps	2017-05-09 09:32:56 +01:00
Paulo Reis	144804e516	Update tests with new methods.	2017-05-08 16:34:17 +01:00
Paulo Reis	d7ac57d054	Use new bulk method.	2017-05-08 16:08:42 +01:00
Paulo Reis	70341e4e80	Add bulk method from range tracker.	2017-05-08 16:08:34 +01:00
Brian Gough	e2f70aca1a	fix tests for redis cluster	2017-05-08 16:02:49 +01:00
Brian Gough	79d8fced49	make history update more atomic	2017-05-08 15:58:29 +01:00
Paulo Reis	e96e6d6a29	Remove unused method.	2017-05-05 15:17:19 +01:00
Paulo Reis	9b8a2c3030	Consolidate methods to accept changes.	2017-05-05 15:12:06 +01:00
James Allen	87a5e30a0b	Bump redis-sharelatex version	2017-05-05 13:37:58 +01:00
James Allen	7456238a71	Bump redis-sharelatex version	2017-05-04 15:42:10 +01:00
Paulo Reis	d530ee9501	Add methods to bulk accept changes.	2017-05-04 15:32:54 +01:00
James Allen	5f93640077	Add scripts for testing cluster failover scenarios	2017-05-04 11:14:17 +01:00
James Allen	4104ca4889	Add in separate redis config for the lock and fix a few web -> realtime	2017-05-03 16:27:32 +01:00
James Allen	5c3661d6d9	Merge remote-tracking branch 'origin/bg-redis-cluster-refactor' into ja-redis-cluster-refactor	2017-05-02 15:39:54 +01:00
James Allen	ed493d8ad3	Rename web -> realtime for consistency with realtime	2017-05-02 15:38:33 +01:00
Brian Gough	64aef0b55a	fix acceptance test to work with redis cluster too	2017-04-27 10:42:43 +01:00
Brian Gough	dc77bc207d	change acceptance test to use redis history client	2017-04-24 16:31:23 +01:00
James Allen	c5449ae282	Split out redis config for real-time and track-changes into separate cluster-compatible configs	2017-04-13 17:00:42 +01:00
James Allen	f21208e841	Use new redis-sharelatex instead of RedisBackend for cluster abstraction	2017-04-12 14:53:03 +01:00
Brian Gough	5583764d05	fix callback prototype in getLock	2017-03-30 15:31:47 +01:00
Brian Gough	c11618b475	improve unlock error handling	2017-03-30 15:31:34 +01:00
Brian Gough	b2b4bc44df	upgrade logger-sharelatex to v1.5.6	2017-03-30 14:16:35 +01:00
Brian Gough	527c02a00b	log op versions pushed to track changes	2017-03-30 11:20:50 +01:00
Shane Kilkelly	64658d0034	Add a .nvmrc file	2017-03-27 14:50:09 +01:00
James Allen	0245bfd031	Support an incoming undo flag for history restores	2017-03-21 11:20:38 +00:00
James Allen	a3a5524778	Fix broken unit test from merge	2017-03-20 10:37:02 +00:00
James Allen	199079a5b7	Merge branch 'master' into ja-cut-and-paste-comments	2017-03-20 10:29:01 +00:00
James Allen	9c5299ec7c	Update RangeTracker to support upserting and moving comments for cut and paste	2017-03-16 15:49:41 +00:00
James Allen	5499a67d78	Add in a consistency check after applying updates that ranges still match	2017-03-15 14:12:06 +00:00
James Allen	b9e4492001	Wrap RangeTracker in try catch	2017-03-06 19:05:11 +00:00
James Allen	a3a9a726d8	Update unit tests	2017-03-06 12:01:18 +00:00
James Allen	d8135749d2	Merge branch 'master' into ja-increase-limits-on-changes ... Conflicts: app/coffee/RedisManager.coffee	2017-03-06 11:57:40 +00:00
James Allen	30294dafb1	Merge pull request #23 from sharelatex/bg-upgrade-raven-logging ... upgrade to logger-sharelatex 1.5.3	2017-03-06 11:53:47 +00:00
James Allen	269433f250	Merge pull request #25 from sharelatex/bg-log-doclines-on-hash-mismatch ... log doclines on hash mismatch	2017-03-06 11:53:24 +00:00
Brian Gough	501d907299	upgrade to logger-sharelatex 1.5.4	2017-03-03 16:08:14 +00:00
Brian Gough	d086e0b61b	log doclines on hash mismatch	2017-03-03 15:57:44 +00:00
Brian Gough	3f13263ecf	upgrade to logger-sharelatex 1.5.3	2017-03-03 15:32:11 +00:00
Brian Gough	b186a01c04	don't log errors from redis backend ... this also picks up errors from RedisManager like "doc ops range is not loaded in redis"	2017-03-03 15:27:42 +00:00
James Allen	d56bb55953	Revert PR #19	2017-03-01 16:49:46 +00:00
James Allen	9cfe651930	Merge pull request #19 from sharelatex/ja-track-changes ... Don't consume partial parts of delete when inserting a change	2017-03-01 10:20:06 +01:00
James Allen	f544814dda	Add in an absolute size limit on the ranges JSON object	2017-02-27 14:34:20 +01:00
James Allen	f211c282b2	Merge branch 'master' into ja-increase-limits-on-changes	2017-02-27 13:50:42 +01:00
James Allen	dd45a8dd7d	Improve logging around hashes and null byte errors	2017-02-27 10:06:24 +01:00
James Allen	60f3393c44	Do a naive check for null bytes after JSON.stringify on doc lines	2017-02-27 09:55:04 +01:00
James Allen	fd3dee0af5	Increase changes count limit to 2000	2017-02-24 14:58:47 +01:00
James Allen	80284e1b01	Only cancel deletes with inserts on undo and reject	2017-02-24 14:21:06 +01:00
James Allen	0420929e2a	Reduce the data logged when updating ranges	2017-02-23 13:57:47 +01:00
James Allen	e1481df76a	Don't consume partial parts of delete when inserting a change	2017-02-23 11:39:19 +01:00
Brian Gough	49271d4ee4	update logger to 1.5.2 - rate limiting on sentry	2017-02-23 10:15:56 +00:00
James Allen	5ab903169a	evert "update logger to 1.5.2 - rate limiting on sentry" ... This reverts commit ba1c4a84959cd2eb33d7d624239d3e621ebae1fd.	2017-02-22 18:26:42 +01:00
James Allen	d6b5cc6389	Increase changes limit to 1200	2017-02-22 18:11:55 +01:00
Brian Gough	ed59ca32fd	update logger to 1.5.2 - rate limiting on sentry	2017-02-22 15:58:40 +00:00
Brian Gough	168bab6960	make hash error logging fine grained	2017-02-22 14:32:51 +00:00
Brian Gough	19e7522d3f	make logging of hash errors optional	2017-02-22 13:46:34 +00:00
Brian Gough	8d1b11f37a	Merge branch 'bg-verify-writes-II'	2017-02-21 16:03:48 +00:00
Brian Gough	5eb339e525	fix utf8 encoding in sha1 hash	2017-02-21 16:03:06 +00:00
Brian Gough	6b808522dc	Merge pull request #18 from sharelatex/bg-verify-writes ... merge redis hash check	2017-02-20 15:55:19 +00:00
Brian Gough	590f8e7ced	Merge pull request #15 from sharelatex/bg-verify-writes ... store sha1 hash of docLines in redis	2017-02-20 15:53:56 +00:00
Brian Gough	62165ddeab	add unit tests	2017-02-20 15:33:19 +00:00
Brian Gough	c57d3ce31c	compute hash on write in redis server	2017-02-20 14:09:09 +00:00
Brian Gough	a3c127e469	added unit tests	2017-02-20 13:53:25 +00:00
Shane Kilkelly	37172a2115	Debug Acceptance Tests: Try without npm rebuild	2017-02-20 09:22:54 +00:00
Shane Kilkelly	28536bfe73	try running mochatest directly without compiling	2017-02-17 16:14:13 +00:00
James Allen	660752dfa6	Increase change limit to 1000	2017-02-17 15:32:44 +01:00
James Allen	89537927b5	Update Ranges Tracker	2017-02-17 09:24:19 +01:00
James Allen	de7f760ea4	Update Ranges Tracker	2017-02-16 17:09:08 +01:00
Brian Gough	b5a4458b68	check sha1 hash value only if present	2017-02-15 14:12:36 +00:00
Brian Gough	bd70aaa76c	add sha1 hash support on writes	2017-02-14 16:11:43 +00:00
Brian Gough	a33d4f505b	log request object on error	2017-02-14 15:20:05 +00:00
Brian Gough	8d1aa56689	enable sentry for docupdater	2017-02-14 14:34:28 +00:00
James Allen	8725aeab03	Allow deleting of comment ranges	2017-01-24 15:57:11 +01:00
James Allen	dc976fd9e9	Use thread id for comment id	2017-01-23 09:45:46 +01:00
James Allen	0705242a55	Increase max op age to 80	2017-01-17 11:45:10 +01:00
James Allen	5fed2424d0	Remove unused redis package reference	2017-01-16 13:05:05 +01:00
James Allen	0706feb26b	Add max limit on number of comments and changes per doc	2017-01-10 16:58:11 +01:00
James Allen	a3d5971d54	Update RangesTRacker	2017-01-10 11:59:09 +01:00
James Allen	540d0f7ec7	Make sure comment ids are unique	2017-01-10 11:55:38 +01:00
James Allen	be19532a1d	Add HTTP end point for accepting changes	2017-01-09 14:41:18 +01:00
James Allen	65f4360738	Consolidate HttpController tests into one file	2017-01-09 14:34:10 +01:00