353 commits

Author	SHA1	Message	Date
Brian Gough	e028de115e	Merge pull request #23 from sharelatex/bg-move-lock ... move lock inside web http calls	2017-03-23 13:42:52 +00:00
Brian Gough	b897afeb1c	Merge pull request #24 from sharelatex/bg-add-profile ... add /profile endpoint	2017-03-23 13:39:12 +00:00
Brian Gough	bc7815f7fc	remove old processUncompressedUpdates method ... replace with new per doc method	2017-03-22 16:59:52 +00:00
Brian Gough	b1c0ebbaae	add withLock to processUncompressedUpdatesForDoc	2017-03-22 16:16:04 +00:00
Brian Gough	0dd668416d	increase request timeout now it is outside lock	2017-03-22 16:02:50 +00:00
Brian Gough	04b82a49ef	add /profile endpoint	2017-03-22 15:38:12 +00:00
Brian Gough	6795820933	move lock inside web http calls	2017-03-21 16:49:23 +00:00
James Allen	d60445adfc	Pass undo flag to doc updater when restoring from history	2017-03-21 11:32:53 +00:00
Brian Gough	ac5d59211d	revert random TTL in favour of delay in archiving ... there could be some issues with newer packs expiring before older ones	2017-01-31 15:07:58 +00:00
Henry Oswald	ad5af5f4dd	ceil not round	2017-01-26 15:14:20 +00:00
Henry Oswald	1d356f93ec	added random up to 30 mins delay to the ttl of mongo objects	2017-01-26 13:07:54 +00:00
James Allen	dadc548f81	Ignore comment updates	2017-01-12 10:04:50 +01:00
James Allen	694be95e66	Swap start_ts and end_ts comparison to correctly break chunks after 5 minutes	2016-10-11 11:03:52 +01:00
James Allen	a197c0d148	Merge pull request #18 from sharelatex/ja-split-on-deletes ... Split update summary on big deletes	2016-10-10 08:46:46 +01:00
James Allen	40ed6fee46	Split update summary on big deletes	2016-10-04 15:13:04 +01:00
Brian Gough	92e36023e5	Merge pull request #17 from sharelatex/ja-consistency-tweaks ... Retry rewind if doc and update versions don't match	2016-09-30 15:14:27 +01:00
James Allen	f8865e616d	Update ensureIndices to reflect reality	2016-09-30 14:34:38 +01:00
James Allen	3c8aeb1262	Log number of retries	2016-09-30 13:38:47 +01:00
James Allen	a7f44bcd01	Add missing return on callback	2016-09-30 13:36:31 +01:00
James Allen	deced1aa15	Fix unit tests with logger.warn stub	2016-09-30 11:41:49 +01:00
James Allen	bddd1fda7d	Retry rewind if doc and update versions don't match	2016-09-30 11:36:47 +01:00
James Allen	cce93b95a5	Fetch updates before doc content when building a diff	2016-09-22 11:19:05 +01:00
James Allen	05a048db9a	Add in a little more logging	2016-09-22 11:19:05 +01:00
James Allen	8dda44f880	Speed up unit tests by only generating big fixtures once up front	2016-09-16 11:51:41 +01:00
James Allen	dfe26262ec	Return a No-op if diff returns nothing	2016-09-16 11:50:44 +01:00
James Allen	80375ae2dd	Run a diff against big delete - insert changes which are likely copy-pastes	2016-09-16 11:33:36 +01:00
Brian Gough	6e5eadfa86	include a timeout on WebApiManager requests	2016-04-13 16:42:36 +01:00
Brian Gough	a55b72871f	don't let s3 errors stop archive worker	2016-04-13 14:39:11 +01:00
Brian Gough	b343be844e	added metrics to pack worker for archiving	2016-04-08 10:29:04 +01:00
Brian Gough	76fe194815	add a metric for append-pack	2016-04-07 15:56:11 +01:00
Brian Gough	25365789b4	use latest sharelatex-metrics package	2016-04-07 15:53:10 +01:00
Brian Gough	78100e40c8	add missing metrics file	2016-04-07 15:33:20 +01:00
Brian Gough	1a1fa8798d	log attempted update when throwing error in DiffGenerator	2016-04-07 15:16:50 +01:00
Brian Gough	6db310bf6b	add insert/archive/unarchive metrics	2016-04-07 15:16:50 +01:00
Brian Gough	d0e08039da	don't modify expiry for temporary packs	2016-04-07 15:16:50 +01:00
Brian Gough	fd49601716	preserve existing history when user upgrades	2016-04-07 15:16:38 +01:00
Brian Gough	e292de5eb0	fix to avoid ever appending permanent changes to expiring packs	2016-04-06 17:00:16 +01:00
Brian Gough	8b7bdd345b	consider all packs for archiving	2016-04-06 15:17:31 +01:00
Brian Gough	ef47337c78	remove additional fields	2016-04-06 15:17:20 +01:00
Brian Gough	0b9a0730c0	mark temporary packs with a last_checked date in the far future ... they do not need to be checked for archiving	2016-04-06 14:29:49 +01:00
Brian Gough	08fc151eee	avoid unnecessary call to insert packs into index	2016-04-06 14:29:21 +01:00
Brian Gough	719e0291aa	consider all packs for processing ... to allow finalisation of old head packs	2016-04-06 14:27:44 +01:00
Brian Gough	79baa99634	clean up logging	2016-04-06 14:26:54 +01:00
Brian Gough	6ab75795a2	archive head packs after sufficient time	2016-04-06 13:30:09 +01:00
Brian Gough	7a0c2900ab	add error handler in tests	2016-04-06 13:29:19 +01:00
Brian Gough	6e18d49736	support archiving from list of project_ids/doc_ids	2016-04-04 17:00:19 +01:00
Brian Gough	6a2ecbac06	upgrade sentry logging	2016-04-04 16:30:17 +01:00
Brian Gough	537a1189e7	update mongodb-core version	2016-03-24 16:10:27 +00:00
Brian Gough	31348141d8	increase logging for discarded updates and version mismatch	2016-03-24 11:55:29 +00:00
Brian Gough	181cebecef	avoid call to fetch packs unnecessarily	2016-03-24 11:55:29 +00:00