17 commits

Author	SHA1	Message	Date
Brian Gough	e1aa436286	respect mongo bulk operations limit of 1000 operations	2016-01-05 11:13:13 +00:00
Brian Gough	dc0044020f	only archive entries older than the current update ... to avoid a stale version of the current update ever being pulled back from S3	2015-09-23 14:33:40 +01:00
Brian Gough	696a866b67	pause the stream of ops, not the download ... the download is buffered in the lineStream so a lot comes out even after pausing the S3 download.	2015-09-23 13:38:57 +01:00
Henrique Dias	aa66c5ee8c	improve size function	2015-09-17 10:41:53 -03:00
Henrique Dias	3f712c452a	add size bulk limit	2015-09-17 09:23:13 -03:00
Brian Gough	9d39012b49	add error handler to each stage of download pipeline	2015-09-16 16:00:37 +01:00
Brian Gough	d9085a5e5e	add error handler to each stage of upload pipeline	2015-09-16 16:00:25 +01:00
Brian Gough	dfa0036507	pause stream while writing to mongo	2015-09-16 15:32:36 +01:00
Henrique Dias	c5a8a249c6	add unarchive acceptance tests	2015-09-03 08:36:32 -03:00
Henrique Dias	d2b1243701	split MongoAWS files	2015-09-02 15:45:29 -03:00
Henrique Dias	20c3e15f93	fix bulk insert limit	2015-08-14 19:58:38 -03:00
Henrique Dias	26c8048729	change mongo stream method (still have a bug in bulk insert limit)	2015-08-14 19:19:54 -03:00
Henrique Dias	fd4afb3574	Archive changes, care about: versioin, expiresAt and Lock	2015-08-14 15:07:16 -03:00
Henrique Dias	3bc5380468	handle inS3 flag	2015-08-09 17:50:15 -03:00
Henrique Dias	daa42bcea0	change s3Stream lib	2015-08-09 15:47:47 -03:00
Henrique Dias	bca48ac117	add unarchive doc track from s3	2015-08-06 17:09:36 -03:00
Henrique Dias	438c4f4d0c	using mongoexport for s3 archive	2015-08-06 15:46:44 -03:00