Commit graph

641 commits

Author SHA1 Message Date
James Allen
05a048db9a Add in a little more logging 2016-09-22 11:19:05 +01:00
James Allen
a3abd2a4a5 Merge pull request #15 from sharelatex/ja-copy-paste-diff
Run a diff against big delete - insert changes which are likely copy-…
2016-09-19 11:07:51 +01:00
James Allen
8dda44f880 Speed up unit tests by only generating big fixtures once up front 2016-09-16 11:51:41 +01:00
James Allen
dfe26262ec Return a No-op if diff returns nothing 2016-09-16 11:50:44 +01:00
James Allen
80375ae2dd Run a diff against big delete - insert changes which are likely copy-pastes 2016-09-16 11:33:36 +01:00
Brian Gough
6e5eadfa86 include a timeout on WebApiManager requests 2016-04-13 16:42:36 +01:00
Brian Gough
a55b72871f don't let s3 errors stop archive worker 2016-04-13 14:39:11 +01:00
Brian Gough
b343be844e added metrics to pack worker for archiving 2016-04-08 10:29:04 +01:00
Brian Gough
76fe194815 add a metric for append-pack 2016-04-07 15:56:11 +01:00
Brian Gough
25365789b4 use latest sharelatex-metrics package 2016-04-07 15:53:10 +01:00
Brian Gough
78100e40c8 add missing metrics file 2016-04-07 15:33:20 +01:00
Brian Gough
1a1fa8798d log attempted update when throwing error in DiffGenerator 2016-04-07 15:16:50 +01:00
Brian Gough
6db310bf6b add insert/archive/unarchive metrics 2016-04-07 15:16:50 +01:00
Brian Gough
d0e08039da don't modify expiry for temporary packs 2016-04-07 15:16:50 +01:00
Brian Gough
fd49601716 preserve existing history when user upgrades 2016-04-07 15:16:38 +01:00
Brian Gough
e292de5eb0 fix to avoid ever appending permanent changes to expiring packs 2016-04-06 17:00:16 +01:00
Brian Gough
8b7bdd345b consider all packs for archiving 2016-04-06 15:17:31 +01:00
Brian Gough
ef47337c78 remove additional fields 2016-04-06 15:17:20 +01:00
Brian Gough
0b9a0730c0 mark temporary packs with a last_checked date in the far future
they do not need to be checked for archiving
2016-04-06 14:29:49 +01:00
Brian Gough
08fc151eee avoid unnecessary call to insert packs into index 2016-04-06 14:29:21 +01:00
Brian Gough
719e0291aa consider all packs for processing
to allow finalisation of old head packs
2016-04-06 14:27:44 +01:00
Brian Gough
79baa99634 clean up logging 2016-04-06 14:26:54 +01:00
Brian Gough
6ab75795a2 archive head packs after sufficient time 2016-04-06 13:30:09 +01:00
Brian Gough
7a0c2900ab add error handler in tests 2016-04-06 13:29:19 +01:00
Brian Gough
6e18d49736 support archiving from list of project_ids/doc_ids 2016-04-04 17:00:19 +01:00
Brian Gough
6a2ecbac06 upgrade sentry logging 2016-04-04 16:30:17 +01:00
Brian Gough
537a1189e7 update mongodb-core version 2016-03-24 16:10:27 +00:00
Brian Gough
31348141d8 increase logging for discarded updates and version mismatch 2016-03-24 11:55:29 +00:00
Brian Gough
181cebecef avoid call to fetch packs unnecessarily 2016-03-24 11:55:29 +00:00
Brian Gough
98683de3ae temporarily disable ttl behaviour
allow existing packs without temporary flag to expire
2016-03-24 11:38:59 +00:00
Brian Gough
3f388fb0ac only change ttl on cached packs, not temporary ones
temporary = without versioning feature enabled
cached = permanent versioned retrieved from s3
2016-03-24 11:38:09 +00:00
Brian Gough
8d900013d9 record whether a pack is temporary in the pack itself
using the expiresAt field no longer determines if the pack is
temporary because archived packs have an expiresAt field added when
they are retrieved from s3
2016-03-24 11:02:58 +00:00
Brian Gough
9543c21dd7 added npm-shrinkwrap.json to avoid bug in mongodb-core 2016-03-11 09:43:07 +00:00
Brian Gough
10932eb4a9 remove debug logs 2016-03-10 15:20:32 +00:00
Brian Gough
5ea7a31ad9 update archiving acceptance tests 2016-03-10 15:15:57 +00:00
Brian Gough
98738d1344 fix for acceptance test 2016-03-10 15:15:29 +00:00
Brian Gough
f01bf99682 acceptance tests - work in progress 2016-03-09 16:56:49 +00:00
Brian Gough
e2e8292590 update tests 2016-03-09 15:24:28 +00:00
Brian Gough
6d43273f04 working on packmanager tests 2016-03-09 14:44:59 +00:00
Brian Gough
f6367e21b8 give separate error for archive in progress vs completed 2016-03-09 14:44:59 +00:00
Brian Gough
7350ab531d exclude already cached packs from archival 2016-03-09 14:44:59 +00:00
Brian Gough
28b184e0ca fix incorrect use of _.union (argument must be array) 2016-03-09 14:44:59 +00:00
Brian Gough
8922b97bd7 avoid duplicate filling of UserInfo in getDocUpdates 2016-03-09 14:44:59 +00:00
Brian Gough
bfc9494cc1 update UpdatesManager tests 2016-03-09 14:44:59 +00:00
Brian Gough
4e63c50a75 update MongoManager tests 2016-03-09 14:44:59 +00:00
Brian Gough
b0d2e22c58 update config file for changes s3 settings 2016-03-09 13:28:22 +00:00
Brian Gough
7e6ea2793b remove startup dependency on s3 settings 2016-03-09 13:28:02 +00:00
Brian Gough
b780df6801 removed unused tests 2016-03-04 15:47:59 +00:00
Brian Gough
4d58c14573 updated MongoAWS tests 2016-03-04 15:47:47 +00:00
Brian Gough
1419d20b1f fix indentation 2016-03-04 15:43:32 +00:00