Henry Oswald
ad5af5f4dd
ceil not round
2017-01-26 15:14:20 +00:00
Henry Oswald
1d356f93ec
added random up to 30 mins delay to the ttl of mongo objects
2017-01-26 13:07:54 +00:00
James Allen
dadc548f81
Ignore comment updates
2017-01-12 10:04:50 +01:00
James Allen
694be95e66
Swap start_ts and end_ts comparison to correctly break chunks after 5 minutes
2016-10-11 11:03:52 +01:00
James Allen
a197c0d148
Merge pull request #18 from sharelatex/ja-split-on-deletes
...
Split update summary on big deletes
2016-10-10 08:46:46 +01:00
James Allen
40ed6fee46
Split update summary on big deletes
2016-10-04 15:13:04 +01:00
Brian Gough
92e36023e5
Merge pull request #17 from sharelatex/ja-consistency-tweaks
...
Retry rewind if doc and update versions don't match
2016-09-30 15:14:27 +01:00
James Allen
f8865e616d
Update ensureIndices to reflect reality
2016-09-30 14:34:38 +01:00
James Allen
3c8aeb1262
Log number of retries
2016-09-30 13:38:47 +01:00
James Allen
a7f44bcd01
Add missing return on callback
2016-09-30 13:36:31 +01:00
James Allen
deced1aa15
Fix unit tests with logger.warn stub
2016-09-30 11:41:49 +01:00
James Allen
bddd1fda7d
Retry rewind if doc and update versions don't match
2016-09-30 11:36:47 +01:00
James Allen
cce93b95a5
Fetch updates before doc content when building a diff
2016-09-22 11:19:05 +01:00
James Allen
05a048db9a
Add in a little more logging
2016-09-22 11:19:05 +01:00
James Allen
8dda44f880
Speed up unit tests by only generating big fixtures once up front
2016-09-16 11:51:41 +01:00
James Allen
dfe26262ec
Return a No-op if diff returns nothing
2016-09-16 11:50:44 +01:00
James Allen
80375ae2dd
Run a diff against big delete - insert changes which are likely copy-pastes
2016-09-16 11:33:36 +01:00
Brian Gough
6e5eadfa86
include a timeout on WebApiManager requests
2016-04-13 16:42:36 +01:00
Brian Gough
a55b72871f
don't let s3 errors stop archive worker
2016-04-13 14:39:11 +01:00
Brian Gough
b343be844e
added metrics to pack worker for archiving
2016-04-08 10:29:04 +01:00
Brian Gough
76fe194815
add a metric for append-pack
2016-04-07 15:56:11 +01:00
Brian Gough
25365789b4
use latest sharelatex-metrics package
2016-04-07 15:53:10 +01:00
Brian Gough
78100e40c8
add missing metrics file
2016-04-07 15:33:20 +01:00
Brian Gough
1a1fa8798d
log attempted update when throwing error in DiffGenerator
2016-04-07 15:16:50 +01:00
Brian Gough
6db310bf6b
add insert/archive/unarchive metrics
2016-04-07 15:16:50 +01:00
Brian Gough
d0e08039da
don't modify expiry for temporary packs
2016-04-07 15:16:50 +01:00
Brian Gough
fd49601716
preserve existing history when user upgrades
2016-04-07 15:16:38 +01:00
Brian Gough
e292de5eb0
fix to avoid ever appending permanent changes to expiring packs
2016-04-06 17:00:16 +01:00
Brian Gough
8b7bdd345b
consider all packs for archiving
2016-04-06 15:17:31 +01:00
Brian Gough
ef47337c78
remove additional fields
2016-04-06 15:17:20 +01:00
Brian Gough
0b9a0730c0
mark temporary packs with a last_checked date in the far future
...
they do not need to be checked for archiving
2016-04-06 14:29:49 +01:00
Brian Gough
08fc151eee
avoid unnecessary call to insert packs into index
2016-04-06 14:29:21 +01:00
Brian Gough
719e0291aa
consider all packs for processing
...
to allow finalisation of old head packs
2016-04-06 14:27:44 +01:00
Brian Gough
79baa99634
clean up logging
2016-04-06 14:26:54 +01:00
Brian Gough
6ab75795a2
archive head packs after sufficient time
2016-04-06 13:30:09 +01:00
Brian Gough
7a0c2900ab
add error handler in tests
2016-04-06 13:29:19 +01:00
Brian Gough
6e18d49736
support archiving from list of project_ids/doc_ids
2016-04-04 17:00:19 +01:00
Brian Gough
6a2ecbac06
upgrade sentry logging
2016-04-04 16:30:17 +01:00
Brian Gough
537a1189e7
update mongodb-core version
2016-03-24 16:10:27 +00:00
Brian Gough
31348141d8
increase logging for discarded updates and version mismatch
2016-03-24 11:55:29 +00:00
Brian Gough
181cebecef
avoid call to fetch packs unnecessarily
2016-03-24 11:55:29 +00:00
Brian Gough
98683de3ae
temporarily disable ttl behaviour
...
allow existing packs without temporary flag to expire
2016-03-24 11:38:59 +00:00
Brian Gough
3f388fb0ac
only change ttl on cached packs, not temporary ones
...
temporary = without versioning feature enabled
cached = permanent versioned retrieved from s3
2016-03-24 11:38:09 +00:00
Brian Gough
8d900013d9
record whether a pack is temporary in the pack itself
...
using the expiresAt field no longer determines if the pack is
temporary because archived packs have an expiresAt field added when
they are retrieved from s3
2016-03-24 11:02:58 +00:00
Brian Gough
9543c21dd7
added npm-shrinkwrap.json to avoid bug in mongodb-core
2016-03-11 09:43:07 +00:00
Brian Gough
10932eb4a9
remove debug logs
2016-03-10 15:20:32 +00:00
Brian Gough
5ea7a31ad9
update archiving acceptance tests
2016-03-10 15:15:57 +00:00
Brian Gough
98738d1344
fix for acceptance test
2016-03-10 15:15:29 +00:00
Brian Gough
f01bf99682
acceptance tests - work in progress
2016-03-09 16:56:49 +00:00
Brian Gough
e2e8292590
update tests
2016-03-09 15:24:28 +00:00