Commit graph

370 commits

Author SHA1 Message Date
Brian Gough
b3f0dc4157 Merge pull request #12 from sharelatex/pack-in-place
Pack in place
2016-01-20 13:38:04 +00:00
Brian Gough
ae61d1261e added tests for pack updates 2016-01-19 15:58:09 +00:00
Brian Gough
61d22f027a added test for inserting as pack 2016-01-19 15:58:09 +00:00
Brian Gough
399a8f0a29 update tests to assume packs are created 2016-01-19 15:58:09 +00:00
Brian Gough
78b3412ca8 decrease delay when packing 2016-01-19 15:58:09 +00:00
Brian Gough
679a81564e respect mongo 3 limit of 1000 bulk operations 2016-01-19 15:58:09 +00:00
Brian Gough
f592611cac always create a new pack, never keep as op 2016-01-19 15:58:09 +00:00
Brian Gough
c6be12f3d5 set v_end on pack creation 2016-01-19 15:58:09 +00:00
Brian - Work
f64969c784 added comment about query memory usage for toArray 2016-01-19 15:58:09 +00:00
Brian Gough
0532a4daaa use compound index to replace separate index for packs 2016-01-19 15:56:09 +00:00
Brian Gough
0ba00a9eb7 expire temporary packs and roll over to a new pack each day 2016-01-19 15:56:09 +00:00
Brian Gough
5e830cbbdb put all new ops into packs 2016-01-19 15:56:09 +00:00
Brian Gough
184204130c Merge pull request #11 from sharelatex/archive-all-data-in-history
Archive all data in history
2016-01-15 16:43:54 +00:00
Brian Gough
dc564fd5d0 archiving document history now sends all changes to s3 2016-01-15 15:54:46 +00:00
Brian Gough
5153ed8217 make peekLastUpdate alway return lastVersion when available 2016-01-15 15:54:44 +00:00
Brian Gough
933b6c50e7 add memory logger to track changes 2016-01-14 16:24:13 +00:00
Brian Gough
8e53d66079 log the key for lock timeouts 2016-01-12 10:47:15 +00:00
Brian Gough
6199532d08 increase logging on s3 operations 2016-01-12 10:36:00 +00:00
Brian Gough
ca1f1dc944 handle exception in parsing retrieved json from aws 2016-01-12 09:26:29 +00:00
Brian Gough
b8862ca5af switch to node-byline module to avoid buffering problem with readline-stream
for lines > 64k the readline-stream module is affected by
https://github.com/jahewson/node-byline/issues/30 which is fixed in
node-byline (readline-stream was an earlier fork of the byline module)
2016-01-11 16:51:35 +00:00
Brian Gough
cb109a27a6 allow PackWorker to shut down cleanly 2016-01-06 09:43:10 +00:00
Brian Gough
ffe30962c9 add a close() method to LockManager to allow clean shutdown 2016-01-06 09:34:39 +00:00
Brian Gough
05163837cb add sentry error reporting to PackWorker 2016-01-05 16:00:52 +00:00
Brian Gough
ee8931bc34 log the request to sentry when an error occurs 2016-01-05 15:41:06 +00:00
Brian Gough
bad923e073 fix acceptance tests for mongojs 1.x driver 2016-01-05 12:10:57 +00:00
Brian Gough
6754bdca1c log timestamp in human-readable form for inconsistent ops 2016-01-05 11:30:24 +00:00
Brian Gough
e1aa436286 respect mongo bulk operations limit of 1000 operations 2016-01-05 11:13:13 +00:00
Brian Gough
bb7153c6c1 workaround for mongojs db.close issue
https://github.com/mafintosh/mongojs/issues/224
2015-12-22 15:36:15 +00:00
Brian Gough
d3583b4ef6 respect limit of 1000 ops in bulk operation with mongojs 1.x 2015-12-22 14:38:04 +00:00
Brian Gough
c7b4062412 remove unsupported options argument in count() method of mongojs 1.x 2015-12-22 14:20:34 +00:00
Brian Gough
d49997d9f3 fix usage of BSON module 2015-12-21 16:56:49 +00:00
Brian Gough
b7de6f2f71 don't try to compress updates across point of broken history 2015-12-21 13:52:26 +00:00
Brian Gough
4a6374efe8 fix read order when retrieving diffs 2015-12-18 12:38:42 +00:00
Brian Gough
911883a566 upgrade logger-sharelatex 2015-12-18 10:38:12 +00:00
Brian Gough
9f69c95192 Merge branch 'upgrade-mongojs' 2015-12-17 16:31:04 +00:00
Brian Gough
4a82dfe618 add setting trackchanges.continueOnError to allow recovery from missing ops 2015-12-17 16:28:02 +00:00
Brian Gough
b84a9e6e91 upgrade mongojs 2015-12-17 14:11:44 +00:00
Brian Gough
54d1036e37 skip ops marked as broken in database 2015-12-09 15:13:37 +00:00
Brian Gough
2a7c33d7ca added /check endpoint for documents 2015-12-09 14:57:04 +00:00
Brian Gough
c51d249221 remove explicit sentry release number
use environment variable SENTRY_RELEASE instead
2015-12-08 10:44:59 +00:00
Brian Gough
6483233f48 added /oops endpoint for generating dummy exception 2015-12-07 16:17:17 +00:00
Brian Gough
e7af602d32 add sentry logging 2015-12-07 15:52:21 +00:00
Brian Gough
23c43b8042 skip any broken ops when viewing history diffs 2015-12-04 15:17:28 +00:00
Brian Gough
be2136de7c fix update-in-place bug for array ops 2015-12-04 15:17:28 +00:00
Brian Gough
11be8c3733 increase timeout in lock manager test
on heavily loaded machines 20ms isn't long enough to consistently get
multiple calls to the lock manager
2015-11-27 14:25:25 +00:00
Brian Gough
3842f0d1cc Merge pull request #9 from sharelatex/only-delete-applied-ops
Only delete applied ops
2015-11-27 12:45:51 +00:00
Brian Gough
dedb02596f Merge pull request #10 from sharelatex/update-in-place
modify last compressed op in place
2015-11-27 12:45:43 +00:00
Brian Gough
8ebc069ddb modify last compressed op in place 2015-11-26 16:17:18 +00:00
Brian Gough
3432d9e91a added comments for redis delete 2015-11-26 15:16:54 +00:00
Brian Gough
e65549099c only delete the applied ops from redis 2015-11-25 16:01:07 +00:00