James Allen
b8d0eb2823
Merge branch 'master' into ja-return-latest-doc-on-no-version
2017-08-04 09:31:19 +02:00
James Allen
c5252f893b
Don't return all updates if no version is given
2017-06-28 15:38:31 +01:00
Brian Gough
e8dfe3099f
add missing callback for null byte check
2017-06-02 14:13:33 +01:00
Brian Gough
b71a15a6a0
null byte check for JSON.stringify in archiving
2017-06-01 13:36:13 +01:00
Brian Gough
d9763eb105
log corrupt updates
2017-05-23 13:55:10 +01:00
James Allen
c6d9df95f3
Merge pull request #38 from sharelatex/ja-redis-cluster
...
Migrate lock to be redis-cluster compatible
2017-05-16 14:16:43 +01:00
James Allen
fb6e028182
Migrate lock to be redis-cluster compatible
2017-05-15 10:34:24 +01:00
Brian Gough
2781d9fd80
keep track of processed/total docs in archiving
2017-05-09 14:00:16 +01:00
Brian Gough
c26bccd390
make arguments handling for packworker more robust
2017-05-09 14:00:16 +01:00
Brian Gough
cac7556ad5
only check packs for archiving once each week
2017-05-09 14:00:15 +01:00
Brian Gough
c6e83c6cb8
allow archiving to exit properly on hard timeout
2017-05-09 14:00:15 +01:00
Brian Gough
9936710a1a
replace error with warning for archiving timeout
2017-05-09 14:00:15 +01:00
Brian Gough
b82567ef79
support scan operations on redis cluster
2017-05-05 11:30:55 +01:00
Brian Gough
534bd8f7ad
add flushing code with error check for cluster
2017-05-02 14:19:49 +01:00
Brian Gough
3b5a07d530
fix update to work with redis cluster
...
the items do not have the same tag, so we must delete the entry from the
docsWithHistoryOps set outside the multi.
2017-04-26 16:25:34 +01:00
Brian Gough
3045becbb6
use redis-sharelatex with redis cluster config
2017-04-24 14:23:32 +01:00
Brian Gough
df38893538
remove old single redis flushing code
2017-04-24 11:36:31 +01:00
Brian Gough
2a68b81cc1
randomise order of flush
2017-04-20 14:32:23 +01:00
Brian Gough
9eb36f9692
improve log message for flushing
2017-04-20 11:55:32 +01:00
Brian Gough
43f51355b2
add limit parameter to flushAll
2017-04-20 11:01:46 +01:00
Brian Gough
976f19f763
remove spurious call to getAllDocIdsWithHistoryOps
2017-04-20 10:14:44 +01:00
Brian Gough
70ce3a5271
Merge pull request #29 from sharelatex/bg-simple-flush-all
...
simple flush for all projects
2017-04-19 16:32:09 +01:00
Brian Gough
b5b61b98d0
avoid blocking when fetching redis keys
...
use scan instead of keys method
2017-04-19 15:39:33 +01:00
Brian Gough
9ce6d77cca
add check for dangling updates
2017-04-13 11:31:45 +01:00
Brian Gough
1ec9f87542
update timeouts to allow for 5 second retry delay
...
now using requestretry module, which has a five second delay between
retries. Need to increase the timeout on the health checks to allow for
this, as the timeout is currently shorter than the retry delay.
2017-04-12 16:47:34 +01:00
Brian Gough
d9ed026d91
simple flush for all projects
...
does not work with redis cluster, only single redis
2017-04-12 16:34:28 +01:00
Shane Kilkelly
96f5802462
Merge branch 'sk-upgrade-metrics'
2017-04-04 16:16:37 +01:00
Shane Kilkelly
9a9f0f5c4d
Remove the Metrics module, use metrics-sharelatex
2017-04-03 15:17:29 +01:00
Brian Gough
5d6d33966b
Merge pull request #27 from sharelatex/bg-retry-web-api-request
...
use requestretry to work around web api timeouts
2017-04-03 14:05:07 +01:00
Brian Gough
7237467cb4
use requestretry to work around web api timeouts
2017-04-03 11:06:50 +01:00
Brian Gough
b0b05d1e1b
improve error handling of releaseLock
...
ported from docupdater
2017-03-31 15:17:13 +01:00
Brian Gough
8bb3dd07a4
continue when ops are out of order
2017-03-30 13:37:11 +01:00
Brian Gough
d29141d583
return error when out-of-order ops detected
2017-03-30 11:48:26 +01:00
Brian Gough
28d2ec93b8
Revert "Revert "Merge pull request #23 from sharelatex/bg-move-lock""
...
This reverts commit 85bc45099f047aa8dfa6d189f02f4b4327c9d602.
2017-03-28 12:52:49 +01:00
Brian Gough
a73c8ba7b8
Revert "Merge pull request #23 from sharelatex/bg-move-lock"
...
This reverts commit 72e6756415e390a29fa784529453f926e5144358, reversing
changes made to 8f507dcc9fbcd51072047c82a443137aaac0f377.
2017-03-24 09:17:11 +00:00
Brian Gough
e028de115e
Merge pull request #23 from sharelatex/bg-move-lock
...
move lock inside web http calls
2017-03-23 13:42:52 +00:00
Brian Gough
bc7815f7fc
remove old processUncompressedUpdates method
...
replace with new per doc method
2017-03-22 16:59:52 +00:00
Brian Gough
b1c0ebbaae
add withLock to processUncompressedUpdatesForDoc
2017-03-22 16:16:04 +00:00
Brian Gough
0dd668416d
increase request timeout now it is outside lock
2017-03-22 16:02:50 +00:00
Brian Gough
6795820933
move lock inside web http calls
2017-03-21 16:49:23 +00:00
James Allen
d60445adfc
Pass undo flag to doc updater when restoring from history
2017-03-21 11:32:53 +00:00
Shane Kilkelly
fa7ee739bb
Update to new metrics api
2017-03-17 14:58:35 +00:00
Shane Kilkelly
c30e672549
Merge branch 'master' into sk-upgrade-metrics
2017-03-16 15:17:57 +00:00
Shane Kilkelly
c303a1a386
wip: upgrade metrics
2017-03-16 15:17:38 +00:00
Brian Gough
ac5d59211d
revert random TTL in favour of delay in archiving
...
there could be some issues with newer packs expiring before older ones
2017-01-31 15:07:58 +00:00
Henry Oswald
ad5af5f4dd
ceil not round
2017-01-26 15:14:20 +00:00
Henry Oswald
1d356f93ec
added random up to 30 mins delay to the ttl of mongo objects
2017-01-26 13:07:54 +00:00
James Allen
dadc548f81
Ignore comment updates
2017-01-12 10:04:50 +01:00
James Allen
694be95e66
Swap start_ts and end_ts comparison to correctly break chunks after 5 minutes
2016-10-11 11:03:52 +01:00
James Allen
a197c0d148
Merge pull request #18 from sharelatex/ja-split-on-deletes
...
Split update summary on big deletes
2016-10-10 08:46:46 +01:00