github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-12-29 03:42:01 +00:00
Code Issues Projects Releases Packages Wiki Activity
497 commits 3 branches 7 tags 206 MiB
Commit graph

459 commits

Author SHA1 Message Date
Brian Gough
ff76d1cf62 update to redis-sharelatex v1.0.2 2017-05-10 13:48:43 +01:00
Brian Gough
2781d9fd80 keep track of processed/total docs in archiving 2017-05-09 14:00:16 +01:00
Brian Gough
c26bccd390 make arguments handling for packworker more robust 2017-05-09 14:00:16 +01:00
Brian Gough
cac7556ad5 only check packs for archiving once each week 2017-05-09 14:00:15 +01:00
Brian Gough
c6e83c6cb8 allow archiving to exit properly on hard timeout 2017-05-09 14:00:15 +01:00
Brian Gough
9936710a1a replace error with warning for archiving timeout 2017-05-09 14:00:15 +01:00
Brian Gough
0e8a924f28 Merge pull request #35 from sharelatex/bg-fix-dangling 
flush dangling docs
 2017-05-05 15:03:41 +01:00
Brian Gough
7cd2911a0a Merge pull request #34 from sharelatex/bg-update-redis-sharelatex-to-v1.0.1 
update redis-sharelatex to v1.0.1
 2017-05-05 11:51:33 +01:00
Brian Gough
b82567ef79 support scan operations on redis cluster 2017-05-05 11:30:55 +01:00
Brian Gough
842b91b93a update redis-sharelatex to v1.0.1 
manual edit to npm-shrinkwrap.json to preserve existing versions
 2017-05-05 09:38:58 +01:00
Brian Gough
e0ed321d7b fix shrinkwrap file to add redis-sharelatex 2017-05-03 16:37:12 +01:00
Brian Gough
35810fa702 Revert "update shrinkwrap file" 
This reverts commit d16d3a1db97049cd99bfd6d965e4fab77453ae67.
 2017-05-03 16:32:23 +01:00
Brian Gough
534bd8f7ad add flushing code with error check for cluster 2017-05-02 14:19:49 +01:00
Brian Gough
0966f694da fix acceptance tests to work with redis cluster 2017-04-27 10:14:44 +01:00
Brian Gough
3b5a07d530 fix update to work with redis cluster 
the items do not have the same tag, so we must delete the entry from the
docsWithHistoryOps set outside the multi.
 2017-04-26 16:25:34 +01:00
Brian Gough
7836556d10 update shrinkwrap file 2017-04-24 14:24:02 +01:00
Brian Gough
3045becbb6 use redis-sharelatex with redis cluster config 2017-04-24 14:23:32 +01:00
Brian Gough
df38893538 remove old single redis flushing code 2017-04-24 11:36:31 +01:00
Brian Gough
a7ff15ec7d remove logging, fix query 2017-04-24 09:49:11 +01:00
Brian Gough
33fb1ddf57 script to flush dangling ops 2017-04-20 16:58:26 +01:00
Brian Gough
2a68b81cc1 randomise order of flush 2017-04-20 14:32:23 +01:00
Brian Gough
9eb36f9692 improve log message for flushing 2017-04-20 11:55:32 +01:00
Brian Gough
43f51355b2 add limit parameter to flushAll 2017-04-20 11:01:46 +01:00
Brian Gough
976f19f763 remove spurious call to getAllDocIdsWithHistoryOps 2017-04-20 10:14:44 +01:00
Brian Gough
70ce3a5271 Merge pull request #29 from sharelatex/bg-simple-flush-all 
simple flush for all projects
 2017-04-19 16:32:09 +01:00
Brian Gough
b5b61b98d0 avoid blocking when fetching redis keys 
use scan instead of keys method
 2017-04-19 15:39:33 +01:00
Brian Gough
9ce6d77cca add check for dangling updates 2017-04-13 11:31:45 +01:00
Brian Gough
1ec9f87542 update timeouts to allow for 5 second retry delay 
now using requestretry module, which has a five second delay between
retries. Need to increase the timeout on the health checks to allow for
this, as the timeout is currently shorter than the retry delay.
 2017-04-12 16:47:34 +01:00
Brian Gough
d9ed026d91 simple flush for all projects 
does not work with redis cluster, only single redis
 2017-04-12 16:34:28 +01:00
Shane Kilkelly
96f5802462 Merge branch 'sk-upgrade-metrics' 2017-04-04 16:16:37 +01:00
Shane Kilkelly
9a9f0f5c4d Remove the Metrics module, use metrics-sharelatex 2017-04-03 15:17:29 +01:00
Brian Gough
5d6d33966b Merge pull request #27 from sharelatex/bg-retry-web-api-request 
use requestretry to work around web api timeouts
 2017-04-03 14:05:07 +01:00
Brian Gough
7237467cb4 use requestretry to work around web api timeouts 2017-04-03 11:06:50 +01:00
Brian Gough
b0b05d1e1b improve error handling of releaseLock 
ported from docupdater
 2017-03-31 15:17:13 +01:00
Shane Kilkelly
b10f3adf2b mock out metrics in tests 2017-03-31 11:33:25 +01:00
Shane Kilkelly
9a84798293 Upgrade to latest metrics 2017-03-31 11:32:59 +01:00
Brian Gough
8bb3dd07a4 continue when ops are out of order 2017-03-30 13:37:11 +01:00
Brian Gough
d29141d583 return error when out-of-order ops detected 2017-03-30 11:48:26 +01:00
Brian Gough
70bb95fb36 update logger-sharelatex to v1.5.6 2017-03-30 09:41:42 +01:00
Brian Gough
28d2ec93b8 Revert "Revert "Merge pull request #23 from sharelatex/bg-move-lock"" 
This reverts commit 85bc45099f047aa8dfa6d189f02f4b4327c9d602.
 2017-03-28 12:52:49 +01:00
Shane Kilkelly
5288814bd6 Merge branch 'master' of github.com:sharelatex/track-changes-sharelatex 2017-03-27 14:37:47 +01:00
Shane Kilkelly
715ccbe227 Add a .nvmrc file 2017-03-27 14:37:32 +01:00
Brian Gough
e745cce9f5 Revert "Revert "added v8-profiler to shrinkwrap"" 
This reverts commit 642641ff7b97034346d494101124bee616927ade.
 2017-03-27 13:26:55 +01:00
Brian Gough
0130d3513d Revert "Revert "Merge pull request #24 from sharelatex/bg-add-profile"" 
This reverts commit a58cd0d787b51634caf694f9d858e638d27178b2.
 2017-03-27 13:25:59 +01:00
Brian Gough
8bd3dcca75 Revert "Merge pull request #24 from sharelatex/bg-add-profile" 
This reverts commit 8f507dcc9fbcd51072047c82a443137aaac0f377, reversing
changes made to 70abd7a155b396e3b44ce21ff8d36be6ad03a060.
 2017-03-24 09:17:38 +00:00
Brian Gough
a73c8ba7b8 Revert "Merge pull request #23 from sharelatex/bg-move-lock" 
This reverts commit 72e6756415e390a29fa784529453f926e5144358, reversing
changes made to 8f507dcc9fbcd51072047c82a443137aaac0f377.
 2017-03-24 09:17:11 +00:00
Brian Gough
6ada7842a9 Revert "added v8-profiler to shrinkwrap" 
This reverts commit 62d481b31ebed2a1714cdfcb037a9c1a9c053756.
 2017-03-24 09:12:28 +00:00
Brian Gough
f1a66df4ad added v8-profiler to shrinkwrap 2017-03-23 17:01:01 +00:00
Brian Gough
7802bcc911 Revert "update shrinkwrap file for new npm package" 
This reverts commit 68283e2650420c1cedc957b039b52766a8d4ce3c.
 2017-03-23 16:57:48 +00:00
Brian Gough
a4cca1f948 update shrinkwrap file for new npm package 2017-03-23 14:21:39 +00:00
Brian Gough
e028de115e Merge pull request #23 from sharelatex/bg-move-lock 
move lock inside web http calls
 2017-03-23 13:42:52 +00:00
Brian Gough
b897afeb1c Merge pull request #24 from sharelatex/bg-add-profile 
add /profile endpoint
 2017-03-23 13:39:12 +00:00
Brian Gough
bc7815f7fc remove old processUncompressedUpdates method 
replace with new per doc method
 2017-03-22 16:59:52 +00:00
Brian Gough
b1c0ebbaae add withLock to processUncompressedUpdatesForDoc 2017-03-22 16:16:04 +00:00
Brian Gough
0dd668416d increase request timeout now it is outside lock 2017-03-22 16:02:50 +00:00
Brian Gough
04b82a49ef add /profile endpoint 2017-03-22 15:38:12 +00:00
Brian Gough
6795820933 move lock inside web http calls 2017-03-21 16:49:23 +00:00
Shane Kilkelly
d520c78ae3 Update metrics version to 1.7.0 2017-03-21 13:45:15 +00:00
James Allen
d60445adfc Pass undo flag to doc updater when restoring from history 2017-03-21 11:32:53 +00:00
Shane Kilkelly
7fbc08d68a Add logger stub to unit test 2017-03-21 09:27:47 +00:00
Shane Kilkelly
118511ae4a Remove commented-out code 2017-03-20 09:39:51 +00:00
Shane Kilkelly
fa7ee739bb Update to new metrics api 2017-03-17 14:58:35 +00:00
Shane Kilkelly
c30e672549 Merge branch 'master' into sk-upgrade-metrics 2017-03-16 15:17:57 +00:00
Shane Kilkelly
c303a1a386 wip: upgrade metrics 2017-03-16 15:17:38 +00:00
Brian Gough
ac5d59211d revert random TTL in favour of delay in archiving 
there could be some issues with newer packs expiring before older ones
 2017-01-31 15:07:58 +00:00
Henry Oswald
ad5af5f4dd ceil not round 2017-01-26 15:14:20 +00:00
Henry Oswald
1d356f93ec added random up to 30 mins delay to the ttl of mongo objects 2017-01-26 13:07:54 +00:00
James Allen
dadc548f81 Ignore comment updates 2017-01-12 10:04:50 +01:00
James Allen
694be95e66 Swap start_ts and end_ts comparison to correctly break chunks after 5 minutes 2016-10-11 11:03:52 +01:00
James Allen
a197c0d148 Merge pull request #18 from sharelatex/ja-split-on-deletes 
Split update summary on big deletes
 2016-10-10 08:46:46 +01:00
James Allen
40ed6fee46 Split update summary on big deletes 2016-10-04 15:13:04 +01:00
Brian Gough
92e36023e5 Merge pull request #17 from sharelatex/ja-consistency-tweaks 
Retry rewind if doc and update versions don't match
 2016-09-30 15:14:27 +01:00
James Allen
f8865e616d Update ensureIndices to reflect reality 2016-09-30 14:34:38 +01:00
James Allen
3c8aeb1262 Log number of retries 2016-09-30 13:38:47 +01:00
James Allen
a7f44bcd01 Add missing return on callback 2016-09-30 13:36:31 +01:00
James Allen
deced1aa15 Fix unit tests with logger.warn stub 2016-09-30 11:41:49 +01:00
James Allen
bddd1fda7d Retry rewind if doc and update versions don't match 2016-09-30 11:36:47 +01:00
James Allen
cce93b95a5 Fetch updates before doc content when building a diff 2016-09-22 11:19:05 +01:00
James Allen
05a048db9a Add in a little more logging 2016-09-22 11:19:05 +01:00
James Allen
8dda44f880 Speed up unit tests by only generating big fixtures once up front 2016-09-16 11:51:41 +01:00
James Allen
dfe26262ec Return a No-op if diff returns nothing 2016-09-16 11:50:44 +01:00
James Allen
80375ae2dd Run a diff against big delete - insert changes which are likely copy-pastes 2016-09-16 11:33:36 +01:00
Brian Gough
6e5eadfa86 include a timeout on WebApiManager requests 2016-04-13 16:42:36 +01:00
Brian Gough
a55b72871f don't let s3 errors stop archive worker 2016-04-13 14:39:11 +01:00
Brian Gough
b343be844e added metrics to pack worker for archiving 2016-04-08 10:29:04 +01:00
Brian Gough
76fe194815 add a metric for append-pack 2016-04-07 15:56:11 +01:00
Brian Gough
25365789b4 use latest sharelatex-metrics package 2016-04-07 15:53:10 +01:00
Brian Gough
78100e40c8 add missing metrics file 2016-04-07 15:33:20 +01:00
Brian Gough
1a1fa8798d log attempted update when throwing error in DiffGenerator 2016-04-07 15:16:50 +01:00
Brian Gough
6db310bf6b add insert/archive/unarchive metrics 2016-04-07 15:16:50 +01:00
Brian Gough
d0e08039da don't modify expiry for temporary packs 2016-04-07 15:16:50 +01:00
Brian Gough
fd49601716 preserve existing history when user upgrades 2016-04-07 15:16:38 +01:00
Brian Gough
e292de5eb0 fix to avoid ever appending permanent changes to expiring packs 2016-04-06 17:00:16 +01:00
Brian Gough
8b7bdd345b consider all packs for archiving 2016-04-06 15:17:31 +01:00
Brian Gough
ef47337c78 remove additional fields 2016-04-06 15:17:20 +01:00
Brian Gough
0b9a0730c0 mark temporary packs with a last_checked date in the far future 
they do not need to be checked for archiving
 2016-04-06 14:29:49 +01:00
Brian Gough
08fc151eee avoid unnecessary call to insert packs into index 2016-04-06 14:29:21 +01:00
Brian Gough
719e0291aa consider all packs for processing 
to allow finalisation of old head packs
 2016-04-06 14:27:44 +01:00
Brian Gough
79baa99634 clean up logging 2016-04-06 14:26:54 +01:00
Brian Gough
6ab75795a2 archive head packs after sufficient time 2016-04-06 13:30:09 +01:00