Commit graph

593 commits

Author SHA1 Message Date
Brian Gough
27044c2d02 allow migration phase to be modified at runtime for testing 2020-01-06 16:46:35 +00:00
Brian Gough
8ae95ebf60 fix rclient check in migration metrics 2020-01-06 16:45:36 +00:00
Brian Gough
97cbf46160 add metrics for migration 2019-12-16 11:46:35 +00:00
Brian Gough
a2e63d009e fix migration phase check 2019-12-16 09:55:26 +00:00
Brian Gough
a85dffbcef fix acceptance tests 2019-12-16 09:27:00 +00:00
Brian Gough
d0c5eb5698 support migration of project history keys to separate redis instance 2019-12-13 16:38:41 +00:00
Brian Gough
ad19fee667 add setting so that double flush is the default
can be disabled to stop flushing to track-changes
2019-11-25 13:36:25 +00:00
Brian Gough
4f6583bbf2 fix getDocVersion and add tests 2019-11-25 13:28:36 +00:00
Brian Gough
68e12f4d2d add metrics for queue operations 2019-11-25 10:51:10 +00:00
Brian Gough
8b73bb9f13 Merge branch 'master' into bg-filter-track-changes-updates 2019-11-22 10:41:33 +00:00
Brian Gough
b7055eecee add metrics for history flushes 2019-11-22 09:14:32 +00:00
Brian Gough
65cf4cf7c7 make flush to track-changes failsafe 2019-11-21 14:58:35 +00:00
Brian Gough
dcd7649bad filter track-changes updates for projects using project-history 2019-11-19 10:02:56 +00:00
Brian Gough
08ee58abb8 remove console.log from tests 2019-11-15 09:34:13 +00:00
Brian Gough
82a72c9b73 fix missing bodyParser limit 2019-11-14 16:32:59 +00:00
dependabot[bot]
6871889bfb Bump request from 2.25.0 to 2.47.0
Bumps [request](https://github.com/request/request) from 2.25.0 to 2.47.0.
- [Release notes](https://github.com/request/request/releases)
- [Changelog](https://github.com/request/request/blob/master/CHANGELOG.md)
- [Commits](https://github.com/request/request/compare/v2.25.0...v2.47.0)

Signed-off-by: dependabot[bot] <support@github.com>
2019-11-05 11:38:25 +00:00
dependabot[bot]
ccc072e9da Bump express from 3.3.4 to 3.11.0
Bumps [express](https://github.com/expressjs/express) from 3.3.4 to 3.11.0.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/3.3.4...3.11.0)

Signed-off-by: dependabot[bot] <support@github.com>
2019-10-30 13:38:50 +00:00
Nate Stemen
1d3a1d22be bump build script to 1.1.24 2019-10-25 12:50:45 -04:00
Nate Stemen
1620956e2e replace private link with public one 2019-10-25 12:50:33 -04:00
Brian Gough
a9428c9ece Merge pull request #96 from overleaf/bg-add-ioredis-maxretriesperrequest
add ioredis maxRetriesPerRequest
2019-10-16 08:55:36 +01:00
Brian Gough
21a9126847 set maxRetriesPerRequest from environment variable 2019-10-16 08:44:08 +01:00
Brian Gough
781e01b8b4 Merge pull request #95 from overleaf/bg-update-ioredis
update to ioredis 4.14.1
2019-10-14 09:21:39 +01:00
Brian Gough
5a62632cfd add ioredis option maxRetriesPerRequest: 0
see https://github.com/luin/ioredis/issues/965
2019-10-04 11:53:14 +01:00
Brian Gough
51a821c03c remove old unused ioredis keepalive option 2019-10-04 11:51:37 +01:00
Brian Gough
18ccd11224 update to ioredis 4.14.1 2019-10-04 11:32:08 +01:00
Henry Oswald
e9efd6b93b logs out when a redis event occurs 2019-10-03 11:10:31 +01:00
Brian Gough
d82b180b76 avoid project history queues building up with deferred flush 2019-10-03 04:05:24 +01:00
Brian Gough
c1454bc4ac Merge pull request #92 from overleaf/bg-flush-queue-prod-fixes
add continuous background flush
2019-10-02 13:11:00 +01:00
Brian Gough
0c14b7d2f8 add comment about background flush limit 2019-10-01 15:06:01 +01:00
Brian Gough
ae3ebf2db6 start background flush after http server has started 2019-10-01 15:02:37 +01:00
Brian Gough
2845b23b70 add smoothing of delete spikes 2019-10-01 15:01:53 +01:00
Brian Gough
2c22a60052 add random jitter to cutoff time 2019-10-01 15:01:20 +01:00
Brian Gough
a32495d2b4 make background flush more adaptive 2019-10-01 14:09:41 +01:00
Brian Gough
73b4262186 add continuous background flush 2019-09-30 16:05:53 +01:00
Brian Gough
33fadf51c1 fix getDocTimestamps for multiple docs 2019-09-30 13:50:25 +01:00
Brian Gough
89a90399fe Merge pull request #89 from overleaf/bg-queue-deletes
queue deletes for deferred processing
2019-09-30 11:14:08 +01:00
John Lees-Miller
c5a9105c33 Update config 2019-09-28 11:07:33 +01:00
Brian Gough
260923f291 keep flushQueuedProjects in the foreground 2019-09-27 10:46:24 +01:00
Brian Gough
7561e05660 check timestamps array length 2019-09-27 10:39:56 +01:00
Brian Gough
0f0682df43 allow flush to complete in acceptance test 2019-09-26 16:00:11 +01:00
Brian Gough
b7f3b848af remove unused dryRun option
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
2019-09-26 15:50:55 +01:00
Brian Gough
3bc176259b fix log line 2019-09-26 15:46:54 +01:00
Brian Gough
8cdc8c410a fix error logging 2019-09-26 15:46:45 +01:00
Brian Gough
fc62abfcfa run flush of queued projects in the background 2019-09-26 15:46:14 +01:00
Brian Gough
ba35c73cb6 add comment about ZPOPMIN 2019-09-26 15:18:10 +01:00
Brian Gough
a709a0adaa for simplicity keep the cutoff time the same while flushing the queue 2019-09-26 15:05:38 +01:00
Brian Gough
eae4b352ca remove unnecessary check 2019-09-26 14:59:03 +01:00
Brian Gough
9f358ead9f add an acceptance test for flush with queue processing 2019-09-26 10:55:05 +01:00
Brian Gough
b49621b3e9 add comments 2019-09-26 10:14:49 +01:00
Brian Gough
83dd43b809 add metric for queue length 2019-09-25 17:04:36 +01:00