Brian Gough
fc62abfcfa
run flush of queued projects in the background
2019-09-26 15:46:14 +01:00
Brian Gough
ba35c73cb6
add comment about ZPOPMIN
2019-09-26 15:18:10 +01:00
Brian Gough
a709a0adaa
for simplicity keep the cutoff time the same while flushing the queue
2019-09-26 15:05:38 +01:00
Brian Gough
eae4b352ca
remove unnecessary check
2019-09-26 14:59:03 +01:00
Brian Gough
9f358ead9f
add an acceptance test for flush with queue processing
2019-09-26 10:55:05 +01:00
Brian Gough
b49621b3e9
add comments
2019-09-26 10:14:49 +01:00
Brian Gough
83dd43b809
add metric for queue length
2019-09-25 17:04:36 +01:00
Brian Gough
f6b2ac7360
queue deletes for deferred processing
2019-09-25 16:42:49 +01:00
Brian Gough
f05e048203
Merge pull request #88 from overleaf/bg-upgrade-redis-wrapper
...
upgrade redis-sharelatex to 1.0.10
2019-09-19 16:26:57 +01:00
Brian Gough
c32101f0fb
upgrade redis-sharelatex to 1.0.10
2019-09-19 16:22:43 +01:00
Brian Gough
adafbb0cf9
Merge pull request #87 from overleaf/bg-remove-redis-server-hashing
...
remove redis server-side hashing for performance
2019-09-10 14:38:49 +01:00
Brian Gough
912a3a7753
remove redis server-side hashing for performance
...
we still compute the document hash in node, and check it on retrieval
but we don't check the hash at the point of writing it in redis which
was previously done with a redis Lua script.
2019-09-09 15:27:58 +01:00
Henry Oswald
c64cae2248
Merge pull request #86 from overleaf/ho-add-logging
...
Add logging for all project flushing
2019-08-30 09:46:46 +01:00
Henry Oswald
aa15a76059
added log lines for all project flushing
2019-08-30 07:38:53 +01:00
Henry Oswald
0ae838dd2d
add logger into project flusher
2019-08-29 20:36:00 +01:00
Brian Gough
a9ba7937c5
Merge pull request #84 from overleaf/bg-skip-history-flush-on-realtime-shutdown
...
skip history flush when project is cleared by realtime shutdown
2019-08-15 11:06:15 +01:00
Brian Gough
e75f2cc325
add unit tests for skipping history flush
2019-08-15 11:01:45 +01:00
Brian Gough
7493462154
add acceptance test for realtime shutdown
2019-08-15 10:38:07 +01:00
Brian Gough
a76e0dca88
skip history flush when project is cleared by realtime shutdown
...
history is flushed by a background cron job anyway
2019-08-15 09:51:16 +01:00
Simon Detheridge
f49f5cbf05
Merge pull request #79 from overleaf/ho-mongo-connection-string
...
use MONGO_CONNECTION_STRING in settings.defaults.coffee if set
2019-08-08 14:16:49 +01:00
Simon Detheridge
6721b904a7
Merge pull request #82 from overleaf/bg-mongo-health-check
...
add a combined health check for mongo and redis
2019-08-08 14:16:38 +01:00
Simon Detheridge
06444d2cc4
Improve/fix serializers for update logging ( #80 )
...
* Improve/fix serializers for update logging
2019-08-08 14:10:54 +01:00
Brian Gough
40f6494b19
add a combined health check for mongo and redis
2019-08-07 16:25:23 +01:00
Henry Oswald
aa1c74a979
use MONGO_CONNECTION_STRING in settings.defaults.coffee if set
2019-08-07 15:44:57 +01:00
Simon Detheridge
ad485709cf
Merge pull request #78 from overleaf/spd-such-logs
...
Don't log document lines in log messages
2019-08-02 12:27:49 +01:00
Simon Detheridge
df9ca8b272
Add serializer to print only length of large fields in production
2019-07-31 16:42:28 +01:00
Brian Gough
2c0c3dc573
Merge pull request #77 from overleaf/bg-use-individual-channels
...
support per-doc pubsub channels
2019-07-30 09:35:38 +01:00
Brian Gough
618880f99d
remove unnecessary check for doc_id
2019-07-24 16:57:43 +01:00
Brian Gough
c9ccf62d71
support per-doc pubsub channels
2019-07-22 12:20:06 +01:00
Brian Gough
0c8922c769
Merge pull request #76 from overleaf/bg-get-updates-from-cluster
...
fix cluster/sentinel connection with real-time
2019-07-10 09:53:18 +01:00
Brian Gough
97487a077e
fix cluster/sentinel connection with real-time
2019-07-10 09:42:05 +01:00
Henry Oswald
06ad0f7acd
Merge pull request #75 from overleaf/ho-pubsub-connection
...
Remove real time redis connection and consolidate on pubsub
2019-07-08 13:58:41 +01:00
Henry Oswald
374585f47d
Merge pull request #72 from overleaf/sk-dep-upgrades-2019-06
...
Update logger, metrics, and redis
2019-07-08 13:58:28 +01:00
Henry Oswald
94b787f467
Merge pull request #71 from overleaf/ho-detailed-flush-status
...
return failed and succesfully flushed projects when flushing everything
2019-07-08 13:58:17 +01:00
Henry Oswald
3b3b2da0f5
add pubsub redis connection and remove real time redis connection
2019-07-04 13:34:31 +01:00
Henry Oswald
a42aa45a36
Merge branch 'sk-dep-upgrades-2019-06' into ho-pubsub-connection
2019-07-03 10:18:13 +01:00
Henry Oswald
eed8e215d7
bump redis-sharelatex (and io redis) to 1.0.9
2019-07-02 16:58:30 +01:00
Brian Gough
44c36ee63b
Merge pull request #74 from overleaf/revert-73-bg-skip-hash-for-bad-char-replacement
...
Revert "skip hash check when non-BMP characters replaced"
2019-06-27 11:40:33 +01:00
Brian Gough
16fb297043
Revert "skip hash check when non-BMP characters replaced"
2019-06-27 11:39:45 +01:00
Brian Gough
09c69dfdab
Merge pull request #73 from overleaf/bg-skip-hash-for-bad-char-replacement
...
skip hash check when non-BMP characters replaced
2019-06-27 11:17:16 +01:00
Brian Gough
f37860599d
skip hash check when non-BMP characters replaced
2019-06-25 16:36:10 +01:00
Shane Kilkelly
e8a8c446ed
Increase timeout on test?
2019-06-19 11:34:38 +01:00
Shane Kilkelly
080b482e51
Update logger, metrics, and redis
...
Also fix acceptance tests, broken by a change in the redis driver
behaviour. It now returns promises from most operations, which confuses mocha.
2019-06-19 10:25:54 +01:00
Henry Oswald
fdef197271
Merge branch 'master' into ho-detailed-flush-status
2019-06-13 14:33:22 +01:00
Henry Oswald
d9a737f97c
return failed and succesfully flushed projects when flushing everything
2019-06-13 14:21:38 +01:00
Brian Gough
e8dd1aae9c
Merge pull request #70 from overleaf/bg-metric-for-invalid-hash
...
add metric for invalid hash and other sharejs errors
2019-06-12 13:50:34 +01:00
Brian Gough
99f896fa57
Merge pull request #69 from overleaf/bg-fix-hash-check
...
fix hash check
2019-06-12 13:42:19 +01:00
Brian Gough
d50b93df2f
add metric for invalid hash and other sharejs errors
2019-06-11 16:48:06 +01:00
Brian Gough
984b2d38e6
add unit tests
2019-06-11 16:33:14 +01:00
Brian Gough
966478cac4
fix hash check to use 'v' field instead of version
2019-06-11 14:11:46 +01:00
