Henry Oswald
aa15a76059
added log lines for all project flushing
2019-08-30 07:38:53 +01:00
Henry Oswald
0ae838dd2d
add logger into project flusher
2019-08-29 20:36:00 +01:00
Brian Gough
a76e0dca88
skip history flush when project is cleared by realtime shutdown
...
history is flushed by a background cron job anyway
2019-08-15 09:51:16 +01:00
Simon Detheridge
6721b904a7
Merge pull request #82 from overleaf/bg-mongo-health-check
...
add a combined health check for mongo and redis
2019-08-08 14:16:38 +01:00
Simon Detheridge
06444d2cc4
Improve/fix serializers for update logging ( #80 )
...
* Improve/fix serializers for update logging
2019-08-08 14:10:54 +01:00
Brian Gough
40f6494b19
add a combined health check for mongo and redis
2019-08-07 16:25:23 +01:00
Simon Detheridge
df9ca8b272
Add serializer to print only length of large fields in production
2019-07-31 16:42:28 +01:00
Brian Gough
618880f99d
remove unnecessary check for doc_id
2019-07-24 16:57:43 +01:00
Brian Gough
c9ccf62d71
support per-doc pubsub channels
2019-07-22 12:20:06 +01:00
Brian Gough
97487a077e
fix cluster/sentinel connection with real-time
2019-07-10 09:42:05 +01:00
Henry Oswald
06ad0f7acd
Merge pull request #75 from overleaf/ho-pubsub-connection
...
Remove real time redis connection and consolidate on pubsub
2019-07-08 13:58:41 +01:00
Henry Oswald
3b3b2da0f5
add pubsub redis connection and remove real time redis connection
2019-07-04 13:34:31 +01:00
Brian Gough
16fb297043
Revert "skip hash check when non-BMP characters replaced"
2019-06-27 11:39:45 +01:00
Brian Gough
f37860599d
skip hash check when non-BMP characters replaced
2019-06-25 16:36:10 +01:00
Henry Oswald
fdef197271
Merge branch 'master' into ho-detailed-flush-status
2019-06-13 14:33:22 +01:00
Henry Oswald
d9a737f97c
return failed and succesfully flushed projects when flushing everything
2019-06-13 14:21:38 +01:00
Brian Gough
e8dd1aae9c
Merge pull request #70 from overleaf/bg-metric-for-invalid-hash
...
add metric for invalid hash and other sharejs errors
2019-06-12 13:50:34 +01:00
Brian Gough
d50b93df2f
add metric for invalid hash and other sharejs errors
2019-06-11 16:48:06 +01:00
Brian Gough
966478cac4
fix hash check to use 'v' field instead of version
2019-06-11 14:11:46 +01:00
Brian Gough
e95059f98e
handle non-urgent flushes in background
2019-06-03 10:01:10 +01:00
Brian Gough
0bbfa7de27
Merge branch 'master' into bg-downgrade-delete-component-error
2019-05-08 09:07:02 +01:00
Brian Gough
27a8248196
convert "Delete component" errors into warnings
2019-05-07 16:55:17 +01:00
Henry Oswald
daca83a057
add dryRun option to flush all projects
2019-05-02 16:54:22 +01:00
Henry Oswald
d5d1736a5e
adds /flush_all_projects project
2019-05-02 16:43:35 +01:00
Tim Alby
c1c23e4bee
record last author id on document flush
...
This is a multi-steps process:
* get a update's `user_id` from the metadata
* store the `user_id` (`lastUpdatedBy`) and current date (`lastUpdatedAt`) for
the document in Redis on every updates
* fetch `lastUpdatedAt` and `lastUpdatedBy` from Redis on document flush
* send the data to web to be persisted in Mongo
2019-05-02 11:10:02 +01:00
Brian Gough
68e7b9c4e9
Merge pull request #48 from sharelatex/bg-check-incoming-hash
...
check incoming hash when present
2019-04-29 10:15:44 +01:00
James Allen
52f3596e53
Review feedback
2019-04-16 11:05:17 +01:00
James Allen
3d76f4b9bf
Record a snapshot to mongo when a doc's comments/changes get collapsed
2019-04-11 13:27:46 +01:00
Brian Gough
3c635c8d98
check version before it is modified by applyOp
2019-04-09 09:20:48 +01:00
Brian Gough
cc1f3fce5b
check incoming hash when present
2019-04-08 14:12:18 +01:00
Brian Gough
fd1425d83f
include a unique id in every message published to redis
2019-03-21 12:10:15 +00:00
Brian Gough
8c5d74faef
use explicit json content-type to avoid security issues with text/html
2019-02-12 16:45:11 +00:00
Henry Oswald
3bc4cb492a
added log line
2019-02-07 16:30:53 +00:00
Henry Oswald
ecaef6485b
revert the removal of realtime keyspace
2019-02-07 15:27:51 +00:00
Henry Oswald
4e1a2c787c
Revert "turn down logging, use logger.info for less important data"
...
This reverts commit c5f91428e3c7702fbbd3ffd1ef7a772d513f33f2.
2019-02-06 15:29:22 +00:00
Christopher Hoskin
1217d8a80a
Merge branch 'master' into csh-ho-docker-issue-1338-bulk-upgrade
2019-01-04 09:18:40 +00:00
Henry Oswald
2505be283a
turn down logging, use logger.info for less important data
2018-12-13 12:46:10 +00:00
Henry Oswald
25e1ad7491
remove redis.realtime
...
real time pushes data into the document updater redis instance
2018-12-05 16:17:41 +00:00
Brian Gough
aae1352519
ensure that project history is flushed when the project is deleted
2018-10-30 12:03:44 +00:00
Brian Gough
537bd9a0c0
Merge pull request #36 from sharelatex/bg-limit-resync-requests-to-web
...
limit resync requests to web (connects to overleaf/sharelatex#775 )
2018-08-16 12:25:16 +01:00
Brian Gough
910b27357d
add more logging to resync operations
2018-08-16 11:14:11 +01:00
Brian Gough
aa013f0bee
limit parallel resync doc requests to web
2018-08-16 11:13:11 +01:00
Brian Gough
fdcb806518
set a timestamp for the first entry in the projectHistory:Ops queue
2018-07-24 09:17:31 +01:00
Brian Gough
e471730efb
Revert "Merge pull request #33 from sharelatex/bg-add-timestamp-marker-to-project-history-queue"
...
This reverts commit 8ec398b133aa75c45384dd7ceb630cd743f4a15c, reversing
changes made to e3a6c934cf53fd245d7d0df737463cc90f954885.
2018-07-23 16:05:04 +01:00
Brian Gough
6237577f87
set a timestamp for the first entry in the projectHistory:Ops queue
2018-07-20 10:43:31 +01:00
Brian Gough
7e526395fb
fix check for "delete component" message
2018-04-30 09:56:01 +01:00
Brian Gough
37f431f148
also block "delete component" messages from sentry
2018-04-27 16:03:46 +01:00
Brian Gough
64670f272a
stop OpRangeNotAvailable errors going to sentry
2018-04-27 15:45:28 +01:00
Hayden Faulds
af92ca70a1
coerce projectHistoryId to integer after reading from Redis
2018-04-23 15:19:06 +01:00
Hayden Faulds
1b63141e49
Revert "Revert "Add projectHistoryId to updates""
2018-04-23 12:08:04 +01:00
