Henry Oswald
daca83a057
add dryRun option to flush all projects
2019-05-02 16:54:22 +01:00
Henry Oswald
d5d1736a5e
adds /flush_all_projects project
2019-05-02 16:43:35 +01:00
Tim Alby
c1c23e4bee
record last author id on document flush
...
This is a multi-steps process:
* get a update's `user_id` from the metadata
* store the `user_id` (`lastUpdatedBy`) and current date (`lastUpdatedAt`) for
the document in Redis on every updates
* fetch `lastUpdatedAt` and `lastUpdatedBy` from Redis on document flush
* send the data to web to be persisted in Mongo
2019-05-02 11:10:02 +01:00
Brian Gough
68e7b9c4e9
Merge pull request #48 from sharelatex/bg-check-incoming-hash
...
check incoming hash when present
2019-04-29 10:15:44 +01:00
James Allen
52f3596e53
Review feedback
2019-04-16 11:05:17 +01:00
James Allen
3d76f4b9bf
Record a snapshot to mongo when a doc's comments/changes get collapsed
2019-04-11 13:27:46 +01:00
Brian Gough
3c635c8d98
check version before it is modified by applyOp
2019-04-09 09:20:48 +01:00
Brian Gough
cc1f3fce5b
check incoming hash when present
2019-04-08 14:12:18 +01:00
Brian Gough
fd1425d83f
include a unique id in every message published to redis
2019-03-21 12:10:15 +00:00
Brian Gough
8c5d74faef
use explicit json content-type to avoid security issues with text/html
2019-02-12 16:45:11 +00:00
Henry Oswald
3bc4cb492a
added log line
2019-02-07 16:30:53 +00:00
Henry Oswald
ecaef6485b
revert the removal of realtime keyspace
2019-02-07 15:27:51 +00:00
Henry Oswald
4e1a2c787c
Revert "turn down logging, use logger.info for less important data"
...
This reverts commit c5f91428e3c7702fbbd3ffd1ef7a772d513f33f2.
2019-02-06 15:29:22 +00:00
Christopher Hoskin
1217d8a80a
Merge branch 'master' into csh-ho-docker-issue-1338-bulk-upgrade
2019-01-04 09:18:40 +00:00
Henry Oswald
2505be283a
turn down logging, use logger.info for less important data
2018-12-13 12:46:10 +00:00
Henry Oswald
25e1ad7491
remove redis.realtime
...
real time pushes data into the document updater redis instance
2018-12-05 16:17:41 +00:00
Brian Gough
aae1352519
ensure that project history is flushed when the project is deleted
2018-10-30 12:03:44 +00:00
Brian Gough
537bd9a0c0
Merge pull request #36 from sharelatex/bg-limit-resync-requests-to-web
...
limit resync requests to web (connects to overleaf/sharelatex#775 )
2018-08-16 12:25:16 +01:00
Brian Gough
910b27357d
add more logging to resync operations
2018-08-16 11:14:11 +01:00
Brian Gough
aa013f0bee
limit parallel resync doc requests to web
2018-08-16 11:13:11 +01:00
Brian Gough
fdcb806518
set a timestamp for the first entry in the projectHistory:Ops queue
2018-07-24 09:17:31 +01:00
Brian Gough
e471730efb
Revert "Merge pull request #33 from sharelatex/bg-add-timestamp-marker-to-project-history-queue"
...
This reverts commit 8ec398b133aa75c45384dd7ceb630cd743f4a15c, reversing
changes made to e3a6c934cf53fd245d7d0df737463cc90f954885.
2018-07-23 16:05:04 +01:00
Brian Gough
6237577f87
set a timestamp for the first entry in the projectHistory:Ops queue
2018-07-20 10:43:31 +01:00
Brian Gough
7e526395fb
fix check for "delete component" message
2018-04-30 09:56:01 +01:00
Brian Gough
37f431f148
also block "delete component" messages from sentry
2018-04-27 16:03:46 +01:00
Brian Gough
64670f272a
stop OpRangeNotAvailable errors going to sentry
2018-04-27 15:45:28 +01:00
Hayden Faulds
af92ca70a1
coerce projectHistoryId to integer after reading from Redis
2018-04-23 15:19:06 +01:00
Hayden Faulds
1b63141e49
Revert "Revert "Add projectHistoryId to updates""
2018-04-23 12:08:04 +01:00
Hayden Faulds
3cc20d6311
Revert "Add projectHistoryId to updates"
2018-04-23 11:41:29 +01:00
Hayden Faulds
25c0be4eaa
add projectHistryId to project structure updates
2018-04-13 15:22:05 +01:00
Hayden Faulds
a220794d32
add projectHistoryId to resync updates
2018-04-13 15:22:05 +01:00
Hayden Faulds
fb1852a593
cache projectHistoryId with doc in Redis
2018-04-13 15:22:05 +01:00
Brian Gough
dd0f8b880a
change update to projectUpdate in project related methods
2018-03-16 10:54:12 +00:00
Brian Gough
3385d2640a
fix structure ordering bug
2018-03-15 16:27:10 +00:00
Brian Gough
0642e3c8c9
support project version on incoming requests
2018-03-15 16:15:10 +00:00
Hayden Faulds
4a5731440a
Merge pull request #21 from sharelatex/hof-history-sync
...
resync project history endpoint
2018-03-14 10:40:59 +00:00
Hayden Faulds
1226f96fc3
extract ProjectHistoryRedisManager
2018-03-09 15:36:38 +00:00
Hayden Faulds
fc6ef6ea7a
improve logging
2018-03-09 12:21:48 +00:00
Hayden Faulds
088ab6e7ba
rename queueResyncDocContents -> queueResyncDocContent
2018-03-09 12:21:37 +00:00
Hayden Faulds
64d1fe13bc
additional logging for project history resync
2018-03-07 16:24:22 +00:00
Hayden Faulds
152a25e96d
rename resyncProject -> resyncProjectHistory
2018-03-07 13:29:42 +00:00
Hayden Faulds
d46217be52
queue doc content sync updates
2018-03-02 15:02:27 +00:00
Hayden Faulds
52d6b710bd
add ability to resync project structure
2018-03-02 15:01:53 +00:00
James Allen
939eaa2d4b
Don't allow a document to be loaded without a pathname
2018-03-02 10:02:49 +00:00
Hayden Faulds
5a11332aa3
remove unecessary method wrapping
2018-02-22 10:17:59 +00:00
Hayden Faulds
0f87ae1f74
simplify comments
2018-02-22 10:16:29 +00:00
Hayden Faulds
ea0dd9700b
reduce the number of times we flush project history
2018-02-22 10:01:05 +00:00
Michael Walker
241d1b27d5
Remove _shouldFlushHistoryOps wrapper
2018-01-31 11:27:40 +00:00
Michael Walker
d2865f0650
Correctly count project_ops_length
2018-01-26 14:41:28 +00:00
Michael Walker
3028fb9c3d
Only flush project updates when crossing the threshold
2018-01-24 11:57:03 +00:00