Brian Gough
b7de6f2f71
don't try to compress updates across point of broken history
2015-12-21 13:52:26 +00:00
Brian Gough
4a6374efe8
fix read order when retrieving diffs
2015-12-18 12:38:42 +00:00
Brian Gough
9f69c95192
Merge branch 'upgrade-mongojs'
2015-12-17 16:31:04 +00:00
Brian Gough
4a82dfe618
add setting trackchanges.continueOnError to allow recovery from missing ops
2015-12-17 16:28:02 +00:00
Brian Gough
b84a9e6e91
upgrade mongojs
2015-12-17 14:11:44 +00:00
Brian Gough
54d1036e37
skip ops marked as broken in database
2015-12-09 15:13:37 +00:00
Brian Gough
2a7c33d7ca
added /check endpoint for documents
2015-12-09 14:57:04 +00:00
Brian Gough
23c43b8042
skip any broken ops when viewing history diffs
2015-12-04 15:17:28 +00:00
Brian Gough
be2136de7c
fix update-in-place bug for array ops
2015-12-04 15:17:28 +00:00
Brian Gough
3842f0d1cc
Merge pull request #9 from sharelatex/only-delete-applied-ops
...
Only delete applied ops
2015-11-27 12:45:51 +00:00
Brian Gough
8ebc069ddb
modify last compressed op in place
2015-11-26 16:17:18 +00:00
Brian Gough
3432d9e91a
added comments for redis delete
2015-11-26 15:16:54 +00:00
Brian Gough
e65549099c
only delete the applied ops from redis
2015-11-25 16:01:07 +00:00
Brian Gough
992857d6a2
added redis write check to healthcheck
2015-10-29 10:52:23 +00:00
Brian Gough
c44d5b1b3d
added healthcheck
2015-10-19 10:59:39 +01:00
Brian Gough
ad144371d0
gracefully handle updates marked as broken
...
set update.broken == true to allow the user to view history without a
crash
2015-10-16 11:24:50 +01:00
Brian Gough
8961e23954
enhance LockManager to avoid accidental unlocking
2015-10-14 14:42:17 +01:00
Brian Gough
b6dae59655
fix callback logic in compressAndSaveRawUpdates
2015-10-08 16:39:13 +01:00
Brian Gough
8226bf3be4
increase lock time to 5 minutes
2015-10-08 16:11:39 +01:00
Brian Gough
add6a68fe1
add missing callback in compressAndSaveRawUpdates
2015-10-08 10:53:25 +01:00
James Allen
1a4b8f4269
API/service layout deprecation warning
2015-10-07 13:44:40 +01:00
James Allen
2a03591030
Stub out noisy/slow logger-sharelatex and mongojs modules in tests
2015-09-25 13:46:20 +01:00
James Allen
23dfe68cb8
Don't error when rewinding and insert op which is beyond the length of the document.
...
ShareJS will accept an op where p > content.length when applied,
and it applies as though p == content.length. However, the op is
passed to us with the original p > content.length. Detect if that
is the case with this op, and shift p back appropriately to match
ShareJS if so.
2015-09-25 13:44:44 +01:00
Brian Gough
92e0b0f04c
add logging to each stage of archiving
2015-09-24 09:10:06 +01:00
Brian Gough
e683b0275a
bug fix for clear archive in progress flag
2015-09-24 09:09:49 +01:00
Brian Gough
692e8c657c
Revert to the default lock timeout now we have write barriers
...
Revert "increase lock timeouts for archiving"
This reverts commit 9eee1b383772adf058130d6e5eab409f57ce03cd.
2015-09-24 08:53:09 +01:00
Brian Gough
2ab1778dd9
move default value of lastVersion into function body
2015-09-23 16:31:33 +01:00
Brian Gough
dc0044020f
only archive entries older than the current update
...
to avoid a stale version of the current update ever being pulled back
from S3
2015-09-23 14:33:40 +01:00
Brian Gough
696a866b67
pause the stream of ops, not the download
...
the download is buffered in the lineStream so a lot comes out even
after pausing the S3 download.
2015-09-23 13:38:57 +01:00
Brian Gough
847a553344
prevent double archiving by checking if any inS3 field is already present
2015-09-23 13:29:32 +01:00
Brian Gough
e49f260507
allow rollback/locking by setting inS3:false when starting the archive process
2015-09-23 13:28:07 +01:00
Brian Gough
551e8334cf
compressedUpdates are now never inserted with inS3
...
it is now always added later, and a new update is forced for any
addition to an archived update
2015-09-23 13:25:10 +01:00
Brian Gough
d6b827426c
support forcing new compressed update in popLastCompressedUpdate
...
callback with a null update, passing the version as an additional
argument
2015-09-23 13:22:38 +01:00
Brian Gough
a10dc4f898
Merge pull request #6 from heukirne/s3-archive
...
Add S3 archive track changes feature
2015-09-21 11:25:06 +01:00
Brian Gough
0e627c92d8
avoid clobbering global _ in loop
2015-09-18 16:26:05 +01:00
Henrique Dias
aa66c5ee8c
improve size function
2015-09-17 10:41:53 -03:00
Henrique Dias
3f712c452a
add size bulk limit
2015-09-17 09:23:13 -03:00
Brian Gough
7af5050370
add lock to unarchive doc
2015-09-16 16:18:36 +01:00
Brian Gough
18f06a3daf
increase lock timeouts for archiving
2015-09-16 16:09:38 +01:00
Brian Gough
b4ffa7d57e
share the document lock between archiving and packing
2015-09-16 16:03:55 +01:00
Brian Gough
9d39012b49
add error handler to each stage of download pipeline
2015-09-16 16:00:37 +01:00
Brian Gough
d9085a5e5e
add error handler to each stage of upload pipeline
2015-09-16 16:00:25 +01:00
Brian Gough
1c1b1d9595
log the case where there are no entries in the document history
2015-09-16 15:34:30 +01:00
Brian Gough
82d0f4fce8
make unarchive more responsive by downloading documents in parallel
...
unarchive is triggered interactively so we should try to make it
reasonably fast
2015-09-16 15:33:59 +01:00
Brian Gough
dfa0036507
pause stream while writing to mongo
2015-09-16 15:32:36 +01:00
Brian Gough
70200a9cf1
only log document ids, not document content
...
avoid filling the log with large documents
2015-09-16 15:31:43 +01:00
Brian Gough
d3dff28bea
Merge remote-tracking branch 'origin/master' into heukirne-s3-archive
2015-09-15 15:19:43 +01:00
Brian Gough
092f98d3ad
suppress error in normal shutdown case
2015-09-12 11:07:54 +01:00
Shane Kilkelly
eab8b4b6c8
Null safe access of id property, needed as user can be null.
2015-09-11 14:07:06 +01:00
Shane Kilkelly
0ad374556d
Add a comment for clarity.
2015-09-10 16:43:40 +01:00
