Brian Gough
d9763eb105
log corrupt updates
2017-05-23 13:55:10 +01:00
James Allen
fb6e028182
Migrate lock to be redis-cluster compatible
2017-05-15 10:34:24 +01:00
Brian Gough
2a68b81cc1
randomise order of flush
2017-04-20 14:32:23 +01:00
Brian Gough
43f51355b2
add limit parameter to flushAll
2017-04-20 11:01:46 +01:00
Brian Gough
976f19f763
remove spurious call to getAllDocIdsWithHistoryOps
2017-04-20 10:14:44 +01:00
Brian Gough
9ce6d77cca
add check for dangling updates
2017-04-13 11:31:45 +01:00
Brian Gough
d9ed026d91
simple flush for all projects
...
does not work with redis cluster, only single redis
2017-04-12 16:34:28 +01:00
Brian Gough
8bb3dd07a4
continue when ops are out of order
2017-03-30 13:37:11 +01:00
Brian Gough
d29141d583
return error when out-of-order ops detected
2017-03-30 11:48:26 +01:00
Brian Gough
28d2ec93b8
Revert "Revert "Merge pull request #23 from sharelatex/bg-move-lock""
...
This reverts commit 85bc45099f047aa8dfa6d189f02f4b4327c9d602.
2017-03-28 12:52:49 +01:00
James Allen
694be95e66
Swap start_ts and end_ts comparison to correctly break chunks after 5 minutes
2016-10-11 11:03:52 +01:00
James Allen
40ed6fee46
Split update summary on big deletes
2016-10-04 15:13:04 +01:00
Brian Gough
31348141d8
increase logging for discarded updates and version mismatch
2016-03-24 11:55:29 +00:00
Brian Gough
98738d1344
fix for acceptance test
2016-03-10 15:15:29 +00:00
Brian Gough
28b184e0ca
fix incorrect use of _.union (argument must be array)
2016-03-09 14:44:59 +00:00
Brian Gough
8922b97bd7
avoid duplicate filling of UserInfo in getDocUpdates
2016-03-09 14:44:59 +00:00
Brian Gough
3d9dfeccc3
remove pack worker
...
remove the op-specific code
remove tests for ops, now only packing
remove unused packing code
work in progress
store index for completed packs only
support archiving and unarchiving of individual packs
remove support for archiving whole document history
split out ArchiveManager, IndexManager
remove old DocArchive code
remove docHistoryStats collection
comment about archiving
added method to look at index when last pack has been archived
added start of iterator for project results
use a proper iterator
added heap module
getting it working
increase pack size since bulk operations no longer needed
remove unused MongoAWSexternal
cleanup
added doc iterator
remove old query code
added missing files
cleanup
clean upclean up
started adding pack worker for archiving
work in progress
work in progress
getting pack worker working
updating worker
getting packworker working
added lock
use correct key name for track changes aws access
use correct key name for track changes aws access
always send back users array
fix up comparison of retrieved objects
handle op ids inside packs
log when s3 download completes
comments
cleanup, remove finalisation ideacleanup, remove finalisation idea
remove logging
2016-03-01 10:10:02 +00:00
Brian Gough
b44a7b9aa6
reject very large ops
2016-01-26 14:52:40 +00:00
Brian Gough
b3ddd839e6
add logging of raw updates
2016-01-26 11:28:02 +00:00
Brian Gough
29c7c5e249
enable packs by default for new docs
2016-01-25 09:55:55 +00:00
Brian Gough
9b2cd11cd4
don't try to append to packs when using the old op code
2016-01-22 10:45:24 +00:00
Brian Gough
84ace7f4c7
use packs only for temporary ops
2016-01-20 14:22:13 +00:00
Brian Gough
5e830cbbdb
put all new ops into packs
2016-01-19 15:56:09 +00:00
Brian Gough
5153ed8217
make peekLastUpdate alway return lastVersion when available
2016-01-15 15:54:44 +00:00
Brian Gough
6754bdca1c
log timestamp in human-readable form for inconsistent ops
2016-01-05 11:30:24 +00:00
Brian Gough
b7de6f2f71
don't try to compress updates across point of broken history
2015-12-21 13:52:26 +00:00
Brian Gough
4a82dfe618
add setting trackchanges.continueOnError to allow recovery from missing ops
2015-12-17 16:28:02 +00:00
Brian Gough
be2136de7c
fix update-in-place bug for array ops
2015-12-04 15:17:28 +00:00
Brian Gough
3842f0d1cc
Merge pull request #9 from sharelatex/only-delete-applied-ops
...
Only delete applied ops
2015-11-27 12:45:51 +00:00
Brian Gough
8ebc069ddb
modify last compressed op in place
2015-11-26 16:17:18 +00:00
Brian Gough
3432d9e91a
added comments for redis delete
2015-11-26 15:16:54 +00:00
Brian Gough
e65549099c
only delete the applied ops from redis
2015-11-25 16:01:07 +00:00
Brian Gough
b6dae59655
fix callback logic in compressAndSaveRawUpdates
2015-10-08 16:39:13 +01:00
Brian Gough
add6a68fe1
add missing callback in compressAndSaveRawUpdates
2015-10-08 10:53:25 +01:00
Brian Gough
2ab1778dd9
move default value of lastVersion into function body
2015-09-23 16:31:33 +01:00
Brian Gough
e49f260507
allow rollback/locking by setting inS3:false when starting the archive process
2015-09-23 13:28:07 +01:00
Brian Gough
d6b827426c
support forcing new compressed update in popLastCompressedUpdate
...
callback with a null update, passing the version as an additional
argument
2015-09-23 13:22:38 +01:00
Brian Gough
a10dc4f898
Merge pull request #6 from heukirne/s3-archive
...
Add S3 archive track changes feature
2015-09-21 11:25:06 +01:00
Brian Gough
0e627c92d8
avoid clobbering global _ in loop
2015-09-18 16:26:05 +01:00
Brian Gough
d3dff28bea
Merge remote-tracking branch 'origin/master' into heukirne-s3-archive
2015-09-15 15:19:43 +01:00
Shane Kilkelly
eab8b4b6c8
Null safe access of id property, needed as user can be null.
2015-09-11 14:07:06 +01:00
Shane Kilkelly
0ad374556d
Add a comment for clarity.
2015-09-10 16:43:40 +01:00
Shane Kilkelly
8387383cb4
In _summarizeUpdates, allow null users through.
...
A null value represents a deleted or otherwise missing user record.
2015-09-10 14:32:47 +01:00
Henrique Dias
efff026a79
handle easier propagation
2015-08-25 16:52:28 -03:00
Henrique Dias
f910e63e90
fix null case
2015-08-24 12:22:17 -03:00
Henrique Dias
fcbe4aa925
fix inS3 propagation
2015-08-24 12:19:19 -03:00
Henrique Dias
04ec45529f
restore updates from S3 when exists
...
fix: avoid rearchiving
2015-08-18 17:11:19 -03:00
Henrique Dias
6bc9c9010a
handle auto unarchive track changes
2015-08-09 19:52:32 -03:00
Brian Gough
8e810bab14
report error for inconsistent history results
2015-02-05 16:37:06 +00:00
James Allen
eb7bcc6922
Remove old history deletion method
2014-05-29 15:37:16 +01:00
