James Allen
a197c0d148
Merge pull request #18 from sharelatex/ja-split-on-deletes
...
Split update summary on big deletes
2016-10-10 08:46:46 +01:00
James Allen
40ed6fee46
Split update summary on big deletes
2016-10-04 15:13:04 +01:00
James Allen
a7f44bcd01
Add missing return on callback
2016-09-30 13:36:31 +01:00
James Allen
deced1aa15
Fix unit tests with logger.warn stub
2016-09-30 11:41:49 +01:00
James Allen
bddd1fda7d
Retry rewind if doc and update versions don't match
2016-09-30 11:36:47 +01:00
James Allen
8dda44f880
Speed up unit tests by only generating big fixtures once up front
2016-09-16 11:51:41 +01:00
James Allen
dfe26262ec
Return a No-op if diff returns nothing
2016-09-16 11:50:44 +01:00
James Allen
80375ae2dd
Run a diff against big delete - insert changes which are likely copy-pastes
2016-09-16 11:33:36 +01:00
Brian Gough
6e5eadfa86
include a timeout on WebApiManager requests
2016-04-13 16:42:36 +01:00
Brian Gough
fd49601716
preserve existing history when user upgrades
2016-04-07 15:16:38 +01:00
Brian Gough
e292de5eb0
fix to avoid ever appending permanent changes to expiring packs
2016-04-06 17:00:16 +01:00
Brian Gough
e2e8292590
update tests
2016-03-09 15:24:28 +00:00
Brian Gough
6d43273f04
working on packmanager tests
2016-03-09 14:44:59 +00:00
Brian Gough
bfc9494cc1
update UpdatesManager tests
2016-03-09 14:44:59 +00:00
Brian Gough
4e63c50a75
update MongoManager tests
2016-03-09 14:44:59 +00:00
Brian Gough
b780df6801
removed unused tests
2016-03-04 15:47:59 +00:00
Brian Gough
4d58c14573
updated MongoAWS tests
2016-03-04 15:47:47 +00:00
Brian Gough
ebd5628e53
temporary workaround to get tests passing
...
temporary workaround to get tests passing
2016-03-01 10:10:12 +00:00
Brian Gough
3d9dfeccc3
remove pack worker
...
remove the op-specific code
remove tests for ops, now only packing
remove unused packing code
work in progress
store index for completed packs only
support archiving and unarchiving of individual packs
remove support for archiving whole document history
split out ArchiveManager, IndexManager
remove old DocArchive code
remove docHistoryStats collection
comment about archiving
added method to look at index when last pack has been archived
added start of iterator for project results
use a proper iterator
added heap module
getting it working
increase pack size since bulk operations no longer needed
remove unused MongoAWSexternal
cleanup
added doc iterator
remove old query code
added missing files
cleanup
clean upclean up
started adding pack worker for archiving
work in progress
work in progress
getting pack worker working
updating worker
getting packworker working
added lock
use correct key name for track changes aws access
use correct key name for track changes aws access
always send back users array
fix up comparison of retrieved objects
handle op ids inside packs
log when s3 download completes
comments
cleanup, remove finalisation ideacleanup, remove finalisation idea
remove logging
2016-03-01 10:10:02 +00:00
Brian Gough
a23ddf31c0
allow packing of temporary ops
2016-01-29 12:36:03 +00:00
Brian Gough
612079ef8a
started adding more pack tests and packing of temp ops
2016-01-28 16:56:32 +00:00
Brian - Work
666a07e5ba
move lock check into HealthChecker
...
to avoid dependency of HttpController on LockManager in unit tests
2016-01-27 16:04:55 +00:00
Brian Gough
b7a4c72f9c
avoid compressing updates if the result would be too big
2016-01-26 12:23:21 +00:00
Brian Gough
ed0aaa189d
add test for non-overlapping insert-delete case
2016-01-26 12:13:43 +00:00
Brian Gough
ae61d1261e
added tests for pack updates
2016-01-19 15:58:09 +00:00
Brian Gough
61d22f027a
added test for inserting as pack
2016-01-19 15:58:09 +00:00
Brian Gough
5e830cbbdb
put all new ops into packs
2016-01-19 15:56:09 +00:00
Brian Gough
dc564fd5d0
archiving document history now sends all changes to s3
2016-01-15 15:54:46 +00:00
Brian Gough
5153ed8217
make peekLastUpdate alway return lastVersion when available
2016-01-15 15:54:44 +00:00
Brian Gough
8e53d66079
log the key for lock timeouts
2016-01-12 10:47:15 +00:00
Brian Gough
c7b4062412
remove unsupported options argument in count() method of mongojs 1.x
2015-12-22 14:20:34 +00:00
Brian Gough
23c43b8042
skip any broken ops when viewing history diffs
2015-12-04 15:17:28 +00:00
Brian Gough
be2136de7c
fix update-in-place bug for array ops
2015-12-04 15:17:28 +00:00
Brian Gough
11be8c3733
increase timeout in lock manager test
...
on heavily loaded machines 20ms isn't long enough to consistently get
multiple calls to the lock manager
2015-11-27 14:25:25 +00:00
Brian Gough
3842f0d1cc
Merge pull request #9 from sharelatex/only-delete-applied-ops
...
Only delete applied ops
2015-11-27 12:45:51 +00:00
Brian Gough
8ebc069ddb
modify last compressed op in place
2015-11-26 16:17:18 +00:00
Brian Gough
e65549099c
only delete the applied ops from redis
2015-11-25 16:01:07 +00:00
Brian Gough
8961e23954
enhance LockManager to avoid accidental unlocking
2015-10-14 14:42:17 +01:00
James Allen
2a03591030
Stub out noisy/slow logger-sharelatex and mongojs modules in tests
2015-09-25 13:46:20 +01:00
James Allen
23dfe68cb8
Don't error when rewinding and insert op which is beyond the length of the document.
...
ShareJS will accept an op where p > content.length when applied,
and it applies as though p == content.length. However, the op is
passed to us with the original p > content.length. Detect if that
is the case with this op, and shift p back appropriately to match
ShareJS if so.
2015-09-25 13:44:44 +01:00
Brian Gough
bdf1d267f0
fix tests
2015-09-23 15:34:36 +01:00
Brian Gough
d6b827426c
support forcing new compressed update in popLastCompressedUpdate
...
callback with a null update, passing the version as an additional
argument
2015-09-23 13:22:38 +01:00
Brian Gough
dc35ef5cda
fix tests for archiving
2015-09-21 13:24:06 +01:00
Henrique Dias
aa66c5ee8c
improve size function
2015-09-17 10:41:53 -03:00
Henrique Dias
3f712c452a
add size bulk limit
2015-09-17 09:23:13 -03:00
Henrique Dias
a4575f178d
fix unit test
2015-09-16 18:39:07 -03:00
Brian Gough
12c5098b48
fix lock in test
2015-09-16 16:50:36 +01:00
Brian Gough
d3dff28bea
Merge remote-tracking branch 'origin/master' into heukirne-s3-archive
2015-09-15 15:19:43 +01:00
Shane Kilkelly
39f528bcbc
Add a test to check that users are summarised properly even when a null user occurs earlier in the update list.
2015-09-11 14:12:01 +01:00
Shane Kilkelly
8387383cb4
In _summarizeUpdates, allow null users through.
...
A null value represents a deleted or otherwise missing user record.
2015-09-10 14:32:47 +01:00
Shane Kilkelly
575bdc62ec
Add a test for when the user can't be found.
2015-09-10 14:32:18 +01:00
Henrique Dias
7de103af68
fix unit scope error
2015-09-02 17:00:32 -03:00
Henrique Dias
0c16fbed88
add more unit tests
2015-09-02 15:39:19 -03:00
Henrique Dias
1abcea1a66
add some unit test
2015-08-31 18:13:18 -03:00
Brian Gough
76d1350593
fix unit tests
...
packManager now uses LockManager, which tries to talk to redis and
needs to be replaced by a stub in the unit tests.
2015-06-05 10:59:46 +01:00
Brian Gough
7d6811559c
test pack for getDocUpdates with time
2015-02-16 16:43:27 +00:00
Brian Gough
e55f2a0ff7
test pack for getDocUpdates with version
2015-02-16 15:59:58 +00:00
Brian Gough
f5a9184b57
update the tests to work with packs
2015-02-16 15:38:40 +00:00
Henry Oswald
1452eb4572
fixed tests
2014-09-26 17:51:41 +01:00
James Allen
eb7bcc6922
Remove old history deletion method
2014-05-29 15:37:16 +01:00
James Allen
29ad81c134
Use expiresAt rather than tempCreatedAt
2014-05-16 16:41:14 +01:00
James Allen
8b0b79bc32
Use TTL index to auto-delete updates after a week when versioning is not enabled
2014-05-16 15:59:12 +01:00
James Allen
24343a38cd
Add error handling of mismatched versions back in
2014-05-01 10:26:01 +01:00
James Allen
eebfa6499e
Be more forgiving with version mismatches
2014-04-15 08:02:49 +01:00
James Allen
953081c39d
Delete updates that are older than a week, unless versioning or preserveHistory is enabled
2014-03-28 16:01:34 +00:00
James Allen
9f1efe6dca
Add in getProjectDetails to WebApiManager
2014-03-28 13:05:16 +00:00
James Allen
22e3aac5b0
Don't trip up over empty ops
2014-03-25 11:40:48 +00:00
James Allen
bd4bb3d3cf
Add in project flushing end point
2014-03-21 15:57:17 +00:00
James Allen
8cae726838
Backport project_id onto doc updates that don't have one yet
2014-03-21 14:40:51 +00:00
James Allen
2cd85fefec
Flush all docs in a project before getting updates
2014-03-21 13:48:14 +00:00
James Allen
cd9cb51027
Delete doc_id from DocsWithHistoryOps set when processing
2014-03-21 13:17:58 +00:00
James Allen
6aaa7ba8d5
Group toV and fromV by the doc they correspond to
2014-03-20 13:37:23 +00:00
James Allen
5595910122
Return a summary of the entire project changes
2014-03-20 12:10:04 +00:00
James Allen
b7de4c9c6c
Implement project update getter methods
2014-03-19 17:44:16 +00:00
James Allen
962fc18329
Store project_id alongside each change
2014-03-19 16:40:55 +00:00
James Allen
d32b0ee12f
Fetch updates up to the limit, even after summarising
2014-03-18 18:09:25 +00:00
James Allen
81811d7cc5
Move update summarizing logic into UpdatesManager
2014-03-18 11:41:48 +00:00
James Allen
22a806a200
Fix issues with consuming delete updates beyond the end of the diff
2014-03-17 14:54:02 +00:00
James Allen
9977a418c3
Compare diffs with user.id, not user_id
2014-03-12 14:04:33 +00:00
James Allen
575afae048
Compress adjacent inserts and deletes into one diff item
2014-03-12 13:27:18 +00:00
James Allen
7fa88f6b41
Summarise updates when returning them
2014-03-11 17:40:07 +00:00
James Allen
4ecb17b16e
Put multiple ops in one update
2014-03-11 15:24:38 +00:00
James Allen
064bdc3eea
Take user id in from request header and pass to doc updater
2014-03-11 13:01:07 +00:00
James Allen
3d1d962501
Don't try to fetch user when id doesn't exist
2014-03-11 11:45:25 +00:00
James Allen
83b2aa3082
add in restore end point
2014-03-10 16:58:39 +00:00
James Allen
7eb8699b93
Refactor diff manager
2014-03-10 16:03:03 +00:00
James Allen
866084ee6b
Pop last version, not last timestamp
2014-03-07 14:02:16 +00:00
James Allen
b45db3aa2b
Return user details like email and name in updates
2014-03-06 18:04:00 +00:00
James Allen
5fc139db00
Return version numbers of updates
2014-03-06 11:00:49 +00:00
James Allen
aadce232a1
Use version numbers for sorting and querying, not dates
2014-03-06 10:45:51 +00:00
James Allen
2e1307bd82
Doc is always one version ahead of latest op version
2014-03-05 16:31:38 +00:00
James Allen
3660253fd4
Add in /updates end point to get updates
2014-03-05 15:59:40 +00:00
James Allen
a46963a349
Refactor arguments to MongoManager.getUpdatesBetweenDates
2014-03-05 15:06:46 +00:00
James Allen
188d620ce1
Log out and put the last update back when an error occurs
2014-03-05 13:22:38 +00:00
James Allen
e4d8cc7a11
Add in acceptance tests for getting a diff
2014-03-04 15:27:03 +00:00
James Allen
8b71d222d4
Create DiffManager.getDiff
2014-03-04 14:05:17 +00:00
James Allen
1d1dcdfa2f
Add in methods for retrieving updates and doc version
2014-03-04 13:02:48 +00:00
James Allen
20d70859aa
Create buildDiff function
2014-03-03 17:39:59 +00:00
James Allen
e0402692cf
Add in diff generating functions
2014-03-01 13:31:34 +00:00
James Allen
76cdd5cf98
Delete only the updates that were processed, not a full batch
2014-03-01 11:42:31 +00:00
