James Allen
af5d01e440
Flush track changes when unloading data from redis
2016-01-20 15:05:31 +00:00
James Allen
184c9031cd
Delete doc from redis after a full set if it wasn't in redis before
2016-01-20 14:31:25 +00:00
Brian Gough
6e97521971
Revert "Merge pull request #4 from sharelatex/flush-ops-from-redis"
...
This reverts commit 84a9ad8b67b860a92271cf7bb669cf21cc0b5b86, reversing
changes made to a6d4649f4f2db06862e8cbc5e132a073ce8c20b2.
2015-12-01 15:58:40 +00:00
Brian Gough
64f5d5526e
Revert "update tests"
...
This reverts commit 501e891760fd5c5bd0e6ffc75f866fd565561f31.
2015-12-01 15:48:31 +00:00
Brian Gough
4bf90afe0e
update tests
2015-11-30 15:22:21 +00:00
Brian Gough
529a061567
Merge pull request #4 from sharelatex/flush-ops-from-redis
...
push ops to track changes when document is flushed
2015-11-30 10:46:21 +00:00
James Allen
03e9d7390f
Add in mongo health check end point
2015-08-31 15:57:26 +01:00
James Allen
d27d2b90a2
Merge pull request #5 from sharelatex/utf16_fix
...
Replace UTF-16 surrogate characters with 'replacement character'
2015-06-12 10:18:34 +01:00
James Allen
a3847d21d5
Replace UTF-16 surrogate characters with 'replacement character'
...
In Javascript, characters are 16-bits wide. It does not understand surrogates as characters.
From Wikipedia (http://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane ):
"The High Surrogates (U+D800–U+DBFF) and Low Surrogate (U+DC00–U+DFFF) codes are reserved
for encoding non-BMP characters in UTF-16 by using a pair of 16-bit codes: one High Surrogate
and one Low Surrogate. A single surrogate code point will never be assigned a character.""
The main offender seems to be \uD835 as a stand alone character, which would be the first
16-bit character of a blackboard bold character (http://www.fileformat.info/info/unicode/char/1d400/index.htm ).
Something must be going on client side that is screwing up the encoding and splitting the
two 16-bit characters so that \uD835 is standalone.
2015-06-12 10:14:35 +01:00
Brian
60db8bf5de
flush ops when document is flushed
2015-05-28 14:48:44 +01:00
Brian Gough
85a4bf0da6
add tests calling zlib.gzip directly (for debugging)
2015-05-19 14:39:47 +01:00
Brian Gough
85eab2e967
modify DispatchManager test to allow for slow shutdown
2015-05-19 14:23:32 +01:00
Brian Gough
57f6919485
modify LockManager test to avoid dependence on timing
2015-05-18 09:03:51 +01:00
Brian Gough
e61beed92f
suppress logging in ZipManager tests
2015-03-30 16:58:00 +01:00
Brian Gough
495af5d568
remove console.logs from tests
2015-03-30 16:56:07 +01:00
Brian Gough
8e8ee5b3da
fix tests
2015-03-30 16:41:23 +01:00
Brian Gough
86505047a3
added unit test (work in progress)
2015-03-27 17:04:38 +00:00
Brian Gough
27d466aa85
added acceptance test for redis compression
2015-03-27 13:17:38 +00:00
Brian Gough
e3d73d4456
add gzip support for large documents to reduce load on redis
2015-03-25 16:53:20 +00:00
Henry Oswald
c137e05bfd
fixed acceptence tests
2014-10-07 12:31:16 +01:00
Henry Oswald
5ff8983528
implemeneted redis-sharelatex
2014-10-07 12:08:36 +01:00
Henry Oswald
9c85e9b848
moved back to not include sentinal while we upgrade to 0.10.x node
2014-09-29 12:07:32 +01:00
Henry Oswald
7b1b6d27f3
fixed missing new redis depenency
2014-09-28 20:14:14 +01:00
Henry Oswald
d7894c4ea7
fixed tests
2014-09-26 17:57:15 +01:00
James Allen
9da118c809
Process updates in the background so that workers only act as dispatchers
2014-08-11 14:16:05 +01:00
James Allen
cab5509076
Listen on queue via BLPOP rather than pub/sub
2014-08-07 11:45:53 +01:00
James Allen
ffd10a8439
Don't send or get versioning info to/from web
...
The version number is only used by the doc updater and so can be cleanly
encapsulated in a collection that only the doc updater knows about. The version
is already stored under docOps, so continue to tore it there.
2014-05-15 11:13:16 +01:00
James Allen
e25fb7e435
Keeping writing and reading updates from Mongo to cover the deploy period
2014-05-14 16:04:25 +01:00
James Allen
bdfe018cad
Log out warning when using Mongo for version still
2014-05-14 14:16:27 +01:00
James Allen
0199f2e129
Use version from web, with fallback to old mongo collection
2014-05-14 13:28:17 +01:00
James Allen
8973969224
Merge branch 'master' into remove_mongo_doc_ops
...
Conflicts:
app/coffee/DocOpsManager.coffee
test/acceptance/coffee/ApplyingUpdatesToADocTests.coffee
test/acceptance/coffee/FlushingDocsTests.coffee
test/unit/coffee/DocOpsManager/DocOpsManagerTests.coffee
test/unit/coffee/RedisManager/prependDocOpsTests.coffee
test/unit/coffee/RedisManager/pushDocOpTests.coffee
2014-05-14 12:39:40 +01:00
James Allen
d1434f7646
Increase redis lock expiry time to 30 seconds
2014-05-07 09:48:29 +01:00
James Allen
c2ebaaa338
Split lines on Windows line endings too
2014-04-10 12:44:46 +01:00
James Allen
c0be3ef37b
Put doc_ids with history changes into project level set
2014-03-21 12:41:05 +00:00
James Allen
2d28f1903f
Flush to the track changes api using the project id as well
2014-03-19 15:56:44 +00:00
James Allen
4f878e000b
Allow source and user_id to be included when setting a document
2014-03-11 12:47:26 +00:00
James Allen
5d45e191f3
Don't crash when logging out error
2014-03-04 12:39:02 +00:00
James Allen
86195ce7c3
Add in load throttling based on a redis key
2014-02-28 19:09:29 +00:00
James Allen
3d70f9126e
Flush track changes api every 50 updates
2014-02-28 18:29:05 +00:00
James Allen
f3192da87f
Tell track changes api to flush doc when flushing doc to mongo
2014-02-26 15:56:52 +00:00
James Allen
dfd3ec993b
Ensure version is still returned from redis
2014-02-26 14:49:52 +00:00
James Allen
b13f70eadb
Push ops into uncompressedHistoryOps list
2014-02-24 16:52:12 +00:00
James Allen
6f19f46d96
Create method for pushing uncompressed ops into redis
2014-02-24 16:40:29 +00:00
James Allen
7a977ca597
Don't store ops in Mongo, just keep them in redis with a timeout
2014-02-24 16:13:36 +00:00
James Allen
e1a7d4f24a
Initial open sourcing
2014-02-12 10:40:42 +00:00
