github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-07 20:31:06 -05:00
Code Issues Projects Releases Packages Wiki Activity
940 commits 3 branches 7 tags 201 MiB
Commit graph

806 commits

Author SHA1 Message Date
James Allen
1d7cbd8a78 Fix unit tests 2016-05-31 13:32:15 +01:00
James Allen
11667e8250 Return semantic error if the requested ops are not available when getting a document 2016-05-31 13:24:19 +01:00
Shane Kilkelly
048fd19418 Add one to size of line. 
To account for newline characters in the original document
 2016-05-12 09:26:50 +01:00
Shane Kilkelly
82d5a7fafd check total size of lines, rather than content-length 2016-05-11 15:55:21 +01:00
Shane Kilkelly
210a61112b reject payloads larger than 2MB for setDoc 2016-04-29 15:08:21 +01:00
James Allen
4ab4e27724 Include COUNT in lock and make RND per-process incase randomBytes blocks 2016-04-13 16:27:53 +01:00
James Allen
d15a22be26 Remove misleading log line about lock value 2016-04-13 12:20:53 +01:00
James Allen
945c728db2 Use signed locks so only the locking party can remove their lock 2016-04-13 11:59:56 +01:00
James Allen
6c79ab4321 Don't let HTTP calls take longer than 5 seconds since we're inside a 30 second lock 2016-04-12 17:10:39 +01:00
James Allen
3c7c318ea0 Clean up and speed up unit tests by making sure requires are mocked 2016-01-20 17:36:06 +00:00
James Allen
af5d01e440 Flush track changes when unloading data from redis 2016-01-20 15:05:31 +00:00
James Allen
184c9031cd Delete doc from redis after a full set if it wasn't in redis before 2016-01-20 14:31:25 +00:00
Brian Gough
e8f09f3357 added comment about flushing to track changes 2015-12-01 16:03:05 +00:00
Brian Gough
6e97521971 Revert "Merge pull request #4 from sharelatex/flush-ops-from-redis" 
This reverts commit 84a9ad8b67b860a92271cf7bb669cf21cc0b5b86, reversing
changes made to a6d4649f4f2db06862e8cbc5e132a073ce8c20b2.
 2015-12-01 15:58:40 +00:00
Brian Gough
13e22e1802 Revert "only flush to track-changes when ops are queued" 
This reverts commit 0baa8f989481ca263fa71e90af21f13b29c29504.
 2015-12-01 15:48:49 +00:00
Brian Gough
64f5d5526e Revert "update tests" 
This reverts commit 501e891760fd5c5bd0e6ffc75f866fd565561f31.
 2015-12-01 15:48:31 +00:00
Brian Gough
4bf90afe0e update tests 2015-11-30 15:22:21 +00:00
Brian Gough
7a9577e081 only flush to track-changes when ops are queued 2015-11-30 15:17:11 +00:00
Brian Gough
529a061567 Merge pull request #4 from sharelatex/flush-ops-from-redis 
push ops to track changes when document is flushed
 2015-11-30 10:46:21 +00:00
James Allen
2589e2d417 Gracefully return when an op has already been submitted 
It is not a fatal error if an op has already been submitted. We just
need to send an ack back to the client that submitted it and continue.
If we detect a duplicate op, set dup: true on the op and pass it back
to real-time for distributing. The dup: true flag will ensure it only
gets acknowledged to the submitting client, not everyone.
 2015-11-19 10:54:28 +00:00
James Allen
e73890bfc2 Error if update would make document too long 2015-11-06 12:52:03 +00:00
James Allen
03e9d7390f Add in mongo health check end point 2015-08-31 15:57:26 +01:00
James Allen
d27d2b90a2 Merge pull request #5 from sharelatex/utf16_fix 
Replace UTF-16 surrogate characters with 'replacement character'
 2015-06-12 10:18:34 +01:00
James Allen
333591d087 Extra null check 2015-06-12 10:16:33 +01:00
James Allen
a3847d21d5 Replace UTF-16 surrogate characters with 'replacement character' 
In Javascript, characters are 16-bits wide. It does not understand surrogates as characters.

From Wikipedia (http://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane):
"The High Surrogates (U+D800–U+DBFF) and Low Surrogate (U+DC00–U+DFFF) codes are reserved
for encoding non-BMP characters in UTF-16 by using a pair of 16-bit codes: one High Surrogate
and one Low Surrogate. A single surrogate code point will never be assigned a character.""

The main offender seems to be \uD835 as a stand alone character, which would be the first
16-bit character of a blackboard bold character (http://www.fileformat.info/info/unicode/char/1d400/index.htm).
Something must be going on client side that is screwing up the encoding and splitting the
two 16-bit characters so that \uD835 is standalone.
 2015-06-12 10:14:35 +01:00
Brian Gough
e7e82fc89d update port to match default track changes configuration 2015-05-28 16:43:58 +01:00
Brian
60db8bf5de flush ops when document is flushed 2015-05-28 14:48:44 +01:00
Brian Gough
85a4bf0da6 add tests calling zlib.gzip directly (for debugging) 2015-05-19 14:39:47 +01:00
Brian Gough
85eab2e967 modify DispatchManager test to allow for slow shutdown 2015-05-19 14:23:32 +01:00
Brian Gough
57f6919485 modify LockManager test to avoid dependence on timing 2015-05-18 09:03:51 +01:00
Henry Oswald
d862227314 make startup message consistent 2015-04-30 16:01:22 +01:00
Brian Gough
ea92fe0e9e Merge pull request #3 from sharelatex/gzip-large-docs 
Gzip large docs
 2015-03-31 10:58:18 +01:00
Brian Gough
6cdf5615fc remove old unused functions 2015-03-31 10:24:09 +01:00
Brian Gough
36f60d5bce enforce minimum size of 1k for compression 2015-03-31 10:07:39 +01:00
Brian Gough
9bb08d7ba5 add ZipManager comments 2015-03-31 10:07:11 +01:00
Brian Gough
e61beed92f suppress logging in ZipManager tests 2015-03-30 16:58:00 +01:00
Brian Gough
495af5d568 remove console.logs from tests 2015-03-30 16:56:07 +01:00
Brian Gough
c8c12e8b41 fix error in ZipManager writesEnabled setting 2015-03-30 16:41:45 +01:00
Brian Gough
8e8ee5b3da fix tests 2015-03-30 16:41:23 +01:00
Brian Gough
86505047a3 added unit test (work in progress) 2015-03-27 17:04:38 +00:00
Brian Gough
03564b2138 fix variable zip.minsize to match config name zip.minSize 2015-03-27 17:00:30 +00:00
Brian Gough
66fa170ac8 disable compression by default 2015-03-27 15:32:13 +00:00
Brian Gough
27d466aa85 added acceptance test for redis compression 2015-03-27 13:17:38 +00:00
Brian Gough
6bffa4d9e0 don't log docLines when document removed from redis 
they can now be binary gzipped data which messes up the logs
 2015-03-25 16:54:36 +00:00
Brian Gough
e3d73d4456 add gzip support for large documents to reduce load on redis 2015-03-25 16:53:20 +00:00
James Allen
d12341da1d Release version 0.1.4 2015-03-20 14:21:34 +00:00
James Allen
4942038a23 Release version 0.1.3 2015-02-26 11:23:45 +00:00
James Allen
5cf819cd44 Release version 0.1.2 2015-02-10 13:17:14 +00:00
Henry Oswald
e23b0f0076 bump redis package 2014-11-20 12:53:41 +00:00
Henry Oswald
db0322e054 added redis health check 2014-11-20 11:40:52 +00:00
Henry Oswald
4f56d8d4e8 Revert "Use new 'robust' subscription scheme" 
This reverts commit dd4b938365d0f6196206fd7c70056c34477d13d5
 2014-11-19 16:54:02 +00:00
Henry Oswald
566bf72bfd Revert "lock down the redis version" 
This reverts commit 8c942ca7149afff241c968f66669b17c7e36ca4e. and dd4b938365d0f6196206fd7c70056c34477d13d5
 2014-11-19 16:53:50 +00:00
Henry Oswald
5ae364e6c1 lock down the redis version 2014-11-19 16:48:43 +00:00
James Allen
0eab2d99b6 Use new 'robust' subscription scheme 2014-11-19 12:54:47 +00:00
James Allen
c822bb6ea4 Add in missed error catching 2014-11-19 12:51:19 +00:00
Henry Oswald
c137e05bfd fixed acceptence tests 2014-10-07 12:31:16 +01:00
Henry Oswald
5ff8983528 implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
Henry Oswald
9c85e9b848 moved back to not include sentinal while we upgrade to 0.10.x node 2014-09-29 12:07:32 +01:00
Henry Oswald
fd6c2b5616 bump redis-sharelatex 2014-09-29 11:41:26 +01:00
Henry Oswald
228045667c increament redis-sharelatex 2014-09-28 20:35:05 +01:00
Henry Oswald
7b1b6d27f3 fixed missing new redis depenency 2014-09-28 20:14:14 +01:00
Henry Oswald
d7894c4ea7 fixed tests 2014-09-26 17:57:15 +01:00
Henry Oswald
672dd3de1b works with sentinal v1 2014-09-26 17:38:59 +01:00
Henry Oswald
57fc710be6 tried downgrading forever 2014-09-09 12:12:04 +01:00
Henry Oswald
8b7d92b149 Merge branch 'master' of https://github.com/sharelatex/document-updater-sharelatex 2014-09-04 12:40:34 +01:00
Henry Oswald
114a9ba4d6 bumbed forever up to same version as docstore as npm is complaining 2014-09-04 12:40:17 +01:00
James Allen
a4a0eef153 Release version 0.1.0 2014-08-19 14:07:14 +01:00
James Allen
3ffe96f548 Lock down module versions 2014-08-19 14:06:56 +01:00
James Allen
48c5f66af3 Remove acceptance test from travis 2014-08-15 11:06:31 +01:00
James Allen
9da118c809 Process updates in the background so that workers only act as dispatchers 2014-08-11 14:16:05 +01:00
James Allen
773e17df66 Send worker idle waiting times to statsd 2014-08-07 13:16:11 +01:00
James Allen
cab5509076 Listen on queue via BLPOP rather than pub/sub 2014-08-07 11:45:53 +01:00
James Allen
946f451155 Move to default settings file 2014-05-15 16:22:06 +01:00
James Allen
ffd10a8439 Don't send or get versioning info to/from web 
The version number is only used by the doc updater and so can be cleanly
encapsulated in a collection that only the doc updater knows about. The version
is already stored under docOps, so continue to tore it there.
 2014-05-15 11:13:16 +01:00
James Allen
e25fb7e435 Keeping writing and reading updates from Mongo to cover the deploy period 2014-05-14 16:04:25 +01:00
James Allen
bdfe018cad Log out warning when using Mongo for version still 2014-05-14 14:16:27 +01:00
James Allen
0199f2e129 Use version from web, with fallback to old mongo collection 2014-05-14 13:28:17 +01:00
James Allen
8973969224 Merge branch 'master' into remove_mongo_doc_ops 
Conflicts:
	app/coffee/DocOpsManager.coffee
	test/acceptance/coffee/ApplyingUpdatesToADocTests.coffee
	test/acceptance/coffee/FlushingDocsTests.coffee
	test/unit/coffee/DocOpsManager/DocOpsManagerTests.coffee
	test/unit/coffee/RedisManager/prependDocOpsTests.coffee
	test/unit/coffee/RedisManager/pushDocOpTests.coffee
 2014-05-14 12:39:40 +01:00
James Allen
6011ce4783 Use new metrics module 2014-05-08 09:28:13 +01:00
James Allen
f511ebd4b6 Exit cleanly on SIGINT et al 2014-05-07 10:05:07 +01:00
James Allen
d1434f7646 Increase redis lock expiry time to 30 seconds 2014-05-07 09:48:29 +01:00
James Allen
c2ebaaa338 Split lines on Windows line endings too 2014-04-10 12:44:46 +01:00
James Allen
375427bf5e Remove extraneous logging 2014-03-21 13:15:42 +00:00
James Allen
c0be3ef37b Put doc_ids with history changes into project level set 2014-03-21 12:41:05 +00:00
James Allen
2d28f1903f Flush to the track changes api using the project id as well 2014-03-19 15:56:44 +00:00
James Allen
4f878e000b Allow source and user_id to be included when setting a document 2014-03-11 12:47:26 +00:00
James Allen
5d45e191f3 Don't crash when logging out error 2014-03-04 12:39:02 +00:00
James Allen
86195ce7c3 Add in load throttling based on a redis key 2014-02-28 19:09:29 +00:00
James Allen
3d70f9126e Flush track changes api every 50 updates 2014-02-28 18:29:05 +00:00
James Allen
77c5a27e12 Set up acceptance tests in TravisCI 2014-02-26 16:54:35 +00:00
James Allen
f3192da87f Tell track changes api to flush doc when flushing doc to mongo 2014-02-26 15:56:52 +00:00
James Allen
dfd3ec993b Ensure version is still returned from redis 2014-02-26 14:49:52 +00:00
James Allen
b13f70eadb Push ops into uncompressedHistoryOps list 2014-02-24 16:52:12 +00:00
James Allen
6f19f46d96 Create method for pushing uncompressed ops into redis 2014-02-24 16:40:29 +00:00
James Allen
9cf883e59f Merge branch 'master' into remove_mongo_doc_ops 2014-02-24 16:17:30 +00:00
James Allen
7a977ca597 Don't store ops in Mongo, just keep them in redis with a timeout 2014-02-24 16:13:36 +00:00
James Allen
86e42ce1de Update README.md 2014-02-24 14:43:33 +00:00
James Allen
55edef6d3d Update .travis.yml 2014-02-24 14:37:38 +00:00
James Allen
1a7d2643c0 Create .travis.yml 2014-02-24 14:24:44 +00:00
goodbest
45437c6006 change repo URL from git+ssh:// to git+https:// 2014-02-21 23:10:26 +08:00
James Allen
53929197dd Create LICENSE 2014-02-18 17:23:36 +00:00
James Allen
b45a9968b3 Create README.md 2014-02-18 17:23:12 +00:00
James Allen
8e8d25e185 Point custom npm modules to github 2014-02-17 15:08:46 +00:00
James Allen
414b669a2c Point every redis connection to the new config 2014-02-13 10:36:26 +00:00
James Allen
a5d816272e Remove unused DocumentUpdater.js file 2014-02-12 10:52:53 +00:00
James Allen
e1a7d4f24a Initial open sourcing 2014-02-12 10:40:42 +00:00