Commit graph

185 commits

Author SHA1 Message Date
James Allen
d27d2b90a2 Merge pull request #5 from sharelatex/utf16_fix
Replace UTF-16 surrogate characters with 'replacement character'
2015-06-12 10:18:34 +01:00
James Allen
333591d087 Extra null check 2015-06-12 10:16:33 +01:00
James Allen
a3847d21d5 Replace UTF-16 surrogate characters with 'replacement character'
In Javascript, characters are 16-bits wide. It does not understand surrogates as characters.

From Wikipedia (http://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane):
"The High Surrogates (U+D800–U+DBFF) and Low Surrogate (U+DC00–U+DFFF) codes are reserved
for encoding non-BMP characters in UTF-16 by using a pair of 16-bit codes: one High Surrogate
and one Low Surrogate. A single surrogate code point will never be assigned a character.""

The main offender seems to be \uD835 as a stand alone character, which would be the first
16-bit character of a blackboard bold character (http://www.fileformat.info/info/unicode/char/1d400/index.htm).
Something must be going on client side that is screwing up the encoding and splitting the
two 16-bit characters so that \uD835 is standalone.
2015-06-12 10:14:35 +01:00
Brian Gough
e7e82fc89d update port to match default track changes configuration 2015-05-28 16:43:58 +01:00
Brian
60db8bf5de flush ops when document is flushed 2015-05-28 14:48:44 +01:00
Brian Gough
85a4bf0da6 add tests calling zlib.gzip directly (for debugging) 2015-05-19 14:39:47 +01:00
Brian Gough
85eab2e967 modify DispatchManager test to allow for slow shutdown 2015-05-19 14:23:32 +01:00
Brian Gough
57f6919485 modify LockManager test to avoid dependence on timing 2015-05-18 09:03:51 +01:00
Henry Oswald
d862227314 make startup message consistent 2015-04-30 16:01:22 +01:00
Brian Gough
ea92fe0e9e Merge pull request #3 from sharelatex/gzip-large-docs
Gzip large docs
2015-03-31 10:58:18 +01:00
Brian Gough
6cdf5615fc remove old unused functions 2015-03-31 10:24:09 +01:00
Brian Gough
36f60d5bce enforce minimum size of 1k for compression 2015-03-31 10:07:39 +01:00
Brian Gough
9bb08d7ba5 add ZipManager comments 2015-03-31 10:07:11 +01:00
Brian Gough
e61beed92f suppress logging in ZipManager tests 2015-03-30 16:58:00 +01:00
Brian Gough
495af5d568 remove console.logs from tests 2015-03-30 16:56:07 +01:00
Brian Gough
c8c12e8b41 fix error in ZipManager writesEnabled setting 2015-03-30 16:41:45 +01:00
Brian Gough
8e8ee5b3da fix tests 2015-03-30 16:41:23 +01:00
Brian Gough
86505047a3 added unit test (work in progress) 2015-03-27 17:04:38 +00:00
Brian Gough
03564b2138 fix variable zip.minsize to match config name zip.minSize 2015-03-27 17:00:30 +00:00
Brian Gough
66fa170ac8 disable compression by default 2015-03-27 15:32:13 +00:00
Brian Gough
27d466aa85 added acceptance test for redis compression 2015-03-27 13:17:38 +00:00
Brian Gough
6bffa4d9e0 don't log docLines when document removed from redis
they can now be binary gzipped data which messes up the logs
2015-03-25 16:54:36 +00:00
Brian Gough
e3d73d4456 add gzip support for large documents to reduce load on redis 2015-03-25 16:53:20 +00:00
James Allen
d12341da1d Release version 0.1.4 2015-03-20 14:21:34 +00:00
James Allen
4942038a23 Release version 0.1.3 2015-02-26 11:23:45 +00:00
James Allen
5cf819cd44 Release version 0.1.2 2015-02-10 13:17:14 +00:00
Henry Oswald
e23b0f0076 bump redis package 2014-11-20 12:53:41 +00:00
Henry Oswald
db0322e054 added redis health check 2014-11-20 11:40:52 +00:00
Henry Oswald
4f56d8d4e8 Revert "Use new 'robust' subscription scheme"
This reverts commit dd4b938365d0f6196206fd7c70056c34477d13d5
2014-11-19 16:54:02 +00:00
Henry Oswald
566bf72bfd Revert "lock down the redis version"
This reverts commit 8c942ca7149afff241c968f66669b17c7e36ca4e. and dd4b938365d0f6196206fd7c70056c34477d13d5
2014-11-19 16:53:50 +00:00
Henry Oswald
5ae364e6c1 lock down the redis version 2014-11-19 16:48:43 +00:00
James Allen
0eab2d99b6 Use new 'robust' subscription scheme 2014-11-19 12:54:47 +00:00
James Allen
c822bb6ea4 Add in missed error catching 2014-11-19 12:51:19 +00:00
Henry Oswald
c137e05bfd fixed acceptence tests 2014-10-07 12:31:16 +01:00
Henry Oswald
5ff8983528 implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
Henry Oswald
9c85e9b848 moved back to not include sentinal while we upgrade to 0.10.x node 2014-09-29 12:07:32 +01:00
Henry Oswald
fd6c2b5616 bump redis-sharelatex 2014-09-29 11:41:26 +01:00
Henry Oswald
228045667c increament redis-sharelatex 2014-09-28 20:35:05 +01:00
Henry Oswald
7b1b6d27f3 fixed missing new redis depenency 2014-09-28 20:14:14 +01:00
Henry Oswald
d7894c4ea7 fixed tests 2014-09-26 17:57:15 +01:00
Henry Oswald
672dd3de1b works with sentinal v1 2014-09-26 17:38:59 +01:00
Henry Oswald
57fc710be6 tried downgrading forever 2014-09-09 12:12:04 +01:00
Henry Oswald
8b7d92b149 Merge branch 'master' of https://github.com/sharelatex/document-updater-sharelatex 2014-09-04 12:40:34 +01:00
Henry Oswald
114a9ba4d6 bumbed forever up to same version as docstore as npm is complaining 2014-09-04 12:40:17 +01:00
James Allen
a4a0eef153 Release version 0.1.0 2014-08-19 14:07:14 +01:00
James Allen
3ffe96f548 Lock down module versions 2014-08-19 14:06:56 +01:00
James Allen
48c5f66af3 Remove acceptance test from travis 2014-08-15 11:06:31 +01:00
James Allen
9da118c809 Process updates in the background so that workers only act as dispatchers 2014-08-11 14:16:05 +01:00
James Allen
773e17df66 Send worker idle waiting times to statsd 2014-08-07 13:16:11 +01:00
James Allen
cab5509076 Listen on queue via BLPOP rather than pub/sub 2014-08-07 11:45:53 +01:00