github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
overleaf/services/document-updater/test/unit/coffee
History
James Allen a3847d21d5 Replace UTF-16 surrogate characters with 'replacement character' 
In Javascript, characters are 16-bits wide. It does not understand surrogates as characters.

From Wikipedia (http://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane):
"The High Surrogates (U+D800–U+DBFF) and Low Surrogate (U+DC00–U+DFFF) codes are reserved
for encoding non-BMP characters in UTF-16 by using a pair of 16-bit codes: one High Surrogate
and one Low Surrogate. A single surrogate code point will never be assigned a character.""

The main offender seems to be \uD835 as a stand alone character, which would be the first
16-bit character of a blackboard bold character (http://www.fileformat.info/info/unicode/char/1d400/index.htm).
Something must be going on client side that is screwing up the encoding and splitting the
two 16-bit characters so that \uD835 is standalone.
2015-06-12 10:14:35 +01:00
..
DiffCodec Initial open sourcing 2014-02-12 10:40:42 +00:00
DispatchManager implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
DocOpsManager Log out warning when using Mongo for version still 2014-05-14 14:16:27 +01:00
DocumentManager Merge branch 'master' into remove_mongo_doc_ops 2014-05-14 12:39:40 +01:00
HttpController Allow source and user_id to be included when setting a document 2014-03-11 12:47:26 +00:00
LockManager implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
PersistenceManager Don't send or get versioning info to/from web 2014-05-15 11:13:16 +01:00
ProjectManager Initial open sourcing 2014-02-12 10:40:42 +00:00
RedisManager implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
ShareJsDB Ensure version is still returned from redis 2014-02-26 14:49:52 +00:00
TrackChangesManager Put doc_ids with history changes into project level set 2014-03-21 12:41:05 +00:00
UpdateManager Replace UTF-16 surrogate characters with 'replacement character' 2015-06-12 10:14:35 +01:00
AddingDocsToMemory.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
CheckingUpdatesLength.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
GettingDoc.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
GettingListOfPendingUpdates.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
GettingTotalNumberOfDocs.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
RemovingSingleDocFromMemory.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
ShareJsUpdateManagerTests.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00