overleaf/services/document-updater/app/coffee
James Allen a3847d21d5 Replace UTF-16 surrogate characters with 'replacement character'
In Javascript, characters are 16-bits wide. It does not understand surrogates as characters.

From Wikipedia (http://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane):
"The High Surrogates (U+D800–U+DBFF) and Low Surrogate (U+DC00–U+DFFF) codes are reserved
for encoding non-BMP characters in UTF-16 by using a pair of 16-bit codes: one High Surrogate
and one Low Surrogate. A single surrogate code point will never be assigned a character.""

The main offender seems to be \uD835 as a stand alone character, which would be the first
16-bit character of a blackboard bold character (http://www.fileformat.info/info/unicode/char/1d400/index.htm).
Something must be going on client side that is screwing up the encoding and splitting the
two 16-bit characters so that \uD835 is standalone.
2015-06-12 10:14:35 +01:00
..
sharejs Initial open sourcing 2014-02-12 10:40:42 +00:00
DiffCodec.coffee Initial open sourcing 2014-02-12 10:40:42 +00:00
DispatchManager.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
DocOpsManager.coffee Use version from web, with fallback to old mongo collection 2014-05-14 13:28:17 +01:00
DocumentManager.coffee Merge branch 'master' into remove_mongo_doc_ops 2014-05-14 12:39:40 +01:00
Errors.coffee Initial open sourcing 2014-02-12 10:40:42 +00:00
HttpController.coffee Allow source and user_id to be included when setting a document 2014-03-11 12:47:26 +00:00
LockManager.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
Metrics.coffee Use new metrics module 2014-05-08 09:28:13 +01:00
mongojs.coffee Use version from web, with fallback to old mongo collection 2014-05-14 13:28:17 +01:00
PersistenceManager.coffee Don't send or get versioning info to/from web 2014-05-15 11:13:16 +01:00
ProjectManager.coffee Initial open sourcing 2014-02-12 10:40:42 +00:00
RedisKeyBuilder.coffee Put doc_ids with history changes into project level set 2014-03-21 12:41:05 +00:00
RedisManager.coffee Add in missed error catching 2014-11-19 12:51:19 +00:00
ShareJsDB.coffee Ensure version is still returned from redis 2014-02-26 14:49:52 +00:00
ShareJsUpdateManager.coffee implemeneted redis-sharelatex 2014-10-07 12:08:36 +01:00
TrackChangesManager.coffee Remove extraneous logging 2014-03-21 13:15:42 +00:00
UpdateManager.coffee Replace UTF-16 surrogate characters with 'replacement character' 2015-06-12 10:14:35 +01:00