github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
23884 commits 3 branches 7 tags 206 MiB
Commit graph

543 commits

Author SHA1 Message Date
Eric Mc Sween
2b76aa7df9 Merge pull request #16877 from overleaf/em-history-ranges-support 
Store in Redis whether docs support ranges in the history

GitOrigin-RevId: b6dd6ab269172c3ea7674179fdeb2f068f68b18f
 2024-02-09 09:05:32 +00:00
Eric Mc Sween
0ffcbb96f6 Merge pull request #16821 from overleaf/em-promisify-ranges-manager 
Make RangesManager synchronous

GitOrigin-RevId: 46bc85d952474fa0d83f3fd1299c0a3888fcf436
 2024-02-09 09:04:31 +00:00
Eric Mc Sween
8136036c33 Merge pull request #16644 from overleaf/em-promisify-update-manager 
Promisify UpdateManager

GitOrigin-RevId: 2c3e21ee6ef2454f79695ca8623c3d38720ff6bf
 2024-01-30 16:49:23 +00:00
Jakob Ackermann
a7c7f2e69b Merge pull request #16589 from overleaf/jpa-script-redis-mongo-mismatch 
[document-updater] add script for checking for redis vs mongo de-syncs

GitOrigin-RevId: b51b61efd750f37180c09af5ccf5681712719ee2
 2024-01-24 09:03:59 +00:00
Jakob Ackermann
8bc44141be Merge pull request #16590 from overleaf/jpa-redis-lock-60s 
[server-ce] increase the doc lock TTL from 30s to 60s

GitOrigin-RevId: 468f7483cc6a80e8034e3cc8071b674123985deb
 2024-01-19 09:06:06 +00:00
Christopher Hoskin
f5089f596c Merge pull request #15477 from overleaf/csh-td-issue-11625-mongo-ug-5-document-updater 
Upgrade mongodb module for document updater from 4.11.0 to 6.2.0

GitOrigin-RevId: 4154763ded77e04cd3f4002824fee4af8b7f8e8a
 2023-12-15 09:03:38 +00:00
Jakob Ackermann
06b93aac50 Merge pull request #14377 from overleaf/tw-docupdater-flush 
add check for unflushedTime

GitOrigin-RevId: e09d63b4de09e30ceb82792526224ad8b2415119
 2023-09-07 08:04:57 +00:00
Eric Mc Sween
1aff1b64ab Merge pull request #14662 from overleaf/em-revert-docupdater-history-ranges-support 
Revert document-updater changes from #14567

GitOrigin-RevId: 72af1d24c94f7c46f2557700e6425f278356cffd
 2023-09-06 08:06:46 +00:00
Eric Mc Sween
3fa1245860 Merge pull request #14567 from overleaf/em-history-ranges-flag 
Add historyRangesSupport flag to projects

GitOrigin-RevId: 1e3f24a7c6f209bbd34eaaf4caee56dc7061b3da
 2023-09-06 08:06:19 +00:00
Tyna William
277fe530ad Merge pull request #14294 from overleaf/tw-docupdater-flush 
add metric to confirm what fraction of flush operations are unnecessary

GitOrigin-RevId: 29cff245de56da2cf6c10603570378ee97f00eb7
 2023-08-17 08:02:57 +00:00
Miguel Serrano
2ccc1aeea5 Merge pull request #13264 from overleaf/msm-docupdater-cleanup 
[docupdater] Cleanup track-changes code

GitOrigin-RevId: 906a4f57a9e7348f47af67cff44a32ae1bb2debb
 2023-07-17 10:24:47 +00:00
Eric Mc Sween
3831416c2f Merge pull request #12206 from overleaf/em-camel-case-docupdater 
Camel case variables in document-updater

GitOrigin-RevId: 76ad0921cc059878f21639547fad1bff1913bc8b
 2023-03-22 09:04:37 +00:00
Eric Mc Sween
74eeec2ba3 Merge pull request #11680 from overleaf/em-mongo-pool-monitoring 
Mongo connection pool monitoring

GitOrigin-RevId: 050e50e7e67061ccbf39a710ca4532eafd423365
 2023-02-09 14:34:35 +00:00
Eric Mc Sween
f76ab26822 Merge pull request #10998 from overleaf/em-upgrade-mongo-document-updater 
Upgrade Mongo driver in document updater

GitOrigin-RevId: e3c4d86bbcf86dcf16fdfdbfab2aaf0d8eb1037c
 2023-01-11 09:05:28 +00:00
Eric Mc Sween
5083060fbb Merge pull request #10677 from overleaf/em-history-id-string 
Represent history ids as strings instead of integers

GitOrigin-RevId: 18977195b65492836e570c571ec7e8c7e088612b
 2022-12-02 09:04:29 +00:00
Lucie Germain
e7203ed0f8 Merge pull request #10232 from overleaf/jpa-lg-silent-no-op-external-update 
[document-updater] skip emitting no-op external updates

GitOrigin-RevId: 3fc29740fb8c2bc8b1ba21428a0e8d0d34b6eaf8
 2022-11-04 09:03:54 +00:00
Eric Mc Sween
7d099acfdd Merge pull request #9150 from overleaf/em-share-ranges-tracker 
Move RangesTracker to shared lib

GitOrigin-RevId: 62da7208f0b453dd7272c06873c7e415ed887817
 2022-10-05 13:26:01 +00:00
Eric Mc Sween
59cdcccc26 Merge pull request #9647 from overleaf/bg-dropbox-to-overleaf-existing-doc 
handle updates to existing doc in dropbox to overleaf metadata

GitOrigin-RevId: e82955a4a76e62fb649263a94103fdb7f322de85
 2022-09-22 08:04:11 +00:00
Brian Gough
348035f6ff Merge pull request #9633 from overleaf/bg-external-update-metric 
add a metric for external updates in docupdater

GitOrigin-RevId: db407f742bea692e3617eec08db4756e3b8adbd9
 2022-09-16 08:05:44 +00:00
Eric Mc Sween
8b2f8ce243 Merge pull request #9383 from overleaf/em-file-tree-histories 
Track source for all file tree operations

GitOrigin-RevId: ff95ea8e99bfa30203a2a42968519bbaba65e708
 2022-08-26 08:03:30 +00:00
Brian Gough
805ac00090 Merge pull request #9141 from overleaf/bg-disable-track-changes-2 
disable flush to track-changes in docupdater

GitOrigin-RevId: d5ff12d4755783af0a3f71939c07994d43b9d740
 2022-08-08 08:04:13 +00:00
Brian Gough
678148d1b9 Merge pull request #8839 from overleaf/bg-disable-track-changes 
[document-updater]  disable track-changes

GitOrigin-RevId: f97be24d959ec8fd21e39c46a13b124ed9e5b406
 2022-08-08 08:03:31 +00:00
Brian Gough
8960e56e20 Merge pull request #9020 from overleaf/bg-decaff-guard 
decaff cleanup of __guard__ calls

GitOrigin-RevId: 0d61e4d44a2fda19285674040ba92e500deae78d
 2022-07-29 08:03:23 +00:00
Brian Gough
0c63515de6 Merge pull request #8729 from overleaf/bg-return-file-too-large 
[document-updater] return 413 error when doc is too large

GitOrigin-RevId: d66300c78bea4071f29044be160929dba782b728
 2022-07-07 08:03:23 +00:00
Brian Gough
e7e4e49d90 Merge pull request #8546 from overleaf/bg-fix-docupdater-history-redis 
[document-updater] docsWithHistoryOps should use the main rclient

GitOrigin-RevId: a86c081ea0b0f02a9d4c36df0ca76dc9aa3bac57
 2022-06-24 08:03:32 +00:00
Jakob Ackermann
cd11db456d Merge pull request #7835 from overleaf/jpa-redis-locker-package 
[misc] extract document-updaters LockManager into redis-wrapper package

GitOrigin-RevId: 5775ab9df7afb614dfc5f502df526c8735f84a91
 2022-05-19 08:02:57 +00:00
Jakob Ackermann
f0bd6dda23 Merge pull request #7986 from overleaf/jpa-eslint-8 
[misc] upgrade eslint packages to the latest version everywhere

GitOrigin-RevId: f1480d4a171acef82fb26c4aa54be3a6088b0ab3
 2022-05-17 08:05:59 +00:00
Brian Gough
8bffacb2ba Merge pull request #7845 from overleaf/bg-snapshot-lost-tracked-changes 
[document-updater] consider lost tracked-changes as collapsed ranges

GitOrigin-RevId: cf8e15c610f906f1296fdcdd69128ad1e029863a
 2022-05-10 08:04:25 +00:00
Brian Gough
bd541b4c2d Merge pull request #7347 from overleaf/bg-record-snapshot-metrics 
record metrics for docupdater range operations

GitOrigin-RevId: 41c500630dd81670d2798d388f2961a19df76b63
 2022-04-07 09:57:01 +00:00
Eric Mc Sween
4bbf017c94 Merge pull request #6237 from overleaf/em-revert-jumprope 
Revert use of the jumprope library in document-updater

GitOrigin-RevId: bf83f04cf5a53bf710af74a366f4dd86c06a1fd5
 2022-01-06 09:03:11 +00:00
Eric Mc Sween
c8f61e908c Merge pull request #6018 from overleaf/msm-rope-apply 
Speed up OT Apply in docupdater using Ropes

GitOrigin-RevId: 9789c3910d40a4161f1f0d9c16cef8d8ca74788c
 2021-12-16 09:02:56 +00:00
Brian Gough
d0b7cae021 Merge pull request #5963 from overleaf/bg-fix-missing-pathname-check 
[document-updater] fix check for missing pathname

GitOrigin-RevId: 81ebca469c820ffb197020d29f6734821c270e96
 2021-12-01 09:03:54 +00:00
Eric Mc Sween
7a0b40a4bf Merge pull request #5941 from overleaf/em-cleanup-redis-manager 
Decaf cleanup docupdater RedisManager and DocumentManager

GitOrigin-RevId: 6a9634d1882a3a7328cac8cd88537dd9d204b281
 2021-12-01 09:03:12 +00:00
Brian Gough
4f32ddc4cb Merge pull request #5892 from overleaf/tm-large-docs-in-resync 
Do size check on doc before queuing resyncDocContent

GitOrigin-RevId: 315b849375fd17a2936adacaad4a93ccf283d853
 2021-11-26 09:02:57 +00:00
Brian Gough
9d4ed693f5 Merge pull request #5826 from overleaf/bg-use-resync-path 
[document-updater] use paths from project structure update in resync

GitOrigin-RevId: 6ed2a48da76553e2fe5d0d3ec8da535515a35110
 2021-11-23 09:03:14 +00:00
Brian Gough
033b9bd982 Merge pull request #5861 from overleaf/bg-log-missing-pathnames 
log missing pathnames in document-updater

GitOrigin-RevId: bf3bdee5ded278710aa0b80193b2cf884029f07b
 2021-11-23 09:02:38 +00:00
Brian Gough
587cd33d82 Merge pull request #5860 from overleaf/bg-fix-docupdater-delete-doc 
fix docupdater delete operation

GitOrigin-RevId: daab03b7659688d53c54ac80756394796d3c6348
 2021-11-23 09:02:35 +00:00
Jakob Ackermann
669606f797 Merge pull request #5568 from overleaf/jpa-node-no-callback-literal 
[misc] fix eslint violations for node/no-callback-literal

GitOrigin-RevId: af5aace52e1476b1b0ee48cc8be2aabbe04efac6
 2021-10-28 08:03:33 +00:00
Jakob Ackermann
7f9fd00bda Merge pull request #5367 from overleaf/jpa-node-handle-callback-err 
[misc] fix eslint violations for node/handle-callback-err

GitOrigin-RevId: 83a4900e8861010df1917bff49382bd9c93375bd
 2021-10-28 08:03:26 +00:00
Jakob Ackermann
5862359ff0 Merge pull request #5518 from overleaf/jpa-no-empty 
[misc] fix eslint violations for no-empty

GitOrigin-RevId: 8fc1688ac28ccf29b830ba43cd6091e64b5e3b07
 2021-10-27 08:03:14 +00:00
Jakob Ackermann
a6f05109a3 Merge pull request #5352 from overleaf/jpa-no-var 
[misc] fix eslint violations for `no-var`

GitOrigin-RevId: c52e82f3a8a993b8662cc5aa56e7b95ca3c55832
 2021-10-27 08:03:00 +00:00
Brian Gough
5d6a833216 Merge pull request #5338 from overleaf/bg-update-docupdater-logger 
[docupdater] upgrade from logger-sharelatex to @overleaf/logger

GitOrigin-RevId: 26294a1e61d25b907d594a586f9b4714bc36ae4f
 2021-10-07 08:04:03 +00:00
Brian Gough
b4cd9b089d Merge pull request #5327 from overleaf/bg-extend-docupdater-profiler 
[docupdater] record synchronous operations in profiler

GitOrigin-RevId: 994139e7f3fb04645e435273f5143e1fa6091be2
 2021-10-07 08:03:57 +00:00
Brian Gough
2cd7b615bf Merge pull request #5159 from overleaf/bg-docupdater-log-relevelling 
docupdater log re-levelling

GitOrigin-RevId: c27478d48f1cb1eae022086fe7af9fe55dfacfa0
 2021-10-01 08:03:04 +00:00
Brian Gough
865a973426 Merge pull request #5132 from overleaf/bg-fix-size-check 
[docupdater] use more accurate doc size check

GitOrigin-RevId: f66d68a7f7fdc127cc31539abdcab65549823d02
 2021-09-21 08:02:55 +00:00
Eric Mc Sween
9f4541c266 Merge pull request #4761 from overleaf/em-peek-doc 
Do not unarchive docs when resyncing project history

GitOrigin-RevId: c7df75789c01e6c85b464a9b94b14654d8568407
 2021-08-20 08:03:50 +00:00
Brian Gough
0fd24b5133 peek at docs without fetching from mongo 2021-07-30 16:13:48 +01:00
Jakob Ackermann
e8697f7f97 [misc] temporary override a few new/changed eslint rules 2021-07-13 12:24:36 +01:00
Jakob Ackermann
c532376e21 [misc] run format_fix and lint:fix 2021-07-13 12:04:42 +01:00
Jakob Ackermann
519dd274bf [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:15 +01:00
Jakob Ackermann
757e1e98c5 [RedisManager] block updating of too large docs in redis 2021-05-06 18:21:19 +01:00
Jakob Ackermann
c707d0b345 [RedisManager] block inserting of too large docs into redis 2021-05-06 18:21:18 +01:00
Jakob Ackermann
a4ae0ea12f [ShareJsUpdateManager] double check doc size before flushing 2021-05-06 18:20:51 +01:00
Jakob Ackermann
6e551f9b34 [perf] use MGET/MSET/DEL for manipulating multiple keys in one operation 
In some cases we can get rid of MULTI/EXEC operations too.
- putDocInMemory: from 10 down to 2 operations
- removeDocFromMemory: from 14+4 down to 4+4 operations
- updateDoc: from 13 down to 8 operations
 2021-04-13 11:47:10 +01:00
Jakob Ackermann
178440395f [perf] switch write sequence for doc contents and doc tracking 
Doc contents are added only after the tracking has been setup.
All read paths on the tracking have been checked to gracefully handle
 the case of existing doc_id but missing doc contents.

- getDoc: -1 operation

REF: 0a2b47c660c60b95e360d8f3b3e30b862ceb6d79
 2021-04-13 11:46:44 +01:00
Jakob Ackermann
50b24043b7 [perf] use MGET for fetching multiple keys in one operation 
- getDoc: from 13 down to 2 operations
 2021-04-09 08:42:35 +01:00
Eric Mc Sween
4d70bd664f Reintroduce Node 12 and metrics upgrades 
These changes were previously merged, not deployed, and reverted. This
reverts the revert.

This reverts commit a6b8c6c658b33b6eee78b8b99e43308f32211ae2, reversing
changes made to 93c98921372eed4244d22fce800716cb27eca299.
 2021-04-01 15:51:00 -04:00
Eric Mc Sween
4dd1b26b2e Revert "Merge pull request #161 from overleaf/em-upgrade-node-12" 
This reverts commit d44102751b9436ad89c5b3b05e7abdff51fcc78a, reversing
changes made to 6c2f5b8d053b75c677da2b7ddd04f998d2be6fff.
 2021-03-31 12:07:11 -04:00
Eric Mc Sween
1a2235a219 Upgrade to Node 12 2021-03-16 16:48:45 -04:00
Thomas
23738540ed Fix API request errors which could contain API hostname or address (#160) 
Wrap errors produced by failing requests to web API, and remove the url/hostname from thrown error messages. (But keep the URL path for info.)
 2021-02-24 15:09:19 +01:00
Henry Oswald
20a373d95c stop listening on the pending updates channels 10 times 2021-02-23 08:27:29 +00:00
Henry Oswald
c7e57cd28f add Dispatchers running on old queue while we migrate 
revert once migrated
 2021-02-15 14:16:45 +00:00
Henry Oswald
854e24bb57 remove unneeded anonymous func 2021-02-15 14:12:28 +00:00
Henry Oswald
11c8cfc939 shard the pending-updates-list queue 2021-02-02 16:38:25 +00:00
Eric Mc Sween
98f8d7f51c Set the diff-match-patch timeout to 100ms 
This might result in worse diffs, but we don't want to spend a second
blocking the event loop while we figure out nicer diffs when comparing
documents.
 2021-01-14 15:11:15 -05:00
Eric Mc Sween
de247302b1 Use a centralized diff-match-patch package 
We use our own fork of the diff-match-patch npm package, which adds an
optimization for the semantic alignment loop.
 2020-12-07 16:15:19 -05:00
Eric Mc Sween
dce5b8759a Decaf cleanup: capitalize class names 2020-12-07 15:30:02 -05:00
Eric Mc Sween
db4b0a6f38 Decaf cleanup: do not throw strings 2020-12-07 15:28:25 -05:00
Eric Mc Sween
8c70e72bfa Decaf cleanup: unused variable 2020-12-07 15:27:41 -05:00
Eric Mc Sween
9f17f3ea0a Decaf cleanup: remove default callback 2020-12-07 15:27:01 -05:00
Eric Mc Sween
b74e7f6feb Decaf cleanup: unnecessary returns 2020-12-07 15:25:52 -05:00
Eric Mc Sween
a91770e979 Decaf cleanup: remove Array.from() 2020-12-07 15:25:20 -05:00
Jakob Ackermann
5e00684dbb [misc] bump metrics module to 3.4.1 
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
 2020-11-25 11:57:19 +00:00
Jakob Ackermann
08ed5f6c9b [misc] bump @overleaf/redis-wrapper to version 2.0.0 2020-11-11 16:20:32 +00:00
Jakob Ackermann
16ef0d9610 [misc] mongodb: use the new db connector by default 
mongojs was enabling it by default as well.
 2020-09-10 10:40:05 +01:00
Jakob Ackermann
c337cf1c4f [misc] mongodb: refactor the process of setting up the db construct 
Co-Authored-By: John Lees-Miller <jdleesmiller@gmail.com>
 2020-09-07 09:49:06 +01:00
Jakob Ackermann
1d57706d44 [misc] migrate acceptance tests to the native mongo driver, drop mongojs 2020-08-28 17:06:25 +01:00
Jakob Ackermann
f80a92ce46 [misc] migrate the app to the native mongo driver 
acceptance tests to follow in a separate commit
 2020-08-28 15:12:30 +01:00
Eric Mc Sween
1d1f204021 Remove backwards-compat project update API 
The project update endpoint accepted updates both in two array params:
docUpdates and fileUpdates, and in a single array: updates. This commit
removes the docUpdates/fileUpdates params now that web uses the updates
param.
 2020-05-20 16:26:22 -04:00
Eric Mc Sween
9799b94752 Accept ordered doc and file updates 
Add an `updates` parameter to the project update endpoint. It can be
used instead of `docUpdates` and `fileUpdates` to provide a single list
of updates in the order they should be processed.
 2020-05-20 07:57:32 -04:00
Eric Mc Sween
6269ace987 Decaf cleanup: move functions to top level 2020-05-15 09:59:32 -04:00
Eric Mc Sween
3d000bcbe6 Decaf cleanup: use arrow functions for callbacks 2020-05-15 09:59:32 -04:00
Eric Mc Sween
751d9ea718 Decaf cleanup: simplify loops 2020-05-15 09:59:32 -04:00
Eric Mc Sween
7a5a782dc7 Decaf cleanup: camel case variables 2020-05-15 09:59:26 -04:00
Eric Mc Sween
ceb77f4c93 Decaf cleanup: remove unused variables 2020-05-14 16:54:08 -04:00
Eric Mc Sween
6b5760ca28 Decaf cleanup: simplify null checks 2020-05-14 16:53:22 -04:00
Eric Mc Sween
f2c67b66fa Decaf cleanup: remove default callbacks 2020-05-14 16:50:04 -04:00
Eric Mc Sween
2bff83137c Decaf cleanup: unnecessary returns 2020-05-14 16:35:10 -04:00
Eric Mc Sween
819aa378d9 Decaf cleanup: remove Array.from() 2020-05-14 16:32:05 -04:00
Eric Mc Sween
a2a1914a53 Use max_doc_length setting to limit incoming doc size 2020-05-11 11:15:37 -04:00
Eric Mc Sween
cb959ddfc1 Decaf cleanup: use arrow functions for callbacks 2020-05-11 11:14:37 -04:00
Eric Mc Sween
e4ac63dd19 Decaf cleanup: move functions to top level 2020-05-11 11:12:15 -04:00
Eric Mc Sween
64a881461f Decaf cleanup: camel case variables 2020-05-11 11:07:15 -04:00
Eric Mc Sween
fc73bbe1a5 Decaf cleanup: simplify null checks 2020-05-11 10:52:06 -04:00
Eric Mc Sween
80ea49c69c Decaf cleanup: remove __guard__ 2020-05-11 10:47:27 -04:00
Eric Mc Sween
814ac40e07 Decaf cleanup: unnecessary returns 2020-05-11 10:45:39 -04:00
Eric Mc Sween
3385ec5f26 Decaf cleanup: unnecessary Array.from() 2020-05-11 10:43:22 -04:00
Tim Alby
9f6ea07002 fix SyntaxError on export var 2020-05-06 12:17:08 +02:00
Tim Alby
e089cfc93c format config/settings.defaults.js & lib/diff_match_patch.js 2020-05-06 12:16:48 +02:00
Tim Alby
96e7a668b7 disable linting for lib/diff_match_patch.js 2020-05-06 12:16:04 +02:00