Commit graph

1023 commits

Author SHA1 Message Date
Brian Gough
ec433357a4 Merge pull request #20023 from overleaf/bg-exit-from-remove-deleted-docs-script
add process exit after completion in remove_deleted_docs script

GitOrigin-RevId: b1cd1606e94563721fcdb883cd83019d8f059c7f
2024-08-21 08:05:28 +00:00
Brian Gough
6290718b7e Merge pull request #19815 from overleaf/bg-fix-ranges-manager-unit-test
fix RangesManager unit test for mixed track changes

GitOrigin-RevId: 8f12708261ce94bed0c80c29f4a05c94876a4e9d
2024-08-08 08:04:22 +00:00
Jakob Ackermann
d21874b076 Merge pull request #19741 from overleaf/jpa-check-filestore-write
[misc] verify file hash when downloading binary file in project-history

GitOrigin-RevId: 0ef56a0753cdfd55fdea921b3555dea48036766b
2024-08-06 08:04:49 +00:00
Jakob Ackermann
e26b6de51b Merge pull request #19740 from overleaf/jpa-linked-file-metadata
[misc] persist linkedFileData in full project history as file metadata

GitOrigin-RevId: f3e8ba947ea34b6796e210a076a248c57188d148
2024-08-06 08:04:44 +00:00
Jakob Ackermann
8c0a78c7e7 Merge pull request #19480 from overleaf/jpa-fast-path-fetch-for-transform
[document-updater] avoid fetching updates to transform when up-to-date

GitOrigin-RevId: 7962d8903a7bc9b572d7c6adfd8f33ad36f30459
2024-07-24 08:05:14 +00:00
Jakob Ackermann
0f0d562786 Merge pull request #19455 from overleaf/jpa-metrics
[misc] add metrics for document processing/broadcasting

GitOrigin-RevId: d81de0dfb7a91863547631580f3c85f569718130
2024-07-19 08:04:34 +00:00
Brian Gough
9babc70df7 Merge pull request #17362 from overleaf/bg-chai-object-id-tests
fix for chai object id tests

GitOrigin-RevId: 98123dbc930c6ee57be3a118177426120482c5f4
2024-07-16 08:04:46 +00:00
Jakob Ackermann
cbe7a2f5a9 Merge pull request #19282 from overleaf/jpa-filestore-sharding
[filestore] add sharding to test_acceptance in CI

GitOrigin-RevId: dd16a0370878d76a934d277838083e08ae0ad8c2
2024-07-15 09:05:11 +00:00
Eric Mc Sween
adf21125b7 Merge pull request #19263 from overleaf/mj-doc-updater-missing-ranges
[doc-updater] Allow resyncs on docs without ranges property

GitOrigin-RevId: b393e463036990ae18bb18aa93ed24c833a619ea
2024-07-05 08:04:35 +00:00
Eric Mc Sween
e8e31dbdb5 Merge pull request #19041 from overleaf/em-docupdater-block-project
Endpoint for blocking projects from being loaded in docupdater

GitOrigin-RevId: 87d79a0b8ccfa0ed46fbf0c198e8a405c1c1151f
2024-07-03 08:04:40 +00:00
Eric Mc Sween
e73fdfba63 Merge pull request #18906 from overleaf/em-migrate-existing-histories-2
History ranges migration script - second attempt

GitOrigin-RevId: 60a2c04e2a72e76a58e9e179fefc4186a96fde32
2024-06-19 08:05:02 +00:00
Alexandre Bourdin
f265797bb1 Sets lastUpdatedBy to null when it is empty on github sync update (#18788)
GitOrigin-RevId: bce67f7d3dbc59facc626cc63af94df46afd8f24
2024-06-18 08:04:39 +00:00
Domagoj Kriskovic
7e8e2b0585 Send origin metadata through docupdater and project-history when restoring files (#18721)
* add RestoreFileOrigin in overleaf-editor-core

* support source to be an object

* use sourceOrOrigin as param

* rename to originOrSource so the priority is more clear

* get timestamp from version

* fix test

* include version and min_count in getUpdatesFromHistory

* extractOriginOrSource util function

* fix RestoreManagerTests

GitOrigin-RevId: 0ace05a6ade2794c753a9d0bffb4f858ecc6899a
2024-06-18 08:04:34 +00:00
andrew rumble
49b7ebe0d0 Remove instances where async is required but not used
GitOrigin-RevId: a202a2d90c4ce61391aabbde1f24437caef54ff0
2024-06-17 08:04:05 +00:00
andrew rumble
ee31d44f03 Explicitly specify latest version of async
GitOrigin-RevId: 9112007de0f5eb4e75604de9f01c2d77b3b8b980
2024-06-17 08:04:00 +00:00
Eric Mc Sween
6c9d4fb522 Merge pull request #18716 from overleaf/em-tracked-delete-undo
Fix translation of tracked deletes

GitOrigin-RevId: 4124db6953cbed46eea61f62118fc8e1ddfff4a0
2024-06-06 08:04:43 +00:00
Eric Mc Sween
55c342134c Merge pull request #18659 from overleaf/em-crop-comments-tracked-deletes
Crop comments when processing tracked deletes

GitOrigin-RevId: 662c9ed86a8ed4959d1671ce466548487f334f45
2024-06-05 08:03:53 +00:00
Eric Mc Sween
22257cf037 Merge pull request #18583 from overleaf/em-multi-hash-key
Ensure that MULTI commands always go to a single node

GitOrigin-RevId: 223d29986766577df89c82983126dabca5d16eed
2024-05-30 08:05:17 +00:00
Eric Mc Sween
e0f6622519 Merge pull request #18579 from overleaf/em-verbose-multi-error
More verbose error on MULTI

GitOrigin-RevId: c1e194063bfd4b4cc88bfb5775af13e1533100d4
2024-05-30 08:04:52 +00:00
Mathias Jakobsen
f14fdc7b7c Merge pull request #17958 from overleaf/mj-web-revert-file
[web] Initial version of file reverting

GitOrigin-RevId: ef500b2c3ddf21b0acb6650c2526edd10ab2cc18
2024-05-29 08:04:16 +00:00
Eric Mc Sween
8704e430a4 Merge pull request #18468 from overleaf/em-docupdater-resolved-comment-ids
Store resolved comment ids in docupdater

GitOrigin-RevId: 69f09ecf69deedbb9a3682f13356533799025ea8
2024-05-29 08:04:01 +00:00
Andrew Rumble
71187a51ba Merge pull request #18289 from overleaf/ac-ar-eslint-return-await
Add ESLint rule @typescript-eslint/return-await to backend services

GitOrigin-RevId: 75e3e32597827fcc852e69d479515fc72e8f45e4
2024-05-27 10:22:49 +00:00
Eric Mc Sween
5d0190da0f Merge pull request #18342 from overleaf/em-tracking-props-none
Separate handling of "none" tracking type in operations

GitOrigin-RevId: b07ee5320ac1a9c63e3b0059aade1a1564819943
2024-05-17 08:04:46 +00:00
Eric Mc Sween
e87df4a791 Merge pull request #18167 from overleaf/em-simpler-typescript-backend
Simpler Typescript setup in the backend

GitOrigin-RevId: 3a90f26a5ac8029d14203faa98008375ce933e7d
2024-05-07 08:04:00 +00:00
Eric Mc Sween
4114901617 Merge pull request #18142 from overleaf/em-docupdater-resync-ranges
Adjust ranges for tracked deletes when resyncing projects

GitOrigin-RevId: 5f8e6279cdc31e76a2f93cf2129eaca8cac3cb78
2024-05-03 08:04:41 +00:00
Jakob Ackermann
a540754f6e Merge pull request #18116 from overleaf/jpa-bulk-replace-localhost
[misc] bulk replace localhost with 127.0.0.1

GitOrigin-RevId: d238f3635302e8ff5500d611108c4d1bef216726
2024-04-26 08:04:39 +00:00
roo hutton
2469c93da4 Merge pull request #17870 from overleaf/bg-node-upgrade-18-20-2
upgrade node to 18.20.2

GitOrigin-RevId: e4ac30c740ced3885353f1472cf690eb5ec0ece1
2024-04-24 08:05:16 +00:00
roo hutton
d75024a7a1 Merge pull request #17974 from overleaf/rh-express-4192
Upgrade express to 4.19.2

GitOrigin-RevId: 383877c39670093033d802af0fae54ec578ba327
2024-04-23 08:05:01 +00:00
Domagoj Kriskovic
238e63e08c [document-updater] Avoid sending empty ops to project-history (#17920)
GitOrigin-RevId: a5156d99be373feb111bcc956d70c0f47047dd72
2024-04-17 08:04:22 +00:00
Domagoj Kriskovic
c4437c69bc Send operations to project-history when accepting tracked changes (#17599)
* added getHistoryOpForAcceptedChange in RangesManager

* rename adjustHistoryUpdatesMetadata to be treated as public

* handle retain op in UpdateTranslator and updateCompressor

* send op to project-history in acceptChanges

* use promises.queueOps

* use ranges in getHistoryOpForAcceptedChange

* using rangesWithChangeRemoved

* acceptChanges acceptance test

* using change.op.hpos

* Revert "using change.op.hpos"

This reverts commit f53333b5099c840ab8fb8bb08df198ad6cfa2d84.

* use getHistoryOpForAcceptedChanges

* fix historyDocLength

* Revert "rename adjustHistoryUpdatesMetadata to be treated as public"

This reverts commit 2ba9443fd040a5c953828584285887c00dc40ea6.

* fix typescript issues

* sort changes before creating history updates

* fix tests

* sinon spy RangesManager.getHistoryUpdatesForAcceptedChanges

* added unit tests

* sort deletes before inserts

* use getDocLength function

* fix docLength calculation

* fix typo

* allow all retains

* fix lint error

* refactor RangesTests

* fix ts error

* fix history_doc_length calculation in RangesManager

* remove retain tracking check from UpdateCompressor

* use makeRanges() properly in tests

* refactor acceptance tests

GitOrigin-RevId: ab12ec53c5f52c20d44827c6037335e048f2edb0
2024-04-17 08:04:17 +00:00
Domagoj Kriskovic
8bde496da4 Send operations to project-history when resolving/unresolving comments (#17540)
* Send operations to project-history when resolving/unresolving comments

* small fixes

* added doc_id in web unit test

* Revert "added doc_id in web unit test"

This reverts commit f0b8251abfce17965d5e1b0e45d8784fcf1d9eed.

* fix mocked dependency in test

* wip: web unit tests

* document updater, reopen test

* document-updater tests

* format fix

* fix typo

* fix callsArgWith

* fix reopenThread calls in doc updater tests

* fix typos

* log error if chat api resolve failes

* log error when reopening thread

* sendStatus calls done() in tests

* using OError instead of logging

* removed timers

* preserve legacy endpoints

* update after merge

* Remove timer check in HttpControllerTest

* prettier

* added "legacy" in log

* remove metrics.timer

* fix promisify issues

* remove unused cb

GitOrigin-RevId: 849538c86996973a065c727835e93028e5429344
2024-04-10 08:04:08 +00:00
Eric Mc Sween
5d2aed3b1c Merge pull request #17713 from overleaf/em-resync-include-ranges
Add ranges to resync doc update

GitOrigin-RevId: 6998b952028a154c21c8f476e0504496019aea9c
2024-04-09 08:04:44 +00:00
Eric Mc Sween
8e662b33fb Merge pull request #17711 from overleaf/em-resync-tracked-deletes
Include tracked deletes in resync doc update

GitOrigin-RevId: 410f9435f92ff44184f60a2695be27d7b819a1e2
2024-04-09 08:04:40 +00:00
Brian Gough
a9d1ddcb22 Merge pull request #17748 from overleaf/bg-node-security-upgrade-2024-04-05
upgrade node version to 18.20.1

GitOrigin-RevId: 28f8f7c7f63285f7b7355e65713d9ed2dbda8a0e
2024-04-08 08:04:41 +00:00
Eric Mc Sween
9d78fd0945 Merge pull request #17690 from overleaf/em-promisify-project-history-redis-manager
Promisify ProjectHistoryRedisManager

GitOrigin-RevId: 11908c6cc102653b34e2014af230e3912cd8b432
2024-04-04 08:04:40 +00:00
Eric Mc Sween
6632ffc939 Merge pull request #17686 from overleaf/em-promisify-document-manager
Promisify DocumentManager

GitOrigin-RevId: 63a28ef292b5aee708637ebcb53a5cbea7a304e6
2024-04-03 08:04:13 +00:00
Eric Mc Sween
3217870c04 Merge pull request #17678 from overleaf/em-diff-as-share-js-ops
Make diffAsShareJsOps() synchronous

GitOrigin-RevId: 5a3f96cc2f4b81e27a4ee971667f52c48c07e938
2024-04-03 08:04:09 +00:00
Eric Mc Sween
19937e3e7e Merge pull request #17655 from overleaf/em-doc-length-fix
Fix doc length reporting in document-updater

GitOrigin-RevId: 37a9c35e3510f7b5be233fff817c89eeff1cb296
2024-03-28 09:04:25 +00:00
Eric Mc Sween
3ab54b5b14 Merge pull request #17368 from overleaf/em-handle-metadata
Handle tracked changes in project-history

GitOrigin-RevId: 9c790a4dcd874f0d68173fc65cb6823a4da55cc6
2024-03-27 09:04:19 +00:00
Alf Eaton
6cc2db3cdd Merge pull request #17525 from overleaf/ae-upgrade-prettier
Upgrade Prettier to v3

GitOrigin-RevId: 6f1338f196408f3edb4892d5220ad3665ff1a5bc
2024-03-26 09:04:05 +00:00
Domagoj Kriskovic
3e701f1388 Send operations to project-history when deleting comments (#17198) (#17494)
* Make Operation.fromRaw parse AddComment operations

* Send operations to project-history when deleting comments

* remove RedisManager.updateDocument call

* format fix

* Revert "remove RedisManager.updateDocument call"

This reverts commit 5d45f34b5919bf5a85e8475e5a450bcb213d3f82.

* remove versions from deleteComment op

* pass userId to deleteComment operation

* revert userId from chat service

* revert userid from chat tests

* format:fix

* document updater fix tests

* format:fix

* fix web unit test

* deleteThread test fix

* send only if historyRangesSupport is true

* use headers to pass userId

* set historyRangesSupport in tests

* optional headers

* Revert "use headers to pass userId"

This reverts commit e916c469f95b1ac166e30e12e735171eb814af01.

---------

Co-authored-by: Eric Mc Sween <5454374+emcsween@users.noreply.github.com>
GitOrigin-RevId: 14c10669e43d76883dbaaa8ab55e102b5ebadd38
2024-03-12 09:03:55 +00:00
Domagoj Kriskovic
f5b894bc2e Revert "Send operations to project-history when deleting comments (#17198)" (#17476)
This reverts commit 34ab7e4ea6dd1b5fd749f9accaf4a6e16ae1d840.

GitOrigin-RevId: 005de52bcacbacf62d22cd8617d7a765f56bd72e
2024-03-11 09:04:29 +00:00
Domagoj Kriskovic
4a3a0752a4 Send operations to project-history when deleting comments (#17198)
* Make Operation.fromRaw parse AddComment operations

* Send operations to project-history when deleting comments

* remove RedisManager.updateDocument call

* format fix

* Revert "remove RedisManager.updateDocument call"

This reverts commit 5d45f34b5919bf5a85e8475e5a450bcb213d3f82.

* remove versions from deleteComment op

* pass userId to deleteComment operation

* revert userId from chat service

* revert userid from chat tests

* format:fix

* document updater fix tests

* format:fix

* fix web unit test

* deleteThread test fix

* send only if historyRangesSupport is true

* use headers to pass userId

* set historyRangesSupport in tests

* optional headers

* Revert "use headers to pass userId"

This reverts commit e916c469f95b1ac166e30e12e735171eb814af01.

---------

Co-authored-by: Eric Mc Sween <5454374+emcsween@users.noreply.github.com>
GitOrigin-RevId: 34ab7e4ea6dd1b5fd749f9accaf4a6e16ae1d840
2024-03-11 09:04:24 +00:00
Alf Eaton
fe4261e1b7 Merge pull request #17355 from overleaf/ae-redis-cluster
[document-updater] Remove unused `redis_cluster` code

GitOrigin-RevId: 8c586ea31309972d435f7d824d32239d0e47ca91
2024-03-07 16:30:33 +00:00
Mathias Jakobsen
a4b7c5f468 Merge pull request #17135 from overleaf/mj-ranges-tracker-error-logs
[ranges-tracker] Move data out of error message

GitOrigin-RevId: 9745661cf75eaed28a133e1ad5eba0bf77d6ecc3
2024-03-05 09:03:28 +00:00
David
b59326e96f Merge pull request #16873 from overleaf/dp-mongo-command-monitoring
Use mongo command monitoring to add timing metrics

GitOrigin-RevId: e7e5dd5cca1ba3802c02198ccf81058d4da3f1e7
2024-02-23 09:02:55 +00:00
Jakob Ackermann
39189978ac Merge pull request #17170 from overleaf/jpa-node-18-19-1
[misc] upgrade Node.js to 18.19.1

GitOrigin-RevId: 1bd2008682b585c3fccc65f74c90ba7fc9edc023
2024-02-20 09:04:55 +00:00
Eric Mc Sween
b55bdf7ced Merge pull request #17128 from overleaf/em-filter-out-comments
Do not send comments to project-history when ranges support is disabled

GitOrigin-RevId: 0c5e5e2c98ea3c2830ba4d5d114bf4730b440440
2024-02-19 09:04:00 +00:00
Eric Mc Sween
702585a897 Merge pull request #17037 from overleaf/em-comment-ids
Add commentIds property to insert updates sent to history

GitOrigin-RevId: 1fdfc21e1cceff2693975ed6d037e60557106e67
2024-02-15 09:03:52 +00:00
Eric Mc Sween
2f513700ec Merge pull request #17010 from overleaf/em-doc-length
Add history_doc_length property to history updates

GitOrigin-RevId: ccad09f23ae9c038480fb7228a987d8fc6fb6274
2024-02-14 09:06:18 +00:00