github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
23884 commits 3 branches 7 tags 206 MiB
Commit graph

175 commits

Author SHA1 Message Date
Jakob Ackermann
3f94dd4dbf [DocArchiveManager] destroy in small batches 
Also add settings options for PARALLEL_JOBS and DESTROY_BATCH_SIZE.
 2021-04-16 09:51:41 +01:00
Jakob Ackermann
5bcd142b6f [MongoManager] drop name filter from getProjectsDeletedDocs 
All the doc names we had in the projects collection are back-filled now.
A few docs will get dummy values back-filled.
 2021-04-12 17:31:26 +01:00
Jakob Ackermann
e80d013268 [misc] drop the deleteDoc endpoint -- use patchDoc instead 2021-04-12 17:31:18 +01:00
Eric Mc Sween
5efae3bee6 Add a global test setup file 
Configure chai and SandboxedModule globally instead of in every test
file. Also add globals that are required for SandboxedModule to work in
Node 12.
 2021-03-16 08:41:46 -04:00
Jakob Ackermann
8915e1d026 [misc] add a new endpoint for getting deleted docs 2021-02-18 10:10:14 +00:00
Jakob Ackermann
0c169cb242 [HttpController] double down on request validation 2021-02-17 11:31:27 +00:00
Jakob Ackermann
26bc074098 [DocstoreManager] patchDoc: allow updates after deletion 2021-02-17 11:31:01 +00:00
Jakob Ackermann
dd4f4057f4 [misc] add a new endpoint for changing a docs meta data -- incl. deleted 
- Validate the request payload with joi -- includes acceptance tests.
- Reject updates to docs that have been deleted.
 2021-02-15 13:13:48 +00:00
Jakob Ackermann
12bef53645 [DocArchiveManager] optionally do not un-archive soft deleted docs 2021-02-09 11:20:26 +00:00
Jakob Ackermann
708bdfd197 [DocManager] optionally flush docs out of mongo when deleting them 2021-02-09 11:17:46 +00:00
Jakob Ackermann
2cf9ff7b41 Merge pull request #89 from overleaf/jpa-is-doc-deleted 
[misc] add a new endpoint for querying the deleted status of a doc
 2021-02-09 11:14:28 +00:00
Jakob Ackermann
28b1ad3243 [misc] add a new endpoint for querying the deleted status of a doc 
`/project/:project_id/doc/:doc_id/deleted` responds with:

- 404: the doc does not exist
- 200 and body `{"deleted":true}`: doc exists and is deleted
- 200 and body `{"deleted":false}`: doc exists and is not deleted
 2021-01-15 14:49:27 +00:00
Jakob Ackermann
9298f486ad [app] add a new route for archiving a single doc 
There is a minor opportunity for optimizing the mongo call(s) for
 archiving: we should include `{ inS3: false }` in the queries instead
 of filtering locally. It's been like that for years now, so it can
 stay a little longer.
 2021-01-04 13:47:55 +00:00
Simon Detheridge
31c757cce2 Don't throw 404 errors when unarchiving, if the doc is already unarchived 2020-09-25 09:17:01 +01:00
Jakob Ackermann
b37e27f023 [misc] run the codemod for moving mongo projections into options 2020-09-17 15:40:20 +01:00
Jakob Ackermann
8a8a830ad3 [misc] simplify mongodb collection access using a shared db construct 
Resolve the getCollection Promises once and store the result in a shared
 `db` object which can get imported by all the call-sites.

The http server is starting only after a Promise of `waitForDb()`
 resolves. This covers the app code and the acceptance tests.
 2020-09-17 15:39:44 +01:00
Simon Detheridge
ad4531ff01 Upgrade metrics to @overleaf/metrics' 2020-09-10 18:07:14 +01:00
Jakob Ackermann
2fa09928ee [misc] migrate acceptance tests to the native mongo driver, drop mongojs 2020-08-25 09:41:23 +01:00
Jakob Ackermann
df0747ec48 [misc] migrate the app to the native mongo driver 
acceptance tests to follow in a separate commit
 2020-08-25 09:40:59 +01:00
Simon Detheridge
a18146612a Remove unhandled-promise-rejection handler 2020-07-31 16:25:33 +01:00
Simon Detheridge
9724407e31 Export .promises from MongoManager 2020-07-31 16:20:07 +01:00
Simon Detheridge
04ae868fe2 Update unit tests for object-persistor 2020-07-23 19:43:51 +01:00
Simon Detheridge
e302c1d844 Update acceptance tests for object-persistor and gcs 2020-07-23 19:43:33 +01:00
Simon Detheridge
287d705671 Update buildscripts to use fake-gcs instead of S3 credentials 2020-07-23 19:30:13 +01:00
Ersun Warncke
06ec5556b0 [misc] fix express deprecations 
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
 2020-07-02 14:48:11 +01:00
Tim Alby
c854f1f570 prettier auto-format 2020-05-28 15:20:54 +02:00
Jakob Ackermann
80095785b8 [misc] test/unit: fix process usage for node v12 
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2020-04-20 16:50:51 +02:00
Simon Detheridge
785a6cb544 prettier: convert test/acceptance decaffeinated files to Prettier format 2020-02-17 10:49:20 +00:00
Simon Detheridge
7e131e4c71 decaffeinate: rename test/acceptance/coffee to test/acceptance/js 2020-02-17 10:49:18 +00:00
decaffeinate
1391ae96e7 decaffeinate: Run post-processing cleanups on ArchiveDocsTests.coffee and 6 other files 2020-02-17 10:49:16 +00:00
decaffeinate
73b0bc023f decaffeinate: Convert ArchiveDocsTests.coffee and 6 other files to JS 2020-02-17 10:49:14 +00:00
decaffeinate
93c718dae9 decaffeinate: Rename ArchiveDocsTests.coffee and 6 other files from .coffee to .js 2020-02-17 10:49:13 +00:00
Simon Detheridge
088fd29391 prettier: convert test/unit decaffeinated files to Prettier format 2020-02-17 10:49:11 +00:00
Simon Detheridge
75f1245610 decaffeinate: rename test/unit/coffee to test/unit/js 2020-02-17 10:49:08 +00:00
decaffeinate
6e498152fd decaffeinate: Run post-processing cleanups on DocArchiveManagerTests.coffee and 4 other files 2020-02-17 10:49:06 +00:00
decaffeinate
06bdba5c14 decaffeinate: Convert DocArchiveManagerTests.coffee and 4 other files to JS 2020-02-17 10:49:04 +00:00
decaffeinate
5098fa6d40 decaffeinate: Rename DocArchiveManagerTests.coffee and 4 other files from .coffee to .js 2020-02-17 10:49:02 +00:00
Simon Detheridge
03b8a1901d Account for ranges in large json payloads and add line-count limit 2019-08-06 09:50:04 +01:00
Simon Detheridge
5261e1b0be Add endpoint to completely purge a document 2019-07-02 15:02:37 +01:00
Henry Oswald
c1947c5cbd dockerise app, 1.1.3 build scripts 2018-05-23 11:27:31 +01:00
Shane Kilkelly
65697cb2f6 Upgrade sinonjs, and fix unit tests. 
- use `sinon.match.has` for error matching
- restore previous use of `calledWith`
 2017-08-22 15:20:43 +01:00
Shane Kilkelly
1f3b0a3713 Upgrade mongojs dependency to version 2.4.0 
- Upgrade in package.json
- Use new connection syntax
- Fix unit tests, issues with object equality
 2017-08-22 08:26:30 +01:00
Brian Gough
abe90cad10 Revert "Merge pull request #23 from sharelatex/bg-optimise-get-all-docs" 
This reverts commit c1337cf23f37bd9ba48419accf1f54c00390e0ea, reversing
changes made to fbb2fa15bf4239db006db742f906554d61a0eac4.
 2017-06-30 14:54:11 +01:00
Brian Gough
b9b0596d83 optimistically load all docs 2017-06-27 16:58:20 +01:00
James Allen
e10d09d040 Merge pull request #22 from sharelatex/ja-archive-ranges 
Archive ranges as well as doc lines
 2017-05-17 12:02:42 +01:00
Shane Kilkelly
6a6fdc13a5 Merge branch 'sk-upgrade-metrics' 2017-04-04 09:34:45 +01:00
James Allen
7a64f040f1 Test and fix _mongoDocToS3Doc 2017-04-03 16:04:54 +01:00
James Allen
bf187f30ec Add tests for _s3DocToMongo 2017-04-03 15:50:33 +01:00
Shane Kilkelly
b14039715f Stub out logger in tests 2017-03-31 11:25:23 +01:00
James Allen
f94e33a3a9 Merge branch 'master' into ja-archive-ranges 
Conflicts:
	app/coffee/DocArchiveManager.coffee
	test/acceptance/coffee/ArchiveDocsTests.coffee
 2017-03-30 17:19:42 +01:00
James Allen
a9ab7b739d Archive ranges as well as doc lines 2017-03-30 17:13:43 +01:00
Shane Kilkelly
e879f9c742 Upgrade metrics 2017-03-16 16:46:58 +00:00
Henry Oswald
c0e23df82b break getDoc into multiple helpers, stop passing filters though from other calls 2017-03-15 22:09:56 +00:00
Henry Oswald
c2b96c7e1e Merge pull request #17 from sharelatex/ho-always-check-s3 
Always get inS3 when getting doc
 2017-03-10 10:36:59 +00:00
Henry Oswald
f067c039fe always search for inS3 when getting a doc 2017-03-09 18:51:54 +00:00
Henry Oswald
d6146b2970 got failing acceptence test 2017-03-09 18:18:50 +00:00
Henry Oswald
dbfe5ac712 error if doc lines are not array when unarchiving 2017-03-09 17:34:33 +00:00
Shane Kilkelly
d09a4cb275 Lengthen sleep for acceptance tests 2017-03-06 15:11:04 +00:00
Shane Kilkelly
929ffbd83a Don't npm rebuild inside container 2017-02-20 10:07:30 +00:00
James Allen
fb39bb0442 Validate that ranges are present in a set document request 2017-01-19 12:15:14 +01:00
James Allen
5e730b33b2 Support new comment op format 2016-12-16 16:44:08 +00:00
James Allen
0174495f75 Add an end point to get all the ranges in a project 2016-12-09 14:37:24 +00:00
James Allen
6fa8b89154 Be explicit about the data we return from mongo 2016-12-05 17:27:31 +00:00
James Allen
a6ec672d84 Merge branch 'ja-delete-refactor' into ja-track-changes 
Conflicts:
	test/acceptance/coffee/ArchiveDocsTests.coffee
 2016-12-05 16:58:29 +00:00
James Allen
3d195de3e9 Improve testing of archiving and be explicit about include_deleted parameter 2016-12-05 16:48:01 +00:00
James Allen
7717625534 Include deleted docs when archiving 2016-12-05 16:31:51 +00:00
James Allen
2985cb587a Add tests for RangeManager 2016-12-05 15:38:38 +00:00
James Allen
b8a97731e4 Merge branch 'master' into ja-track-changes 
Conflicts:
	test/acceptance/coffee/GettingAllDocsTests.coffee
 2016-12-05 14:58:57 +00:00
James Allen
bd5086c054 Don't return deleted docs with all project docs 2016-12-05 14:37:44 +00:00
James Allen
0a65a5eabf Set and get ranges along with lines 2016-12-05 14:21:49 +00:00
James Allen
ca96d168dd Make attributes that can be set in mongo flexible 2016-12-02 15:22:08 +00:00
James Allen
144bae3a55 Add an error guard around the version being present 2016-12-02 15:17:38 +00:00
James Allen
ab527c3c5d remove code left over from migration 2016-12-02 15:02:54 +00:00
James Allen
8b95a04a36 Don't look up version in mongo if not needed 2016-11-30 15:08:56 +00:00
James Allen
7ede096903 Merge branch 'master' into ja-set-version-in-docstore 2016-11-29 17:24:53 +00:00
James Allen
50f2983912 Log out version when updating doc 2016-11-29 16:49:35 +00:00
James Allen
10008fa9c4 Don't add colours in automated test script 2016-11-29 14:41:12 +00:00
James Allen
7204c08fb2 Rebuild npm modules as part of acceptance test docker run 2016-11-29 14:29:51 +00:00
James Allen
f7ba554658 set up acceptance tests to run via docker 2016-11-28 16:51:43 +00:00
James Allen
eb9a3b314a Set/get version in docOps via docstore HTTP API 2016-11-28 14:55:16 +00:00
Brian Gough
7a28fb58e7 clean up mongo query 2016-09-05 11:14:41 +01:00
Henry Oswald
9cfa4b3f84 Don't put docs which are already in s3 through archive job 
This fixes the bug we saw with 'RangeError: Maximum call stack size exceeded'
if lots of docs are already in s3 cb() gets called synchronously multiple times quickly
which can cause the execption. I am not sure where the recursion is, maybe inside async.

doing setImmediate(cb) also fixes the issue as I beilve it gives the
process chance to clear the stack. Similar to process.nextTick
 2015-09-07 14:11:02 +01:00
Henry Oswald
ae75b855b4 added failing test for trying to archive load of docs which are already 
in s3
 2015-09-07 14:06:20 +01:00
Henry Oswald
aaa2a099f1 change acceptance test to use big naught list of strings 2015-09-01 16:51:10 +01:00
Henry Oswald
192b0af15a added utf-8 char which fails the archive option 2015-09-01 15:47:16 +01:00
Henry Oswald
5dd33ea313 changed back to standard /archive /unarchive 2015-08-18 14:10:53 +01:00
Henry Oswald
5765b59e0c added /deactivate and /reactivate endpoints as web will call those 
also fixed acceptence test
 2015-08-18 13:31:26 +01:00
Henry Oswald
cf07e2fec5 improve archve doc acceptence tests 2015-08-13 22:48:17 +01:00
Henry Oswald
ef4e437c3f improve doc manager so it calls get doc recursivly 2015-08-13 22:47:33 +01:00
Henry Oswald
ddf4692f0d added acceptance test for large doc ~1mb 2015-08-13 13:59:40 +01:00
Henry Oswald
859e8a0e57 - deal with calling archive twice on same project 
- clean up unit tests
- change s3 keys
- use err not error
 2015-08-13 13:00:09 +01:00
Henrique Dias
42f2afc4dd remove unarchived docs from s3 2015-06-03 20:05:15 -03:00
Henrique Dias
9a68bfb393 fix unit tests 2015-06-03 12:00:08 -03:00
Henrique Dias
5ba14522da handling error 2015-06-02 20:08:50 -03:00
Henrique Dias
a26320013d more acceptance tests 2015-06-02 19:24:45 -03:00
Henrique Dias
0cd2120430 more unit tests for archive feature 2015-06-02 18:13:16 -03:00
Henrique Dias
6add9a0da7 init unit tests 2015-06-02 17:12:11 -03:00
Henrique Dias
5c31e80062 more refactoring 2015-06-02 16:29:32 -03:00
Henrique Dias
7c7ff64904 retrieving all docs from s3 2015-06-01 19:36:26 -03:00
Henrique Dias
7f5204f0f7 initial version for archive docs in S3 2015-06-01 18:24:40 -03:00