github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-07 20:31:06 -05:00
Code Issues Projects Releases Packages Wiki Activity
19830 commits 3 branches 7 tags 201 MiB
Commit graph

169 commits

Author SHA1 Message Date
Jakob Ackermann
2fa09928ee [misc] migrate acceptance tests to the native mongo driver, drop mongojs 2020-08-25 09:41:23 +01:00
Jakob Ackermann
df0747ec48 [misc] migrate the app to the native mongo driver 
acceptance tests to follow in a separate commit
 2020-08-25 09:40:59 +01:00
Simon Detheridge
1575cd41eb Don't pass empty array to asyncPool when archiving 2020-08-10 15:44:41 +01:00
Simon Detheridge
cb8c2bf4b5 AsyncPool -> asyncPool 2020-07-31 16:21:59 +01:00
Simon Detheridge
9724407e31 Export .promises from MongoManager 2020-07-31 16:20:07 +01:00
Simon Detheridge
268f436461 Don't call AsyncPool with an empty array 2020-07-24 11:51:18 +01:00
Simon Detheridge
6e04db552a Implement object-persistor support 2020-07-23 19:42:49 +01:00
Ersun Warncke
06ec5556b0 [misc] fix express deprecations 
Co-Authored-By: Jakob Ackermann <jakob.ackermann@overleaf.com>
 2020-07-02 14:48:11 +01:00
Tim Alby
c854f1f570 prettier auto-format 2020-05-28 15:20:54 +02:00
Simon Detheridge
746791766c Additional prettier fix 2020-02-17 10:49:32 +00:00
Simon Detheridge
fb931e206c prettier: convert app/js decaffeinated files to Prettier format 2020-02-17 10:49:00 +00:00
Simon Detheridge
c1805978c4 decaffeinate: rename app/coffee dir to app/js 2020-02-17 10:48:58 +00:00
decaffeinate
d31a747c72 decaffeinate: Run post-processing cleanups on DocArchiveManager.coffee and 7 other files 2020-02-17 10:48:57 +00:00
decaffeinate
afa2577381 decaffeinate: Convert DocArchiveManager.coffee and 7 other files to JS 2020-02-17 10:48:55 +00:00
decaffeinate
88badb15c4 decaffeinate: Rename DocArchiveManager.coffee and 7 other files from .coffee to .js 2020-02-17 10:48:53 +00:00
Simon Detheridge
03b8a1901d Account for ranges in large json payloads and add line-count limit 2019-08-06 09:50:04 +01:00
Simon Detheridge
5261e1b0be Add endpoint to completely purge a document 2019-07-02 15:02:37 +01:00
Henry Oswald
7e3c6ddb3f also delete doc ops 2018-10-19 12:10:33 +01:00
Henry Oswald
2828df74e5 hard delete a doc on health check 2018-10-18 18:06:05 +01:00
James Allen
a4cc02dbc3 Provide hosts as environment settings and add npm run start script 2017-12-29 08:12:31 +00:00
James Allen
d86a964081 Remove another verbose log object 2017-09-20 09:52:28 +02:00
James Allen
05a1939547 Don't log out doc lines and ranges 2017-09-20 09:42:08 +02:00
Shane Kilkelly
1f3b0a3713 Upgrade mongojs dependency to version 2.4.0 
- Upgrade in package.json
- Use new connection syntax
- Fix unit tests, issues with object equality
 2017-08-22 08:26:30 +01:00
Brian Gough
abe90cad10 Revert "Merge pull request #23 from sharelatex/bg-optimise-get-all-docs" 
This reverts commit c1337cf23f37bd9ba48419accf1f54c00390e0ea, reversing
changes made to fbb2fa15bf4239db006db742f906554d61a0eac4.
 2017-06-30 14:54:11 +01:00
Brian Gough
a2ac37ac99 comments about optimistic loading 2017-06-28 10:06:50 +01:00
Brian Gough
b9b0596d83 optimistically load all docs 2017-06-27 16:58:20 +01:00
James Allen
e10d09d040 Merge pull request #22 from sharelatex/ja-archive-ranges 
Archive ranges as well as doc lines
 2017-05-17 12:02:42 +01:00
Shane Kilkelly
6a6fdc13a5 Merge branch 'sk-upgrade-metrics' 2017-04-04 09:34:45 +01:00
James Allen
7a64f040f1 Test and fix _mongoDocToS3Doc 2017-04-03 16:04:54 +01:00
James Allen
f94e33a3a9 Merge branch 'master' into ja-archive-ranges 
Conflicts:
	app/coffee/DocArchiveManager.coffee
	test/acceptance/coffee/ArchiveDocsTests.coffee
 2017-03-30 17:19:42 +01:00
James Allen
a9ab7b739d Archive ranges as well as doc lines 2017-03-30 17:13:43 +01:00
Shane Kilkelly
bc37c844ea update to new metrics api 2017-03-17 15:27:36 +00:00
Shane Kilkelly
e879f9c742 Upgrade metrics 2017-03-16 16:46:58 +00:00
Henry Oswald
c0e23df82b break getDoc into multiple helpers, stop passing filters though from other calls 2017-03-15 22:09:56 +00:00
Henry Oswald
ffd1aa0ca4 just get doc_id for quick check to see if it exists 2017-03-10 11:45:03 +00:00
Henry Oswald
7adc67d13a add missing error handler 2017-03-10 10:37:11 +00:00
Henry Oswald
c2b96c7e1e Merge pull request #17 from sharelatex/ho-always-check-s3 
Always get inS3 when getting doc
 2017-03-10 10:36:59 +00:00
Henry Oswald
f067c039fe always search for inS3 when getting a doc 2017-03-09 18:51:54 +00:00
Henry Oswald
8bd0df3530 also log out lines in error message 2017-03-09 17:36:28 +00:00
Henry Oswald
dbfe5ac712 error if doc lines are not array when unarchiving 2017-03-09 17:34:33 +00:00
James Allen
315a1d5e80 Add ranges into health check 2017-01-31 10:23:51 +01:00
James Allen
fb39bb0442 Validate that ranges are present in a set document request 2017-01-19 12:15:14 +01:00
James Allen
5e730b33b2 Support new comment op format 2016-12-16 16:44:08 +00:00
James Allen
0174495f75 Add an end point to get all the ranges in a project 2016-12-09 14:37:24 +00:00
James Allen
6fa8b89154 Be explicit about the data we return from mongo 2016-12-05 17:27:31 +00:00
James Allen
a6ec672d84 Merge branch 'ja-delete-refactor' into ja-track-changes 
Conflicts:
	test/acceptance/coffee/ArchiveDocsTests.coffee
 2016-12-05 16:58:29 +00:00
James Allen
3d195de3e9 Improve testing of archiving and be explicit about include_deleted parameter 2016-12-05 16:48:01 +00:00
James Allen
7717625534 Include deleted docs when archiving 2016-12-05 16:31:51 +00:00
James Allen
2985cb587a Add tests for RangeManager 2016-12-05 15:38:38 +00:00
James Allen
b8a97731e4 Merge branch 'master' into ja-track-changes 
Conflicts:
	test/acceptance/coffee/GettingAllDocsTests.coffee
 2016-12-05 14:58:57 +00:00
James Allen
bd5086c054 Don't return deleted docs with all project docs 2016-12-05 14:37:44 +00:00
James Allen
0a65a5eabf Set and get ranges along with lines 2016-12-05 14:21:49 +00:00
James Allen
ca96d168dd Make attributes that can be set in mongo flexible 2016-12-02 15:22:08 +00:00
James Allen
144bae3a55 Add an error guard around the version being present 2016-12-02 15:17:38 +00:00
James Allen
ab527c3c5d remove code left over from migration 2016-12-02 15:02:54 +00:00
James Allen
e156d8017c Be explicit in version filter 2016-11-30 15:29:43 +00:00
James Allen
8b95a04a36 Don't look up version in mongo if not needed 2016-11-30 15:08:56 +00:00
James Allen
50f2983912 Log out version when updating doc 2016-11-29 16:49:35 +00:00
James Allen
eb9a3b314a Set/get version in docOps via docstore HTTP API 2016-11-28 14:55:16 +00:00
Brian Gough
7a28fb58e7 clean up mongo query 2016-09-05 11:14:41 +01:00
Brian Gough
0a0a8e23fe remove unused modules 2016-09-05 11:14:41 +01:00
Henry Oswald
1541eb319c add more null checks to health check 2015-12-02 10:10:52 +00:00
Henry Oswald
a964119b5e check err in health check and null check res 2015-12-01 21:50:18 +00:00
Henry Oswald
29c812cece moved the build opts down 2015-10-16 12:57:05 +01:00
Henry Oswald
4f5669591a add timeout to health check requests and always try the delete 2015-10-16 12:30:53 +01:00
Henry Oswald
de9a24f7bb pulled health check into its own file and use project_id from settings 2015-10-16 10:13:54 +01:00
Henry Oswald
f1bf0c97ec make archive and unarchive parallelLimit to speed it up 2015-09-22 09:41:26 +01:00
Henry Oswald
9cfa4b3f84 Don't put docs which are already in s3 through archive job 
This fixes the bug we saw with 'RangeError: Maximum call stack size exceeded'
if lots of docs are already in s3 cb() gets called synchronously multiple times quickly
which can cause the execption. I am not sure where the recursion is, maybe inside async.

doing setImmediate(cb) also fixes the issue as I beilve it gives the
process chance to clear the stack. Similar to process.nextTick
 2015-09-07 14:11:02 +01:00
Henry Oswald
738e33a052 change crypto to always use utf-8 as it works with aws 2015-09-01 16:52:02 +01:00
Henry Oswald
599b9770f4 add un archive endpoint 2015-08-13 22:48:45 +01:00
Henry Oswald
ef4e437c3f improve doc manager so it calls get doc recursivly 2015-08-13 22:47:33 +01:00
Henry Oswald
b8884352f6 added some better error logging 2015-08-13 13:59:22 +01:00
Henry Oswald
859e8a0e57 - deal with calling archive twice on same project 
- clean up unit tests
- change s3 keys
- use err not error
 2015-08-13 13:00:09 +01:00
Henrique Dias
42f2afc4dd remove unarchived docs from s3 2015-06-03 20:05:15 -03:00
Henrique Dias
aa23c1da56 md5 check for doc lines 2015-06-03 11:45:47 -03:00
Henrique Dias
5ba14522da handling error 2015-06-02 20:08:50 -03:00
Henrique Dias
a26320013d more acceptance tests 2015-06-02 19:24:45 -03:00
Henrique Dias
0cd2120430 more unit tests for archive feature 2015-06-02 18:13:16 -03:00
Henrique Dias
5c31e80062 more refactoring 2015-06-02 16:29:32 -03:00
Henrique Dias
ddd26798f1 refactoring 
s3 filter in router
 2015-06-02 15:55:22 -03:00
Henrique Dias
86f16caeef fix some issues 2015-06-02 15:13:28 -03:00
Henry Oswald
ea92f57acc point where we unset doc lines also search on doc rev to make sure 
the document has not been updated
 2015-06-02 12:39:49 +01:00
Henry Oswald
60bc7bf4e1 pulled logic for arching docs arpart a bit, can be moved into its own file 
hacked out (needs tests) the unarchiving of individual docs & doc updates
 2015-06-02 12:36:38 +01:00
Henry Oswald
19f1a9ad82 changed style of async jobs to map to use underscore which is like we 
use in the rest of the stack. Also removed a few => binds which are
not needed
 2015-06-02 10:59:35 +01:00
Henrique Dias
7c7ff64904 retrieving all docs from s3 2015-06-01 19:36:26 -03:00
Henrique Dias
7f5204f0f7 initial version for archive docs in S3 2015-06-01 18:24:40 -03:00
Henry Oswald
d59fb0d590 Merge branch 'master' into doc-collection-only 
Conflicts:
	app/coffee/DocManager.coffee
 2015-02-27 14:58:38 +00:00
Henry Oswald
53480153e3 if the doc has no lines then it is new and needs to be written 2015-02-27 14:50:08 +00:00
Henry Oswald
32829f0e6e remove old rev, not needed any more 2015-02-27 14:06:06 +00:00
Henry Oswald
a73dc90b00 change mongomangers upsert to use an inc 
this is tested and does work with new documents
 2015-02-26 16:01:10 +00:00
Henry Oswald
7242119532 simplified DocManager.updateDoc so it calls mongomanager.findDoc 
makes the logic for dealing new documents that are not in mongo yet
much simpler
 2015-02-26 16:00:28 +00:00
Henry Oswald
5861231548 docstore now only touches doc collection 
- need to run migration 1 before using this for this to work
- added acceptence test so new documents updated should be upserted, not return 404
 2015-02-26 14:54:08 +00:00
Henry Oswald
3cd3c118d1 don't log out when doc lines are [] as this is a new empty doc 2015-02-23 13:17:55 +00:00
Henry Oswald
daf7c4f68e log out when a doc from the project collection is used 2015-02-22 18:45:21 +00:00
Henry Oswald
a0832c79e6 return 404 if doc is deleted and include_deleted is not set to true 2015-02-03 14:05:08 +00:00
Henry Oswald
7b0b72494f write to doc collection first 2015-01-27 15:05:35 +00:00
Henry Oswald
5d2c09bf75 read docs from doc collection first 
- removes include deleted flag as it is irrelivant now
 2015-01-22 15:05:48 +00:00
Henry Oswald
535d4435bf added missing err check 2015-01-21 12:31:06 +00:00
Henry Oswald
704bf53fe5 Write docs to do collection and project collection 
- docs are written to project collection then doc collection

- deletes used to insert into doc collection, now its the same upsert
with deleted = true

- does not set delted = false for new docs as it is not needed, save space

- changed delete to be seperate update on mongo
 2015-01-21 11:52:39 +00:00
Henry Oswald
28990d5a39 changed logging to easier to see when the http request starts 2015-01-20 13:46:57 +00:00