github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
567 commits 3 branches 7 tags 206 MiB
Commit graph

506 commits

Author SHA1 Message Date
Simon Detheridge
a6d9d34fe3 Configure PersistorManager to use the new S3 manager for 'aws-sdk' 2020-01-07 09:30:30 +00:00
Simon Detheridge
2ca74fdf15 Cleanup and refactor S3PersistorManager to use aws-sdk only 2020-01-07 09:30:30 +00:00
Simon Detheridge
473aea4e60 Remove AWSSDKPersistorManager 2020-01-07 09:30:30 +00:00
Simon Detheridge
e27cf4db7b Pass pipeline errors onto 'next' error handler 2020-01-07 09:30:30 +00:00
Simon Detheridge
87b51d0c35 Remove old exception-handling and shutdown-related mechanisms 2020-01-07 09:30:28 +00:00
Simon Detheridge
74b480fc55 Post-decaf cleanup of app.js 2020-01-07 09:29:33 +00:00
Simon Detheridge
85d3c0a852 Remove old exception-handling and shutdown-related mechanisms 2020-01-07 09:27:02 +00:00
Simon Detheridge
6a679023d3 Fix order of .status().send() 2020-01-07 09:27:02 +00:00
Simon Detheridge
fce275e1d4 Post-decaf cleanup of app.js 2020-01-07 09:27:02 +00:00
Simon Detheridge
0329c759dc Remove wrapper function in PersistorManager 2020-01-07 09:26:43 +00:00
Simon Detheridge
dbfacce988 Post-decaf cleanup of PersistorManager 2020-01-07 09:26:43 +00:00
Simon Detheridge
10f7087912 Post-decaf cleanup of KeyBuilderTests 2020-01-07 09:26:43 +00:00
Simon Detheridge
ec80052d9a Decaf cleanup of FileController 
Remove deprecated res.send
 2020-01-07 09:26:23 +00:00
Simon Detheridge
37d44d5d86 Return full path in stubbed 'glob' method 2020-01-07 09:26:01 +00:00
Simon Detheridge
fccb288985 Add comment regarding deleteDirectory being internal-only 2020-01-07 09:26:01 +00:00
Simon Detheridge
f1b6b35c69 Throw errors more consistently in FSPersistorManager 2020-01-07 09:26:00 +00:00
Simon Detheridge
25f1c2bfc4 Delete temporary file when error in writing to stream 2020-01-07 09:26:00 +00:00
Simon Detheridge
039bec02f7 Use directory names in acceptance tests, and add test for getting size 2020-01-07 09:26:00 +00:00
Simon Detheridge
21ba083877 Use 'glob' to determine 'directory' size. 
Check for filtered directory paths in all tests.
 2020-01-07 09:26:00 +00:00
Simon Detheridge
ce90292394 Decaf cleanup for FSPersistorManager 2020-01-07 09:26:00 +00:00
Simon Detheridge
4315824d3c Minor cleanup of FilestoreApp acceptance-test helper 2020-01-07 09:25:04 +00:00
Simon Detheridge
a8158d6c8c Cleanup and promisify acceptance tests. Run tests for all backends. 2020-01-07 09:25:04 +00:00
Simon Detheridge
006f84abeb Decaf and promisify ImageOptimiser 2020-01-07 09:24:29 +00:00
Simon Detheridge
42adc59d01 Clean up and promisify health-check controller and KeyBuilder 2020-01-07 09:23:41 +00:00
Simon Detheridge
ac2d05ecb3 Remove unnecessary logging from FileConverter 2020-01-07 09:22:53 +00:00
Simon Detheridge
b6a1ea6a30 Cleanup and promisify FileConverter 2020-01-07 09:22:53 +00:00
Simon Detheridge
a216be5fd3 Improve error handling in FileHandler 2020-01-07 09:18:59 +00:00
Simon Detheridge
27aaff7843 Decaf cleanup for FileHandler and LocalFileWriter 
Simplified code and tests where possible
 2020-01-07 09:18:56 +00:00
Simon Detheridge
eacad77112 Cleanup SafeExec 2019-12-19 15:42:46 +00:00
Simon Detheridge
f19d82369d Bump eslintrc up again to last-known-compatible version with eslit-prettier 2019-12-18 09:52:18 +00:00
Simon Detheridge
b94df9952b Add OError 2019-12-18 09:34:25 +00:00
Simon Detheridge
42f6d10031 Remove mongo and redis from dependencies 2019-12-17 15:19:19 +00:00
Simon Detheridge
0b3b71b857 Fall back to old AWS env vars if present 2019-12-17 09:57:51 +00:00
Simon Detheridge
b7e098208f Update build files from build_scripts 2019-12-16 17:17:25 +00:00
Simon Detheridge
29f333d60c Placate eslint 2019-12-16 15:38:00 +00:00
Simon Detheridge
7870c8185e Add linting to makefile 2019-12-16 11:58:22 +00:00
Simon Detheridge
230baef8fa Add linting tasks to package.json 2019-12-16 11:32:58 +00:00
Simon Detheridge
35d050a49c Prettier fixes 2019-12-16 11:32:46 +00:00
Simon Detheridge
6974fd640d Remove unusued cluster.coffee 2019-12-16 11:29:25 +00:00
Simon Detheridge
495424a94b Fix metrics includes in SandboxedModule 2019-12-16 11:28:24 +00:00
Simon Detheridge
eae7d28495 Prettier: convert test/unit decaffeinated files to Prettier format 2019-12-16 11:20:29 +00:00
Simon Detheridge
ec60f778e6 Rename test/unit/coffee to test/unit/js 2019-12-16 11:20:27 +00:00
decaffeinate
ff04085bdd decaffeinate: Run post-processing cleanups on AWSSDKPersistorManagerTests.coffee and 12 other files 2019-12-16 11:20:25 +00:00
decaffeinate
5cf7138ef1 decaffeinate: Convert AWSSDKPersistorManagerTests.coffee and 12 other files to JS 2019-12-16 11:20:22 +00:00
decaffeinate
72c83bdaff decaffeinate: Rename AWSSDKPersistorManagerTests.coffee and 12 other files from .coffee to .js 2019-12-16 11:20:20 +00:00
Simon Detheridge
e06c03e536 Fix decaf error in FSPersistorManager 2019-12-16 11:19:56 +00:00
Simon Detheridge
0b6e725137 Remove coffeescript compiles from package.json 2019-12-16 11:19:56 +00:00
Simon Detheridge
9943e2cc8f Prettier: convert individual decaffeinated files to Prettier format 2019-12-16 11:19:56 +00:00
Simon Detheridge
1d295ef7e4 Decaffeinate: convert individual files to js 2019-12-16 11:19:55 +00:00
Simon Detheridge
d70a33c4f5 Rename individual coffee files to js files 2019-12-16 11:19:43 +00:00
Simon Detheridge
e4b1106761 Prettier: convert test/acceptance decaffeinated files to Prettier format 2019-12-16 11:19:43 +00:00
Simon Detheridge
b8e7abd25e Rename test/acceptance/coffee to test/acceptance/js 2019-12-16 11:19:43 +00:00
decaffeinate
738e1d821e decaffeinate: Run post-processing cleanups on FilestoreApp.coffee and 1 other file 2019-12-16 11:19:43 +00:00
decaffeinate
e6e75d4e19 decaffeinate: Convert FilestoreApp.coffee and 1 other file to JS 2019-12-16 11:19:43 +00:00
decaffeinate
1c7d1af4ca decaffeinate: Rename FilestoreApp.coffee and 1 other file from .coffee to .js 2019-12-16 11:19:43 +00:00
Simon Detheridge
6225f2f236 Prettier: convert app/js decaffeinated files to Prettier format 2019-12-16 11:19:36 +00:00
Simon Detheridge
601861ee58 Remove js from dockerignore 2019-12-16 10:54:12 +00:00
Simon Detheridge
3718c90c7a Rename app/coffee dir to app/js 2019-12-16 10:54:08 +00:00
decaffeinate
fcbb13b5bf decaffeinate: Run post-processing cleanups on AWSSDKPersistorManager.coffee and 13 other files 2019-12-16 10:54:06 +00:00
decaffeinate
6bd8452f19 decaffeinate: Convert AWSSDKPersistorManager.coffee and 13 other files to JS 2019-12-16 10:54:03 +00:00
decaffeinate
8d2c87420e decaffeinate: Rename AWSSDKPersistorManager.coffee and 13 other files from .coffee to .js 2019-12-16 10:53:59 +00:00
Simon Detheridge
ae3d25c969 Decaffeinate: add eslint and prettier rc files 2019-12-16 10:53:56 +00:00
Simon Detheridge
1798efd4be Install eslint and prettier 2019-12-16 10:53:52 +00:00
Simon Detheridge
c9106eff95 Upgrade Docker image to node 10 2019-12-16 10:53:42 +00:00
Jakob Ackermann
ed97fcfcd0 [logging] do not overwrite the logger name 
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2019-12-16 10:53:34 +00:00
Jakob Ackermann
c5e1584fcc [FSPersistorManager] fix the stream opening for node10+ 
Attaching a `readable` listener causes the stream to hang otherwise.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2019-12-16 10:53:23 +00:00
Simon Detheridge
dab44f6fec Merge branch 'master' into logging/keep-logger-name 2019-12-16 09:42:18 +00:00
Simon Detheridge
32b7f25404 Merge pull request #49 from das7pad/misc/node10plus-support 
[misc] add support for node 10+
 2019-12-16 09:39:00 +00:00
Simon Detheridge
56b38af678 Explicitly wait for S3 in acceptance tests 2019-12-10 17:43:34 +00:00
Simon Detheridge
237c4113cd Ensure fakes3 is healthy before running tests 2019-12-10 16:36:33 +00:00
Simon Detheridge
cf684dcd98 Fix fakes3 configuration in ci yml 2019-12-10 11:38:19 +00:00
Simon Detheridge
6f326d5650 Use SSL setting based on url protocol 2019-12-09 17:41:20 +00:00
Simon Detheridge
48aa141591 Add metric for s3 ingress 2019-12-09 17:06:00 +00:00
Simon Detheridge
1d1106bc67 Add metric for s3 egress 2019-12-09 17:06:00 +00:00
Simon Detheridge
c01603b1e7 Support custom S3 endpoints 2019-12-09 17:06:00 +00:00
Simon Detheridge
96457597ac Add fake s3 server and initial config 2019-12-09 17:06:00 +00:00
Simon Detheridge
86b9e4b53a Rename request -> s3Request to prevent overwriting main import 2019-12-09 17:05:58 +00:00
Brian Gough
2ec38068aa add comments about aws-sdk and s3 backends 2019-12-06 14:35:13 +00:00
Nate Stemen
aba0d14edd bump build script to 1.1.24 2019-10-25 11:52:54 -04:00
Nate Stemen
4798589213 replace private link with public one 2019-10-25 11:51:58 -04:00
Shane Kilkelly
dbede8509c Merge pull request #54 from overleaf/sk-dep-upgrades-2019-06 
update logger and metrics
 2019-07-09 10:18:16 +01:00
Eric Mc Sween
7d900b57bf Fix createUnbufferedStream() function call 
In 49a21155f642670dfea264ac73fb60241f37cb87, I managed to incorrectly
write the `createUnbufferedStream()` function from the AWS SDK as
`getUnbufferedStream()` and to consistently use that naming in the unit
tests.

This commit fixes that. I have tested again on S3.
 2019-06-19 12:58:17 -04:00
Shane Kilkelly
013400d7a4 Re-add environment variable for conversions 2019-06-19 14:04:57 +01:00
Shane Kilkelly
f865762c29 update logger and metrics 2019-06-19 13:50:58 +01:00
Eric Mc Sween
f2521a29b9 Use AWS SDK for getFileStream() 
The AWS SDK has a retry strategy to deal with rate limiting or transient
unavailability of S3. We hope it will reduce our error rates.
 2019-06-19 08:18:11 -04:00
Eric Mc Sween
3575c89d03 Return file size on HEAD request 
This will be used by the file preview feature when it gets partial
content.
 2019-06-17 08:23:59 -04:00
Timothée Alby
f081546ec0 update Git URL in Jenkinsfile 2019-05-07 17:37:07 +01:00
Timothée Alby
32a54a7e37 Update README.md 2019-05-07 17:22:35 +02:00
Jakob Ackermann
a79adcd325 [logging] do not overwrite the logger name 
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2019-05-02 12:48:43 +02:00
Jakob Ackermann
c59a3db4e8 [FSPersistorManager] fix the stream opening for node10+ 
Attaching a `readable` listener causes the stream to hang otherwise.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2019-05-02 02:47:53 +02:00
Brian Gough
b9d3b1a842 fix beginShutdown to use server.close() 2019-03-15 15:35:28 +00:00
Henry Oswald
37b039abf0 add sentry to settings.defaults.coffee 2019-03-05 17:20:16 +00:00
Henry Oswald
2adae20368 remove a console.log 2019-02-06 11:43:11 +00:00
Henry Oswald
0f48c77e7e enable conversions for local docker compose 2019-02-05 17:19:33 +00:00
Henry Oswald
5eec00db51 don't run conversions test when disabled 2019-02-05 14:06:51 +00:00
Henry Oswald
3fccf79ca8 cleanup 2019-02-05 12:50:08 +00:00
Henry Oswald
ca9231b04e set enable conversions to false by default but enable in compse 2019-02-05 11:19:18 +00:00
Henry Oswald
19007befb2 Merge branch 'master' into ho-docker 2019-02-01 16:26:45 +00:00
Henry Oswald
ea845f4935 bulk upgrade to 1.1.12, moved to npm rather than git 2019-01-31 16:01:20 +00:00
Brian Gough
9d93eee3e8 return a 404 error (instead of a 500) when copying a missing file 2019-01-09 10:31:59 +00:00
Henry Oswald
3614f217e6 add enableConversions flag to disable conversions which we can't do on 
k8
 2019-01-07 15:54:24 +00:00
Christopher Hoskin
0421192449 Ignore any file ending in .map, wherever it is 2019-01-04 16:22:13 +00:00
Christopher Hoskin
c8d146b571 Bump settings to v1.1.0 2019-01-04 16:03:45 +00:00
Christopher Hoskin
b522fe1531 Remove grunt 2019-01-04 16:03:22 +00:00
Christopher Hoskin
6763809748 Bump logger to v1.5.9 2019-01-03 16:27:45 +00:00
Christopher Hoskin
594b403bd3 Move metrics init to start of app.coffee 2019-01-02 09:55:31 +00:00
Christopher Hoskin
3b6eba33ba Remove Metrics.inc "startup" from app.coffee 2019-01-02 09:53:53 +00:00
Christopher Hoskin
aab5a1af6a Merge branch 'master' into csh-ho-docker 2019-01-02 09:51:35 +00:00
Christopher Hoskin
6214256568 Bump logger to v1.5.8 2019-01-02 09:47:38 +00:00
Brian Gough
cf12ec1154 use the aws sdk to copy files in S3PersistorManager 
to work around problems with knox
 2018-12-20 16:11:18 +00:00
Brian Gough
28271655e1 avoid problems with caching of apt-get operations 2018-12-06 08:19:39 +00:00
Henry Oswald
25275212af bump metrics to 2.0.7 2018-12-05 12:36:32 +00:00
Henry Oswald
5d927d30b6 update build scripts to 1.1.10 2018-12-05 10:53:11 +00:00
Henry Oswald
8743527452 bump metrics to 2.0.4 2018-11-29 16:42:26 +00:00
Henry Oswald
e64156cd27 clean npm-shrinkwrap and package.json 2018-11-29 16:41:30 +00:00
Henry Oswald
95f0a02dee stub out metrics in tests & inject metrics endpoint 2018-11-29 13:53:43 +00:00
Henry Oswald
ee1c14a410 bump metrics 2018-11-29 12:46:14 +00:00
Brian Gough
fdc5370f94 Merge pull request #36 from sharelatex/bg-upgrade-logger 
upgrade logger-sharelatex to v1.5.8
 2018-11-13 14:06:44 +00:00
Brian Gough
8951f297aa Merge pull request #37 from sharelatex/bg-reduce-logging-of-not-found-errors 
reduce logging of not found errors
 2018-11-13 14:06:19 +00:00
Brian Gough
0d6de10b5c Merge pull request #34 from sharelatex/bg-make-backends-consistent 
make filestore backends consistent
 2018-11-13 10:12:18 +00:00
Brian Gough
e930c7106b Merge pull request #35 from sharelatex/bg-serve-converted-file-from-disk 
serve file from disk to avoid read-after-write inconsistency
 2018-11-13 10:12:00 +00:00
Brian Gough
23a6d6e81d consider 403 and 404 as NotFound errors 2018-11-09 14:05:38 +00:00
Brian Gough
69b164092c suppress unnecessary error logging for NotFound 2018-11-09 14:05:38 +00:00
Brian Gough
c2bfdd095c upgrade logger-sharelatex to v1.5.8 2018-11-08 16:29:23 +00:00
Brian Gough
8744b3aa4e serve file from disk to avoid read-after-write inconsistency 2018-11-08 15:09:34 +00:00
Brian Gough
78ccf453db update unit tests 2018-11-07 16:51:06 +00:00
Brian Gough
40ac59090b make backends consistent by deleting temporary files 2018-11-07 16:22:09 +00:00
Brian Gough
06bfd4c270 add dockerfile to install packages needed by filestore 2018-11-07 16:12:49 +00:00
Christopher Hoskin
038b3476fe Add some details to the README 2018-10-22 14:09:56 +01:00
Henry Oswald
38f565d35f logout settings 2018-09-28 15:53:57 +01:00
Henry Oswald
aacb54d54c Merge branch 'master' into ho-docker 2018-09-14 17:16:18 +01:00
Brian Gough
96e9d8b0ec fix typo 2018-09-11 10:51:10 +01:00
Brian Gough
f3adf5a653 fix shrinkwrap for upgraded logger-sharelatex module 2018-09-06 17:01:40 +01:00
Brian Gough
f4ec677296 add sentry to logging 2018-09-06 17:01:18 +01:00
Christopher Hoskin
53dacc04ef Bump metrics from 1.3.0 to 1.8.1 2018-07-26 16:06:30 +01:00
Christopher Hoskin
0e9a535e1c Bump build script to 1.1.9 2018-07-26 16:05:11 +01:00
Henry Oswald
cba013cd71 change ordering of docker rmi 2018-07-23 17:06:31 +01:00
Christopher Hoskin
f9f599e3e7 Push container images to overleaf-ops project 2018-07-18 11:30:01 +01:00
Christopher Hoskin
1044069535 Remove csh-gcdm-test and csh-staging repos 2018-07-17 12:03:14 +01:00
Christopher Hoskin
c7733d05cc Remove trailing / from project name 2018-07-17 11:06:33 +01:00
Christopher Hoskin
d182e5329c Add cr-test2 GCR project 2018-07-17 10:52:20 +01:00
Christopher Hoskin
4784ae33a2 Bump build scripts to 1.1.8 2018-07-17 10:51:15 +01:00
Henry Oswald
1e4e68c64a fix settings test 2018-07-10 14:55:40 +01:00
Henry Oswald
089bf3e084 seperate the standard s3 creds from the bucket s3 creds 2018-07-10 14:17:37 +01:00
Michael Mazour
c4e3f9eb02 Amend: tests for populating S3 settings from environment variable 2018-07-06 11:05:40 +01:00
Michael Mazour
3e1ef3af63 Populate S3 settings from environment variable 2018-07-06 11:05:19 +01:00
Michael Mazour
2da15f2eb3 Amend - improve documentation of settings 2018-07-06 10:28:02 +01:00
Michael Mazour
7feafccf31 Amend: safely navigate to bucket credentials 2018-07-06 09:28:09 +01:00
Michael Mazour
03033409c6 Amend: remove unused params 2018-07-06 09:12:59 +01:00
Michael Mazour
336a38ec1e Amend: scrub secrets from logs 
Calls to `getFile` can now include S3 credentials in `opts`, so sanitize before writing to opts to log.
 2018-07-04 16:41:31 +01:00