github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-30 07:45:26 -05:00
Code Issues Projects Releases Packages Wiki Activity
254 commits 3 branches 7 tags 206 MiB
Commit graph

127 commits

Author SHA1 Message Date
Shane Kilkelly
76506ed91e Pass start/end options down to the S3PersistorManager. 
Test that those options cause a 'Range' header to be attached to the
request sent to S3.
 2015-08-28 10:09:41 +01:00
Shane Kilkelly
d88736e3b5 Pass the start and end options down to fs.createReadStream, and test for same. 2015-08-27 16:12:11 +01:00
Shane Kilkelly
57aedefdd3 Begin work towards obeying 'Range' headers, thread the opts down to persistence layer. 2015-08-27 15:21:33 +01:00
Xavier Trochu
f417b9b171 Fix the AWS-SDK Persistor 2015-07-24 15:48:46 +02:00
Xavier Trochu
7ef46a79a0 Fix aws-sdk persistor. 
Also fix some typos.
 2015-07-24 09:06:52 +02:00
Xavier Trochu
41397821b1 Add a manager using the aws-sdk library 
The knox library does not support the AWS Instance Role.
So use the official AWS SDK to connect to S3
 2015-07-22 16:42:45 +02:00
Xavier Trochu
3289451ece Revert "Make S3 Key/Secret pair optional." 
This reverts commit 65122e603099a4f9ef5c5186b7eb8832340a993f.

knox requires the key and secret arguments.
 2015-07-22 16:41:45 +02:00
Xavier Trochu
566e69c6cc Make S3 Key/Secret pair optional. 
When using Instance Role in AWS, the credentials are unneeded. So
make them optional to use.
 2015-07-21 10:00:59 +02:00
Brian Gough
8ebbacee5a move pipe after error handler setup 2015-05-08 15:19:45 +01:00
Brian Gough
7e9f8b885b add support for public files 2015-05-08 15:15:48 +01:00
Henry Oswald
7c2e4b00ed no file subtype can be an issue, added a check on it in the keybuilder 2015-04-13 12:01:49 +01:00
Henry Oswald
158b640b59 Merge branch 'master' of https://github.com/sharelatex/filestore-sharelatex 2015-04-09 17:16:23 +01:00
Henry Oswald
b3432f016c allow proxying of template images though filestore 2015-04-09 17:14:47 +01:00
Brian Gough
07e53bbb2f handle error when listing bucket 2015-03-31 10:35:33 +01:00
Brian Gough
a5e63f7261 Merge pull request #18 from sharelatex/terminate-processes 
Use process group to terminate processes
 2015-03-13 11:32:57 +00:00
Brian Gough
a7b9376919 use SIGTERM instead of SIGKILL to allow process to shut down cleanly 2015-03-13 10:15:37 +00:00
Brian Gough
9aaef729ad cleanup and comments 2015-03-13 10:10:51 +00:00
Brian Gough
63ee4d1e7d use close event instead of exit to capture stdout/stderr correctly 2015-03-13 10:10:21 +00:00
Brian Gough
33d8974d02 kill process group on timeout of convert commands 2015-03-12 17:09:27 +00:00
James Allen
45689fd2b8 Only call getFileStream callback once 2015-02-26 11:32:05 +00:00
Brian Gough
dad209262f execute success callback when filesystem copyfile completed 2014-12-22 11:27:00 +00:00
Brian Gough
72ebaea5e2 log FS delete errors only on failure 2014-12-22 11:11:31 +00:00
Brian Gough
7d96f3aadd use 'response' to send 'NoSuchKey' when local file is not found 2014-12-22 11:11:03 +00:00
Brian Gough
8744cf94a5 delete directory with rimraf 2014-12-19 16:58:31 +00:00
Brian Gough
af4c72150f fix path in directory deletion 2014-12-19 16:57:44 +00:00
Brian Gough
55c4e220fa prevent double callback in getFileStream 2014-12-19 16:28:42 +00:00
Henry Oswald
f83ab25b35 basic acceptence test of sending and getting file back added 2014-06-19 17:07:50 +01:00
c4live
408bf43b98 Update FSPersistorManager.coffee 
Using code 'ENOENT' for error checking instead of errno 34.
 2014-06-05 16:01:38 +02:00
c4live
5bc89c3cbf Update FileHandler.coffee 
Restoring error check.
 2014-06-05 14:10:49 +02:00
c4live
1e42221954 Update FSPersistorManager.coffee 
Do not propagate the error if trying to delete a directory that does not exist.
 2014-06-05 14:10:05 +02:00
c4live
a9be97622b Update FileHandler.coffee: ignore any error that happens while deleting directory during insertFile. 2014-06-05 10:31:05 +02:00
James Allen
df6b5132c2 Tidy up testing/logging 2014-06-02 17:25:40 +01:00
James Allen
e17aaba3ba Catch errors correctly 2014-06-02 17:11:25 +01:00
James Allen
b3f796a093 More logging 2014-06-02 17:02:45 +01:00
James Allen
415d937f84 Add more logging and don't delete temp files 2014-06-02 16:49:49 +01:00
James Allen
e529875245 Revert "Send content-length header when getting file" 
This reverts commit 6f5f7a98fd3ce53091aacc744f43c7704ba73461.
 2014-05-23 13:57:18 +01:00
James Allen
daf4f1ffd4 Send content-length header when getting file 2014-05-23 13:54:20 +01:00
James Allen
de0d79e1bf Use configurable upload directory 2014-05-17 21:01:48 +01:00
Henry Oswald
3bbeff5aff moved all metrics to use the module, deleting old metrics file and added different check in err domain 2014-05-12 15:45:15 +01:00
Henry Oswald
09224007a7 on domain error wait 3 seconds to send 500 to allow for other cleanup. 
Also check inside function if the res has finished. #confused
 2014-05-07 09:06:05 +01:00
Henry Oswald
be8fdcfd6a added null check on send property of res 2014-05-07 08:23:44 +01:00
James Allen
5e5f9cf4e5 Replace all instances of / in file paths in FSPersistorManager 2014-03-31 15:17:17 +01:00
Henry Oswald
191170509f bumped convert timeout to 40 seconds 2014-03-14 08:55:28 +00:00
Henry Oswald
41b32deac3 changed conversions to work with new image magick feature pdf:fit-page= 2014-03-13 14:04:46 +00:00
Henry Oswald
2f22563d59 changed key for templates to put /v/ in for version 2014-03-05 17:39:27 +00:00
Henry Oswald
1e1c14e5fe Merge pull request #11 from cwoac/FSPersistorManager 
Fs persistor manager - merge from cwoac
 2014-03-05 10:39:26 +00:00
Oliver Matthews
f920fd0b16 match refactor_config on cwoac/sharelatex 2014-03-04 15:24:50 +00:00
Oliver Matthews
957df0eb04 Don't dump streams to log files. 2014-03-04 14:45:32 +00:00
Henry Oswald
711e95a82a delete converted file when finished to clean up 2014-03-04 14:30:52 +00:00
Henry Oswald
94a6504635 cranked up kill to child processes to killSignal: "SIGKILL" 2014-03-04 12:44:16 +00:00
Oliver Matthews
40e2cb1c6d fix tabbing 2014-03-01 15:23:11 +00:00
Oliver Matthews
88cc89a0d1 filter /s from key ids 2014-03-01 15:10:47 +00:00
Oliver Matthews
7c5634044f Allow selection of FS persistor manager. Only require the chosen persistor manager. 2014-02-26 15:15:03 +00:00
Oliver Matthews
ff96f37b2d unit test passing version of FSPersistorManager 2014-02-26 15:10:55 +00:00
Oliver Matthews
53bcdccc04 change the method for assuming s3 2014-02-26 11:58:48 +00:00
Oliver Matthews
e0f0bfb0ea revert settings.persistorManager 2014-02-26 11:49:15 +00:00
Oliver Matthews
4b5f9ad4fb make s3 chosen if no config option specified 
rename *wrapper  to FilestoreManager
remove ToS3 from filestore commands
rename configuration option for filestore.
add details of interface to FilestoreWrapper.coffee for reference.
remove api comments
rename settings.filestoreManager to settings.persistorManager
 2014-02-26 10:24:51 +00:00
Henry Oswald
c9af5e5420 Renamed fsWrapper to PersistorManager 
not 100% happy with that still but its the best I can see so far
 2014-02-26 09:15:37 +00:00
Henry Oswald
49ffeb633d fix where cacheWarm param is read from and improved some logging 2014-02-26 09:06:47 +00:00
Oliver Matthews
5f87dfb3bf add unit tests for fsWrapper 
make other unit tests no longer depend on fsWrapper
 2014-02-25 15:52:30 +00:00
Oliver Matthews
c43533b5cb move test wrapper into fsWrapper where appropriate 2014-02-25 10:54:51 +00:00
Oliver Matthews
f68080d56c create mixin to allow switching of backend 2014-02-25 09:10:22 +00:00
Henry Oswald
159670a3d4 reduced timeout to 20 seconds 2014-02-20 11:45:51 +00:00
Henry Oswald
99e15dc12f upped https connections to 300 and added debug logging 2014-02-19 21:03:03 +00:00
Henry Oswald
fada5ecaf4 converted get stream to use knox, test to see if it is more robust 2014-02-19 20:56:45 +00:00
Henry Oswald
3f54688ec8 added max sockets to s3 wrapper, getting some ETIMEDOUT under laod, may be cause 2014-02-19 18:39:25 +00:00
Henry Oswald
b24ae77f4c call callback when error returned from getign status of file 2014-02-19 15:26:00 +00:00
Henry Oswald
0a2ce3b869 fixed missleading log file 2014-02-19 15:24:29 +00:00
Henry Oswald
90ad2c24c6 call the callback when no response is returned from s3 when checking if file exists 2014-02-19 14:58:36 +00:00
Henry Oswald
960b46dda0 increased timeout to 4 mins.... 2014-02-19 14:48:16 +00:00
Henry Oswald
b85017759e added more logging 2014-02-19 14:48:16 +00:00
Henry Oswald
531b83a032 added more logging 2014-02-19 14:48:15 +00:00
Henry Oswald
d0600a4b64 changed optimisation to optipng and added timeouts to conversions 2014-02-19 14:48:15 +00:00
Henry Oswald
f63cdb3515 changed thumbnail and preview to be smaller 2014-02-19 14:48:15 +00:00
Henry Oswald
f99a6dc39b changed file converted to use child process with nice 2014-02-19 14:48:12 +00:00
Henry Oswald
7501046ebc improved logging 2014-02-17 16:31:33 +00:00
James Allen
3199aad601 Initial open source commit 2014-02-14 16:39:05 +00:00