Simon Detheridge
|
f440cec8ec
|
Check if streams are already destroyed before destroying
|
2020-03-31 11:07:31 +01:00 |
|
Simon Detheridge
|
5fadafaf96
|
Destroy streams handled by getReadyPipeline on error
|
2020-03-30 17:14:46 +01:00 |
|
Simon Detheridge
|
87b8f8e194
|
Destroy file-download stream on error
|
2020-03-30 17:13:40 +01:00 |
|
Simon Detheridge
|
4cb4d450be
|
Call end() when handing ERR_STREAM_PREMATURE_CLOSE
|
2020-03-30 15:03:09 +01:00 |
|
Simon Detheridge
|
ae62bb75ca
|
Merge pull request #107 from overleaf/spd-handle-fixes-hopefully
(Hopefully) fix handle leaks
|
2020-03-27 11:37:48 +00:00 |
|
Simon Detheridge
|
64562dffb0
|
eslint-disable => eslint-disable-next-line
|
2020-03-26 22:07:37 +00:00 |
|
Simon Detheridge
|
a1ae68f6b5
|
Tidy up 'getReadyPipeline'
|
2020-03-26 16:57:55 +00:00 |
|
Simon Detheridge
|
1f037ef653
|
Catch errors from pipeline and stream when waiting for streams to be readable
|
2020-03-26 16:57:55 +00:00 |
|
Simon Detheridge
|
238d96ed44
|
Fail acceptance tests on unhandled promise rejection
|
2020-03-26 16:57:55 +00:00 |
|
Simon Detheridge
|
7e45a82c35
|
Use autodestroy on Transform stream
|
2020-03-26 15:09:56 +00:00 |
|
Simon Detheridge
|
82599bce67
|
Merge pull request #105 from overleaf/jpa-logger-1-9-1
[misc] bump logger-sharelatex to 1.9.1
|
2020-03-26 10:31:33 +00:00 |
|
Simon Detheridge
|
345696b067
|
Merge pull request #106 from overleaf/spd-single-pipe
Use single pipeline when calculating md5
|
2020-03-26 10:31:07 +00:00 |
|
Simon Detheridge
|
d073fe75ca
|
Use single pipeline when calculating md5
|
2020-03-25 16:59:51 +00:00 |
|
Jakob Ackermann
|
2ce3c8f97e
|
[misc] bump logger-sharelatex to 1.9.1
|
2020-03-23 16:18:07 +01:00 |
|
Simon Detheridge
|
63b4b3e9a2
|
Merge pull request #104 from overleaf/spd-custom-bucket-fallback
Fix fallback to custom s3 buckets
|
2020-03-19 11:39:55 +00:00 |
|
Simon Detheridge
|
65f19346f0
|
Fix fallback to custom s3 buckets
|
2020-03-19 11:10:31 +00:00 |
|
Simon Detheridge
|
9a03a3406e
|
Merge pull request #86 from overleaf/spd-gcs-persistor
Add GCS Persistor
|
2020-03-19 09:16:47 +00:00 |
|
Simon Detheridge
|
cb4bdd99f4
|
Use an ISODate for deleted file names
|
2020-03-18 20:19:12 +00:00 |
|
Simon Detheridge
|
9d32d4ec16
|
Don't modify 'opts' parameter
|
2020-03-16 15:57:37 +00:00 |
|
Simon Detheridge
|
b37c52fc3a
|
Make GCS delete concurrency configurable
|
2020-03-16 15:54:05 +00:00 |
|
Simon Detheridge
|
9b658dda18
|
Copy-on-delete before unlocking
|
2020-03-16 15:53:45 +00:00 |
|
Simon Detheridge
|
06c4c0f74f
|
Fix incorrect key when deleting projects
|
2020-03-16 11:35:01 +00:00 |
|
Simon Detheridge
|
9f74aac1a0
|
Add acceptance tests for directory deletion
|
2020-03-16 11:34:45 +00:00 |
|
Simon Detheridge
|
ce52f8aa60
|
Update FSPersistor deleteDirectory behaviour to match S3 and GCS
|
2020-03-16 11:33:51 +00:00 |
|
Simon Detheridge
|
47e96a4d94
|
Add endpoint to delete a project
Needs acceptance tests
|
2020-03-14 14:56:29 +00:00 |
|
Simon Detheridge
|
fc80aa3954
|
Move directory key validation into FileHandler
|
2020-03-14 14:31:30 +00:00 |
|
Simon Detheridge
|
58db14456a
|
Add timestamp to files in deleted bucket
|
2020-03-14 14:11:17 +00:00 |
|
Simon Detheridge
|
edf1ce1f7e
|
Delete files from a directory in parallel
|
2020-03-14 14:02:58 +00:00 |
|
Simon Detheridge
|
183cb0179a
|
Add support to GCS persistor for unlocking files and copying on delete
|
2020-03-13 16:18:45 +00:00 |
|
Simon Detheridge
|
28c3fe4a56
|
Validate key names when deleting directory with a configurable regex
|
2020-03-10 17:54:09 +00:00 |
|
Simon Detheridge
|
2509b51883
|
Add optional gcs config to override gcs settings (for fake gcs server)
|
2020-03-05 17:23:47 +00:00 |
|
Simon Detheridge
|
eb93ae4b10
|
Use Bucket.deleteFiles to delete directory contents, instead of iterating
|
2020-03-05 14:12:15 +00:00 |
|
Simon Detheridge
|
460dd96b17
|
Cosmetic clean-up of GCS Persistor & tests
|
2020-03-05 13:45:46 +00:00 |
|
Simon Detheridge
|
3bb956b38e
|
Use http for the fake GCS server
|
2020-03-04 17:01:20 +00:00 |
|
Simon Detheridge
|
30114cd79b
|
Remove unnecessary try/catch around 'createReadStream'
|
2020-03-04 16:38:05 +00:00 |
|
Simon Detheridge
|
a7198764cb
|
Improve/add some comments for clarity
|
2020-03-04 16:25:11 +00:00 |
|
Simon Detheridge
|
def383574e
|
Simplify getMeteredStream to record metric directly
|
2020-03-04 16:17:36 +00:00 |
|
Simon Detheridge
|
76243fd75a
|
'bucket' -> 'bucketName' in GCS Persistor
|
2020-03-04 16:04:26 +00:00 |
|
Simon Detheridge
|
6979b8638a
|
Add 'Buffer' global for GCS unit tests
|
2020-03-04 15:42:16 +00:00 |
|
Simon Detheridge
|
12274e1427
|
Add unit tests for GCS Persistor
|
2020-03-04 15:38:18 +00:00 |
|
Simon Detheridge
|
e58284aefe
|
Move base64/hex methods to PersistorHelper
Also add some null-safety checks
|
2020-03-04 15:38:18 +00:00 |
|
Simon Detheridge
|
d9c9d74994
|
Remove unnecessary test for S3 file deletion
S3 does not throw a not-found error when deleting a file that does not exist
|
2020-03-04 15:38:18 +00:00 |
|
Simon Detheridge
|
e2f3dd23c9
|
Switch back to official fake-gcs-server image
|
2020-03-04 15:38:18 +00:00 |
|
Simon Detheridge
|
9dddf25209
|
Add note on gcs config to config file
|
2020-03-04 15:38:18 +00:00 |
|
Simon Detheridge
|
2cfab8d313
|
Add GCS-specific acceptance tests
|
2020-03-04 15:38:16 +00:00 |
|
Simon Detheridge
|
366ce97169
|
Add GCS Persistor
|
2020-03-04 15:09:14 +00:00 |
|
Simon Detheridge
|
e6cf0687a9
|
Add gcs client library
|
2020-03-04 15:09:14 +00:00 |
|
Simon Detheridge
|
7292d93dab
|
Add fake GCS server, and fix health checks
|
2020-03-04 15:08:19 +00:00 |
|
Simon Detheridge
|
0610b74cc9
|
Merge pull request #102 from overleaf/spd-integration-node12
Merge node 12 integration branch to master
|
2020-03-04 13:10:55 +00:00 |
|
Simon Detheridge
|
6c5826e527
|
Merge pull request #103 from overleaf/spd-bump-metrics
Bump metrics to 2.5.0
|
2020-03-04 11:03:48 +00:00 |
|