Miguel Serrano
e014aeb100
Fixed regular expression to insert/delete files not supporting templates api
2020-05-26 17:21:21 +02:00
Simon Detheridge
3637cd70ae
Add support for redirecting to signed URLs
2020-04-15 12:22:54 +01:00
Simon Detheridge
13ab7b5ca1
Fix unit tests for pipeline changes
2020-04-02 16:49:45 +01:00
Simon Detheridge
ccf5f8b9e8
Add acceptance test for leaked sockets on aborted connections
2020-04-02 15:56:19 +01:00
Simon Detheridge
4489670497
Only delete the converted-cache folder if conversions are enabled
2020-03-31 15:38:42 +01:00
Simon Detheridge
64562dffb0
eslint-disable => eslint-disable-next-line
2020-03-26 22:07:37 +00:00
Simon Detheridge
a1ae68f6b5
Tidy up 'getReadyPipeline'
2020-03-26 16:57:55 +00:00
Simon Detheridge
1f037ef653
Catch errors from pipeline and stream when waiting for streams to be readable
2020-03-26 16:57:55 +00:00
Simon Detheridge
238d96ed44
Fail acceptance tests on unhandled promise rejection
2020-03-26 16:57:55 +00:00
Simon Detheridge
d073fe75ca
Use single pipeline when calculating md5
2020-03-25 16:59:51 +00:00
Simon Detheridge
65f19346f0
Fix fallback to custom s3 buckets
2020-03-19 11:10:31 +00:00
Simon Detheridge
cb4bdd99f4
Use an ISODate for deleted file names
2020-03-18 20:19:12 +00:00
Simon Detheridge
06c4c0f74f
Fix incorrect key when deleting projects
2020-03-16 11:35:01 +00:00
Simon Detheridge
9f74aac1a0
Add acceptance tests for directory deletion
2020-03-16 11:34:45 +00:00
Simon Detheridge
ce52f8aa60
Update FSPersistor deleteDirectory behaviour to match S3 and GCS
2020-03-16 11:33:51 +00:00
Simon Detheridge
47e96a4d94
Add endpoint to delete a project
...
Needs acceptance tests
2020-03-14 14:56:29 +00:00
Simon Detheridge
fc80aa3954
Move directory key validation into FileHandler
2020-03-14 14:31:30 +00:00
Simon Detheridge
58db14456a
Add timestamp to files in deleted bucket
2020-03-14 14:11:17 +00:00
Simon Detheridge
edf1ce1f7e
Delete files from a directory in parallel
2020-03-14 14:02:58 +00:00
Simon Detheridge
183cb0179a
Add support to GCS persistor for unlocking files and copying on delete
2020-03-13 16:18:45 +00:00
Simon Detheridge
28c3fe4a56
Validate key names when deleting directory with a configurable regex
2020-03-10 17:54:09 +00:00
Simon Detheridge
eb93ae4b10
Use Bucket.deleteFiles to delete directory contents, instead of iterating
2020-03-05 14:12:15 +00:00
Simon Detheridge
460dd96b17
Cosmetic clean-up of GCS Persistor & tests
2020-03-05 13:45:46 +00:00
Simon Detheridge
3bb956b38e
Use http for the fake GCS server
2020-03-04 17:01:20 +00:00
Simon Detheridge
30114cd79b
Remove unnecessary try/catch around 'createReadStream'
2020-03-04 16:38:05 +00:00
Simon Detheridge
def383574e
Simplify getMeteredStream to record metric directly
2020-03-04 16:17:36 +00:00
Simon Detheridge
76243fd75a
'bucket' -> 'bucketName' in GCS Persistor
2020-03-04 16:04:26 +00:00
Simon Detheridge
6979b8638a
Add 'Buffer' global for GCS unit tests
2020-03-04 15:42:16 +00:00
Simon Detheridge
12274e1427
Add unit tests for GCS Persistor
2020-03-04 15:38:18 +00:00
Simon Detheridge
d9c9d74994
Remove unnecessary test for S3 file deletion
...
S3 does not throw a not-found error when deleting a file that does not exist
2020-03-04 15:38:18 +00:00
Simon Detheridge
e2f3dd23c9
Switch back to official fake-gcs-server image
2020-03-04 15:38:18 +00:00
Simon Detheridge
2cfab8d313
Add GCS-specific acceptance tests
2020-03-04 15:38:16 +00:00
Simon Detheridge
7292d93dab
Add fake GCS server, and fix health checks
2020-03-04 15:08:19 +00:00
Simon Detheridge
9807568de8
Add missing 'process' global for settings tests
2020-03-04 10:08:31 +00:00
Simon Detheridge
8905d393f7
Merge pull request #95 from overleaf/jpa-test-cleanup
...
[misc] test cleanup
2020-03-04 10:01:39 +00:00
Jakob Ackermann
2b15729658
[misc] promisify FileHandler and remove dependency on async
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 18:23:17 +01:00
Brian Gough
ebabe52632
fix settings unit test
2020-02-28 15:27:29 +00:00
Jakob Ackermann
2b9d0868c5
[misc] test/acceptance: retrieve ingress metrics just before using it
...
The upload request can bump the ingress metric.
The content hash validation might require a full download
in case the ETag field of the upload response is not a md5 sum.
2020-02-28 12:27:06 +01:00
Jakob Ackermann
516102e6fb
[misc] test/acceptance: do not hard code fake credentials
2020-02-28 12:26:46 +01:00
Jakob Ackermann
847f124d7b
[misc] test/acceptance: skip the shutdown in case we did not start yet
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:26:24 +01:00
Jakob Ackermann
6589aa6ae0
[misc] test/acceptance: harden the startup check for s3
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:26:24 +01:00
Jakob Ackermann
94b8a7f891
[misc] test/unit: KeybuilderTests: use a unique suite label
2020-02-28 12:26:23 +01:00
Jakob Ackermann
54e3b57772
[misc] test/unit: add missing globals that are lazy loaded
2020-02-28 12:26:23 +01:00
Jakob Ackermann
7920570dd8
[misc] test/unit: add missing require stubs for metrics and settings
2020-02-28 12:26:22 +01:00
Simon Detheridge
8acc9ef0ae
Make S3 partSize a setting
2020-02-17 14:06:15 +00:00
Simon Detheridge
9e7eec7a19
Add acceptance test for large files
2020-02-17 14:05:56 +00:00
Simon Detheridge
2f2a819b74
Re-fetch file to calculate md5 if etag is not in correct format
2020-02-17 14:05:43 +00:00
Simon Detheridge
5adfb3e2c0
Use large upload chunks
2020-02-17 14:05:31 +00:00
Simon Detheridge
4e507197fc
Revert "Merge pull request #91 from overleaf/revert-78-spd-migration-persistor"
...
This reverts commit 0cce792604c258538cf012486e7329fa1be7da0f, reversing
changes made to 08b606e22f855121b2d8a2832a5d7e2f45836c3f.
2020-02-17 14:04:42 +00:00
Simon Detheridge
df2ddbe0e1
Revert "Add Migration Persistor, to send missing file requests to a fallback persistor"
2020-02-14 14:26:33 +00:00
