Jakob Ackermann
e3d7afeb96
[misc] run format_fix and lint:fix
2021-07-13 12:04:46 +01:00
Jakob Ackermann
3152bf361d
[misc] switch from settings-sharelatex to @overleaf/settings
2021-07-12 17:47:19 +01:00
Jakob Ackermann
95f3bcef21
[misc] bump metrics module to 3.4.1
...
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
2020-11-25 11:57:23 +00:00
Jakob Ackermann
f0551307d2
[misc] bump the dev-env to 3.3.2
2020-08-10 17:01:12 +01:00
Simon Detheridge
14834abf83
Move config settings to correct place for new persistor module
2020-07-08 10:32:41 +01:00
Simon Detheridge
6c853de5d5
Migrate to new object-persistor module ( #122 )
...
* Migrate to new object-persistor module
* Support updated persistor module using ES6 classes
* Update object-persistor
* Upgrade new persistor code to OError3
* Update to latest minor release for outdated packages
* Update package hashes to sha512
* Point object-persistor at master branch
2020-07-07 13:49:54 +01:00
Ersun Warncke
046c8d916b
remove un-needed config change in unit test
2020-07-06 09:27:12 -04:00
Ersun Warncke
0e4be81406
upgrade to o-error v3
2020-06-24 22:40:04 -04:00
Simon Detheridge
7b6299e05a
Update GcsPersistor range unit test
2020-06-09 13:55:03 +01:00
Miguel Serrano
e014aeb100
Fixed regular expression to insert/delete files not supporting templates api
2020-05-26 17:21:21 +02:00
Simon Detheridge
3637cd70ae
Add support for redirecting to signed URLs
2020-04-15 12:22:54 +01:00
Simon Detheridge
13ab7b5ca1
Fix unit tests for pipeline changes
2020-04-02 16:49:45 +01:00
Simon Detheridge
4489670497
Only delete the converted-cache folder if conversions are enabled
2020-03-31 15:38:42 +01:00
Simon Detheridge
a1ae68f6b5
Tidy up 'getReadyPipeline'
2020-03-26 16:57:55 +00:00
Simon Detheridge
1f037ef653
Catch errors from pipeline and stream when waiting for streams to be readable
2020-03-26 16:57:55 +00:00
Simon Detheridge
d073fe75ca
Use single pipeline when calculating md5
2020-03-25 16:59:51 +00:00
Simon Detheridge
06c4c0f74f
Fix incorrect key when deleting projects
2020-03-16 11:35:01 +00:00
Simon Detheridge
ce52f8aa60
Update FSPersistor deleteDirectory behaviour to match S3 and GCS
2020-03-16 11:33:51 +00:00
Simon Detheridge
47e96a4d94
Add endpoint to delete a project
...
Needs acceptance tests
2020-03-14 14:56:29 +00:00
Simon Detheridge
fc80aa3954
Move directory key validation into FileHandler
2020-03-14 14:31:30 +00:00
Simon Detheridge
edf1ce1f7e
Delete files from a directory in parallel
2020-03-14 14:02:58 +00:00
Simon Detheridge
183cb0179a
Add support to GCS persistor for unlocking files and copying on delete
2020-03-13 16:18:45 +00:00
Simon Detheridge
28c3fe4a56
Validate key names when deleting directory with a configurable regex
2020-03-10 17:54:09 +00:00
Simon Detheridge
eb93ae4b10
Use Bucket.deleteFiles to delete directory contents, instead of iterating
2020-03-05 14:12:15 +00:00
Simon Detheridge
460dd96b17
Cosmetic clean-up of GCS Persistor & tests
2020-03-05 13:45:46 +00:00
Simon Detheridge
30114cd79b
Remove unnecessary try/catch around 'createReadStream'
2020-03-04 16:38:05 +00:00
Simon Detheridge
def383574e
Simplify getMeteredStream to record metric directly
2020-03-04 16:17:36 +00:00
Simon Detheridge
76243fd75a
'bucket' -> 'bucketName' in GCS Persistor
2020-03-04 16:04:26 +00:00
Simon Detheridge
6979b8638a
Add 'Buffer' global for GCS unit tests
2020-03-04 15:42:16 +00:00
Simon Detheridge
12274e1427
Add unit tests for GCS Persistor
2020-03-04 15:38:18 +00:00
Simon Detheridge
d9c9d74994
Remove unnecessary test for S3 file deletion
...
S3 does not throw a not-found error when deleting a file that does not exist
2020-03-04 15:38:18 +00:00
Simon Detheridge
9807568de8
Add missing 'process' global for settings tests
2020-03-04 10:08:31 +00:00
Simon Detheridge
8905d393f7
Merge pull request #95 from overleaf/jpa-test-cleanup
...
[misc] test cleanup
2020-03-04 10:01:39 +00:00
Jakob Ackermann
2b15729658
[misc] promisify FileHandler and remove dependency on async
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 18:23:17 +01:00
Brian Gough
ebabe52632
fix settings unit test
2020-02-28 15:27:29 +00:00
Jakob Ackermann
94b8a7f891
[misc] test/unit: KeybuilderTests: use a unique suite label
2020-02-28 12:26:23 +01:00
Jakob Ackermann
54e3b57772
[misc] test/unit: add missing globals that are lazy loaded
2020-02-28 12:26:23 +01:00
Jakob Ackermann
7920570dd8
[misc] test/unit: add missing require stubs for metrics and settings
2020-02-28 12:26:22 +01:00
Simon Detheridge
8acc9ef0ae
Make S3 partSize a setting
2020-02-17 14:06:15 +00:00
Simon Detheridge
2f2a819b74
Re-fetch file to calculate md5 if etag is not in correct format
2020-02-17 14:05:43 +00:00
Simon Detheridge
5adfb3e2c0
Use large upload chunks
2020-02-17 14:05:31 +00:00
Simon Detheridge
4e507197fc
Revert "Merge pull request #91 from overleaf/revert-78-spd-migration-persistor"
...
This reverts commit 0cce792604c258538cf012486e7329fa1be7da0f, reversing
changes made to 08b606e22f855121b2d8a2832a5d7e2f45836c3f.
2020-02-17 14:04:42 +00:00
Simon Detheridge
df2ddbe0e1
Revert "Add Migration Persistor, to send missing file requests to a fallback persistor"
2020-02-14 14:26:33 +00:00
Simon Detheridge
93cd55fb79
Refactor persistors to use a helper for common things
2020-02-12 13:27:12 +00:00
Simon Detheridge
9e0b378948
Remove minipass as dependency and refactor to make things clearer
2020-02-12 13:21:52 +00:00
Simon Detheridge
b4b7fd226e
Add mechanisms to transfer files with md5-based integrity checks
...
Fix error in settings and tidy up tests
Remove unused variable declaration
Remove .only from tests and update eslint rules to catch it in future
Use to catch errors more safely getting md5 hash
Avoid unnecessary call to S3 to get md5 response
2020-02-12 13:21:52 +00:00
Simon Detheridge
2625e03a31
Add MigrationPersistor for sending 404 requests to a fallback persistor
2020-02-12 13:19:48 +00:00
Simon Detheridge
f877f51775
Rename *PersistorManager to *Persistor
2020-02-12 13:19:48 +00:00
Simon Detheridge
28fb998719
Refactor request logger into a class
2020-01-14 12:02:39 +00:00
Simon Detheridge
9615a06e0f
Improve logging middleware to add info methods to request object
2020-01-10 17:22:06 +00:00