Simon Detheridge
def383574e
Simplify getMeteredStream to record metric directly
2020-03-04 16:17:36 +00:00
Simon Detheridge
76243fd75a
'bucket' -> 'bucketName' in GCS Persistor
2020-03-04 16:04:26 +00:00
Simon Detheridge
6979b8638a
Add 'Buffer' global for GCS unit tests
2020-03-04 15:42:16 +00:00
Simon Detheridge
12274e1427
Add unit tests for GCS Persistor
2020-03-04 15:38:18 +00:00
Simon Detheridge
e58284aefe
Move base64/hex methods to PersistorHelper
...
Also add some null-safety checks
2020-03-04 15:38:18 +00:00
Simon Detheridge
d9c9d74994
Remove unnecessary test for S3 file deletion
...
S3 does not throw a not-found error when deleting a file that does not exist
2020-03-04 15:38:18 +00:00
Simon Detheridge
e2f3dd23c9
Switch back to official fake-gcs-server image
2020-03-04 15:38:18 +00:00
Simon Detheridge
9dddf25209
Add note on gcs config to config file
2020-03-04 15:38:18 +00:00
Simon Detheridge
2cfab8d313
Add GCS-specific acceptance tests
2020-03-04 15:38:16 +00:00
Simon Detheridge
366ce97169
Add GCS Persistor
2020-03-04 15:09:14 +00:00
Simon Detheridge
e6cf0687a9
Add gcs client library
2020-03-04 15:09:14 +00:00
Simon Detheridge
7292d93dab
Add fake GCS server, and fix health checks
2020-03-04 15:08:19 +00:00
Simon Detheridge
0610b74cc9
Merge pull request #102 from overleaf/spd-integration-node12
...
Merge node 12 integration branch to master
2020-03-04 13:10:55 +00:00
Simon Detheridge
6c5826e527
Merge pull request #103 from overleaf/spd-bump-metrics
...
Bump metrics to 2.5.0
2020-03-04 11:03:48 +00:00
Simon Detheridge
d06b74b0f8
Bump metrics to 2.5.0
2020-03-04 10:55:01 +00:00
Simon Detheridge
79ea76be52
Merge pull request #101 from overleaf/spd-node12-upgrade
...
Upgrade to node 12
2020-03-04 10:23:42 +00:00
Simon Detheridge
9807568de8
Add missing 'process' global for settings tests
2020-03-04 10:08:31 +00:00
Simon Detheridge
a3782422b7
Upgrade to node 12
2020-03-04 10:06:18 +00:00
Simon Detheridge
8905d393f7
Merge pull request #95 from overleaf/jpa-test-cleanup
...
[misc] test cleanup
2020-03-04 10:01:39 +00:00
Simon Detheridge
b5e4345855
Merge pull request #96 from overleaf/jpa-dependencies-cleanup
...
[misc] dependencies cleanup
2020-03-04 10:00:56 +00:00
Simon Detheridge
467af9885a
Merge pull request #97 from overleaf/jpa-3rd-party-s3-via-env
...
[config] add support for third party s3 which uses path-style buckets
2020-03-04 09:59:48 +00:00
Simon Detheridge
62b2aa8f02
Merge pull request #98 from overleaf/jpa-promisify-filehandler
...
[misc] promisify FileHandler and remove dependency on async
2020-03-04 09:59:15 +00:00
Simon Detheridge
2e9dd5ccf5
Merge pull request #100 from overleaf/spd-prettierfix
...
Fix prettier task
2020-03-04 09:58:35 +00:00
Jakob Ackermann
2b15729658
[misc] promisify FileHandler and remove dependency on async
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 18:23:17 +01:00
Jakob Ackermann
ada6ac0764
[misc] revert eslint rules
2020-03-02 18:12:55 +01:00
Jakob Ackermann
1276fb426a
[misc] bump buildscript version to 1.3.6
2020-03-02 17:50:45 +01:00
Simon Detheridge
0312b00150
Fix prettier task
2020-03-02 16:31:35 +00:00
Jakob Ackermann
4b7c6fafd2
[misc] drop unused dependency pngcrush
2020-03-02 17:11:56 +01:00
Jakob Ackermann
3c61e53918
[misc] move mocha to the devDependencies
2020-03-02 17:11:39 +01:00
Jakob Ackermann
6f27f7a1bf
[misc] drop the /heapdump route and related heapdump package
2020-03-02 17:09:35 +01:00
Jakob Ackermann
32557ab1d7
[SafeExec] replace _.once with lodash.once
2020-03-02 17:08:42 +01:00
Jakob Ackermann
0db4a17a14
[HealthCheckController] use fs.copyFile instead of fs-extra.copy
...
the fs-extra method has a HUGE overhead of JS code and also syscalls for
no particular benefit in this case: just copy the tiny.pdf file.
Here is an overview of the major operations: paths are relative to
https://github.com/jprichardson/node-fs-extra/blob/1.0.0
We start in /lib/copy/copy.js
- sys: check that the source file exists
- sys: check that the source file has an existing parent directory?!
Continue in /lib/copy/ncp.js
- sys: more stat calls on both source and dest to determine permissions
- read/write streams to pipe the file content through the process
- sys: chmod on the destination to match the source permissions
What we actually need is a call to the binding and let node/the os
figure out the best way to copy the contents.
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 17:02:35 +01:00
Simon Detheridge
428dcdf0ee
Merge pull request #99 from overleaf/bg-update-npm-packages
...
run npm update
2020-03-02 11:28:27 +00:00
Simon Detheridge
c13d7f4197
Bump aws-sdk version in package.json
2020-03-02 11:18:06 +00:00
Brian Gough
ebabe52632
fix settings unit test
2020-02-28 15:27:29 +00:00
Brian Gough
e3011b2521
run npm update
2020-02-28 14:51:57 +00:00
Jakob Ackermann
b4e3d7de62
[config] add support for third party s3 which uses path-style buckets
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:48:52 +01:00
Jakob Ackermann
2b9d0868c5
[misc] test/acceptance: retrieve ingress metrics just before using it
...
The upload request can bump the ingress metric.
The content hash validation might require a full download
in case the ETag field of the upload response is not a md5 sum.
2020-02-28 12:27:06 +01:00
Jakob Ackermann
516102e6fb
[misc] test/acceptance: do not hard code fake credentials
2020-02-28 12:26:46 +01:00
Jakob Ackermann
847f124d7b
[misc] test/acceptance: skip the shutdown in case we did not start yet
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:26:24 +01:00
Jakob Ackermann
6589aa6ae0
[misc] test/acceptance: harden the startup check for s3
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:26:24 +01:00
Jakob Ackermann
94b8a7f891
[misc] test/unit: KeybuilderTests: use a unique suite label
2020-02-28 12:26:23 +01:00
Jakob Ackermann
54e3b57772
[misc] test/unit: add missing globals that are lazy loaded
2020-02-28 12:26:23 +01:00
Jakob Ackermann
7920570dd8
[misc] test/unit: add missing require stubs for metrics and settings
2020-02-28 12:26:22 +01:00
Simon Detheridge
00d949ba26
Merge pull request #94 from overleaf/spd-unhandled-errors
...
Write to error log on unhandled rejection/exception
2020-02-27 10:37:31 +00:00
Simon Detheridge
6e19a650d2
Write to error log on unhandled rejection/exception
2020-02-27 10:33:48 +00:00
Simon Detheridge
0cdcac4cd2
Merge pull request #93 from overleaf/spd-migration-persistor-redux
...
Add migration persistor with md5 fix
2020-02-18 10:52:18 +00:00
Simon Detheridge
492f8abc9f
Update config/settings.defaults.coffee
...
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
2020-02-18 10:24:29 +00:00
Simon Detheridge
8acc9ef0ae
Make S3 partSize a setting
2020-02-17 14:06:15 +00:00
Simon Detheridge
7c4bf97a7f
Avoid fetching file a second time when source md5 can't be fetched
2020-02-17 14:06:05 +00:00