Simon Detheridge
366ce97169
Add GCS Persistor
2020-03-04 15:09:14 +00:00
Simon Detheridge
e6cf0687a9
Add gcs client library
2020-03-04 15:09:14 +00:00
Simon Detheridge
7292d93dab
Add fake GCS server, and fix health checks
2020-03-04 15:08:19 +00:00
Simon Detheridge
0610b74cc9
Merge pull request #102 from overleaf/spd-integration-node12
...
Merge node 12 integration branch to master
2020-03-04 13:10:55 +00:00
Simon Detheridge
6c5826e527
Merge pull request #103 from overleaf/spd-bump-metrics
...
Bump metrics to 2.5.0
2020-03-04 11:03:48 +00:00
Simon Detheridge
d06b74b0f8
Bump metrics to 2.5.0
2020-03-04 10:55:01 +00:00
Simon Detheridge
79ea76be52
Merge pull request #101 from overleaf/spd-node12-upgrade
...
Upgrade to node 12
2020-03-04 10:23:42 +00:00
Simon Detheridge
9807568de8
Add missing 'process' global for settings tests
2020-03-04 10:08:31 +00:00
Simon Detheridge
a3782422b7
Upgrade to node 12
2020-03-04 10:06:18 +00:00
Simon Detheridge
8905d393f7
Merge pull request #95 from overleaf/jpa-test-cleanup
...
[misc] test cleanup
2020-03-04 10:01:39 +00:00
Simon Detheridge
b5e4345855
Merge pull request #96 from overleaf/jpa-dependencies-cleanup
...
[misc] dependencies cleanup
2020-03-04 10:00:56 +00:00
Simon Detheridge
467af9885a
Merge pull request #97 from overleaf/jpa-3rd-party-s3-via-env
...
[config] add support for third party s3 which uses path-style buckets
2020-03-04 09:59:48 +00:00
Simon Detheridge
62b2aa8f02
Merge pull request #98 from overleaf/jpa-promisify-filehandler
...
[misc] promisify FileHandler and remove dependency on async
2020-03-04 09:59:15 +00:00
Simon Detheridge
2e9dd5ccf5
Merge pull request #100 from overleaf/spd-prettierfix
...
Fix prettier task
2020-03-04 09:58:35 +00:00
Jakob Ackermann
2b15729658
[misc] promisify FileHandler and remove dependency on async
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 18:23:17 +01:00
Jakob Ackermann
ada6ac0764
[misc] revert eslint rules
2020-03-02 18:12:55 +01:00
Jakob Ackermann
1276fb426a
[misc] bump buildscript version to 1.3.6
2020-03-02 17:50:45 +01:00
Simon Detheridge
0312b00150
Fix prettier task
2020-03-02 16:31:35 +00:00
Jakob Ackermann
4b7c6fafd2
[misc] drop unused dependency pngcrush
2020-03-02 17:11:56 +01:00
Jakob Ackermann
3c61e53918
[misc] move mocha to the devDependencies
2020-03-02 17:11:39 +01:00
Jakob Ackermann
6f27f7a1bf
[misc] drop the /heapdump route and related heapdump package
2020-03-02 17:09:35 +01:00
Jakob Ackermann
32557ab1d7
[SafeExec] replace _.once with lodash.once
2020-03-02 17:08:42 +01:00
Jakob Ackermann
0db4a17a14
[HealthCheckController] use fs.copyFile instead of fs-extra.copy
...
the fs-extra method has a HUGE overhead of JS code and also syscalls for
no particular benefit in this case: just copy the tiny.pdf file.
Here is an overview of the major operations: paths are relative to
https://github.com/jprichardson/node-fs-extra/blob/1.0.0
We start in /lib/copy/copy.js
- sys: check that the source file exists
- sys: check that the source file has an existing parent directory?!
Continue in /lib/copy/ncp.js
- sys: more stat calls on both source and dest to determine permissions
- read/write streams to pipe the file content through the process
- sys: chmod on the destination to match the source permissions
What we actually need is a call to the binding and let node/the os
figure out the best way to copy the contents.
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 17:02:35 +01:00
Simon Detheridge
428dcdf0ee
Merge pull request #99 from overleaf/bg-update-npm-packages
...
run npm update
2020-03-02 11:28:27 +00:00
Simon Detheridge
c13d7f4197
Bump aws-sdk version in package.json
2020-03-02 11:18:06 +00:00
Brian Gough
ebabe52632
fix settings unit test
2020-02-28 15:27:29 +00:00
Brian Gough
e3011b2521
run npm update
2020-02-28 14:51:57 +00:00
Jakob Ackermann
b4e3d7de62
[config] add support for third party s3 which uses path-style buckets
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:48:52 +01:00
Jakob Ackermann
2b9d0868c5
[misc] test/acceptance: retrieve ingress metrics just before using it
...
The upload request can bump the ingress metric.
The content hash validation might require a full download
in case the ETag field of the upload response is not a md5 sum.
2020-02-28 12:27:06 +01:00
Jakob Ackermann
516102e6fb
[misc] test/acceptance: do not hard code fake credentials
2020-02-28 12:26:46 +01:00
Jakob Ackermann
847f124d7b
[misc] test/acceptance: skip the shutdown in case we did not start yet
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:26:24 +01:00
Jakob Ackermann
6589aa6ae0
[misc] test/acceptance: harden the startup check for s3
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:26:24 +01:00
Jakob Ackermann
94b8a7f891
[misc] test/unit: KeybuilderTests: use a unique suite label
2020-02-28 12:26:23 +01:00
Jakob Ackermann
54e3b57772
[misc] test/unit: add missing globals that are lazy loaded
2020-02-28 12:26:23 +01:00
Jakob Ackermann
7920570dd8
[misc] test/unit: add missing require stubs for metrics and settings
2020-02-28 12:26:22 +01:00
Simon Detheridge
00d949ba26
Merge pull request #94 from overleaf/spd-unhandled-errors
...
Write to error log on unhandled rejection/exception
2020-02-27 10:37:31 +00:00
Simon Detheridge
6e19a650d2
Write to error log on unhandled rejection/exception
2020-02-27 10:33:48 +00:00
Simon Detheridge
0cdcac4cd2
Merge pull request #93 from overleaf/spd-migration-persistor-redux
...
Add migration persistor with md5 fix
2020-02-18 10:52:18 +00:00
Simon Detheridge
492f8abc9f
Update config/settings.defaults.coffee
...
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
2020-02-18 10:24:29 +00:00
Simon Detheridge
8acc9ef0ae
Make S3 partSize a setting
2020-02-17 14:06:15 +00:00
Simon Detheridge
7c4bf97a7f
Avoid fetching file a second time when source md5 can't be fetched
2020-02-17 14:06:05 +00:00
Simon Detheridge
9e7eec7a19
Add acceptance test for large files
2020-02-17 14:05:56 +00:00
Simon Detheridge
2f2a819b74
Re-fetch file to calculate md5 if etag is not in correct format
2020-02-17 14:05:43 +00:00
Simon Detheridge
5adfb3e2c0
Use large upload chunks
2020-02-17 14:05:31 +00:00
Simon Detheridge
4e507197fc
Revert "Merge pull request #91 from overleaf/revert-78-spd-migration-persistor"
...
This reverts commit 0cce792604c258538cf012486e7329fa1be7da0f, reversing
changes made to 08b606e22f855121b2d8a2832a5d7e2f45836c3f.
2020-02-17 14:04:42 +00:00
Simon Detheridge
bbea064687
Merge pull request #91 from overleaf/revert-78-spd-migration-persistor
...
Revert "Add Migration Persistor, to send missing file requests to a fallback persistor"
2020-02-14 14:34:29 +00:00
Simon Detheridge
df2ddbe0e1
Revert "Add Migration Persistor, to send missing file requests to a fallback persistor"
2020-02-14 14:26:33 +00:00
Simon Detheridge
8f86002bb4
Merge pull request #89 from overleaf/spd-error-logging-fix
...
Move error handler to be the last middleware
2020-02-14 11:07:07 +00:00
Simon Detheridge
e4926e09a7
Move error handler to be the last middleware
2020-02-14 10:58:46 +00:00
Simon Detheridge
2f67ade3ec
Merge pull request #88 from overleaf/dependabot-lodash-4.17.15
...
Bump lodash from 4.17.11 to 4.17.15
2020-02-14 09:45:12 +00:00