Simon Detheridge
2e9dd5ccf5
Merge pull request #100 from overleaf/spd-prettierfix
...
Fix prettier task
2020-03-04 09:58:35 +00:00
Jakob Ackermann
2b15729658
[misc] promisify FileHandler and remove dependency on async
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 18:23:17 +01:00
Jakob Ackermann
ada6ac0764
[misc] revert eslint rules
2020-03-02 18:12:55 +01:00
Jakob Ackermann
1276fb426a
[misc] bump buildscript version to 1.3.6
2020-03-02 17:50:45 +01:00
Simon Detheridge
0312b00150
Fix prettier task
2020-03-02 16:31:35 +00:00
Jakob Ackermann
4b7c6fafd2
[misc] drop unused dependency pngcrush
2020-03-02 17:11:56 +01:00
Jakob Ackermann
3c61e53918
[misc] move mocha to the devDependencies
2020-03-02 17:11:39 +01:00
Jakob Ackermann
6f27f7a1bf
[misc] drop the /heapdump route and related heapdump package
2020-03-02 17:09:35 +01:00
Jakob Ackermann
32557ab1d7
[SafeExec] replace _.once with lodash.once
2020-03-02 17:08:42 +01:00
Jakob Ackermann
0db4a17a14
[HealthCheckController] use fs.copyFile instead of fs-extra.copy
...
the fs-extra method has a HUGE overhead of JS code and also syscalls for
no particular benefit in this case: just copy the tiny.pdf file.
Here is an overview of the major operations: paths are relative to
https://github.com/jprichardson/node-fs-extra/blob/1.0.0
We start in /lib/copy/copy.js
- sys: check that the source file exists
- sys: check that the source file has an existing parent directory?!
Continue in /lib/copy/ncp.js
- sys: more stat calls on both source and dest to determine permissions
- read/write streams to pipe the file content through the process
- sys: chmod on the destination to match the source permissions
What we actually need is a call to the binding and let node/the os
figure out the best way to copy the contents.
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 17:02:35 +01:00
Simon Detheridge
428dcdf0ee
Merge pull request #99 from overleaf/bg-update-npm-packages
...
run npm update
2020-03-02 11:28:27 +00:00
Simon Detheridge
c13d7f4197
Bump aws-sdk version in package.json
2020-03-02 11:18:06 +00:00
Brian Gough
ebabe52632
fix settings unit test
2020-02-28 15:27:29 +00:00
Brian Gough
e3011b2521
run npm update
2020-02-28 14:51:57 +00:00
Jakob Ackermann
b4e3d7de62
[config] add support for third party s3 which uses path-style buckets
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:48:52 +01:00
Jakob Ackermann
2b9d0868c5
[misc] test/acceptance: retrieve ingress metrics just before using it
...
The upload request can bump the ingress metric.
The content hash validation might require a full download
in case the ETag field of the upload response is not a md5 sum.
2020-02-28 12:27:06 +01:00
Jakob Ackermann
516102e6fb
[misc] test/acceptance: do not hard code fake credentials
2020-02-28 12:26:46 +01:00
Jakob Ackermann
847f124d7b
[misc] test/acceptance: skip the shutdown in case we did not start yet
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:26:24 +01:00
Jakob Ackermann
6589aa6ae0
[misc] test/acceptance: harden the startup check for s3
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-02-28 12:26:24 +01:00
Jakob Ackermann
94b8a7f891
[misc] test/unit: KeybuilderTests: use a unique suite label
2020-02-28 12:26:23 +01:00
Jakob Ackermann
54e3b57772
[misc] test/unit: add missing globals that are lazy loaded
2020-02-28 12:26:23 +01:00
Jakob Ackermann
7920570dd8
[misc] test/unit: add missing require stubs for metrics and settings
2020-02-28 12:26:22 +01:00
Simon Detheridge
00d949ba26
Merge pull request #94 from overleaf/spd-unhandled-errors
...
Write to error log on unhandled rejection/exception
2020-02-27 10:37:31 +00:00
Simon Detheridge
6e19a650d2
Write to error log on unhandled rejection/exception
2020-02-27 10:33:48 +00:00
Simon Detheridge
0cdcac4cd2
Merge pull request #93 from overleaf/spd-migration-persistor-redux
...
Add migration persistor with md5 fix
2020-02-18 10:52:18 +00:00
Simon Detheridge
492f8abc9f
Update config/settings.defaults.coffee
...
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
2020-02-18 10:24:29 +00:00
Simon Detheridge
8acc9ef0ae
Make S3 partSize a setting
2020-02-17 14:06:15 +00:00
Simon Detheridge
7c4bf97a7f
Avoid fetching file a second time when source md5 can't be fetched
2020-02-17 14:06:05 +00:00
Simon Detheridge
9e7eec7a19
Add acceptance test for large files
2020-02-17 14:05:56 +00:00
Simon Detheridge
2f2a819b74
Re-fetch file to calculate md5 if etag is not in correct format
2020-02-17 14:05:43 +00:00
Simon Detheridge
5adfb3e2c0
Use large upload chunks
2020-02-17 14:05:31 +00:00
Simon Detheridge
4e507197fc
Revert "Merge pull request #91 from overleaf/revert-78-spd-migration-persistor"
...
This reverts commit 0cce792604c258538cf012486e7329fa1be7da0f, reversing
changes made to 08b606e22f855121b2d8a2832a5d7e2f45836c3f.
2020-02-17 14:04:42 +00:00
Simon Detheridge
bbea064687
Merge pull request #91 from overleaf/revert-78-spd-migration-persistor
...
Revert "Add Migration Persistor, to send missing file requests to a fallback persistor"
2020-02-14 14:34:29 +00:00
Simon Detheridge
df2ddbe0e1
Revert "Add Migration Persistor, to send missing file requests to a fallback persistor"
2020-02-14 14:26:33 +00:00
Simon Detheridge
8f86002bb4
Merge pull request #89 from overleaf/spd-error-logging-fix
...
Move error handler to be the last middleware
2020-02-14 11:07:07 +00:00
Simon Detheridge
e4926e09a7
Move error handler to be the last middleware
2020-02-14 10:58:46 +00:00
Simon Detheridge
2f67ade3ec
Merge pull request #88 from overleaf/dependabot-lodash-4.17.15
...
Bump lodash from 4.17.11 to 4.17.15
2020-02-14 09:45:12 +00:00
dependabot[bot]
31f2572e57
Bump lodash from 4.17.11 to 4.17.15
...
Bumps [lodash](https://github.com/lodash/lodash ) from 4.17.11 to 4.17.15.
- [Release notes](https://github.com/lodash/lodash/releases )
- [Commits](https://github.com/lodash/lodash/compare/4.17.11...4.17.15 )
Signed-off-by: dependabot[bot] <support@github.com>
2020-02-14 09:39:06 +00:00
Simon Detheridge
485cfce6d3
Merge pull request #85 from overleaf/jpa-dependencies-cleanup
...
[misc] rename npm-shrinkwrap.json to package-lock.json and run npm i
2020-02-14 09:24:43 +00:00
Simon Detheridge
16d1674eab
Merge pull request #84 from overleaf/bg-remove-travis-yml
...
Remove unused .travis.yml file
2020-02-14 09:21:03 +00:00
Jakob Ackermann
84a80930eb
[misc] rename npm-shrinkwrap.json to package-lock.json and run npm i
2020-02-14 09:19:29 +00:00
Simon Detheridge
44fbc6a733
Merge pull request #82 from overleaf/spd-error-handler-error
...
Fix log info assignment in error handler
2020-02-14 09:18:16 +00:00
Simon Detheridge
aae277e16e
Merge pull request #78 from overleaf/spd-migration-persistor
...
Add Migration Persistor, to send missing file requests to a fallback persistor
2020-02-14 09:17:31 +00:00
Simon Detheridge
9cf473ba7b
Merge pull request #79 from overleaf/spd-helper-refactor
...
Refactor persistors to use a helper for common things
2020-02-13 16:59:35 +00:00
Simon Detheridge
3b011258d2
Tidy up md5 hash generation
2020-02-12 13:27:12 +00:00
Simon Detheridge
49ad408b30
Remove unused imports and format correctly
2020-02-12 13:27:12 +00:00
Simon Detheridge
93cd55fb79
Refactor persistors to use a helper for common things
2020-02-12 13:27:12 +00:00
Simon Detheridge
304fdfd35c
Explicitly resume stream after adding listener
2020-02-12 13:21:53 +00:00
Simon Detheridge
6dcf351377
Remove unnecessary 'async'
2020-02-12 13:21:53 +00:00
Simon Detheridge
f4a16cd972
Update tests to properly check for copied files
2020-02-12 13:21:53 +00:00