github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-30 00:15:23 -05:00
Code Issues Projects Releases Packages Wiki Activity
514 commits 3 branches 7 tags 206 MiB
Commit graph

514 commits

This branch
This branch
All branches
Author SHA1 Message Date
Jakob Ackermann
0db4a17a14 [HealthCheckController] use fs.copyFile instead of fs-extra.copy 
the fs-extra method has a HUGE overhead of JS code and also syscalls for
 no particular benefit in this case: just copy the tiny.pdf file.

Here is an overview of the major operations: paths are relative to
  https://github.com/jprichardson/node-fs-extra/blob/1.0.0

We start in /lib/copy/copy.js
- sys: check that the source file exists
- sys: check that the source file has an existing parent directory?!
Continue in /lib/copy/ncp.js
- sys: more stat calls on both source and dest to determine permissions
- read/write streams to pipe the file content through the process
- sys: chmod on the destination to match the source permissions

What we actually need is a call to the binding and let node/the os
 figure out the best way to copy the contents.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2020-03-02 17:02:35 +01:00
Simon Detheridge
428dcdf0ee Merge pull request #99 from overleaf/bg-update-npm-packages 
run npm update
 2020-03-02 11:28:27 +00:00
Simon Detheridge
c13d7f4197 Bump aws-sdk version in package.json 2020-03-02 11:18:06 +00:00
Brian Gough
ebabe52632 fix settings unit test 2020-02-28 15:27:29 +00:00
Brian Gough
e3011b2521 run npm update 2020-02-28 14:51:57 +00:00
Simon Detheridge
00d949ba26 Merge pull request #94 from overleaf/spd-unhandled-errors 
Write to error log on unhandled rejection/exception
 2020-02-27 10:37:31 +00:00
Simon Detheridge
6e19a650d2 Write to error log on unhandled rejection/exception 2020-02-27 10:33:48 +00:00
Simon Detheridge
0cdcac4cd2 Merge pull request #93 from overleaf/spd-migration-persistor-redux 
Add migration persistor with md5 fix
 2020-02-18 10:52:18 +00:00
Simon Detheridge
492f8abc9f Update config/settings.defaults.coffee 
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
 2020-02-18 10:24:29 +00:00
Simon Detheridge
8acc9ef0ae Make S3 partSize a setting 2020-02-17 14:06:15 +00:00
Simon Detheridge
7c4bf97a7f Avoid fetching file a second time when source md5 can't be fetched 2020-02-17 14:06:05 +00:00
Simon Detheridge
9e7eec7a19 Add acceptance test for large files 2020-02-17 14:05:56 +00:00
Simon Detheridge
2f2a819b74 Re-fetch file to calculate md5 if etag is not in correct format 2020-02-17 14:05:43 +00:00
Simon Detheridge
5adfb3e2c0 Use large upload chunks 2020-02-17 14:05:31 +00:00
Simon Detheridge
4e507197fc Revert "Merge pull request #91 from overleaf/revert-78-spd-migration-persistor" 
This reverts commit 0cce792604c258538cf012486e7329fa1be7da0f, reversing
changes made to 08b606e22f855121b2d8a2832a5d7e2f45836c3f.
 2020-02-17 14:04:42 +00:00
Simon Detheridge
bbea064687 Merge pull request #91 from overleaf/revert-78-spd-migration-persistor 
Revert "Add Migration Persistor, to send missing file requests to a fallback persistor"
 2020-02-14 14:34:29 +00:00
Simon Detheridge
df2ddbe0e1 Revert "Add Migration Persistor, to send missing file requests to a fallback persistor" 2020-02-14 14:26:33 +00:00
Simon Detheridge
8f86002bb4 Merge pull request #89 from overleaf/spd-error-logging-fix 
Move error handler to be the last middleware
 2020-02-14 11:07:07 +00:00
Simon Detheridge
e4926e09a7 Move error handler to be the last middleware 2020-02-14 10:58:46 +00:00
Simon Detheridge
2f67ade3ec Merge pull request #88 from overleaf/dependabot-lodash-4.17.15 
Bump lodash from 4.17.11 to 4.17.15
 2020-02-14 09:45:12 +00:00
dependabot[bot]
31f2572e57 Bump lodash from 4.17.11 to 4.17.15 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.11 to 4.17.15.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.11...4.17.15)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-02-14 09:39:06 +00:00
Simon Detheridge
485cfce6d3 Merge pull request #85 from overleaf/jpa-dependencies-cleanup 
[misc] rename npm-shrinkwrap.json to package-lock.json and run npm i
 2020-02-14 09:24:43 +00:00
Simon Detheridge
16d1674eab Merge pull request #84 from overleaf/bg-remove-travis-yml 
Remove unused .travis.yml file
 2020-02-14 09:21:03 +00:00
Jakob Ackermann
84a80930eb [misc] rename npm-shrinkwrap.json to package-lock.json and run npm i 2020-02-14 09:19:29 +00:00
Simon Detheridge
44fbc6a733 Merge pull request #82 from overleaf/spd-error-handler-error 
Fix log info assignment in error handler
 2020-02-14 09:18:16 +00:00
Simon Detheridge
aae277e16e Merge pull request #78 from overleaf/spd-migration-persistor 
Add Migration Persistor, to send missing file requests to a fallback persistor
 2020-02-14 09:17:31 +00:00
Simon Detheridge
9cf473ba7b Merge pull request #79 from overleaf/spd-helper-refactor 
Refactor persistors to use a helper for common things
 2020-02-13 16:59:35 +00:00
Simon Detheridge
3b011258d2 Tidy up md5 hash generation 2020-02-12 13:27:12 +00:00
Simon Detheridge
49ad408b30 Remove unused imports and format correctly 2020-02-12 13:27:12 +00:00
Simon Detheridge
93cd55fb79 Refactor persistors to use a helper for common things 2020-02-12 13:27:12 +00:00
Simon Detheridge
304fdfd35c Explicitly resume stream after adding listener 2020-02-12 13:21:53 +00:00
Simon Detheridge
6dcf351377 Remove unnecessary 'async' 2020-02-12 13:21:53 +00:00
Simon Detheridge
f4a16cd972 Update tests to properly check for copied files 2020-02-12 13:21:53 +00:00
Simon Detheridge
42a5d168df Remove unused packages 2020-02-12 13:21:52 +00:00
Simon Detheridge
9e0b378948 Remove minipass as dependency and refactor to make things clearer 2020-02-12 13:21:52 +00:00
Simon Detheridge
5d5d325691 Preserve all error information when cleanup of copied file fails 2020-02-12 13:21:52 +00:00
Simon Detheridge
b4b7fd226e Add mechanisms to transfer files with md5-based integrity checks 
Fix error in settings and tidy up tests

Remove unused variable declaration

Remove .only from tests and update eslint rules to catch it in future

Use  to catch errors more safely getting md5 hash

Avoid unnecessary call to S3 to get md5 response
 2020-02-12 13:21:52 +00:00
Simon Detheridge
2625e03a31 Add MigrationPersistor for sending 404 requests to a fallback persistor 2020-02-12 13:19:48 +00:00
Simon Detheridge
f877f51775 Rename *PersistorManager to *Persistor 2020-02-12 13:19:48 +00:00
Brian Gough
a2e1591e93 remove unused .travis.yml file 2020-02-12 12:37:00 +00:00
Jakob Ackermann
03246041cd Merge pull request #83 from overleaf/jpa-build-scripts-1-3-5 
[misc] update the build scripts to 1.3.5
 2020-02-12 13:21:36 +01:00
Simon Detheridge
e3613e52db Fix log info assignment in error handler 2020-02-12 10:34:56 +00:00
Jakob Ackermann
1341af04f5 [misc] update the build scripts to 1.3.5 2020-02-11 18:58:55 +01:00
Jakob Ackermann
ca94225c19 Merge pull request #80 from overleaf/jpa-cleanup-install-deps 
[misc] cleanup install deps
 2020-02-10 17:16:44 +01:00
Brian Gough
14ba1ea1c1 update to node 10.19.0 2020-02-07 14:16:31 +00:00
Jakob Ackermann
ac4361d560 [misc] dockerignore: hide the local data dirs 2020-01-31 15:50:53 +01:00
Jakob Ackermann
c1d81dc1ea [misc] install a maintained ghostscript version and delete package lists 
also do not install vim and bail out in case any command exited with a
 non zero exit code.
 2020-01-31 15:50:53 +01:00
Simon Detheridge
2f99c220f8 Merge pull request #77 from overleaf/spd-simplified-logging 
Simplify logging to only log once per request
 2020-01-20 10:52:57 +00:00
Simon Detheridge
997434c930 Remove redundant 'res.logInfo' 2020-01-14 20:55:33 +00:00
Simon Detheridge
035364b62e Fix RequestLogger object lifecycle 2020-01-14 17:15:01 +00:00