Jakob Ackermann
06c839a230
Merge pull request #17237 from overleaf/jpa-s3-iam
...
[object-persistor] add support for IAM credentials in S3Persistor
GitOrigin-RevId: 4ff34082ca557b17d8a7803f700ee704a3d184c6
2024-02-22 09:04:52 +00:00
Christopher Hoskin
6d86e14b05
Merge pull request #15318 from overleaf/csh-issue-11625-mongoug-5-object-persistor
...
Upgrade mongodb for object-persistor from 3.5.9 to 6.1.0
GitOrigin-RevId: e6b97ae93092a77c46f043979dd9eb5caf234866
2023-10-19 08:03:19 +00:00
Jakob Ackermann
6781f7d0ac
Merge pull request #13781 from overleaf/em-fetch-utils-logger
...
Use fetch-utils in logger library
GitOrigin-RevId: f12067a774856469979475a3be63bebbabcd25a6
2023-09-08 08:04:12 +00:00
Christopher Hoskin
5fbd5e21a2
Merge pull request #13274 from overleaf/csh-issue-13004-cleanup-custom-code
...
Remove apiScheme from GCS
GitOrigin-RevId: c948287cb00c7bc432c797bfdfa92a79b84d1bed
2023-06-02 08:05:10 +00:00
Jakob Ackermann
2760ad35fc
Merge pull request #12899 from overleaf/jpa-object-persistor-pipe
...
[object-persistor] move away from manual .pipe()
GitOrigin-RevId: 5407d48fbbb026ba81f43c891499bd3a5ff59518
2023-05-23 08:05:12 +00:00
Jakob Ackermann
0b82db0abe
Merge pull request #11804 from overleaf/jpa-fix-fs-persistor-nested-layout
...
[object-persistor] forward settings.useSubdirectories to FSPersistor
GitOrigin-RevId: 92cef51c392cfeaebbcca2597740562c1f3fc95f
2023-02-13 14:41:08 +00:00
Eric Mc Sween
4325f1d947
Merge pull request #11179 from overleaf/em-upgrade-gcs-lib
...
Upgrade @google-cloud/storage library to 6.9.0
GitOrigin-RevId: 0e80dcccca4679ffe28d5b16512a691e58694e2b
2023-01-18 09:05:00 +00:00
Eric Mc Sween
be081856a8
Merge pull request #10261 from overleaf/em-object-persistor-md5
...
Compute the md5 hash as we receive the stream
GitOrigin-RevId: 6c0c0a31165c5e1320e7f7a6dbf59a0d3576d3c4
2022-11-11 09:03:42 +00:00
Eric Mc Sween
3ee794da47
Merge pull request #10259 from overleaf/em-object-persistor-tempfile
...
Atomic update of files in FS backend of object-persistor
GitOrigin-RevId: b57c0c1a7d6299affd00e174cb7ae75ae711c5d3
2022-11-11 09:03:35 +00:00
Eric Mc Sween
a98f752b99
Merge pull request #10249 from overleaf/em-object-persistor-subdirs
...
Add useSubdirectories option to object-persistor FS backend
GitOrigin-RevId: dc4f0a57e892ffa5be0c88b8baf7efce683ddfc1
2022-11-11 09:03:27 +00:00
Eric Mc Sween
8fb3edbecd
Merge pull request #10245 from overleaf/em-object-persistor-promises-api
...
Use Node 16 promises APIs in object-persistor
GitOrigin-RevId: 8520da0c0678c17b22a9164d15d61c2b57af8f4e
2022-11-11 09:03:19 +00:00
Jakob Ackermann
f0bd6dda23
Merge pull request #7986 from overleaf/jpa-eslint-8
...
[misc] upgrade eslint packages to the latest version everywhere
GitOrigin-RevId: f1480d4a171acef82fb26c4aa54be3a6088b0ab3
2022-05-17 08:05:59 +00:00
Jakob Ackermann
81b935e317
Merge pull request #7977 from overleaf/jpa-use-local-libraries
...
[misc] use local libraries
GitOrigin-RevId: ec16907c0ce4f9a79689c522f37d52502559e7b2
2022-05-17 08:05:52 +00:00
Jakob Ackermann
4b308553be
Merge pull request #6120 from overleaf/jpa-same-linting-packages
...
[misc] move the linting setup to the root of the monorepo
GitOrigin-RevId: 1633e2a58598add0b727738cd3bfba0ab7bae781
2021-12-17 09:03:06 +00:00
Brian Gough
d630d4a37c
Merge pull request #5258 from overleaf/revert-5214-bg-migrate-node-uuid-to-uuid
...
Revert "migrate node-uuid to uuid"
GitOrigin-RevId: f512eb1040e2ce6d87774de9c1fc884594480cef
2021-09-28 08:04:11 +00:00
Brian Gough
4c04448d1c
Merge pull request #5226 from overleaf/bg-migrate-node-uuid-to-uuid
...
[object-persistor] fix formatting
GitOrigin-RevId: 3ce374b6ec7291658de347d82b98a3678ba57dd2
2021-09-27 08:03:01 +00:00
Brian Gough
9ee1e29286
Merge pull request #5214 from overleaf/bg-migrate-node-uuid-to-uuid
...
migrate node-uuid to uuid
GitOrigin-RevId: 742fd4d3f039c6a026b3d35c3b0a7d54c00ca183
2021-09-27 08:02:54 +00:00
Brian Gough
d4563c8786
Merge pull request #4866 from overleaf/bg-gcs-delete-directory-batch
...
make object-persistor delete gcs files in batches
GitOrigin-RevId: 8ebc892c5f6eb30507ec41d5d3a108e650af5cac
2021-08-27 08:03:25 +00:00
Jakob Ackermann
a383dc316c
[misc] run format:fix
2021-07-27 16:53:06 +01:00
Eric Mc Sween
9089d332ab
Add a "unsignedUrls" option to the GCS persistor
...
This boolean option disables URL signing. Instead, getRedirectUrl()
returns the plain GCS download URL.
2020-09-22 15:59:13 -04:00
Eric Mc Sween
967dde280e
Disable auto-decompression in GCS persistor
...
The GCS library automatically decompresses files with Content-encoding:
gzip. This is very useful, but the S3 library doesn't do that, so we
need to disable the feature in GCS so that it behaves the same way other
persistors behave.
2020-08-18 16:04:54 -04:00
Eric Mc Sween
6521837993
Add S3 options: httpOptions, maxRetries
2020-07-09 17:50:05 -04:00
Eric Mc Sween
3e7e4369af
Implement getRedirectUrl() for the S3 backend
2020-07-08 17:13:20 -04:00
Eric Mc Sween
523ff9c4cd
Support metadata when uploading objects
...
Add contentType and contentEncoding options to sendStream(). These
options will set the corresponding metadata on the object.
This changes the API. The fourth argument to sendStream() used to be the
source md5. Now, it's an options object and the source md5 is a property
of that object.
2020-07-08 08:13:53 -04:00
Simon Detheridge
7369e2f339
Add .circleci/config.yml and fix errors
2020-07-03 11:59:34 +01:00
Simon Detheridge
e92b75a2f8
Create new module from overleaf/filestore persistors ( #1 )
...
* Create new module from overleaf/filestore persistors
* Convert persistors to ES6 classes with local settings
* Update README.md
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
* Update README.md
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
* Update .gitignore
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
* Switch to AGPL license
* Paginate S3 list-object results
* Remove S3 client caching
* Clean up S3 md5-verification mechanism
* Update README for recent changes
* Update README.md
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
* Remove package-lock
* Remove comment about FileHandler
* Add directory marker to FSPersistor.deleteDirectory
* Don't copy opts in GcsPersistor.getObjectStream
* Use Date.now instead of getTime
* Catch errors in migration persistor
* Check that settings.buckets exists
* Don't mutate options in ObserverStream constructor
* Update src/PersistorHelper.js
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
* Lint and format fixes
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
2020-07-02 14:19:45 +01:00
