20 commits

Author	SHA1	Message	Date
Simon Detheridge	1f037ef653	Catch errors from pipeline and stream when waiting for streams to be readable	2020-03-26 16:57:55 +00:00
Simon Detheridge	d073fe75ca	Use single pipeline when calculating md5	2020-03-25 16:59:51 +00:00
Simon Detheridge	cb4bdd99f4	Use an ISODate for deleted file names	2020-03-18 20:19:12 +00:00
Simon Detheridge	9d32d4ec16	Don't modify 'opts' parameter	2020-03-16 15:57:37 +00:00
Simon Detheridge	b37c52fc3a	Make GCS delete concurrency configurable	2020-03-16 15:54:05 +00:00
Simon Detheridge	9b658dda18	Copy-on-delete before unlocking	2020-03-16 15:53:45 +00:00
Simon Detheridge	fc80aa3954	Move directory key validation into FileHandler	2020-03-14 14:31:30 +00:00
Simon Detheridge	58db14456a	Add timestamp to files in deleted bucket	2020-03-14 14:11:17 +00:00
Simon Detheridge	edf1ce1f7e	Delete files from a directory in parallel	2020-03-14 14:02:58 +00:00
Simon Detheridge	183cb0179a	Add support to GCS persistor for unlocking files and copying on delete	2020-03-13 16:18:45 +00:00
Simon Detheridge	28c3fe4a56	Validate key names when deleting directory with a configurable regex	2020-03-10 17:54:09 +00:00
Simon Detheridge	eb93ae4b10	Use Bucket.deleteFiles to delete directory contents, instead of iterating	2020-03-05 14:12:15 +00:00
Simon Detheridge	460dd96b17	Cosmetic clean-up of GCS Persistor & tests	2020-03-05 13:45:46 +00:00
Simon Detheridge	3bb956b38e	Use http for the fake GCS server	2020-03-04 17:01:20 +00:00
Simon Detheridge	30114cd79b	Remove unnecessary try/catch around 'createReadStream'	2020-03-04 16:38:05 +00:00
Simon Detheridge	a7198764cb	Improve/add some comments for clarity	2020-03-04 16:25:11 +00:00
Simon Detheridge	def383574e	Simplify getMeteredStream to record metric directly	2020-03-04 16:17:36 +00:00
Simon Detheridge	76243fd75a	'bucket' -> 'bucketName' in GCS Persistor	2020-03-04 16:04:26 +00:00
Simon Detheridge	e58284aefe	Move base64/hex methods to PersistorHelper ... Also add some null-safety checks	2020-03-04 15:38:18 +00:00
Simon Detheridge	366ce97169	Add GCS Persistor	2020-03-04 15:09:14 +00:00