128 commits

Author	SHA1	Message	Date
Brian Gough	cf12ec1154	use the aws sdk to copy files in S3PersistorManager ... to work around problems with knox	2018-12-20 16:11:18 +00:00
Brian Gough	8951f297aa	Merge pull request #37 from sharelatex/bg-reduce-logging-of-not-found-errors ... reduce logging of not found errors	2018-11-13 14:06:19 +00:00
Brian Gough	0d6de10b5c	Merge pull request #34 from sharelatex/bg-make-backends-consistent ... make filestore backends consistent	2018-11-13 10:12:18 +00:00
Brian Gough	23a6d6e81d	consider 403 and 404 as NotFound errors	2018-11-09 14:05:38 +00:00
Brian Gough	69b164092c	suppress unnecessary error logging for NotFound	2018-11-09 14:05:38 +00:00
Brian Gough	8744b3aa4e	serve file from disk to avoid read-after-write inconsistency	2018-11-08 15:09:34 +00:00
Brian Gough	40ac59090b	make backends consistent by deleting temporary files	2018-11-07 16:22:09 +00:00
Henry Oswald	089bf3e084	seperate the standard s3 creds from the bucket s3 creds	2018-07-10 14:17:37 +01:00
Michael Mazour	7feafccf31	Amend: safely navigate to bucket credentials	2018-07-06 09:28:09 +01:00
Michael Mazour	03033409c6	Amend: remove unused params	2018-07-06 09:12:59 +01:00
Michael Mazour	336a38ec1e	Amend: scrub secrets from logs ... Calls to `getFile` can now include S3 credentials in `opts`, so sanitize before writing to opts to log.	2018-07-04 16:41:31 +01:00
Michael Mazour	600ab3ce67	Amend: remove problematic ampersand	2018-07-04 16:39:41 +01:00
Michael Mazour	cfbf0d81ba	Amend: fix params retrieval	2018-07-04 12:11:09 +01:00
Michael Mazour	ece650741a	Amend per several review comments ... - Removed unused vars - Label the metric with the bucket name	2018-07-04 12:02:09 +01:00
Michael Mazour	feca8933f1	Add endpoint for arbitrary bucket fetch ... Add `/bucket/:bucket/key/*`, which fetches the file from the given bucket at the given path. Uses auth stored at `settings.filestore.s3.{{bucketName}}` if present, and otherwise default auth.	2018-07-04 11:22:36 +01:00
Shane Kilkelly	23fe93ccfd	Prevent double-logging	2017-06-22 15:28:08 +01:00
Shane Kilkelly	ba9477cdb3	Re-work error checking	2017-06-22 09:41:36 +01:00
Shane Kilkelly	d0bbe8beae	If optipng is sigkilled by timeout, then ignore that error	2017-06-21 15:18:36 +01:00
Shane Kilkelly	b12f69722f	Begin fixing health-check	2017-03-08 14:59:34 +00:00
Henry Oswald	e0f39da616	null check fsPath on unlink file	2016-12-19 14:48:09 +00:00
Henry Oswald	bf93fc3f50	run health check max 1 a at a time	2016-12-19 13:26:21 +00:00
Henry Oswald	fd99887753	running check in background	2016-12-19 12:29:08 +00:00
Henry Oswald	d41bf2e974	result path not in final cb	2016-12-14 18:11:28 +00:00
Henry Oswald	4bba56ed28	added missing files	2016-12-14 17:14:15 +00:00
Henry Oswald	b166c86197	added missing originalFsPath	2016-12-13 14:29:04 +00:00
Henry Oswald	b38e7d3078	* delete both file and converted file if there is an error ... * increase timeout of optpng	2016-12-13 12:43:57 +00:00
James Allen	be43330208	Allow convert command to be prefixed by security commands	2016-05-09 11:37:35 +01:00
Henrique Santos	0bb12f32fa	remove project controller / handler ... add unit test scaffolding	2016-03-13 16:22:14 -03:00
Henrique Santos	1f20744a68	add size function for each persitor	2016-03-12 11:08:07 -03:00
Henrique Santos	5d2fc89137	scaffolding project methods	2016-03-12 04:35:49 -03:00
Henrique Santos	099d0fef2f	scaffolding project size	2016-03-11 22:31:46 -03:00
James Allen	bc6c8d6afc	Don't send multiple status codes on insertFile error	2015-11-25 16:07:51 +00:00
Xavier Trochu	81a7203315	Fix the checkIfFileExists API.	2015-11-25 15:40:35 +01:00
Xavier Trochu	3b6270236c	Replace indentation from 2 space to tabs.	2015-11-20 14:38:23 +01:00
Xavier Trochu	0219900933	Forward the options argument to getFileStream	2015-11-20 12:07:04 +01:00
Xavier Trochu	bfd41fdaf9	Add aws-sdk unit test. ... Fix Aws-Sdk persistor to return a correct error on file not found. Fix FileHandler after some change were lost on a previous merge	2015-11-20 12:02:22 +01:00
Xavier Trochu	476db58c3f	Add the opts parameter to getFileStream and implement Range requests	2015-11-17 12:52:25 +01:00
Xavier Trochu	26523b011a	small fixes	2015-11-16 09:59:27 +01:00
Xavier Trochu	63aef6b832	Fix postmerge issue	2015-11-05 12:11:02 +01:00
Xavier Trochu	a367190a21	Merge remote-tracking branch 'sharelatex/master'	2015-11-04 15:20:15 +01:00
Shane Kilkelly	0d450b4e33	Tighten up error handling.	2015-09-10 11:10:30 +01:00
Shane Kilkelly	fcfc595c35	Handle potential error before calling LocalFileWriter.writeStream.	2015-09-10 09:12:49 +01:00
Shane Kilkelly	da5a538095	Fix calls to PersistorManager.getFileStream which were missed in initial refactor. ... This should fix a breakage in the template publishing workflow. Ref: b70b837b7ba29b0631f3105ec1dd5f6cdebbd195	2015-09-08 09:23:08 +01:00
Shane Kilkelly	9b8f11f517	Fix the check for a success response from S3. ... We need to handle both a 200 and 206 (partial content) response.	2015-09-01 12:25:10 +01:00
Shane Kilkelly	82af1be756	Improve testing of error conditions in the FSPersistorManager.	2015-09-01 12:10:30 +01:00
Shane Kilkelly	edc06c82e5	Correct usage of '==' rather than '=' when checking err.code.	2015-09-01 12:10:05 +01:00
Shane Kilkelly	8bbf81f4bc	If the S3 service produces an error, handle it.	2015-09-01 11:36:28 +01:00
Shane Kilkelly	47fdf3c5c9	Ensure that Filestore responds with 404 when a file does not exist.	2015-08-31 16:47:16 +01:00
Shane Kilkelly	8a631e70f7	Refactor, and log when we have a 'range' header.	2015-08-28 12:16:41 +01:00
Shane Kilkelly	387e96ab67	Refactor to make _get_range testable.	2015-08-28 12:02:50 +01:00