Commit graph

18049 commits

Author SHA1 Message Date
Brian Gough
92e0b0f04c add logging to each stage of archiving 2015-09-24 09:10:06 +01:00
Brian Gough
e683b0275a bug fix for clear archive in progress flag 2015-09-24 09:09:49 +01:00
Brian Gough
692e8c657c Revert to the default lock timeout now we have write barriers
Revert "increase lock timeouts for archiving"

This reverts commit 9eee1b383772adf058130d6e5eab409f57ce03cd.
2015-09-24 08:53:09 +01:00
Brian Gough
2ab1778dd9 move default value of lastVersion into function body 2015-09-23 16:31:33 +01:00
Brian Gough
bdf1d267f0 fix tests 2015-09-23 15:34:36 +01:00
Brian Gough
dc0044020f only archive entries older than the current update
to avoid a stale version of the current update ever being pulled back
from S3
2015-09-23 14:33:40 +01:00
Brian Gough
696a866b67 pause the stream of ops, not the download
the download is buffered in the lineStream so a lot comes out even
after pausing the S3 download.
2015-09-23 13:38:57 +01:00
Brian Gough
847a553344 prevent double archiving by checking if any inS3 field is already present 2015-09-23 13:29:32 +01:00
Brian Gough
e49f260507 allow rollback/locking by setting inS3:false when starting the archive process 2015-09-23 13:28:07 +01:00
Brian Gough
551e8334cf compressedUpdates are now never inserted with inS3
it is now always added later, and a new update is forced for any
addition to an archived update
2015-09-23 13:25:10 +01:00
Brian Gough
d6b827426c support forcing new compressed update in popLastCompressedUpdate
callback with a null update, passing the version as an additional
argument
2015-09-23 13:22:38 +01:00
Henry Oswald
27c02f5ef5 added some robots to the blocked list 2015-09-22 14:27:14 +01:00
Shane Kilkelly
30a63d1e42 Remove ng-click on output files, which was causing a double-request for download. 2015-09-22 10:11:03 +01:00
Henry Oswald
f1bf0c97ec make archive and unarchive parallelLimit to speed it up 2015-09-22 09:41:26 +01:00
Shane Kilkelly
3217b1d58f When serving output files, intelligently determine the appropriate content-type.
cherry pick 6fa3fda3ed28239cf3ac9720629f9707663aa197 from datajoy.
2015-09-21 16:59:35 +01:00
Brian Gough
dc35ef5cda fix tests for archiving 2015-09-21 13:24:06 +01:00
Brian Gough
a10dc4f898 Merge pull request #6 from heukirne/s3-archive
Add S3 archive track changes feature
2015-09-21 11:25:06 +01:00
Brian Gough
0e627c92d8 avoid clobbering global _ in loop 2015-09-18 16:26:05 +01:00
James Allen
195e22b45e added mongodb migrations
added imagemagick
make sure release is up to date
2015-09-17 14:07:50 +00:00
Henrique Dias
aa66c5ee8c improve size function 2015-09-17 10:41:53 -03:00
Henrique Dias
3f712c452a add size bulk limit 2015-09-17 09:23:13 -03:00
Henry Oswald
a427634cbb lock down metrics and spelling 2015-09-17 10:42:36 +01:00
Henry Oswald
cf28ee4f9b locked down packages 2015-09-17 10:34:27 +01:00
Henry Oswald
ddcfc3ee61 lock down smoke test and metrics version 2015-09-17 10:30:12 +01:00
Henry Oswald
011f628de8 lock down translations version 2015-09-17 10:26:29 +01:00
Henrique Dias
01cbfd5338 fix minor issues in acceptance test 2015-09-16 19:33:23 -03:00
Henrique Dias
a4575f178d fix unit test 2015-09-16 18:39:07 -03:00
Brian Gough
12c5098b48 fix lock in test 2015-09-16 16:50:36 +01:00
Henry Oswald
40f9a59507 remove extra logging in migration and change file paths in migration 2 2015-09-16 16:29:26 +01:00
Henry Oswald
ffd21cd080 remove process.exit from migrations 2015-09-16 16:26:51 +01:00
Brian Gough
7af5050370 add lock to unarchive doc 2015-09-16 16:18:36 +01:00
Brian Gough
18f06a3daf increase lock timeouts for archiving 2015-09-16 16:09:38 +01:00
Brian Gough
b4ffa7d57e share the document lock between archiving and packing 2015-09-16 16:03:55 +01:00
Brian Gough
9d39012b49 add error handler to each stage of download pipeline 2015-09-16 16:00:37 +01:00
Brian Gough
d9085a5e5e add error handler to each stage of upload pipeline 2015-09-16 16:00:25 +01:00
Henry Oswald
da42dff40e change timeout in migrations-1 to 0ms 2015-09-16 15:58:38 +01:00
Brian Gough
1c1b1d9595 log the case where there are no entries in the document history 2015-09-16 15:34:30 +01:00
Brian Gough
82d0f4fce8 make unarchive more responsive by downloading documents in parallel
unarchive is triggered interactively so we should try to make it
reasonably fast
2015-09-16 15:33:59 +01:00
Brian Gough
dfa0036507 pause stream while writing to mongo 2015-09-16 15:32:36 +01:00
Brian Gough
70200a9cf1 only log document ids, not document content
avoid filling the log with large documents
2015-09-16 15:31:43 +01:00
Henry Oswald
0ef14ce759 bring down migration timeout in delete doc lines to 0ms 2015-09-16 14:44:53 +01:00
Henry Oswald
856404099b renamed first migration to move_doc_lines_to_doc_collection 2015-09-16 14:22:54 +01:00
Henry Oswald
0218f9624e add the migrate task into grunt 2015-09-16 13:41:09 +01:00
Henry Oswald
24007affa7 change admin link to /admin/user
this is the page for the admin panel, if the panel is installed
then it will load, otherwise a redirect to register users has been
added in.
2015-09-16 12:17:58 +01:00
Brian Gough
d3dff28bea Merge remote-tracking branch 'origin/master' into heukirne-s3-archive 2015-09-15 15:19:43 +01:00
Henry Oswald
70611da833 added a null check to getRequestUserAndProject 2015-09-15 12:47:48 +01:00
Henry Oswald
5ffd76c670 Merge branch 'master' of https://github.com/sharelatex/web-sharelatex 2015-09-14 15:36:08 +01:00
Henry Oswald
9a72667e91 added word count UI feature. 2015-09-14 15:28:19 +01:00
Shane Kilkelly
5e157bcd16 Fix a case where a null user would cause displayName to crash. 2015-09-14 12:27:27 +01:00
Shane Kilkelly
77ac6c1baf Merge branch 'master' of github.com:sharelatex/web-sharelatex 2015-09-14 12:08:11 +01:00