Commit graph

521 commits

Author SHA1 Message Date
Christopher Hoskin
1217d8a80a Merge branch 'master' into csh-ho-docker-issue-1338-bulk-upgrade 2019-01-04 09:18:40 +00:00
Henry Oswald
2505be283a turn down logging, use logger.info for less important data 2018-12-13 12:46:10 +00:00
Henry Oswald
25e1ad7491 remove redis.realtime
real time pushes data into the document updater redis instance
2018-12-05 16:17:41 +00:00
Henry Oswald
9437cf8b02 explicity split redis config values 2018-12-05 15:16:55 +00:00
Henry Oswald
5f046ed329 install metrics route and bump metrics version 2018-12-05 12:35:46 +00:00
Brian Gough
aae1352519 ensure that project history is flushed when the project is deleted 2018-10-30 12:03:44 +00:00
Henry Oswald
a11d6dcc48 Merge branch 'master' into ho-docker 2018-10-15 15:17:40 +01:00
Alasdair Smith
c128e0ab1c Use setting instead of hard-coding port 2018-10-09 11:46:27 +01:00
Henry Oswald
ff673c71d5 set redis port via env var 2018-09-28 17:04:34 +01:00
Henry Oswald
93cf574251 upgrade build scripts 1.1.9 2018-09-28 15:13:47 +01:00
Henry Oswald
3db7377d74 add process.env["REDIS_PASSWORD"] 2018-09-28 15:11:17 +01:00
Brian Gough
537bd9a0c0 Merge pull request #36 from sharelatex/bg-limit-resync-requests-to-web
limit resync requests to web (connects to overleaf/sharelatex#775)
2018-08-16 12:25:16 +01:00
Brian Gough
910b27357d add more logging to resync operations 2018-08-16 11:14:11 +01:00
Brian Gough
aa013f0bee limit parallel resync doc requests to web 2018-08-16 11:13:11 +01:00
Brian Gough
fdcb806518 set a timestamp for the first entry in the projectHistory:Ops queue 2018-07-24 09:17:31 +01:00
Brian Gough
e471730efb Revert "Merge pull request #33 from sharelatex/bg-add-timestamp-marker-to-project-history-queue"
This reverts commit 8ec398b133aa75c45384dd7ceb630cd743f4a15c, reversing
changes made to e3a6c934cf53fd245d7d0df737463cc90f954885.
2018-07-23 16:05:04 +01:00
Brian Gough
33e898f671 update to 1.1.9 build scripts 2018-07-23 15:35:54 +01:00
Brian Gough
6237577f87 set a timestamp for the first entry in the projectHistory:Ops queue 2018-07-20 10:43:31 +01:00
Hayden Faulds
62a10f25f2 move tk call to before sandboxed module call 2018-06-05 13:54:30 +01:00
Henry Oswald
581963a4af Dockerised env. 1.1.3 build scripts 2018-05-23 13:52:20 +01:00
Brian Gough
7e526395fb fix check for "delete component" message 2018-04-30 09:56:01 +01:00
Brian Gough
37f431f148 also block "delete component" messages from sentry 2018-04-27 16:03:46 +01:00
Brian Gough
64670f272a stop OpRangeNotAvailable errors going to sentry 2018-04-27 15:45:28 +01:00
Hayden Faulds
af92ca70a1 coerce projectHistoryId to integer after reading from Redis 2018-04-23 15:19:06 +01:00
Hayden Faulds
1b63141e49 Revert "Revert "Add projectHistoryId to updates"" 2018-04-23 12:08:04 +01:00
Hayden Faulds
3cc20d6311 Revert "Add projectHistoryId to updates" 2018-04-23 11:41:29 +01:00
Hayden Faulds
9629d0a11e Merge pull request #26 from sharelatex/hof-deleted-project-history
Add projectHistoryId to updates
2018-04-23 11:01:53 +01:00
Alberto Fernández Capel
7d597fe59e Do not specify node version in .travis.yml
So travis picks instead the version from .nvmrc
2018-04-19 14:03:35 +01:00
Hayden Faulds
25c0be4eaa add projectHistryId to project structure updates 2018-04-13 15:22:05 +01:00
Hayden Faulds
a220794d32 add projectHistoryId to resync updates 2018-04-13 15:22:05 +01:00
Hayden Faulds
fb1852a593 cache projectHistoryId with doc in Redis 2018-04-13 15:22:05 +01:00
Brian Gough
dd0f8b880a change update to projectUpdate in project related methods 2018-03-16 10:54:12 +00:00
Brian Gough
779f00f912 add acceptance test for ordering of project structure changes 2018-03-15 16:27:14 +00:00
Brian Gough
3385d2640a fix structure ordering bug 2018-03-15 16:27:10 +00:00
Brian Gough
75a5428cbf update acceptance tests 2018-03-15 16:24:14 +00:00
Brian Gough
b3887fd984 update unit tests for incoming project versions 2018-03-15 16:24:11 +00:00
Brian Gough
0642e3c8c9 support project version on incoming requests 2018-03-15 16:15:10 +00:00
Hayden Faulds
4a5731440a Merge pull request #21 from sharelatex/hof-history-sync
resync project history endpoint
2018-03-14 10:40:59 +00:00
Hayden Faulds
1226f96fc3 extract ProjectHistoryRedisManager 2018-03-09 15:36:38 +00:00
Hayden Faulds
fc6ef6ea7a improve logging 2018-03-09 12:21:48 +00:00
Hayden Faulds
088ab6e7ba rename queueResyncDocContents -> queueResyncDocContent 2018-03-09 12:21:37 +00:00
Hayden Faulds
64d1fe13bc additional logging for project history resync 2018-03-07 16:24:22 +00:00
Hayden Faulds
a72d228354 add unit tests 2018-03-07 13:29:53 +00:00
Hayden Faulds
152a25e96d rename resyncProject -> resyncProjectHistory 2018-03-07 13:29:42 +00:00
Hayden Faulds
039f997e80 update project history resync url 2018-03-07 11:37:26 +00:00
Hayden Faulds
d46217be52 queue doc content sync updates 2018-03-02 15:02:27 +00:00
Hayden Faulds
52d6b710bd add ability to resync project structure 2018-03-02 15:01:53 +00:00
James Allen
939eaa2d4b Don't allow a document to be loaded without a pathname 2018-03-02 10:02:49 +00:00
Hayden Faulds
5a11332aa3 remove unecessary method wrapping 2018-02-22 10:17:59 +00:00
Hayden Faulds
0f87ae1f74 simplify comments 2018-02-22 10:16:29 +00:00
Hayden Faulds
ea0dd9700b reduce the number of times we flush project history 2018-02-22 10:01:05 +00:00
James Allen
3d5740fd7d Merge pull request #18 from sharelatex/msw-flush-project-ops
Flush project structure changes every 500
2018-02-16 10:30:00 +00:00
James Allen
5425fdef8a update to 1.0.1 build scripts 2018-02-15 16:54:21 +00:00
James Allen
657da70d45 Fix unit test namespacing from mocha upgrade 2018-02-15 16:28:58 +00:00
James Allen
5d1659457e Update to build 1.0.0 build scripts 2018-02-15 16:28:40 +00:00
James Allen
f6c79f3203 Fix and extend unit tests 2018-02-15 15:55:12 +00:00
Michael Walker
6c57317f8d Add a test that sending too few updates does not flush history 2018-01-31 11:41:08 +00:00
Michael Walker
241d1b27d5 Remove _shouldFlushHistoryOps wrapper 2018-01-31 11:27:40 +00:00
Michael Walker
772ee4083a Turn a unit test into an acceptance test 2018-01-31 11:17:56 +00:00
Michael Walker
bdaa2ffa77 Add an acceptance test for flushing project changes 2018-01-26 14:41:47 +00:00
Michael Walker
d2865f0650 Correctly count project_ops_length 2018-01-26 14:41:28 +00:00
Michael Walker
3028fb9c3d Only flush project updates when crossing the threshold 2018-01-24 11:57:03 +00:00
Michael Walker
24c74db0dc Flush project changes when queue hits limit 2018-01-23 11:18:20 +00:00
Michael Walker
3d313ebc53 Add method to get number of queued project updates 2018-01-23 11:10:54 +00:00
James Allen
ccf6cb1a18 Provide hosts as environment settings, add npm run start script and update coffeescript 2017-12-29 08:13:16 +00:00
Brian Gough
0b583dd4e6 fix log message 2017-12-20 13:27:10 +00:00
Brian Gough
a8b14552c0 allow flush to succeed when doc is already deleted 2017-12-19 12:27:57 +00:00
Brian Gough
41f15c4fa8 fix unit test 2017-12-14 14:46:27 +00:00
Brian Gough
947e18eccf Merge branch 'bg-limit-ops-in-lock' 2017-12-14 14:26:44 +00:00
James Allen
6abf9c227d Calculate and send doc length before each update 2017-12-11 17:26:32 +00:00
Hayden Faulds
bd45fcb5f8 Merge pull request #12 from sharelatex/hof-version-entity-creation
version entity creation; Connects to sharelatex/web-sharelatex-internal#104
2017-11-24 10:02:51 +00:00
Brian Gough
fb572670cd Merge pull request #11 from sharelatex/bg-use-separate-flush-count-for-projects
use a separate parameter for doc and project flush
2017-11-21 13:54:03 +00:00
Brian Gough
0bb9f147aa fix whitespace 2017-11-13 11:56:08 +00:00
Brian Gough
a4bbf5cf52 use a separate parameter for doc and project flush 2017-11-13 11:53:39 +00:00
Hayden Faulds
d0e56e5054 add acceptance tests for entity additions 2017-11-10 15:01:37 +00:00
Hayden Faulds
a3420b1236 version entity additions 2017-11-10 14:54:56 +00:00
Hayden Faulds
944e633bac add acceptance test for entity renaming 2017-11-08 09:20:38 +00:00
Hayden Faulds
7e86afe55e version file renames 2017-11-08 09:20:38 +00:00
Hayden Faulds
6d571e6d23 version document renames 2017-11-08 09:20:37 +00:00
Hayden Faulds
b8052e7612 allow settings.redis.project_history to be undefined 2017-10-25 13:56:38 +01:00
Hayden Faulds
5b7be93c79 Merge pull request #2 from sharelatex/hof-doc-pathname
Add metadata to updates for history
2017-10-25 13:13:00 +01:00
Brian Gough
e590543e42 Merge pull request #4 from sharelatex/bg-fix-listen-in-acceptance-tests
exit if mock servers fail to start
2017-10-25 09:13:48 +01:00
Brian Gough
f3098f7470 upgrade ioredis to 3.2.1 via redis-sharelatex 2017-10-24 12:20:14 +01:00
Brian Gough
367ed1e941 Merge pull request #8 from sharelatex/bg-upgrade-to-node-6x
upgrade to node 6.9.5
2017-10-24 11:11:23 +01:00
Brian Gough
56fd6e3d48 upgrade to node 6.9.5 2017-10-24 11:08:20 +01:00
Brian Gough
d9d58393d7 remove unused ioredis package
it is loaded from redis-sharelatex, not here.
2017-10-24 10:39:00 +01:00
Hayden Faulds
d1f0c4ceae add acceptance tests for project history API 2017-10-24 09:06:23 +01:00
Brian Gough
14f3d22071 upgrade to latest async ^2.5.0 2017-10-23 16:02:24 +01:00
Brian Gough
20655b6eeb remove unused requires of async 2017-10-23 15:59:32 +01:00
Hayden Faulds
04ecd1e7ee set pathname in acceptance tests 2017-10-23 15:32:25 +01:00
Hayden Faulds
be41a1614d decorate ops with pathname and doc_length 2017-10-23 15:32:25 +01:00
Hayden Faulds
962520fca8 flush track-changes and project-history in HistoryManager 2017-10-23 15:32:25 +01:00
Hayden Faulds
d003aef31c conditionally enqueue history updates for project 2017-10-23 15:32:25 +01:00
Hayden Faulds
c27df0bfef split apart multi and rclient in tests 2017-10-23 14:24:23 +01:00
Hayden Faulds
81f998afe9 return pathname from DocumentManager.getDoc 2017-10-23 14:23:21 +01:00
Hayden Faulds
748315aadc handle pathname in DocumentManager.getDoc 2017-10-23 14:23:21 +01:00
Hayden Faulds
cbdace7386 store pathname in Redis cache 2017-10-23 14:23:21 +01:00
Hayden Faulds
05b93a629a return pathname from PersistenceManager 2017-10-23 14:17:14 +01:00
Brian Gough
02d3d1bd17 fix unit tests
timekeeper must be called before SandboxedModule.require
2017-10-23 14:02:53 +01:00
Brian Gough
7c2a28c161 fix use of timekeeper in unit tests 2017-10-23 13:33:14 +01:00