github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
23871 commits 3 branches 7 tags 206 MiB
Commit graph

675 commits

Author SHA1 Message Date
Jakob Ackermann
7fa9061015 [misc] stop processing requests as we detect a client disconnect 
v2 expose `client.connected`; v0 exposes `client.disconnected`

(cherry-picked from commit a9d70484343ca9be367d45bf7bb949e4be449647)
 2020-06-05 11:32:13 +01:00
Jakob Ackermann
0840700ffd [Router] validate the callback argument 2020-06-05 10:59:01 +01:00
Jakob Ackermann
c6d08647c7 [misc] socket.io: use a secondary publicId for public facing usages 2020-06-04 17:18:07 +01:00
Brian Gough
f973b377f0 update node to v10.21.0 2020-06-03 09:12:21 +01:00
Tim Alby
94d57f50c0 add fake lint and format targets 
Highly hacky!
Lint and format steps are coming very soon thanks to the decaf, but in the
meantime we need steps to pass CI. Updating the build scripts after the
decaf will undo this change.
 2020-05-28 16:09:27 +02:00
Jakob Ackermann
d13acb8ca3 [ChannelManager] port related and still mostly valid test from v2 
I skipped the bulk of verifyConsistentBehaviour tests which are not
 valid for the new implementation -- there is no optimization and
 always cleanup.
 2020-05-15 18:39:33 +02:00
Jakob Ackermann
41debfae0f [ChannelManager] rework (un)subscribing to redis 
- send a subscribe request on every request
- wait for a pending unsubscribe request before subscribing
- wait for a pending subscribe request before unsubscribing

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
 2020-05-15 18:34:33 +02:00
Jakob Ackermann
1095851dfe [misc] test/unit: fix typos and assertion of error messages 
Sinon does not check the contents of the passed error when checked via
 sinon.stub().calledWith.
```
callback = sinon.stub()
callback(new Error("some message"))
  .calledWith(new Error("completely different message"))
  === true
```

Cherry-pick plus an additional patch for the joinProject bail-out.
(cherry picked from commit d9570fee70701a5f431c39fdbec5f8bc5a7843fe)
 2020-05-15 14:46:05 +02:00
Jakob Ackermann
17d04b9041 [misc] bump sinon to 2.x for promise support with stubs 
(cherry picked from commit 3c92b937f9430175d7c51660e03c507635448e88)
 2020-05-15 12:01:22 +02:00
Jakob Ackermann
b713beb7f0 Merge pull request #135 from overleaf/jpa-skip-leave-project-for-invalid-clients 
[WebsocketController] skip leaveProject when joinProject didn't complete
 2020-05-13 15:34:41 +02:00
Jakob Ackermann
55af5e502f [WebsocketController] skip leaveProject when joinProject didn't complete 
Also drop dead code:
 - user_id bailout

   There is a check on a completed joinProject call now. It will always
    set a user_id, see Router.coffee which has a fallback `{_id:"..."}`.

 - late project_id bailout

   WebsocketLoadBalancer.emitToRoom will not work without a project_id.
   We have to bail out before the call.
 2020-05-12 17:15:08 +02:00
Jakob Ackermann
684cb3903c [WebsocketController] handle redis subscribe error on joinProject 
joinProject should not complete when the redis pub/sub subscribe request
 failed.
 2020-05-12 13:03:50 +02:00
Brian Gough
5c28da1031 add metric for pendingUpdates queue 2020-04-07 11:53:54 +01:00
Brian Gough
5765884f38 Merge branch 'master' into jpa-pub-sub-metrics 2020-04-07 11:46:23 +01:00
Henry Oswald
8711abdb66 bump redis to 1.0.12 2020-03-31 14:04:33 +01:00
Jakob Ackermann
56628a16c6 [misc] track redis pub/sub payload sizes on publish 2020-03-30 11:31:44 +02:00
Jakob Ackermann
a9b8e9be3b [misc] upgrade metrics-sharelatex to 2.6.2 2020-03-30 10:47:01 +02:00
Jakob Ackermann
69569e3571 [misc] config: add headroom for JSON serialization in maxUpdateSize 2020-03-24 16:21:29 +01:00
Jakob Ackermann
af53d3b603 [misc] skip duplicate JSON serialization for size check 2020-03-24 11:22:28 +01:00
Jakob Ackermann
cb675d38c2 [misc] SafeJsonParse: align the size limit with the frontend->rt limit 
frontend -> real-time and doc-updater -> real-time should be in sync.
Otherwise we can send a payload to doc-updater, but can not receive the
 confirmation of it -- and the client will send it again in a loop.

Also log the size of the payload.
 2020-03-24 09:14:15 +01:00
Jakob Ackermann
15244a54be [misc] WebsocketController: limit the update size to 7mb 
bail out early on -- especially do not push the update into redis for
 doc-updater to discard it.

Confirm the update silently, otherwise the frontend will send it again.
Broadcast a 'otUpdateError' message and disconnect the client, like
 doc-updater would do.
 2020-03-24 09:12:12 +01:00
Jakob Ackermann
d320c2d5f3 [misc] let proxys observe an upcoming shutdown before starting to drain 
Otherwise clients may be routed to the same pod upon reconnecting.
 2020-03-17 16:41:48 +01:00
Jakob Ackermann
2b1c5bf436 Merge pull request #100 from overleaf/jpa-dependencies-cleanup 
[misc] cleanup dependencies
 2020-02-14 10:56:11 +01:00
Jakob Ackermann
43013e0820 [misc] cleanup unused dependency on mongo 2020-02-12 14:44:01 +01:00
Jakob Ackermann
902b4fca46 [misc] rename npm-shrinkwrap.json to package-lock.json and run npm i 2020-02-12 14:39:53 +01:00
Brian Gough
1ad8315437 remove unused .travis.yml file 2020-02-12 12:37:00 +00:00
Jakob Ackermann
24d46e9d4b [misc] update the build scripts to 1.3.5 2020-02-11 12:27:56 +01:00
Brian Gough
9a4124ee11 Merge pull request #97 from overleaf/bg-update-node 
Update to node:10.19.0 docker image
 2020-02-10 10:00:43 +00:00
Brian Gough
98b29889bd Merge pull request #95 from overleaf/bg-revert-unnecessary-changes 
Revert debugging PRs for node 10 upgrade
 2020-02-10 10:00:17 +00:00
Brian Gough
e0e2090a42 update node version in nvmrc and buildscripts 2020-02-07 14:41:12 +00:00
Brian Gough
04a9d66784 use public node:10.19.0 image 2020-02-07 14:15:48 +00:00
Brian Gough
abe4d1d525 update to gcr.io/overleaf-ops/node:10.19.0 2020-02-06 03:34:30 +00:00
Brian Gough
64bd739a87 Revert "Merge pull request #91 from overleaf/spd-trycatch-all-the-things" 
This reverts commit 2bf7f14f9d050c58f141f465633bb6e274b903dd, reversing
changes made to 989240812532ca43a52513339f4dda8f44a80a64.
 2020-02-05 10:05:36 +00:00
Brian Gough
4ec82b1baa upgrade to local node:10.18.1 image 2020-02-04 15:02:15 +00:00
Simon Detheridge
4102aa0580 Add more detail to metric 2020-02-04 14:03:56 +00:00
Simon Detheridge
8e45a62e32 Handle ECONNRESET in the same way as EPIPE 2020-02-04 13:58:45 +00:00
Simon Detheridge
7663e9689e Merge pull request #91 from overleaf/spd-trycatch-all-the-things 
Add try/catch around all client emissions
 2020-02-04 13:21:49 +00:00
Simon Detheridge
fbff3fe727 Don't shut down on uncaught EPIPE 2020-02-04 12:56:43 +00:00
Simon Detheridge
9fd4699098 Merge remote-tracking branch 'origin/bg-avoid-emit-when-client-not-connected' into spd-trycatch-all-the-things 2020-02-04 12:35:35 +00:00
Simon Detheridge
216a977922 Add try/catch around all client emissions 2020-02-04 12:13:03 +00:00
Brian Gough
ebb83e4633 use diconnected property, not connected 2020-02-04 11:59:25 +00:00
Brian Gough
7380d523d5 avoid emitting when client not connected 
the emit is happening asynchronously after the client list is computed,
so clients may have disconnected in the intervening time.
 2020-02-04 11:39:37 +00:00
Brian Gough
e263d37476 pass the signal correctly to the shutdown handler 2020-02-04 11:14:53 +00:00
Brian Gough
1fc8cc44c3 log shutdown messages as warnings 2020-02-04 11:14:14 +00:00
Simon Detheridge
c7e2b99a7b Update hybi-16 patch to work with socket.io 0.9.19 2020-02-04 10:43:06 +00:00
Simon Detheridge
ef852dfa33 Update socket.io to latest patch release 2020-02-04 10:32:54 +00:00
Brian Gough
4f94110225 Merge pull request #84 from overleaf/spd-catch-errors 
Bump to Node 10 and add error handlers for socket.io
 2020-02-03 14:59:04 +00:00
Brian Gough
49a8e1214b use a separate field for client errors 2020-02-03 14:47:45 +00:00
Brian Gough
e04b6e1e49 Update app/coffee/Router.coffee 
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
 2020-02-03 14:46:14 +00:00
Shane Kilkelly
e63c6f4395 Merge pull request #87 from overleaf/sk-restricted-users 
Filter "comments" if restricted user.
 2019-10-31 10:21:11 +00:00
Shane Kilkelly
6df88ebc49 Filter "comments" if restricted user. 2019-10-30 13:54:40 +00:00
Shane Kilkelly
403caa65e8 Revert "Revert "Track the isRestrictedUser flag on clients"" 
This reverts commit 651e392a7c644403f199e1b03e7494b61ce71d0c.
 2019-10-30 13:52:36 +00:00
Nate Stemen
9a838bd071 bump build script to 1.1.24 2019-10-25 13:23:13 -04:00
Nate Stemen
3dc7c357a5 add public link to contributing docs 2019-10-25 13:22:58 -04:00
Simon Detheridge
925a8651c1 Revert "Track the isRestrictedUser flag on clients" 2019-10-22 10:17:38 +01:00
Simon Detheridge
c31c2d292d Merge pull request #81 from overleaf/sk-restricted-users 
Track the `isRestrictedUser` flag on clients
 2019-10-22 09:45:00 +01:00
Simon Detheridge
ce366fdbee Bump Dockerfile to node 10 2019-10-17 12:46:07 +01:00
Simon Detheridge
7543f2fcbd Catch errors from socket.io and attempt graceful cleanup 2019-10-17 12:45:56 +01:00
Brian Gough
dff4d66209 Merge pull request #83 from overleaf/bg-upgrade-ioredis 
upgrade ioredis to v4.14.1
 2019-10-14 11:18:45 +01:00
Brian Gough
f028148fe2 upgrade ioredis to v4.14.1 2019-10-14 11:10:20 +01:00
Shane Kilkelly
2cc2be3d9c send messages to clients with concurrency of 2 2019-10-11 10:01:21 +01:00
Shane Kilkelly
06aa578bdc Make it an error when we get no data from joinProject 2019-10-11 09:57:16 +01:00
Simon Detheridge
85b23d7da7 Add maxRetriesPerRequest setting for redis 2019-10-10 16:56:58 +01:00
Shane Kilkelly
df6cd4a054 Also block getConnectedUsers for restricted users. 
Plus refactor to use a pass list instead of a deny list.
 2019-10-04 13:41:49 +01:00
Shane Kilkelly
6765d03339 Track the isRestrictedUser flag on clients 
Then, don't send new chat messages and new comments to those restricted clients.
We do this because we don't want to leak private information (email addresses
and names) to "restricted" users, those who have read-only access via a
shared token.
 2019-10-04 10:30:24 +01:00
Simon Detheridge
21e294c6eb Generate retryable error when hitting rate limits in web 2019-09-02 11:27:04 +01:00
Brian Gough
fe2e7b3065 minimal fix for undefined connected users 2019-08-16 10:07:30 +01:00
Henry Oswald
38ed780d80 add log line to draining 2019-08-15 14:41:22 +01:00
Brian Gough
b0f0fb64ac clean up unused variable, convert setting to number 2019-08-15 09:48:42 +01:00
Brian Gough
a7a161556f Merge branch 'bg-status-on-shutdown' 2019-08-15 09:42:00 +01:00
Brian Gough
fa94e3d5e3 Merge pull request #69 from overleaf/ho-drain-connections-timewindow 
add shutdownDrainTimeWindow, drains all connections within time range
 2019-08-15 09:22:10 +01:00
Brian Gough
022e47b5c8 Merge pull request #73 from overleaf/bg-connected-client-metrics 
add connected client count metric
 2019-08-15 09:20:03 +01:00
Brian Gough
3552fa40c2 Merge pull request #72 from overleaf/bg-refresh-client-list 
refresh client list
 2019-08-15 09:19:20 +01:00
Henry Oswald
78629610d5 add health check endpoint and http route logger 2019-08-14 15:38:02 +01:00
Brian Gough
f13e66b453 fix client count so that result is zero when all clients have left 2019-08-14 15:34:23 +01:00
Brian Gough
8270c14d86 add connected client count metric 2019-08-14 15:22:03 +01:00
Brian Gough
d57b229e17 update tests 2019-08-14 13:03:14 +01:00
Brian Gough
d3171e4e2e remove unwanted argument 2019-08-14 13:03:06 +01:00
Henry Oswald
2ae4c8c174 Merge branch 'ho-drain-connections-timewindow' of github.com:overleaf/real-time into ho-drain-connections-timewindow 2019-08-14 11:52:22 +01:00
Henry Oswald
4a984f533e remove forceDrainMsDelay 
as soon as a pod is marked as being killed we should start draining
 2019-08-14 11:51:25 +01:00
Brian Gough
20d442120f notify docupdate if the flush is from a shutdown 2019-08-13 17:36:53 +01:00
Brian Gough
7db882f339 fix unit tests 2019-08-13 17:26:49 +01:00
Brian Gough
0708f717fd reject connections when shutdown in progress 
send a message to the client to reconnect immediately
 2019-08-13 16:59:15 +01:00
Brian Gough
53431953fc make shutDownInProgress available via settings 2019-08-13 16:56:48 +01:00
Brian Gough
b3e5709b64 enforce a minimum drain rate 2019-08-13 16:15:30 +01:00
Henry Oswald
00cca29d9e add shutdownDrainTimeWindow, drains all connections within time range 2019-08-13 14:21:47 +01:00
Brian Gough
5b54d36b37 fail readiness check when shutting down 2019-08-13 10:41:35 +01:00
Brian Gough
2000f478a7 refresh the client list on demand 2019-08-13 10:40:03 +01:00
Miguel Serrano
49c7bde799 Merge pull request #66 from overleaf/msm-patched-eventemitter-socketio-node7 
Patched EventEmitter for socket.io compatibility with Node >= 7
 2019-08-12 11:30:11 +02:00
Brian Gough
478a727c61 ignore spurious requests to leave other docs 2019-07-29 15:19:08 +01:00
Brian Gough
04a171171f fix async behaviour of join/leave 2019-07-29 11:54:02 +01:00
mserranom
cf0df28f4c Patched EventEmitter for socket.io compatibility with Node >= 7 2019-07-25 09:22:24 +00:00
Brian Gough
22d722f3e8 add metric for RoomEvents listeners 2019-07-24 16:25:45 +01:00
Brian Gough
277ec71a5b subscribe to doc updates before requesting doc content 2019-07-24 15:49:29 +01:00
Brian Gough
1c74cbbc4e add comments 2019-07-24 15:49:29 +01:00
Brian Gough
273af3f3aa refactor subscribe resolution 2019-07-24 14:30:48 +01:00
Brian Gough
e14a94906a update naming from Set -> Map 2019-07-24 14:18:15 +01:00
Brian Gough
cb53bfafd6 remove unnecessary require 2019-07-24 09:52:31 +01:00
Brian Gough
61b3a000b4 fix whitespace 2019-07-24 09:52:20 +01:00
Brian Gough
159b39c491 ensure redis channel is subscribed when joining room 2019-07-23 17:02:09 +01:00
Brian Gough
84e6ff616f whitespace fix 2019-07-22 12:25:41 +01:00
Brian Gough
bb629c27a1 rename unit test ChannelManager to ChannelManagerTests 2019-07-22 11:28:49 +01:00
Brian Gough
1afebd12a1 unit tests 2019-07-22 11:23:43 +01:00
Brian Gough
92e6910180 cleanup 2019-07-22 11:23:33 +01:00
Brian Gough
8c7b73480f upgrade sinon to 1.17.7 for onCall support 2019-07-22 11:23:02 +01:00
Brian Gough
9f7df5f10c wip unit tests 2019-07-19 11:58:40 +01:00
Brian Gough
a538d10488 extend comment re disconnection 2019-07-19 08:56:38 +01:00
Brian Gough
616014e05d add comment about automatically leaving rooms 2019-07-19 08:50:43 +01:00
Brian Gough
40353a410f fix unit tests 2019-07-19 08:49:57 +01:00
Brian Gough
3bf5dd5d6b clarify errors for subscribe/unsubscribe 2019-07-18 14:25:25 +01:00
Brian Gough
f6f6f549d9 don't publish on individual channels until explicitly set 2019-07-18 12:55:23 +01:00
Brian Gough
804f4c2bd2 listen on separate channels for each project/doc 2019-07-18 12:55:23 +01:00
Brian Gough
ae512dc9fb Merge pull request #62 from overleaf/bg-patch-socket-io-frame-bug 
monkeypatch socket.io to fix frame handler in v0.9.16
 2019-07-17 13:48:04 +01:00
Brian Gough
9ecce32ff9 Merge pull request #63 from overleaf/bg-log-out-of-order-events 
log out of order events now that the rate is lower
 2019-07-17 13:47:45 +01:00
Brian Gough
0c6ba4c1a8 monkeypatch socket.io to fix frame handler in v0.9.16 2019-07-16 14:02:52 +01:00
Brian Gough
8a7804f0a7 make event order check a configuration setting 2019-07-15 13:45:34 +01:00
Brian Gough
24a4709cff log out of order events now that the rate is lower 2019-07-15 11:14:48 +01:00
Brian Gough
e632f9f29d only create per-client metrics when there are multiple redis clients 2019-07-11 11:35:48 +01:00
Brian Gough
80f8f2465e remove unused pubsub client 2019-07-11 11:10:33 +01:00
Brian Gough
689a75f397 add logging for redis clients at start up 2019-07-09 14:18:39 +01:00
Brian Gough
dd54789e2b fix build problems 2019-07-09 12:20:59 +01:00
Brian Gough
580b100362 only publish to one redis client in WebsocketLoadBalancer 
but listen to all of them
 2019-07-09 12:03:13 +01:00
Brian Gough
999cbd8ee6 add a per-client metric 2019-07-09 12:01:58 +01:00
Brian Gough
cb289f2dec make redis client list dynamic based on settings 2019-07-09 11:45:00 +01:00
Brian Gough
b5f9bc422b support multple redis instances for pubsub 2019-07-08 15:56:25 +01:00
Henry Oswald
520857cf7a simplify redis continual traffic 
we can't send double health check events to same redis, it causes
health check duplicate errors. Commit just sends health check data to
pub sub pair and then sends non health check traffic to cluster to keep
the connection open
 2019-07-08 12:07:28 +01:00
Henry Oswald
1038c5cd0d send health check to pubsub channel and use different var name 2019-07-08 11:53:42 +01:00
Henry Oswald
eadef7b133 Merge pull request #59 from overleaf/ho-redis-natmap-sentinel 
Move pubsub to seperate connection
 2019-07-08 11:18:12 +01:00
Henry Oswald
9953c933ee Update package.json 2019-07-08 11:18:02 +01:00
Henry Oswald
487865fad3 Merge pull request #57 from overleaf/sk-dep-upgrades-2019-06 
update logger and metrics
 2019-07-08 11:17:29 +01:00
Henry Oswald
42e5d2fb6e Update app.coffee 2019-07-08 11:17:08 +01:00
Henry Oswald
8042a415ec move pubsub traffic over to a pubsub redis connection string 2019-07-04 13:38:39 +01:00
Henry Oswald
b268285ff6 bump redis-sharelatex (and io redis) to 1.0.9 2019-07-02 16:55:07 +01:00
Henry Oswald
670ce61da1 require underscore 2019-07-02 15:36:17 +01:00
Henry Oswald
88b75b8baa send health check data to all redis backends 2019-07-02 14:56:50 +01:00
Henry Oswald
92f60690f3 add redis set 2019-07-02 14:46:58 +01:00
Henry Oswald
922a1a1aba bump redis driver 2019-07-02 12:04:19 +01:00
Shane Kilkelly
283a6066d4 update logger and metrics 2019-06-21 11:26:28 +01:00
miguel
46dfe56b05 Downgraded unathorised log to warning 2019-06-21 07:30:12 +02:00
Brian Gough
a579a83389 Merge pull request #53 from overleaf/em-doc-updater-404s 
Do not log error on doc updater 404
 2019-06-03 10:55:04 +01:00
Brian Gough
d4f003fe0f Merge pull request #55 from overleaf/bg-allow-fractional-drain-rate 
allow fractional drain rate
 2019-06-03 10:53:57 +01:00
Brian Gough
9da98614c7 Merge pull request #54 from overleaf/bg-use-background-flush-on-disconnect 
use background flush on disconnect
 2019-06-03 10:53:43 +01:00
Brian Gough
40f3456b1c update unit test 2019-06-03 10:34:59 +01:00
Brian Gough
63f052192e use background flush on disconnect 2019-06-03 09:42:12 +01:00
Eric Mc Sween
acf850bce9 Do not log error on doc updater 404 
Document updater 404s are not indicative of a problem, but just of the client
trying to connect to a deleted document, which is easy to trigger. We
log a warning instead.
 2019-05-31 17:32:21 -04:00
Henry Oswald
421a914e72 log out when health check manager fails a check 2019-05-31 09:15:49 +01:00
Henry Oswald
2c9b222437 fix breaking tests from using ioredis with inbuilt promises 
because a promise is returned from ioredis it errors in mocha as it
can't take a promise and a callback
 2019-05-30 10:58:05 +01:00
Henry Oswald
47e0cb44ce bump redis to 1.0.8 2019-05-30 10:29:34 +01:00
Brian Gough
74db743ffa allow fractional drain rate 2019-05-24 15:23:01 +01:00
Henry Oswald
78372119f8 Revert "Csh issue 1118 node 10.15.3" 2019-05-24 15:21:48 +01:00
Brian Gough
20d5cc69a4 filter invalid updates 2019-05-24 10:19:02 +01:00
Christopher Hoskin
fd8ac10bf7 Merge pull request #46 from das7pad/misc/node7plus-support 
[misc] add support for node 7+
 2019-05-13 11:58:27 +01:00
Christopher Hoskin
20683f3094 Update Node from 6.15.1 to 10.15.3 2019-05-13 11:55:42 +01:00
Christopher Hoskin
5f045d9792 Update buildscripts to 1.1.21 2019-05-13 11:54:37 +01:00
Timothée Alby
8dc41da0ba update Git URL in Jenkinsfile 2019-05-07 17:45:08 +01:00
Jakob Ackermann
79a314d5fd [misc] disable the flash transport 
We do not use flash on the website and the policy file provider is not
 compatible with node7+.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2019-05-01 01:48:01 +02:00
Jakob Ackermann
6601e94db7 [misc] bump socket.io to 0.9.19 for node7+ support 
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
 2019-05-01 01:48:00 +02:00
Brian Gough
2828cc3d7f Merge pull request #45 from sharelatex/bg-update-request 
update request module
 2019-04-15 15:37:14 +01:00
Brian Gough
2277872022 update package.json to current version of request 2019-04-15 15:26:04 +01:00
Brian Gough
15c7c911f9 update request module 2019-04-15 14:46:58 +01:00
Brian Gough
2dbdcf5bc8 add health check to pubsub channels 2019-04-15 14:17:55 +01:00
Brian Gough
e72acacf17 downsample logging 2019-04-11 16:25:42 +01:00
Brian Gough
9eee098af4 Merge pull request #43 from sharelatex/bg-update-ioredis 
update to redis-sharelatex v1.0.6 for latest ioredis
 2019-04-11 15:50:56 +01:00
Brian Gough
cefdd15c5e update to redis-sharelatex v1.0.6 for latest ioredis 2019-04-11 15:49:19 +01:00
Brian Gough
ef9e97e7d1 add metric for applied-ops events 2019-04-11 15:39:28 +01:00
Brian Gough
6374a641d5 fix unit tests 2019-04-11 15:07:42 +01:00
Brian Gough
c6225d614e add /debug/events endpoint 2019-04-11 15:00:25 +01:00
Brian Gough
2a31139254 log and skip duplicate events 2019-04-11 12:53:43 +01:00
Brian Gough
893515e83f handle duplicate entries in io.sockets.clients 2019-04-09 14:48:00 +01:00
Brian Gough
936311f1fb fix eventlogger tests to use name with dashes 2019-03-22 11:20:38 +00:00
Brian Gough
efa83b4cde stub out eventlogger in unit test 2019-03-22 11:19:31 +00:00
Brian Gough
6c71ae172b return out of order events for consistency 2019-03-22 11:19:08 +00:00
Brian Gough
c5a09fcf0a add comment about downsampling 2019-03-22 11:18:34 +00:00
Brian Gough
695472a8aa fix event id parsing to allow for dashes in keys 2019-03-22 11:18:19 +00:00
Brian Gough
1ab5e52699 down-sample valid events by 1000 2019-03-21 15:52:53 +00:00
Brian Gough
e91b967bdb use per-channel event metrics 2019-03-21 14:59:39 +00:00
Brian Gough
8c82faa966 check order of messages on applied-ops channel 2019-03-21 14:59:39 +00:00
Brian Gough
57c5ec14bd check for a valid counter value in event log 2019-03-21 14:59:19 +00:00
Brian Gough
9b25374cd3 use time-based cleaning of event log 2019-03-21 14:59:19 +00:00
Brian Gough
befe4be517 add check for duplicate events 2019-03-19 14:22:11 +00:00
Henry Oswald
b9e3853a47 add sentry into settings.defaults 2019-03-05 17:33:52 +00:00
Henry Oswald
5119db43a7 Merge pull request #37 from sharelatex/ho-continual-pub-sub-traffic 
setup continualPubsubTraffic to keep pub sub happy
 2019-02-19 11:28:06 +00:00
Henry Oswald
26e903f384 setup continualPubsubTraffic 
this keeps the pub sub channel ticking along happily
 2019-02-15 17:04:49 +00:00
Chrystal Griffiths
507c4e5ce2 Fix test to accommodate changes to anonymous users 2019-02-15 15:18:17 +00:00
Chrystal Maria Griffiths
0b88a63444 Merge pull request #35 from sharelatex/cmg-anonymous-display-name 
Nameless logged in user labelled as anonymous
 2019-02-15 14:53:18 +00:00
Henry Oswald
6fb6086ba1 remove console.log 2019-02-12 14:28:42 +00:00
Chrystal Griffiths
26acdfd072 Add comment explaining why not sending anon data up 2019-02-12 14:06:59 +00:00
Chrystal Griffiths
bb06f82e04 Still send cursorData for logged in users 2019-02-12 14:00:47 +00:00
Chrystal Griffiths
2ec760403f Revert to method not sending cursorData because of duplication 2019-02-11 11:52:14 +00:00
Chrystal Griffiths
cb12e1c6f6 Send an empty string for every nameless user 2019-02-08 15:39:51 +00:00
Henry Oswald
1fc1b4206e add shutDownInProgress check into sig listening 2019-02-07 13:57:38 +00:00
Henry Oswald
d85bf5cedb remove extra logging line 2019-02-06 15:26:12 +00:00
Henry Oswald
37d1c96051 add logging 2019-02-04 16:59:36 +00:00
Henry Oswald
a07e516f65 update redis key 2019-02-04 10:22:26 +00:00
Henry Oswald
0939a558d7 point real time to web api 2019-02-01 19:27:56 +00:00
Henry Oswald
6cdfd5b659 added http auth to config 2019-02-01 17:25:45 +00:00
Henry Oswald
23e0ce678e call app real-time 2019-01-31 15:33:11 +00:00
Henry Oswald
2e46cfb3eb bump node redis to 1.0.5 2019-01-25 16:38:42 +00:00
Henry Oswald
08e48afcb9 more logging 2019-01-25 10:57:36 +00:00
Henry Oswald
338bee061c remove the cluster key as well 2019-01-25 10:50:34 +00:00
Henry Oswald
01d3d9d13e remove natmap 2019-01-25 10:37:00 +00:00
Henry Oswald
2b1f67b6fa remove natmap from defaults config 2019-01-25 10:30:31 +00:00
Henry Oswald
b193045a2a log out real time settings 2019-01-25 10:04:26 +00:00
Henry Oswald
e1b742c215 add logging 2019-01-23 10:15:00 +00:00
Henry Oswald
73b2c1ec05 auto wrap redis from env vars 2019-01-23 10:11:26 +00:00
Henry Oswald
397b65abf7 put settings on all redis's 2019-01-22 17:06:43 +00:00
Henry Oswald
52014c851b bump redis 2019-01-22 17:00:48 +00:00
Henry Oswald
2cd3e6f18f added logging in settings 2019-01-22 16:49:22 +00:00
Henry Oswald
150483eece put redis keys back in for web 2019-01-22 16:44:39 +00:00
Henry Oswald
9517aac86d bump redis version 2019-01-22 16:02:13 +00:00
Henry Oswald
e3618acf20 mvp2 for redis-cluster 2019-01-22 15:53:24 +00:00
Christopher Hoskin
3288d6d321 Add **/*.map to .gitignore 2019-01-08 14:28:10 +00:00
Christopher Hoskin
4a3711aba8 Bump settings to v1.1.0 2019-01-08 14:26:05 +00:00
Christopher Hoskin
c6fa764a6a Add app.js.map to .gitignore 2019-01-03 18:04:50 +00:00
Christopher Hoskin
b7e8bf3c4c Bump logger to 1.5.7 2019-01-03 17:58:10 +00:00
Christopher Hoskin
cda65ad1ca Bump build scripts to 1.1.10 2019-01-03 16:21:33 +00:00
Christopher Hoskin
cd362f22be Move to v2 metrics 2019-01-03 16:17:31 +00:00
Christopher Hoskin
ba24e2686e Merge branch 'master' into csh-ho-docker-issue-1338-bulk-upgrade 2019-01-03 16:08:40 +00:00
Christopher Hoskin
9e7f84cd0d Bump node version to 6.15.1 2018-12-27 08:23:18 +00:00
Henry Oswald
bbd88e75eb fix broken tests 2018-12-05 15:41:12 +00:00
Henry Oswald
7a7f1aed91 remove console.log 2018-12-05 15:39:27 +00:00
Henry Oswald
d4e8bc1d4b remove metrics.set as we don't support it atm 2018-12-05 15:29:23 +00:00
Henry Oswald
5ad7482385 update config to take explicit redis configs 2018-12-05 14:43:45 +00:00
Henry Oswald
cdf605e171 inject metrics root and bump lib to 2.0.8 2018-12-05 14:01:15 +00:00
Henry Oswald
78b779a338 try different metrics 2018-12-04 16:21:20 +00:00
Henry Oswald
258617fbd4 bump metrics 2018-12-04 15:58:42 +00:00
Henry Oswald
fa3dd21d0c Merge branch 'master' into ho-docker 2018-12-04 15:22:03 +00:00
Henry Oswald
b834049eeb improve delay ms logging 2018-12-04 14:49:34 +00:00
Henry Oswald
05611de15e use FORCE_DRAIN_MS_DELAY 2018-12-04 14:31:07 +00:00
Henry Oswald
2418e5db57 use delayExitUntilDrained 2018-12-04 14:22:06 +00:00
Henry Oswald
8073cdea75 improve logging 2018-12-04 14:17:59 +00:00
Henry Oswald
7f2decae4d start drain after 3 hours 2018-12-04 14:06:39 +00:00
Henry Oswald
9a851d6cce mvp for safe shutdown 2018-12-04 13:47:04 +00:00
Henry Oswald
fc2d2405f4 log out io clients 2018-12-04 13:39:16 +00:00
Henry Oswald
2282518c90 print out req.query 2018-12-04 13:26:47 +00:00
Henry Oswald
64f3d32c6c log out status calls 2018-12-04 13:22:09 +00:00
Henry Oswald
034e627a8a bump metrics and shrinkwrap 2018-12-04 13:14:50 +00:00
Henry Oswald
57cd7c734a add debugging 2018-12-03 14:34:46 +00:00
Henry Oswald
ca2af0af32 make cookies configurable via env vars 2018-12-03 14:05:03 +00:00
Henry Oswald
bbc11ae1b5 add DOCUMENT_UPDATER_HOST as option 2018-10-19 19:42:33 +01:00
Henry Oswald
477c446ea8 add web-api host as ana option 2018-10-19 19:31:12 +01:00
Henry Oswald
63d7bb501b return a 200 for root path for google health check 2018-10-19 16:44:40 +01:00
Alasdair Smith
4a495dbd1f Use setting instead of hard-coding port 2018-10-09 11:53:49 +01:00
Henry Oswald
ff8afb6c24 pass redis port 2018-10-03 15:01:31 +01:00
Henry Oswald
e322e46900 update build scripts 2018-10-03 14:21:02 +01:00
Henry Oswald
3a89bf0957 pass redis password through as env var 2018-10-03 14:20:20 +01:00
Henry Oswald
327fa79f1c update build scripts, acceptence tests now pass. includes dockerfile 1.1.3 2018-05-23 15:00:46 +01:00
Alberto Fernández Capel
8c10c737dc Setup travis ci 2018-04-19 14:37:46 +01:00
James Allen
3d050f647b Provide hosts as environment settings and add npm run start script 2017-12-29 08:15:32 +00:00
Shane Kilkelly
7295342ec2 fix existance checks for first_name and last_name 2017-12-18 11:13:19 +00:00
Shane Kilkelly
675814f1b1 Handle the case where the user has only a last_name set 2017-12-13 10:28:35 +00:00
Shane Kilkelly
06c8729ce7 If a user has only their first_name set, don't label as Anonymous 2017-12-12 15:27:50 +00:00
Brian Gough
c8ad331551 fix unit tests 2017-11-13 09:41:04 +00:00
Brian Gough
b734f7a3f7 convert errors to warnings 2017-11-10 15:01:23 +00:00
Shane Kilkelly
3b39464aa5 Merge branch 'master' into sk-unlisted-projects 2017-11-02 10:58:41 +00:00
Brian Gough
5c8291a8da fix sentry initialisation 2017-10-30 16:14:44 +00:00
Brian Gough
009f8a3eae add sentry support 2017-10-30 16:14:29 +00:00
Brian Gough
17ac7719c3 Merge pull request #23 from sharelatex/bg-upgrade-ioredis 
upgrade ioredis
 2017-10-30 14:06:18 +00:00
Brian Gough
0568aada17 upgrade to node 6 2017-10-30 13:46:37 +00:00
Brian Gough
cc118fa230 remove unnecessary ioredis package 2017-10-30 13:44:09 +00:00
Brian Gough
510ee6bf4f use ioredis 3 via redis-sharelatex 2017-10-30 13:43:36 +00:00
Shane Kilkelly
c5e602c0a9 Fix typo 2017-10-26 16:00:06 +01:00
Brian Gough
168d646328 exit if mock servers fail to start 2017-10-20 15:19:20 +01:00
Shane Kilkelly
76673d5f0a Change anonToken to anonymousAccessToken 2017-10-20 10:10:58 +01:00
Shane Kilkelly
438bb28c04 Get anonToken from joinProject payload 2017-09-29 16:32:46 +01:00
Shane Kilkelly
d4c735c3ea Pass anonymous-read token along as header to web-api 2017-09-26 14:21:41 +01:00
Alasdair Smith
6470577c3f Add acceptance tests for joinDoc variadic 2017-09-22 11:01:11 +01:00
Alasdair Smith
785d6e2eea Add tests for comment & change encoding 2017-09-22 10:56:08 +01:00
Alasdair Smith
b8d3f34e54 Fix joinDoc tests not passing options arg and restructure tests 2017-09-22 10:56:08 +01:00
Alasdair Smith
ed252db25e Merge pull request #18 from sharelatex/as-encode-ranges 
Encoding changes & comments in ranges
 2017-09-22 09:48:42 +01:00
Alasdair Smith
a0505afb23 Be defensive on comment text 2017-09-22 09:34:10 +01:00
Alasdair Smith
c67150ea10 Ensure falsy value doesn't fail conditional 2017-09-22 09:33:29 +01:00
Alasdair Smith
937bf82a2f Return callback with guard 2017-09-22 09:25:24 +01:00
Alasdair Smith
5d8e201732 Don't return obj 2017-09-21 16:58:03 +01:00
Alasdair Smith
a299d7335d Fix incorrect var 2017-09-21 16:56:18 +01:00
Alasdair Smith
90d05dc6dd Make args order consistent 2017-09-21 16:56:09 +01:00
Alasdair Smith
3966e2f85b Make variadic options more explicit 2017-09-21 16:55:49 +01:00
Alasdair Smith
790b9ea8ed Switch order of args 2017-09-21 15:19:19 +01:00
Alasdair Smith
55c880e1dd DRY up a bit 2017-09-21 15:07:15 +01:00
Alasdair Smith
b796879c9f Handle options not being passed 2017-09-21 14:58:49 +01:00
Alasdair Smith
aa6e0d0d69 Only encode ranges if option passed 2017-09-21 14:23:16 +01:00
Alasdair Smith
9f503f1e9f First pass at encoding changes & comments in ranges 2017-09-21 13:25:55 +01:00
Joe Green
dc8f4ffc2a Update Jenkinsfile 2017-09-21 11:03:07 +01:00
Joe Green
0b41d32cbb Update Jenkinsfile 2017-09-21 10:59:41 +01:00
Joe Green
72ef354206 build.txt 2017-09-04 14:54:05 +01:00
Joe Green
3dae68a4f0 Create Jenkinsfile 2017-08-11 14:24:46 +01:00
Brian Gough
ab6fe1d948 check for null bytes from JSON.stringify 2017-06-01 11:27:56 +01:00
James Allen
642134da79 Don't return a user if there is no entry 
ioredis returns a blank object, {}, if there is no key with hgetall.
Previously, node-redis returned nil. So we need to check for a blank
object as well as a nil object.
 2017-05-16 11:06:05 +01:00
James Allen
a8917b933f Remove multi call to make compatible with redis-cluster 2017-05-11 17:27:28 +01:00
James Allen
01d0b63f2c Update config for websessions 2017-05-10 15:52:35 +01:00
James Allen
d04be1c000 Update redis-sharelatex 2017-05-09 17:09:00 +01:00
James Allen
720f24427a Use new redis-sharelatex with support for cluster 2017-05-02 15:51:17 +01:00
Shane Kilkelly
ed76d57bf8 Add a .nvmrc file 2017-03-27 14:51:46 +01:00
James Allen
d8ff8ba098 Merge pull request #11 from sharelatex/ja-allow-readonly-comments 
Allow readonly users to submit comments
 2017-03-20 14:25:03 +00:00
James Allen
8766646149 Allow users to send a comment update if they are read-only 2017-03-15 15:45:52 +00:00
James Allen
d939f6cd65 Remove some old logging 2017-03-15 15:45:18 +00:00
Brian Gough
d468f662ac handle disconnects of unauthenticated users 2017-02-23 12:04:36 +00:00
Shane Kilkelly
50930cd7b1 Don't npm rebuild inside container 2017-02-20 10:14:41 +00:00
James Allen
9fd099c24f Update getDoc signature to match reality 2017-01-16 17:09:44 +01:00
James Allen
f66d631886 Merge branch 'master' into ja-track-changes 2017-01-13 13:40:42 +01:00
Brian Gough
e5160d9a36 log client id when disconnecting on otUpdateError 
only log errors for connected clients
 2016-12-16 14:43:51 +00:00
James Allen
bf2620ee0c Return ranges from docupdater to client 2016-12-08 11:37:31 +00:00
James Allen
5d377713d6 Try to fix issue with acceptance tests timing out 2016-12-08 11:25:25 +00:00
James Allen
7bade0a8bb Merge branch 'master' of github.com:sharelatex/real-time-sharelatex 2016-12-08 11:16:22 +00:00
James Allen
9cf0eb5540 Add in acceptance test script 2016-12-08 11:14:27 +00:00
James Allen
4cff89becc Fix acceptance tests 2016-12-08 11:12:07 +00:00
Shane Kilkelly
de18231ef1 clarify purpose of redis client 2016-11-09 12:09:15 +00:00
Shane Kilkelly
41868ddda3 Make real-time work with web sessions in redis-cluster 2016-11-09 12:06:32 +00:00
Brian Gough
2e0f5b74db send connectionAccepted/Rejected events on connect 
let the client know whether it has successfully authenticated
 2016-10-28 15:41:57 +01:00
James Allen
6fa2a81baf Make breakout logic after draining N clients more clear 2016-10-24 16:54:56 +01:00
James Allen
7107d9adcd Delete logging that will be noisy in production 2016-10-24 16:40:10 +01:00
James Allen
59d042e264 Add end point to start draining clients 2016-10-24 16:36:09 +01:00
Shane Kilkelly
185bc7e635 Update session code 2016-09-07 08:58:35 +01:00
Brian Gough
8ffec68250 add comment about fallback case 2016-09-05 12:46:52 +01:00
Brian Gough
ef85bce3b8 track permissions when clients join and leave docs 2016-09-05 12:46:52 +01:00
Brian Gough
9ab19c5d03 avoid double callback 2016-09-02 16:34:14 +01:00
James Allen
51939512ac Return semantic error if doc ops range is not loaded 2016-05-31 14:21:23 +01:00
James Allen
f4a465ea69 Return a 'not authorized' error if the user is not logged in/authorized 2016-05-31 11:49:51 +01:00
James Allen
343ec9d708 Add in flags that track how often each callback is called 2016-05-26 15:46:45 +01:00
James Allen
030abc5340 Don't flush to track changes now that this happens in doc updater 2016-01-20 17:51:24 +00:00
James Allen
b28e5ac6b2 Use configurable app name if present 2015-12-07 11:49:55 +00:00
James Allen
73cd1a3e92 Use latest version of metrics with suitable event loop monitoring 2015-12-03 16:50:56 +00:00
James Allen
5b17764da2 Pin down logger version number 2015-12-01 17:28:26 +00:00
James Allen
3580e3ba6b Update to latest logger 2015-12-01 17:18:45 +00:00
James Allen
830d676f4f Add in limit on all JSON parsing 2015-12-01 11:05:49 +00:00
James Allen
92d18d7e2e Reduce limit to actuall 1Mb, not 1Gb 2015-11-30 15:40:03 +00:00
James Allen
0372fa3203 Add in extra logging about size of messages 2015-11-30 15:25:09 +00:00
James Allen
503b766dcc For duplicate ops only send ack to submitting client 
When a duplicate op is received, we only need to ack it to client
that sent it. Only that client is having trouble, and all other clients
will already have received it.
 2015-11-19 10:58:28 +00:00
James Allen
669cb0c085 Merge branch 'master' of github.com:sharelatex/real-time-sharelatex 2015-08-31 14:04:59 +01:00
James Allen
da28b0fc70 Use updated metrics with unlimited socket config 2015-08-31 14:04:54 +01:00
Henry Oswald
3c70acd560 added metrics into realtime around doc update manager 2015-08-29 08:26:16 +01:00
James Allen
d79793c34f Monitor get document times 2015-08-29 08:22:43 +01:00
Henry Oswald
42e7d5d4b6 make startup message consistent 2015-04-30 15:05:31 +01:00
James Allen
985abe42fe Release version 0.1.4 2015-03-20 14:21:17 +00:00
James Allen
5da5c5c435 Accept arrays of messages to send to client 2015-03-12 14:32:35 +00:00
James Allen
536118b0cf Relay messages received via HTTP into the project 2015-03-03 17:15:19 +00:00
James Allen
5ee71a423d Release version 0.1.3 2015-02-26 11:23:01 +00:00
James Allen
f0462f0b1f Bump version to 0.1.2 2015-02-10 13:14:13 +00:00
David Renshaw
f72fa9de34 'request' is a production dependency 2015-02-09 17:34:01 -05:00
James Allen
79cd0e6a5c Record user id correctly when updating position 2015-02-05 13:41:31 +00:00
James Allen
99ac814c7d Truncate error stack traces to 10 lines 2014-11-25 09:17:26 +00:00
James Allen
7a9f7f0870 Don't print massive stack trace when there is no room_id 2014-11-24 22:28:50 +00:00
James Allen
d62dc7ca3a Don't be so verbose with client update errors 2014-11-24 15:42:26 +00:00
James Allen
43a008c0bc Stub logger in unit tests 2014-11-24 15:42:13 +00:00
James Allen
0ed7d0c811 Ignore messages with null room_id 2014-11-24 12:09:12 +00:00
James Allen
57a34e940e Authorize users before updating their cursor positions 2014-11-24 12:05:05 +00:00
James Allen
a48c8aad92 Support anonymous access 2014-11-21 11:48:59 +00:00
James Allen
2a05045600 Add in redis health check 2014-11-20 16:56:09 +00:00
James Allen
4d691cf543 Delete client.coffee 2014-11-17 15:09:55 +00:00
James Allen
cddb4e7279 Add cookie parser dependency 2014-11-17 15:01:37 +00:00