github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity
22697 commits 3 branches 7 tags 206 MiB
Commit graph

690 commits

Author SHA1 Message Date
Eric Mc Sween
971a768c4e Install prettier and eslint 2020-09-11 15:59:33 -04:00
John Lees-Miller
ea2d76de51 Improve tagging for a (maybe) null error (#12) 
* Document a way of handling an error that might be null

* Make typecheck strict

* Test some edge cases

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
 2020-09-11 10:05:54 +01:00
Brian Gough
c530422f1e update package version to 3.1.0 2020-09-04 15:52:05 +01:00
Jakob Ackermann
71ba4150fe [misc] enable compression for the /metrics route 2020-09-04 15:32:48 +01:00
Brian Gough
b6b65b9638 Merge pull request #39 from overleaf/jpa-res-headers-public-api 
[http] use public node api for getting the response content-length
 2020-09-04 14:52:20 +01:00
Eric Mc Sween
967dde280e Disable auto-decompression in GCS persistor 
The GCS library automatically decompresses files with Content-encoding:
gzip. This is very useful, but the S3 library doesn't do that, so we
need to disable the feature in GCS so that it behaves the same way other
persistors behave.
 2020-08-18 16:04:54 -04:00
Jakob Ackermann
6f4829e817 [misc] fix module importing 2020-08-17 14:43:08 +01:00
Jakob Ackermann
37fda043d4 [http] use public node api for getting the response content-length 2020-08-17 11:59:18 +01:00
Christopher Hoskin
a66708a319 Ignore error 2020-08-11 10:56:51 +01:00
Christopher Hoskin
8649277c9f Format:fix 2020-08-11 10:55:03 +01:00
Christopher Hoskin
ea502a0623 Put LoggingManager.initialize back as it was in tests 2020-08-11 10:52:53 +01:00
Christopher Hoskin
20f2e1eb1d Don't catch checkLogLevel 2020-08-11 10:48:44 +01:00
Christopher Hoskin
9b21292b44 Ensure fetchResponse.text is a string in sinon stub 2020-08-11 10:47:31 +01:00
Christopher Hoskin
46b22b30c7 Fix test description 2020-08-11 09:46:52 +01:00
Christopher Hoskin
3a2ff4e7b5 Merge branch 'master' into csh-metadata-fallback-node-fetch 2020-08-11 09:44:28 +01:00
Simon Detheridge
1115e1a1e6 Add default parameter for 'opts' in MigrationPersistor.getObjectStream 2020-08-10 16:44:37 +01:00
Brian Gough
e5747fefd2 remove package-lock.json (not needed for library) 2020-08-06 14:52:07 +01:00
Brian Gough
5b39d49358 use scoped package name @overleaf/metrics 2020-08-05 11:27:56 +01:00
Jakob Ackermann
397d5c8f1f [misc] fix prettier-eslint glob for real -- include tests 2020-07-31 13:26:28 +01:00
Christopher Hoskin
e32411eaaa Clean up acceptance tests 2020-07-30 15:23:04 +01:00
Christopher Hoskin
57ef07dc67 lint and format 2020-07-30 15:08:22 +01:00
Christopher Hoskin
8376cf1ca2 Get acceptance tests working 2020-07-30 15:06:32 +01:00
Christopher Hoskin
6917847b10 catch 2020-07-30 14:45:49 +01:00
Christopher Hoskin
74e2a5eaa1 WIP 2020-07-30 13:53:48 +01:00
Christopher Hoskin
1ca6269aa5 Bump version to 2.2.0 2020-07-24 12:18:33 +01:00
Christopher Hoskin
d42bd2da50 Tidy up 2020-07-24 12:17:31 +01:00
Christopher Hoskin
6bc0d15d78 LOG_LEVEL_SOURCE env taking values file, gce_metadata or none, default file 2020-07-24 11:59:23 +01:00
Christopher Hoskin
0a15fedb34 Remove debug logs 2020-07-24 09:12:59 +01:00
Christopher Hoskin
02356c7f27 All logging manager tests working 2020-07-24 09:12:59 +01:00
Christopher Hoskin
6961f41488 Promisfy log level checks 2020-07-24 09:12:59 +01:00
Christopher Hoskin
6b8dd86a26 WIP 2020-07-24 09:12:59 +01:00
Christopher Hoskin
14bac39222 Add MOCHA_GREP for testing 2020-07-24 09:12:59 +01:00
Christopher Hoskin
40a08bebdb WIP 2020-07-24 09:12:59 +01:00
Christopher Hoskin
50d2856841 Re-instate metadata request as fallback for CLSI 2020-07-24 09:12:59 +01:00
Simon Detheridge
8037208800 Bump dependency versions 2020-07-23 10:33:01 +01:00
dependabot[bot]
98e4444585 Bump lodash from 4.17.15 to 4.17.19 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-07-18 15:48:52 +00:00
dependabot[bot]
2c088b3d20 Bump lodash from 4.17.15 to 4.17.19 
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-07-17 17:09:14 +00:00
Brian Gough
d415ae0cbe remove gruntfile 2020-07-17 16:23:01 +01:00
Brian Gough
a0f856cff2 fix tests 2020-07-17 16:17:18 +01:00
Brian Gough
747a80b545 decaffeinate 2020-07-17 16:01:58 +01:00
Brian Gough
e31a819636 remove statsd 2020-07-17 15:36:37 +01:00
Eric Mc Sween
6521837993 Add S3 options: httpOptions, maxRetries 2020-07-09 17:50:05 -04:00
Eric Mc Sween
3e7e4369af Implement getRedirectUrl() for the S3 backend 2020-07-08 17:13:20 -04:00
Eric Mc Sween
523ff9c4cd Support metadata when uploading objects 
Add contentType and contentEncoding options to sendStream(). These
options will set the corresponding metadata on the object.

This changes the API. The fourth argument to sendStream() used to be the
source md5. Now, it's an options object and the source md5 is a property
of that object.
 2020-07-08 08:13:53 -04:00
Simon Detheridge
990bacf3c2 Update to OError3 2020-07-07 10:40:37 +01:00
Eric Mc Sween
e197e4dc11 2.7.0 2020-07-06 08:20:02 -04:00
Eric Mc Sween
d69195eaa9 Log requests that don't have a route property 
The v1 history service has its routes set up via swagger-tools, which
doesn't write a route property on the request. This prevents us to send
request metrics based on the route, but we can still log the request.
 2020-07-03 16:38:29 -04:00
Simon Detheridge
7369e2f339 Add .circleci/config.yml and fix errors 2020-07-03 11:59:34 +01:00
Simon Detheridge
e92b75a2f8 Create new module from overleaf/filestore persistors (#1) 
* Create new module from overleaf/filestore persistors

* Convert persistors to ES6 classes with local settings

* Update README.md

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* Update README.md

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* Update .gitignore

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* Switch to AGPL license

* Paginate S3 list-object results

* Remove S3 client caching

* Clean up S3 md5-verification mechanism

* Update README for recent changes

* Update README.md

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* Remove package-lock

* Remove comment about FileHandler

* Add directory marker to FSPersistor.deleteDirectory

* Don't copy opts in GcsPersistor.getObjectStream

* Use Date.now instead of getTime

* Catch errors in migration persistor

* Check that settings.buckets exists

* Don't mutate options in ObserverStream constructor

* Update src/PersistorHelper.js

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* Lint and format fixes

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
 2020-07-02 14:19:45 +01:00
Jakob Ackermann
e61b1124bb [misc] bump eslint to 7.3.1 2020-07-02 12:15:22 +01:00
Jakob Ackermann
97407980db [misc] monkey patch prettier-eslint basepath 2020-07-02 12:12:07 +01:00
Christopher Hoskin
102c7524c9 Re-generate package-lock.json 2020-07-02 11:12:07 +01:00
Christopher Hoskin
db5084d30f Updrade runtime dependencies 2020-07-02 11:08:10 +01:00
Christopher Hoskin
bcaee67a5c Bump sinon and sandboxed-module 2020-07-02 11:04:21 +01:00
Christopher Hoskin
fcd4a60657 Make prettier 2020-07-02 11:01:23 +01:00
Christopher Hoskin
4c8d29af59 Update mocha and prettier 2020-07-02 11:00:28 +01:00
Christopher Hoskin
edbf7220df 2.1.1 2020-07-02 10:53:42 +01:00
Christopher Hoskin
d695a40a61 Lint 2020-07-02 10:51:37 +01:00
Christopher Hoskin
b932a8127b Bump eslint plugins 2020-07-02 10:46:37 +01:00
Christopher Hoskin
52649c9d0b 2.1.0 2020-06-30 14:00:15 +01:00
Christopher Hoskin
fa8df5cc0b Format:fix 2020-06-30 13:56:13 +01:00
Christopher Hoskin
9e54503d1b Add prettier-eslint-cli to dev-dependencies 2020-06-30 13:55:24 +01:00
Christopher Hoskin
c93ee44879 Update acorn dependency (via eslint) https://github.com/advisories/GHSA-6chw-6frg-f759 2020-06-30 13:53:34 +01:00
Christopher Hoskin
9437266dcb Merge branch 'master' into jpa-csh-tracing-end-time-from-fs 2020-06-19 11:41:15 +01:00
Miguel Serrano
12e3afb174 Bumped major version 2020-06-03 10:20:43 +02:00
Miguel Serrano
da0f2d5e75 Updated .nvmrc to latest LTS 2020-06-03 10:19:58 +02:00
Miguel Serrano
c602d536d2 updated OError to v3 2020-06-03 10:19:45 +02:00
Brian Gough
cd6e36d7dc upgrade ioredis to 4.17.3 2020-06-01 09:11:57 +01:00
John Lees-Miller
6cc03cad0c Add coverage 2020-05-15 17:18:01 +01:00
John Lees-Miller
c9d9982615 Update package.json metadata 2020-05-15 16:50:37 +01:00
Jakob Ackermann
5aad5db40f [checkLogLevel] read the updated tracing end time from disk 2020-05-15 13:39:35 +02:00
John Lees-Miller
d2f47e69a8 Bump version to 3.0.0 2020-05-15 11:25:12 +01:00
John Lees-Miller
fd6ea2bb70 Add typescript types 2020-05-15 11:24:14 +01:00
John Lees-Miller
497f2a7936 Make tag check stricter and use a map 2020-05-15 11:07:08 +01:00
John Lees-Miller
4b0060f0b1 Remove HTTPErrors from v3 2020-05-15 10:45:08 +01:00
John Lees-Miller
90494fd75f Fix demo require path 2020-05-15 10:36:00 +01:00
John Lees-Miller
8af22ae878 Improve assertion messages in test utility 2020-05-15 10:32:18 +01:00
John Lees-Miller
c44054a31b Add cast for the fake TaggedError 2020-05-05 09:06:15 +01:00
John Lees-Miller
fc197630be Avoid capturing stack trace twice 
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
 2020-05-05 09:02:11 +01:00
John Lees-Miller
6078aa963d Add typecheck to CI 2020-04-30 10:29:23 +01:00
John Lees-Miller
f6eb185f84 Update README for v3 2020-04-30 10:29:23 +01:00
John Lees-Miller
09cd72d51c Rework to favor tagging over wrapping 2020-04-29 21:08:58 +01:00
John Lees-Miller
1a38f4e4ff Add typescript for type checking 2020-04-29 21:03:41 +01:00
John Lees-Miller
c46075aaab Make constructor take only message and info 2020-04-17 15:45:08 +01:00
John Lees-Miller
c000bdc43c Add standard and fix complaints 2020-04-17 10:59:27 +01:00
John Lees-Miller
4b903626b4 Apply linting rules 2020-04-17 09:14:35 +01:00
John Lees-Miller
f80753fe40 Add linting and formatting 2020-04-17 09:14:35 +01:00
John Lees-Miller
113c2cc5ec Add CI badge 2020-04-17 09:13:04 +01:00
John Lees-Miller
ad6b25e2c1 Add .circleci/config.yml 2020-04-17 09:10:47 +01:00
Jakob Ackermann
ea767920fc [misc] fix unit tests 2020-04-01 16:40:15 +02:00
Henry Oswald
264e549fc5 bump io redis to 4.16.1 2020-03-31 09:40:44 +01:00
Eric Mc Sween
8131c5ac91 2.6.2 2020-03-25 09:53:54 -04:00
Eric Mc Sween
ef7e3b0e7a Fix HTTP request timing metrics 
The conversion between high resolution time and milliseconds was
incorrect.
 2020-03-25 09:39:21 -04:00
Jakob Ackermann
2137c7a42c [misc] version bump to 1.9.1 2020-03-23 13:06:32 +01:00
Jakob Ackermann
b049331777 [misc] fix decaff cleanup for reportedToSentry 
Previously it would bailout of the loop after processing ANY attribute.
REF: 6fe4dcbf32720282821d42140ab92593866e0772
 2020-03-23 10:48:55 +01:00
Eric Mc Sween
a17843f3bf 2.6.1 2020-03-20 07:47:05 -04:00
Eric Mc Sween
25448bfef4 Downgrade all request logs to INFO 
Commit 9056143fe36f1347a1ff985ef8592a1de7d798dd added logic to log
requests with different error levels depending on the status code. The
intention was to make the 5xx and 4xx requests stand out in Stackdriver.
Unfortunately, this also creates a lot of noise in Sentry since we log
the errors separately from the requests.

This commit brings back the former behaviour of logging all requests at
the INFO level. We can revisit this if we integrate the strategy
implemented in filestore of logging once per request.
 2020-03-18 08:41:11 -04:00
Eric Mc Sween
62e82d7469 Send the status code with the HTTP request size metric 2020-03-12 06:24:46 -04:00
Eric Mc Sween
4a92be80ea Send HTTP request size metric 
The metric is a "summary" called http_request_size_bytes.
 2020-03-11 16:39:49 -04:00
Brian Gough
03e81153db avoid step effects in summary metrics 
reduce the window size from 10 minutes to 1 minute,  so that short
spikes do not cause a 10 minute long "table" graph.
 2020-03-10 15:01:09 +00:00
Simon Detheridge
74c2afc12d Bump package version 2020-03-04 10:31:12 +00:00
Simon Detheridge
9a8cddbbb6 Don't set UV_THREADPOOL_SIZE if already set 2020-03-03 17:09:35 +00:00
mserranom
2e35a7ccf9 removed grunt 2020-02-17 16:24:52 +01:00
mserranom
c8617ff55e updated paths 2020-02-17 16:17:53 +01:00
mserranom
fbbee7d499 removed coffeescript 2020-02-17 16:17:05 +01:00
decaffeinate
ec32a2b1e2 decaffeinate: Run post-processing cleanups on AccessTokenEncryptorTests.coffee 2020-02-17 16:05:42 +01:00
decaffeinate
d8832849aa decaffeinate: Convert AccessTokenEncryptorTests.coffee to JS 2020-02-17 16:05:42 +01:00
decaffeinate
45717e21fe decaffeinate: Rename AccessTokenEncryptorTests.coffee from .coffee to .js 2020-02-17 16:05:41 +01:00
decaffeinate
7df0e74feb decaffeinate: Run post-processing cleanups on AccessTokenEncryptor.coffee 2020-02-17 16:04:54 +01:00
decaffeinate
054bbee912 decaffeinate: Convert AccessTokenEncryptor.coffee to JS 2020-02-17 16:04:53 +01:00
decaffeinate
b51f05bf5e decaffeinate: Rename AccessTokenEncryptor.coffee from .coffee to .js 2020-02-17 16:04:53 +01:00
John Lees-Miller
59156ec89e Fix incorrect expect 2020-01-27 10:45:31 +00:00
Brian Gough
2273978e7b fix gauge usage 2019-12-16 11:42:10 +00:00
Brian Gough
3a5374c6f9 increase minor version for backwards compatible addition 2019-12-16 10:22:50 +00:00
Brian Gough
93df87eff3 allow options for count 2019-12-16 10:18:30 +00:00
Simon Detheridge
620573c0ad Add an explicit test for a 2019 string 2019-12-09 16:07:25 +00:00
Simon Detheridge
b8d5f6baeb Split methods to reduce code duplication 2019-12-09 15:35:22 +00:00
Simon Detheridge
73f293cf49 Remove unneeded 'async' dependency 
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
 2019-12-09 15:01:31 +00:00
Simon Detheridge
a9e39af24f Keep oldmethods and use old method when app uses an older label 2019-12-09 14:49:23 +00:00
Simon Detheridge
c1cd5c65e7 Update dev dependencies 2019-12-05 08:58:49 +00:00
Simon Detheridge
120890bb08 Update lib/coffee/AccessTokenEncryptor.coffee 
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com>
 2019-12-05 08:57:19 +00:00
Simon Detheridge
c2fc2ed178 Replace deprecated createCipher with createCipheriv for new tokens 
Co-authored-by: Alasdair Smith <ali@alasdairsmith.co.uk>
 2019-12-04 10:09:39 +00:00
Simon Detheridge
4d6ed4fd3a Run tests via 'npm test' 2019-12-04 10:02:20 +00:00
Simon Detheridge
53aa2490f5 Merge pull request #20 from overleaf/spd-metrics-ttl 
Add mechanism to expire old prometheus metrics
 2019-10-28 17:21:57 +00:00
Simon Detheridge
feecda8ea8 Use map instead of hash for metrics 2019-10-28 14:39:53 +00:00
Simon Detheridge
e0cf10a886 Fix typo, gague -> gauge 2019-10-28 12:34:04 +00:00
Eric Mc Sween
7b7b6d0793 2.3.0 2019-10-25 07:17:43 -04:00
Eric Mc Sween
364cdf1fdb 1.9.0 2019-10-25 07:15:35 -04:00
Eric Mc Sween
166211b278 Stackdriver logging 
When the environment variable STACKDRIVER_LOGGING is set to true, send
request logs in a format that Stackdriver knows how to interpret. Also,
set the log level accordint to the status code. 4xx responses are logged
as warnings and 5xx responses are logged as errors.
 2019-10-24 22:05:12 -04:00
Eric Mc Sween
cb300b79ea Add option to log to Stackdriver 
When the environment variable STACKDRIVER_LOGGING is set to true, logs
are sent to Stackdriver in addition to stdout.
 2019-10-24 17:15:29 -04:00
Eric Mc Sween
905d65931a Small cleanup, satisfy linter, add prettier 2019-10-24 17:15:29 -04:00
Simon Detheridge
07e4eb4dfb Add mechanism to expire old prometheus metrics 
Adds a wrapper around the prometheus client, which keeps track of the
last time a metrics was accessed, and removes old ones once they have
not been accessed for a period of time.
 2019-10-23 17:07:45 +01:00
Shane Kilkelly
1735e8f56b Upgrade to node 10, specify digest in pbkdf2 2019-10-08 15:53:08 +01:00
Brian Gough
c8d50d0ee9 upgrade ioredis to v4.14.1 2019-10-01 16:44:04 +01:00
John Lees-Miller
626511fa73 Bump version number 2019-09-19 16:11:59 +01:00
Brian Gough
d9214b6eeb check full reply in health check 2019-09-19 15:41:43 +01:00
Brian Gough
2435099766 make health check more robust 2019-09-19 15:05:32 +01:00
John Lees-Miller
f0d59bc192 Bump lodash version 2019-07-30 09:34:49 +01:00
Tim Alby
9229c9b3a9 add common HTTP errors 2019-07-29 18:25:31 +02:00
John Lees-Miller
1482e6841b Bump to 1.8.0 and include license 2019-07-11 13:07:47 +01:00
Miguel Serrano
7e6b562981 Fix test setup (#15) 2019-07-11 13:25:09 +02:00
Miguel Serrano
6b3e7e4ffc replace bunyan error serializer with custom one (#14) 2019-07-11 12:11:09 +02:00
John Lees-Miller
0dd8b124bd Remove private from package.json 2019-07-08 13:24:03 +01:00
John Lees-Miller
bc224b43c9 Add repository to package.json 2019-07-08 13:22:49 +01:00
John Lees-Miller
79804a8e2f Remove unreachable code 2019-07-08 11:57:28 +01:00
John Lees-Miller
9725b1bf0c Bump to 2.0.0 2019-07-08 11:52:59 +01:00
John Lees-Miller
96ab30883a Remove ES5 interface 2019-07-08 11:52:59 +01:00
John Lees-Miller
bdde7fac65 Move shrinkwrap to package-lock 2019-07-08 11:46:11 +01:00
John Lees-Miller
295a603ca5 Update readme for @overleaf/o-error 2019-07-08 11:45:41 +01:00
John Lees-Miller
970dcb4e8e Update metadata for publishing 2019-07-08 11:35:29 +01:00
Tim Alby
d63812c557 update tests 2019-07-05 18:55:53 +02:00
Tim Alby
9afd8e4e5d change exports 2019-07-05 18:45:43 +02:00
Tim Alby
626e6267ae update README with OError naming 2019-07-05 15:50:58 +02:00
Tim Alby
76b13e7d35 rename to OError 2019-07-05 15:25:22 +02:00
Tim Alby
9c1b48a5e4 remove ErrorWithStatusCode 2019-07-05 15:24:11 +02:00
Tim Alby
7bb8a5bd11 handle custom errors created without info 2019-07-04 10:38:56 +02:00
Henry Oswald
630a957d7b bump ioredis to have naptmap for sentinal 2019-07-02 11:56:21 +01:00
John Lees-Miller
c167ad9bdb Upgrade mocha version 2019-07-01 11:35:24 +01:00
John Lees-Miller
ed7dc939ac Add support for errors with ES6 classes 2019-07-01 11:34:15 +01:00
Brian Gough
286eb747ad add status label to gauges 2019-06-06 10:34:55 +01:00
Henry Oswald
d694674b78 put host in error message 2019-05-30 14:36:54 +01:00
Henry Oswald
6600c2e050 add redis options inc host ip to error message 2019-05-30 10:13:05 +01:00
Henry Oswald
b2d6fb77db bump to 1.0.7 2019-05-30 10:12:40 +01:00
Brian Gough
a4bc5ba5b6 fix tests for ioredis as default 2019-05-20 16:15:54 +01:00
Timothée Alby
6d33d383d0 Update README.md 
- change app name
 2019-05-07 16:16:11 +01:00
Shane Kilkelly
f030823744 Add readme 2019-05-07 10:44:58 +01:00
Henry Oswald
d1dc6b8ce7 bump ioredis 2019-05-03 11:37:32 +01:00
Henry Oswald
db590651ee make ioredis the driver for standard connections 2019-05-03 11:33:24 +01:00
Henry Oswald
816c49daf2 bump metrics to 2.1.2 2019-04-12 12:36:47 +01:00
Henry Oswald
d4faaaa60e use console.log not logger.log 
we don't habe verbiouse lossing enabled that early in the apps lifecycle
also best not to require modules before enabling profiler
 2019-04-12 12:34:17 +01:00
Brian Gough
f4b2f7742e allow patch updates to ioredis 4.6.0 2019-04-11 15:24:27 +01:00
Ersun Warncke
1809d6e403 filter out error logs in buffer 2019-03-12 08:23:37 -04:00
Ersun Warncke
cb22e98766 default ring buffer size to zero 2019-03-11 10:27:35 -04:00
Ersun Warncke
b31022e329 remove jsx and react from eslint config and add eslint modules 2019-03-08 12:16:21 -04:00
Ersun Warncke
a2b5793f49 add eslint and prettier 2019-03-08 12:06:24 -04:00
Ersun Warncke
83597e8723 use ring buffer for logging and include with error 2019-03-08 10:09:44 -04:00
Ersun Warncke
5a6e00e12b decaf 2019-03-06 16:08:38 -04:00
Henry Oswald
15d14d8e2b add injectMetricsRoute into statsd so it doens't blow up 2019-02-07 09:47:29 +00:00
Henry Oswald
bf18c6e513 MVP for running both statsd and prom side by side 
statsd code is from v1.8.1
 2019-01-30 11:11:37 +00:00
Ersun Warncke
128c9f7c39 lower case prod check, bump version 2019-01-29 12:00:14 -04:00
Ersun Warncke
8764c7378b check dynamic log level in gce metadata 2019-01-29 10:13:26 -04:00
Henry Oswald
85011ed0e7 add path into inc options 2019-01-28 14:37:54 +00:00
Henry Oswald
dd3b8eae45 bump ioredis to 4.6.0 which lets us use natMap, package goes to 1.0.5 2019-01-22 15:45:41 +00:00
Christopher Hoskin
02907fd2e7 Fix Register II 2019-01-15 16:16:12 +00:00
Christopher Hoskin
20e45b7a2e Fix Register 2019-01-15 16:12:15 +00:00
Christopher Hoskin
d4caa48118 Bump package version 2019-01-15 15:44:01 +00:00
Christopher Hoskin
93bef54c39 Make register public so we can support other servers e.g. hapi 2019-01-15 15:36:35 +00:00
Henry Oswald
a2ed182942 add process.env['LOG_LEVEL'] varable to let us change the logging level 2018-12-14 14:30:01 +00:00
Henry Oswald
dfead32d69 Update package.json 2018-12-13 09:19:05 +00:00
Henry Oswald
1a34f3db1b Update package.json 2018-12-13 08:46:58 +00:00
Henry Oswald
287effb139 use ENABLE_PROFILE_AGENT 2018-12-12 21:17:12 +00:00
Henry Oswald
db4ae84bda require logger 2018-12-12 21:05:33 +00:00
Henry Oswald
a86c4d8abd add profiler 2018-12-12 20:11:40 +00:00
Henry Oswald
9eac49ad84 add some logging in 2018-12-11 16:07:34 +00:00
Henry Oswald
c257482e15 add ENABLE_DEBUG_AGENT and don't require modules unless they are enabled 2018-12-11 15:46:29 +00:00
Christopher Hoskin
6f82309829 Bump version to 2.0.10 2018-12-11 12:52:38 +00:00
Christopher Hoskin
61e6cf0493 Add host label to timing metrics 2018-12-11 12:01:22 +00:00
Henry Oswald
425a6f55ff set build version via env var and bump build version 2018-12-10 22:02:25 +00:00
Henry Oswald
7a227adaeb add if statment around traceagenet and bump to 2.0.8 2018-12-05 13:58:40 +00:00
Henry Oswald
02071584ae bump to 2.0.7 2018-12-05 12:31:16 +00:00
Henry Oswald
b2f49351c0 few tidy up changes 2018-12-05 11:03:40 +00:00
Henry Oswald
93f4a7eeaf small cleanup 2018-12-04 17:01:30 +00:00
Henry Oswald
85d4b03bcb bump to 2.0.6 2018-12-04 16:29:09 +00:00
Henry Oswald
2d4283fdf0 add DEBUG_METRICS env var 2018-12-04 16:20:52 +00:00
Henry Oswald
31fa5cef51 add logging 2018-12-04 16:09:09 +00:00
Henry Oswald
27e6db1e51 inc process_startup on init 2018-12-04 15:57:19 +00:00
Henry Oswald
752541b7f1 remove logging and bump to 2.0.4 2018-11-29 16:10:55 +00:00
Henry Oswald
9a737bef2f bump to 2.0.3 2018-11-28 10:45:11 +00:00
Henry Oswald
917afa3edc have default opts in inc 2018-11-27 16:12:12 +00:00
Henry Oswald
8adfc49af7 v 2.0.1 2018-11-27 15:48:17 +00:00
Henry Oswald
39f924f73c bump to 2.0.0 2018-11-27 15:15:45 +00:00
Henry Oswald
4e370ef24d clean up 2018-11-27 12:07:26 +00:00
Henry Oswald
a8cfa97463 use modified prefix and method name 2018-11-27 11:55:38 +00:00
Henry Oswald
2a6839f48c roll back async method for moment 2018-11-27 11:27:16 +00:00
Henry Oswald
fdd4db25a3 big refactor remoing statsd and converting keys to prom keys 2018-11-27 10:36:57 +00:00
Henry Oswald
b9f3a3f987 improve mongo metrics to be more promethious like 2018-11-26 13:31:21 +00:00
Henry Oswald
4806e6fd87 use lables 2018-11-26 09:46:26 +00:00
Henry Oswald
f7deba6de9 Merge branch 'master' into ho-stackdriver2 2018-11-23 15:29:06 +00:00
Christopher Hoskin
725abdce3b Remove spurious blank lines 2018-11-21 12:42:40 +00:00
Christopher Hoskin
00aaa3f3d4 Merge branch 'master' into ho-stackdriver 2018-11-21 12:41:34 +00:00
Henry Oswald
7f3cd3c77c just do debugging and tracing 2018-11-21 10:30:18 +00:00
Henry Oswald
f63b84983d add count to prom 2018-11-21 10:28:32 +00:00
Henry Oswald
d01ff95b46 bump to 1.9.1 2018-11-21 10:20:58 +00:00
Henry Oswald
32da84a1d7 proper null check 2018-11-21 10:20:33 +00:00
Henry Oswald
70a75113cb refactor key building 2018-11-21 10:15:43 +00:00
Henry Oswald
c36e3d74b5 replace - with _ 2018-11-21 08:57:00 +00:00
Henry Oswald
cf920a86f7 fix metrics.this 2018-11-21 08:50:33 +00:00
Henry Oswald
5599521c09 consolodate on metrics.timing 2018-11-20 21:52:09 +00:00
Henry Oswald
7fc49d1eb5 metrics of different types can clash, share promMetrics 2018-11-20 17:50:54 +00:00
Henry Oswald
bd8fd1141e client -> prom 2018-11-20 17:25:37 +00:00
Henry Oswald
6854a64b73 fix this.key 2018-11-20 17:20:49 +00:00
Henry Oswald
6f708fd26a add summaries 2018-11-20 17:17:02 +00:00
Henry Oswald
5039287ee4 don't default to adding underscore to name 2018-11-20 16:28:36 +00:00
Henry Oswald
816c9348f6 don't put dots in key name 2018-11-20 16:13:40 +00:00
Henry Oswald
f4af82282f use buildKey for prefixing name and host 2018-11-20 16:06:02 +00:00
Henry Oswald
1a3b427315 bump to 1.9.0 2018-11-20 14:54:09 +00:00
Henry Oswald
c576a86c14 Update metrics.coffee 2018-11-20 13:50:04 +00:00
Christopher Hoskin
65fccf8abe Ensure gauge values are numeric, not string 2018-11-07 16:08:31 +00:00
Christopher Hoskin
d0e1324dba Sanitize metric keys for Prometheus 2018-11-07 12:44:10 +00:00
Christopher Hoskin
bb3cff5744 Add prom-client to package.json 2018-11-07 11:21:45 +00:00
Christopher Hoskin
56eaae89f9 Add Gauges 2018-11-06 16:15:41 +00:00
Christopher Hoskin
6d4d05957e Get counter metric working 2018-11-06 14:22:03 +00:00
Christopher Hoskin
f604fb92e5 Get default Prometheus metrics working 2018-11-06 11:14:26 +00:00
Brian Gough
cd81a602fd increment version to 1.5.8 2018-11-06 10:56:58 +00:00
Brian Gough
df3b889f50 remove the path in the message from fs errors 2018-11-06 10:50:12 +00:00
Brian Gough
3676f78ff9 avoid reporting the same error object more than once 2018-11-02 15:53:21 +00:00
Henry Oswald
7e359c9df2 add trace and debug agent to metrics 2018-10-16 17:19:21 +01:00
Henry Oswald
4b075db038 non tested promethius counters added 2018-10-16 16:47:12 +01:00
Christopher Hoskin
90b6e5afce Release version v1.8.1 2018-07-12 11:07:43 +01:00
Christopher Hoskin
06e8450694 Allow remote statsd to be specified by environment variable 2018-07-06 13:57:52 +01:00
Brian Gough
00fdea922d allow a global gauge not specific to a host 2018-05-18 15:09:11 +01:00
Brian Gough
c8cc1e1bfe handle undefined logger in event loop monitor 2018-05-10 10:10:34 +01:00
John Lees-Miller
138b339c0f Tidy up for release 2018-04-23 08:11:08 +01:00
Alberto Fernández Capel
006cfb0142 Upgrade minor version 
The last change introduced a new but compatible feature.
 2018-04-11 14:08:18 +01:00
Alberto Fernández Capel
98eb92a0a9 Allow .js setting files 2018-04-10 14:01:32 +01:00
James Allen
a94b34f37f Initial commit of stand-alone module 2018-03-07 16:42:39 +00:00
James Allen
86ea1d3b57 Bump version to 1.5.7 2017-12-07 09:36:43 +00:00
James Allen
f707c835b9 Add debug level 2017-12-07 09:36:12 +00:00
Brian Gough
1737ddbf29 update ioredis to 3.2.1 2017-10-24 11:30:03 +01:00
Brian Gough
66c09a770e separate out devDependencies 2017-10-24 10:46:38 +01:00
Brian Gough
bda88c0356 remove the key_schema from options passed to redis 2017-06-19 15:46:21 +01:00
Brian Gough
bce937f562 add support for passing options to cluster 2017-06-16 15:28:55 +01:00
James Allen
90e0b4b273 Gracefully handle missing callback to exec in ioredis 2017-05-05 13:36:39 +01:00
James Allen
da85837e6f Update package.json 2017-05-04 15:41:43 +01:00
James Allen
92f22bc1a1 Fix redis cluster health check 2017-05-04 15:38:07 +01:00
Brian Gough
6923ca0d5f add test for ioredis monkey patching 2017-04-25 15:34:41 +01:00
James Allen
89397e4c50 Get unit tests running and add one for ioredis 2017-04-12 16:58:59 +01:00
James Allen
3b58e390fd Include ioredis as a driver to be able to handle cluster configs 2017-04-12 14:53:34 +01:00
Brian Gough
9d6101e365 Merge pull request #3 from sharelatex/bg-rate-limit 
rate limit on logging
 2017-03-29 16:26:16 +01:00
Brian Gough
9c7494eb67 fix off-by-one error in error count 2017-03-29 13:38:07 +01:00
Brian Gough
c7a6f44b11 add unit tests 2017-03-29 13:38:07 +01:00
Shane Kilkelly
7cf65d072d Add a .nvmrc file 2017-03-27 15:28:36 +01:00
Shane Kilkelly
bd1d6aa9c3 Add a .nvmrc file 2017-03-27 14:59:09 +01:00
Shane Kilkelly
d0eaa235a3 bump package version 2017-03-23 15:32:12 +00:00
Shane Kilkelly
6dbcc34df6 reduce UV thread-pool size to 16 2017-03-23 15:31:05 +00:00
Brian Gough
013c2a693c rate limit on logging v1.5.6 2017-03-23 13:54:44 +00:00
Shane Kilkelly
c930f6c680 bump to version 1.5.5 2017-03-22 14:59:40 +00:00
Shane Kilkelly
27c6390f4c upgrade to coffee-script 1.12.4 2017-03-22 14:46:42 +00:00
Shane Kilkelly
7bdba3756b Don't bother checking for error properties on error object 2017-03-21 14:19:12 +00:00
Shane Kilkelly
f2ebdd1662 Increment stats for success/failure of wrapped async calls 2017-03-20 16:54:40 +00:00
Shane Kilkelly
1f77cc0fd3 If function is called without callback, apply with original args 2017-03-20 16:37:53 +00:00
Shane Kilkelly
fbe19cd97d Don't return early in case where last arg is not a function 2017-03-20 16:25:10 +00:00
Shane Kilkelly
31235beee5 Don't throw error if the function is not invoked with callback. 
Instead, log the error and return early.
 2017-03-20 16:17:22 +00:00
Shane Kilkelly
9846703be5 Bump version 2017-03-20 10:17:09 +00:00
Shane Kilkelly
be2c6a96af Log args and their indexes, if they look like object ids 2017-03-17 14:20:13 +00:00
Shane Kilkelly
e7f71a25d8 Use an explicit prefix 2017-03-17 11:47:38 +00:00
Shane Kilkelly
f397678589 Clean up, don't allocate an extra date 2017-03-16 15:07:25 +00:00
Shane Kilkelly
5ea83947dd remove stray compiled js file 2017-03-16 10:10:40 +00:00
Shane Kilkelly
1f9d4950a2 Update gitignore 2017-03-16 10:09:16 +00:00
Shane Kilkelly
772f950d7c Test failure when wrapped method is not async/callback 2017-03-16 10:07:52 +00:00
Shane Kilkelly
a5aec5b812 Test failure to wrap method 2017-03-16 10:02:19 +00:00
Shane Kilkelly
40b238271d Add tests for methods producing errors, and logger 2017-03-16 09:55:07 +00:00
Shane Kilkelly
c3b18618bf Add unit tests 2017-03-16 09:49:45 +00:00
Shane Kilkelly
e99a7f6a87 refactor 2017-03-15 16:07:36 +00:00
Shane Kilkelly
3cb0ab2784 Add a 'timeAsyncMethod' helper 2017-03-15 15:06:54 +00:00
James Allen
aebaff829b Update package.json 2017-03-06 12:09:37 +00:00
Brian Gough
dfb2a069c0 use try catch around sentry logging 2017-03-03 16:05:56 +00:00
Brian Gough
5fc5a86b31 upgrade to latest raven 2017-03-03 14:43:41 +00:00
Brian Gough
68b941357e avoid uninitialised last error timestamp 2017-02-23 09:42:01 +00:00
Brian Gough
6e3f102fb2 rate limit on sentry reporting 2017-02-23 09:29:52 +00:00
James Allen
caeac717fc Set UV_THREADPOOL_SIZE to 128 for all processes 2016-10-24 10:50:44 +01:00
Brian Gough
cf0e106153 release 1.5.1 2016-04-01 15:52:42 +01:00
Brian Gough
24e097d263 don't force message over exception but include it as description 2016-04-01 15:52:28 +01:00
Brian Gough
7bdf2ef26b release 1.5 2016-04-01 15:35:54 +01:00
Brian Gough
fdc1ad2898 use logger.error message as title of sentry report 2016-04-01 15:35:14 +01:00
Brian Gough
04529c0019 release 1.4 2016-03-30 14:30:05 +01:00
Brian Gough
e04e6332e8 add fatal exception reporting 
split out capture exception code into separate method
 2016-03-29 12:53:16 +01:00
Shane Kilkelly
9c6df30031 remove deps 2016-03-25 14:51:52 +00:00
Shane Kilkelly
e6e3fb6948 change to be a class, with settings passed in 2016-03-25 14:33:38 +00:00
Shane Kilkelly
7786ef9ed8 set package to private 2016-03-25 11:06:11 +00:00
Shane Kilkelly
fc3c0d1384 init (copied from thirdpartydatastore-sharelatex) 2016-03-25 11:04:46 +00:00
Brian Gough
2df5845444 updated version to 1.5.0 2016-03-17 09:40:40 +00:00
Brian Gough
622bbe3123 return timeSpan from timers 
to allow additional calculation
 2016-03-15 13:52:40 +00:00
Brian Gough
29177f8de8 add support for statsd count method 2016-03-15 13:52:32 +00:00
Brian Gough
e8b8cfd9e0 release version 1.3.1 2016-01-26 16:42:08 +00:00
Brian Gough
17cf288e4c recreate Error objects from plain objects when logging to sentry 2016-01-26 16:41:52 +00:00
Brian Gough
2e21da050c add logging of request parameters to sentry 2016-01-05 13:34:50 +00:00
Brian Gough
fd70f23e5c updated version to 1.2.1 2015-12-18 10:58:03 +00:00
Brian Gough
06662f869f capture id's as tags, and pass attributes to sentry 2015-12-17 16:41:32 +00:00
Brian Gough
f1abe1e03b allow passing options to sentry 2015-12-08 10:30:18 +00:00
James Allen
8db30020ae Monitor event loop by looking for skew 
If we monitor with setImmediate, we miss big blocking loops. For example,
suppose we have 1000 1ms loops then a single bad 1000ms loop. setImmediate
will only be called at the right time 1/1000 of the time (it has to be the
loop just before the bad one). So this monitoring method gives a good average
if the std dev is low, but doesn't pick up spikes.

Instead, we can monitor the skew from the expected time between setIntervals.
In the case above, with a setInterval for 1000ms, we will pick up a skew
proportional to the amount of time that it overlaps the bad loop. So 50%
change of picking up skew > 500ms, and thus getting a good sense of any
spikes.
 2015-12-03 16:32:20 +00:00
Henry Oswald
30bfae5b35 upgraded bunyan 2015-12-01 20:51:08 +00:00
James Allen
738363a6de Set maxSockets to Infinity for all services 2015-08-31 14:02:03 +01:00
Brian Gough
0215b12a5f log memory usage every minute 2015-08-20 17:03:58 +01:00
Brian Gough
f237f7c3cc remove bug in optional argument handling 2015-08-18 11:23:10 +01:00
Brian Gough
27c382416a fix whitespace 2015-08-17 15:19:13 +01:00
Brian Gough
e3e8d80466 remove non-working metrics from graphite 2015-08-17 15:19:07 +01:00
Brian Gough
d2af7b24a0 remove randomisation to avoid shutdown problems 2015-08-17 15:18:18 +01:00
Brian Gough
577a3759c0 bugfix for memory chunk size 2015-08-14 15:44:24 +01:00
Brian Gough
46ec20ef9c add memory check and periodic gc 2015-08-14 14:38:24 +01:00
Brian Gough
175e3efd5f update package version to 1.2.0 2015-08-06 09:40:58 +01:00
Brian Gough
349b499f85 add compatibility with v2 mongo driver 2015-07-30 08:57:43 +01:00
Brian Gough
ffa523bced added monitoring of event loop time 
should indicate if node is blocking on libuv threads
as described in https://nodejs.org/api/dns.html#dns_dns_lookup
 2015-06-23 10:51:48 +01:00
James Allen
90e6c85e54 Send errors to Sentry if configured 2015-06-16 16:50:18 +01:00
Henry Oswald
5637678b48 bumped bunyan version to work with node 0.12 2015-06-08 10:01:45 +01:00
Henry Oswald
9329249bc9 Revert "reduce memory capture in http logger" 
This reverts commit fc2e043b20204e04f240814d4efc05762db7df96.

Had to revert this because req.route.path is not set until a matching
route has been hit, so it was always null inside res.end meaning
statsd data was never sent over.

This commit did not actually stop the memory leak so reverting it
has not short term repocusion
 2015-05-14 16:14:24 +01:00
Brian Gough
1e0a991fcd reduce memory capture in http logger 
only capture the properties of 'req' that we need, to avoid leaking
the whole req object for responses that never call res.end()
 2015-05-05 10:50:59 +01:00
Brian Gough
fa7e068ebb update minor version, due to addition of new close() method 2015-01-05 16:46:44 +00:00
Brian Gough
30070f23b8 add a close() method to terminate the module cleanly 
closes the statsd connection and cancels registered interval timers
prevents express from hanging when trying to shutdown
 2015-01-05 16:45:32 +00:00
Henry Oswald
3334a55944 bump package 2014-11-20 12:50:31 +00:00
Henry Oswald
917ce27a2c add a double message check 2014-11-20 12:50:19 +00:00
Henry Oswald
119ae80853 bump package.json 2014-11-20 11:09:57 +00:00
Henry Oswald
ef0ea2ace2 if redis fails once mark it to always fail 2014-11-20 11:09:44 +00:00
Henry Oswald
e133c7101e incremented version and added basic health check function in 2014-11-19 23:18:56 +00:00
Henry Oswald
6da4f3eef5 bump package 2014-11-19 16:41:28 +00:00
Henry Oswald
4cd3c249ec removed reconnect stuff 2014-11-19 16:39:06 +00:00
James Allen
1ee287ed98 add createMonitoredSubscriptionClient method 2014-11-19 14:05:16 +00:00
James Allen
42cbaecbe0 Let the client take care of reconnecting only when the stream is actually ended 2014-11-19 12:30:10 +00:00
James Allen
861eafd88d Handle reconnect logic better to (hopefully) avoid duplicate connections 2014-11-19 11:52:40 +00:00
James Allen
db3537a8eb Reset heartbeat timer after triggering a reconnect 2014-11-19 11:25:09 +00:00