Commit graph

449 commits

Author SHA1 Message Date
Jakob Ackermann
7f6c2afc92 Merge pull request #20980 from overleaf/jpa-extend-create-project
[web] scripts/create_project: fix and extend

GitOrigin-RevId: 4df41d02138a2214c83e7237794995d534e3c603
2024-10-14 11:08:04 +00:00
Andrew Rumble
e05c181af4 Ignore test settings file
GitOrigin-RevId: 5f9e901f4358bb631299ec165b72305ea814e343
2024-10-14 11:07:18 +00:00
Miguel Serrano
ce130a8bc5 Merge pull request #20909 from overleaf/msm-check-esm-test
[scripts] lookup `test` dirs on `esm-check-migration.js`

GitOrigin-RevId: 79a0961e3b018b440cdb15980a194b63b2211aec
2024-10-14 11:06:50 +00:00
roo hutton
8f593757e9 Merge pull request #20801 from overleaf/rh-odc-never-latex
Add script for extracting ODC users without LaTeX experience

GitOrigin-RevId: 5680a4e8ad1e7331c9b22752b7292b23bb169869
2024-10-14 11:02:19 +00:00
Liangjun Song
9e15c73228 Merge pull request #20732 from overleaf/revert-20731-revert-20480-ls-convert-migration-scripts-to-esm
Revert "Revert "Convert migration scripts to ESM""

GitOrigin-RevId: 0430a3cd02b9d23bf0f4573346351dcf4ee17fa6
2024-10-14 10:58:50 +00:00
Jakob Ackermann
39ee8de1a5 Merge pull request #20756 from overleaf/jpa-clear-admin-sessions
[web] add script for clearing admin sessions

GitOrigin-RevId: c5103b233073db62276698067b2262d7a785592b
2024-10-14 10:58:12 +00:00
Miguel Serrano
a6016ff984 [web] esm-check-migration to lookup scripts and migrations (#20736)
* [web] `esm-check-migration` to lookup scripts and migrations

* Remove unnecessary minimist default

GitOrigin-RevId: 55b0900d2d1dc2669e297ddc778dcd236c2912a7
2024-10-14 10:57:37 +00:00
Liangjun Song
b69ff9417a Merge pull request #20731 from overleaf/revert-20480-ls-convert-migration-scripts-to-esm
Revert "Convert migration scripts to ESM"

GitOrigin-RevId: ca9ddda6ddee8964f72797d3b653ef9cee6a7a49
2024-10-14 10:57:02 +00:00
Liangjun Song
492fe24a75 Merge pull request #20480 from overleaf/ls-convert-migration-scripts-to-esm
Convert migration scripts to ESM

GitOrigin-RevId: 46f04a1837ebb8244b1156af2d58162d024b6b2d
2024-10-14 10:56:57 +00:00
Antoine Clausse
6c2cf20125 Merge pull request #20552 from overleaf/ac-update-node-fetch-2
Reapply "Upgrade node-fetch to 2.7.0", Fix fetch-utils tests

GitOrigin-RevId: b42a2d2c50ce73f474e39755845e4df065f30b48
2024-10-01 08:05:18 +00:00
andrew rumble
0c428e149f Update mongoose count usages to countDocuments
count was removed see:
https://mongoosejs.com/docs/migrating_to_8.html#removed-count.

GitOrigin-RevId: 308c8f4b681269c29669c45b5f2022404b3af463
2024-09-24 08:06:32 +00:00
Antoine Clausse
2179d62d74 Merge pull request #20549 from overleaf/revert-20165-ac-update-node-fetch
Revert "Upgrade node-fetch to 2.7.0"

GitOrigin-RevId: 928df97c6fbc81f21097aaf9c331b0c70eeac002
2024-09-24 08:04:44 +00:00
Antoine Clausse
da3553d800 Upgrade node-fetch to 2.7.0 (#20165)
* Set `node-fetch` to `^2.7.0`

* Update package-lock.json

```
# root
bin/npm update node-fetch
bin/npm update cross-fetch

# in other path in docker
npm update node-fetch
```

* Update node-fetch patch

* [fetch-utils] Skip the test: destroys the request body if it doesn't get consumed

```
  1) fetch-utils
       fetchJson
         destroys the request body if it doesn't get consumed:
     FetchError: Invalid response body while trying to fetch http://example.com:30001/json/ignore-request: write EPIPE
      at PassThrough.<anonymous> (/overleaf/node_modules/node-fetch/lib/index.js:400:12)
      at PassThrough.emit (node:events:529:35)
      at emitErrorNT (node:internal/streams/destroy:151:8)
      at emitErrorCloseNT (node:internal/streams/destroy:116:3)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
```

* [fetch-utils] Delete the test: destroys the request body if it doesn't get consumed

* Remove the `setTimeout` in the node-fetch patch

Fixes a test and doesn't break filestore acceptance tests

* Update node-fetch patch again: bring changes from https://github.com/node-fetch/node-fetch/blob/e87b093/src/index.js

* Update node-fetch patch again: bring changes from https://github.com/node-fetch/node-fetch/blob/e87b093/src/index.js

* Update node-fetch patches back to single lines

Per https://github.com/overleaf/internal/pull/20165#discussion_r1739035513

GitOrigin-RevId: 945e5a12e838673b7bc87b588b7aca1bcd9109e2
2024-09-24 08:04:39 +00:00
Miguel Serrano
8ca24b104b Script to track ES Modules migration progress (#20448)
* Script to track ES Modules migration progress

GitOrigin-RevId: 8582f529313c40c26d27d7c2f1542b1828c5a7e4
2024-09-19 08:05:16 +00:00
Jimmy Domagala-Tang
707790e51e Merge pull request #20256 from overleaf/jdt-enable-writefull-unset
Differentiate between unset and disabled Writefull

GitOrigin-RevId: 3cf8f12ede851dab5a8067bdbcddba6c69870573
2024-09-06 08:05:52 +00:00
Alexandre Bourdin
4997b7f1ee Merge pull request #20062 from overleaf/ab-saas-authentication-module-renaming
[web] Rename overleaf-integration module to saas-authentication

GitOrigin-RevId: 7d69e06a19da9855fa9a0962114ad97b45ad8875
2024-08-23 08:04:43 +00:00
Miguel Serrano
0346ba2698 [web] CE script to verify TexLive versions on startup (#19653)
* [web] CE script to verify TexLive versions on startup

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: b99001d38468a775991a7284611aa333e956b919
2024-08-22 14:01:07 +00:00
andrew rumble
032deaf05c Switch to mongodb-legacy
GitOrigin-RevId: 11e09528c153de6b7766d18c3c90d94962190371
2024-08-21 08:04:24 +00:00
Alf Eaton
901d79dd55 Merge pull request #19757 from overleaf/ae-eslint-upgrade
Upgrade `@typescript-eslint` dependencies

GitOrigin-RevId: b953e795d705b817ec3669b04cd6a289be12b20d
2024-08-08 08:04:09 +00:00
Jakob Ackermann
e08b0b3d9e Merge pull request #19686 from overleaf/jpa-fix-ce
[web] sync overleafModuleImports between environments and check in CI

GitOrigin-RevId: c36c41d87d1b6fd687942bca8284a4725bb6c300
2024-08-01 08:04:37 +00:00
Alexandre Bourdin
dcf6e502b9 Merge pull request #19443 from overleaf/ab-overleaf-integration-refacto-move-institutions
[web] Move onboarding related code to onboarding module

GitOrigin-RevId: 405d4c3588f3911867fecd02b36e55fcd7633615
2024-07-30 08:04:31 +00:00
roo hutton
523e340a0d Merge pull request #19398 from overleaf/rh-rm-ieee-notification
[web] Remove IEEE Collabratec notification

GitOrigin-RevId: 14ec3e50ed4913b815620f5215df59b17fc03054
2024-07-18 08:04:41 +00:00
Lucie Germain
71a6353dfc Merge pull request #13265 from overleaf/ab-more-fr-translations
[web] French translations batch

GitOrigin-RevId: 28bdacada29615398c9b1d69004f5fa78b589614
2024-07-15 09:05:20 +00:00
Eric Mc Sween
fbdf245517 Merge pull request #19273 from overleaf/em-history-migration-concurrency
Add concurrency option to history ranges support migration script

GitOrigin-RevId: 8707abc9b76116090332b6abb11030adb17ceb4e
2024-07-15 09:00:46 +00:00
Eric Mc Sween
a95c0bbfc3 Merge pull request #19151 from overleaf/em-history-ranges-quick-migration
Add quick history ranges support migration

GitOrigin-RevId: 8446beb6bcd7384c32fc1b216e4b72d8f5d91500
2024-07-03 08:04:44 +00:00
Antoine Clausse
ebec84540f Add waitForDb in scripts/resync_subscriptions.js (#19154)
GitOrigin-RevId: 608bce2654d2b6dbeb662ad0da8bb5c5ce3d68f3
2024-06-28 08:04:29 +00:00
Antoine Clausse
01188589f8 Add some JSDoc types to @overleaf/logger (#19153)
* Add some JSDoc types to `@overleaf/logger`

* Update `logger.error` calls

* Fixup `logger.err` JSDoc

* Update `logger.err` calls

* Fix `args` type

* Remove "Error message" description

* Replace `arguments` by actual arguments of the method

* Fix: "ESLint: Unnecessary '.apply()'.(no-useless-call)"

* Add JSDoc params to `debug` `info` `warn`

* Remove extra `args` param in JSDoc so developers aren't invited to use it

Not sure if this is the best thing to do because it creates a warning in the IDE: "Parameter args is not described in JSDoc"

* Add comment about serialization of `err` `req` `res`

* Allow strings as first param in `debug` `info` `warn`

* Fix syntax for optional parameters in JSDoc

* Add 2 signatures, to avoid "string, string" params

* Fix `@signature` names copy-pastes

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* Revert the double `@param attributes`. It doesn't work

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 086dee8bbf30d577c5e1f844a9df5e518c46aca7
2024-06-28 08:04:25 +00:00
Antoine Clausse
de959fb7b9 Fix institution metrics sendAll cron (#19127)
* Don't send status 200 in the beginning: we may throw later

This was causing `Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client`
https://cloudlogging.app.goo.gl/Bchegw2iCYLxhPNk8

* Continue when failing to send one email, but log the error

* Revert "Don't send status 200...", log error instead

This reverts commit 3e13559704f822e330a18a68567f46491649f222.

* Fix `logger.error` params

* Fixup: good syntax for `logger.error`

* Remove useless `OError` import

* Update services/web/modules/metrics/app/src/MetricsEmailController.js

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* Update services/web/modules/metrics/app/src/MetricsEmailController.js

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* Update services/web/modules/metrics/app/src/MetricsEmailController.js

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* Update services/web/scripts/recurly/collect_paypal_past_due_invoice.js

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: c68bc677326729fc6c39b7c2f027e35ebb776bd2
2024-06-26 08:05:11 +00:00
Jimmy Domagala-Tang
50b19c7793 Merge pull request #18850 from overleaf/ac-promisify-resync-subscriptions
Promisify resync_subscriptions

GitOrigin-RevId: c23a99683a22104815525b179d4e46d3ad568f94
2024-06-26 08:04:57 +00:00
Mathias Jakobsen
94694a6385 Merge pull request #19057 from overleaf/mj-benchmark-seed-random
[web] Add quick and dirty PRNG for seeding benchmark runs

GitOrigin-RevId: 079f9b565f17b44d7062f6b93c26f694e486c6b9
2024-06-24 12:04:46 +00:00
ilkin-overleaf
ea6f627490 Merge pull request #18912 from overleaf/ii-remove-deleted-users-from-token-access-refs-script-fix
[web] Improve the removal of deleted users from token access refs

GitOrigin-RevId: 7b79e327388e37603a6c4f227f1d4bba743d4730
2024-06-20 08:05:22 +00:00
Eric Mc Sween
8d35177b52 Merge pull request #18985 from overleaf/em-migrate-bulk
Bulk history ranges migration

GitOrigin-RevId: 0d1846b412cfcddead63a7bc15bd06a82fbb47f3
2024-06-20 08:05:05 +00:00
Eric Mc Sween
e73fdfba63 Merge pull request #18906 from overleaf/em-migrate-existing-histories-2
History ranges migration script - second attempt

GitOrigin-RevId: 60a2c04e2a72e76a58e9e179fefc4186a96fde32
2024-06-19 08:05:02 +00:00
Jessica Lawshe
c117a64ebb Merge pull request #18902 from overleaf/ar-web-script-for-salesforce-data
[web] Add script for loading v2 salesforce data

GitOrigin-RevId: 15d3c30be166d011702d1a2d4a1d960cd4b2fa8c
2024-06-18 08:05:11 +00:00
Jakob Ackermann
e36de5a62d Merge pull request #18856 from overleaf/jpa-server-ce-shutdown
[server-ce] improve shutdown procedure

GitOrigin-RevId: 5a99868d17f597c366e42625cd39f05146dcb682
2024-06-18 08:04:04 +00:00
ilkin-overleaf
b34be6bea4 Merge pull request #18653 from overleaf/ii-invite-token-create-hmac
[web] Add HMAC tokens for project invitations

GitOrigin-RevId: 02fa01e24790c9a87f57ff9346f5346658d4dd46
2024-06-17 08:04:17 +00:00
Antoine Clausse
262a92083a Don't throw cron job when some PayPal collection fails (#18795)
* Don't throw cron job when some PayPal collection fails

Follow-up of https://github.com/overleaf/internal/pull/18414 and https://github.com/overleaf/internal/pull/18572

This was causing `Heartbeat [cron-web-collect-paypal-prod] is expired.`
And the cron to rerun (altogether three times a day, instead of once a day)

https://cloudlogging.app.goo.gl/W4qBPFDeTUkRQ8J27

* Update tests

GitOrigin-RevId: a6a29cc84c0c72fd86b2e3a9739669d3a5fb0be5
2024-06-11 08:04:04 +00:00
Antoine Clausse
01e1286a8b In PayPal collect invoices script: Update throw unsuccessful invoices collections (#18572)
* Remove throw on `INVOICES_COLLECTED_SUCCESS.length === 0`

Effectively reverts 038377b511

See: https://digital-science.slack.com/archives/C20TZCMMF/p1716973110408049

* Update tests so they don't expect rejections

* Reject when some invoice collection failed

GitOrigin-RevId: aa37f7fa37c96b8624e87d94be675d115e3250a9
2024-05-30 08:04:31 +00:00
Antoine Clausse
554be73a36 In collect_paypal_past_due_invoice.js, iterate over each page instead of gathering data from all pages at first (#18414)
* Create `getPaginatedEndpointIterator` to iterate each page

* Create `waitMs` util, it will replace `slowCallback`

* Make `handleAPIError` async

* Make `isAccountUsingPaypal` async

* Make `attemptInvoiceCollection` async

* Make `attemptInvoicesCollection` async

* Use `await` instead of `new Promise`

* Remove unused callbackified `attemptInvoiceCollection`

* Run `attemptInvoiceCollection` for each page instead of gathering all pages in the beginning

* Add test on fetching multiple pages of invoice

GitOrigin-RevId: 2674b18c6ca5732b873fb2bc71b515909006f93d
2024-05-27 10:23:18 +00:00
Eric Mc Sween
876ee4d967 Merge pull request #18225 from overleaf/em-typescript-eslint
Add typescript-eslint rule: no-floating-promises

GitOrigin-RevId: 8c3decdff537c885f5bfeb5250b7805480bc6602
2024-05-27 10:22:20 +00:00
Antoine Clausse
3300811d3a [web] Simplify removeSessionsFromRedis signature (#18440)
* Simplify `removeSessionsFromRedis` signature

* Update usage of `removeSessionsFromRedis`

* Fix tests around `removeSessionsFromRedis`

* Add comments "remove all sessions except the current session"

GitOrigin-RevId: 03bf99c14faf2c8e403bc4bcc16463a70e031284
2024-05-27 10:21:30 +00:00
Antoine Clausse
78a0bc2b05 [web] Convert RecurlyWrapper functions to async (#18384)
* Rename `RecurlyWrapper` to `promises`, as it will only contain the promises soon

* Update `apiRequest`

* Update `_parseXml`

* Update `_parseXmlAndGetAttribute`

* Update `_parse*Xml`

* Update `updateAccountEmailAddress`

* Update `checkAccountExists`

* Update `createAccount`

* Update `createBillingInfo`

* Update `setAddressAndCompanyBillingInfo`

* Update `createSubscription`

* Update `_createPaypalSubscription`

* Update `_handle422Response`

* Update `_createCreditCardSubscription`

* Update `createSubscription`

* Update `getSubscriptions`

* Update `getSubscription`

* Update `getPaginatedEndpoint`

* Update `getAccount`

* Update `getAccountActiveCoupons`

* Update `getCoupon`

* Update `getBillingInfo`

* Update `getAccountPastDueInvoices`

* Update `attemptInvoiceCollection`

* Update `updateSubscription`

* Update `createFixedAmmountCoupon`

* Update `lookupCoupon`

* Update `redeemCoupon`

* Update `extendTrial`

* Update `listAccountActiveSubscriptions`

* To find which functions to add as callbackified, I used this Regex:
`RecurlyWrapper\.(?!promises)[^.\s]*`

And after adding callbackified functions, we're left with no results with the Regex:
`RecurlyWrapper\.(?!promises|apiUrl|_buildXml|_parseXml|attemptInvoiceCollection|createFixedAmmountCoupon|getAccountActiveCoupons|getBillingInfo|getPaginatedEndpoint|getSubscription|updateAccountEmailAddress)[^.\s]*`

* Update unit tests

* Test `getSubscription` both as "promise" and as "callback"

I'm not sure if we want to generalize this.

* Fix: add missing `await`s (!!)

* Change `apiRequest` to reject errors instead of resolving it in an object

* Fixup for CollectPayPalPastDueInvoice test

* Fix: callbackify `getSubscriptions` (!!)

* Replace `.then(...)` chain by multiple `await`

* Fixup `attemptInvoicesCollection`: prevent reading length of undefined

* Use `return await` when returning promises

Per https://github.com/overleaf/internal/pull/18384#pullrequestreview-2065738771

GitOrigin-RevId: ceda755b24fd29f97a27e60ac5db9bc7e369f932
2024-05-27 10:21:26 +00:00
Antoine Clausse
25d8e053be [web] Update revokeAllUserSessions and rename it to removeSessionsFromRedis (#18360)
* Fix `revokeAllUserSessions` call in `_cleanupUser`

The user object should be passed, not the _id

* Change `revokeAllUserSessions` signature, take `req` and `stayLoggedIn` arguments

* Update uses of `revokeAllUserSessions`

* Fix promisified `revokeAllUserSessions` args

* Update tests

* Destroy or Regenerate the session in the end of `revokeAllUserSessions`

Per https://github.com/overleaf/internal/issues/17036#issuecomment-1938398570

* Revert "Destroy or Regenerate the session in the end of `revokeAllUserSessions`"

This reverts commit fe30734dbe45b27d2931d2e43a711d591bb85787.

* Rename `revokeAllUserSessions` to `removeSessionsFromRedis`

* Fixup tests

* Fix: add optional chaining in `req.sessionID` (!!)

GitOrigin-RevId: d41676bf00f463230af495e09c65fb9ee521f49f
2024-05-20 08:04:12 +00:00
Antoine Clausse
d7f0c66d86 [web] Rename team name IEEECollabratec to IEEEPublications (#18387)
* Replaces all `IEEECollabratec` by `IEEEPublications`

* Revert change in migration

GitOrigin-RevId: f0445b726abdc5f65b7a0613d9979b081332685e
2024-05-17 08:05:06 +00:00
Brian Gough
598eaf08fd Merge pull request #17816 from overleaf/bg-eslint-rule-for-find-with-await
add eslint rule for find with await

GitOrigin-RevId: 7e78104e610073ff3c151d7314753c8301a8c787
2024-05-17 08:04:58 +00:00
Antoine Clausse
9419cc3b37 [web] Add tests to collect_paypal_past_due_invoice.js + update logging (#18310)
* Fix: Invoices collected array length comparison

Update the code with the correct condition to respect the intent of the previous implementation ("exit with non zero code when no invoicess were processed").
See 5476f39984

However, I'm not sure if erroring when no invoices are collected is actually what we want to do.

* Wrap `collect_paypal_past_due_invoice` script and export the function

* Fixup typo `accoutns`

* Log invoices collection data before throwing

* Add note: `handleAPIError` is silencing the errors

* Create a test on `collect_paypal_past_due_invoice`

* Replace `console.log` by `@overleaf/logger` (bunyan)

Our `console.warn` show up as Errors (in red) in GCP. For example the following is an error in GCP:
```
Errors in attemptInvoiceCollection with id=2693634 OError: Recurly API returned with status code: 400
```
https://github.com/overleaf/internal/blob/5476f39/services/web/scripts/recurly/collect_paypal_past_due_invoice.js#L9

---

Does it correctly set the levels as warnings if we use `@overleaf/logger`

GitOrigin-RevId: 37c8bdf4afd8cef4706700aafb44480ec8966a74
2024-05-15 08:04:46 +00:00
Antoine Clausse
a26c655220 Delete 3 migration scripts for compile-timeouts (#18163)
Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 2d66052994159b6d902b807f02488095d65562e1
2024-05-01 08:05:00 +00:00
Antoine Clausse
cdd79e8ec0 Fix: unset recent users featuresUpdatedAt after wrong update (#18149)
* Copy previous script

* Remove `featuresUpdatedAt` that was wrongly set on recent users

* Fix! `signupDate` -> `signUpDate`

* Add test on `migration_compile_timeout_60s_to_20s_fixup_new_users.js`

* style: `$unset: { featuresUpdatedAt: 1 }` -> `$unset: { featuresUpdatedAt: '' }`

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

* Add comment on test (https://github.com/overleaf/internal/pull/18149#discussion_r1582999534)

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>

---------

Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com>
GitOrigin-RevId: 408f5c7d48e60722aba736167b8e8858e9570d99
2024-04-30 08:04:33 +00:00
Antoine Clausse
711d50a2f1 [web] Create script to update forgotten featuresUpdatedAt after the migration to 20s compile timeout (#18113)
* Copy `migration_compile_timeout_60s_to_20s.js` script

* Update `featuresUpdatedAt`

* Add a comment about `featuresUpdatedAt` in migration_compile_timeout_60s_to_20s.js

* Fix test on migration_compile_timeout_60s_to_20s.js

* Fix: Include users having `featuresUpdatedAt` undefined in the update

* Add test on `migration_compile_timeout_60s_to_20s_fixup_features_updated_at`

GitOrigin-RevId: 4b2baf955a6a9f39bf9ce00b7839af551064c6cb
2024-04-30 08:04:28 +00:00
roo hutton
2469c93da4 Merge pull request #17870 from overleaf/bg-node-upgrade-18-20-2
upgrade node to 18.20.2

GitOrigin-RevId: e4ac30c740ced3885353f1472cf690eb5ec0ece1
2024-04-24 08:05:16 +00:00