overleaf/services/web
Jakob Ackermann 7db7cd4a49 [misc] merge pdf caching into main (#4033)
* [frontend] WIP: pdf caching using service worker -- squashed

Ref: 920fbaa00b31530f7c457a2d93bad5e553798057
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
Co-Authored-By: Eric Mc Sween <eric.mcsween@overleaf.com>

* [misc] add contentId into the URL for protecting PDF stream contents

* [misc] gracefully handle missing ranges in serviceWorker

* [misc] support PDF stream caching for anonymous users

* [misc] polish header names and add URL to error message when fetch fails

* [misc] polish event handler registration

* [misc] limit serviceWorker scope to /project/ -- trailing slash

This will block the service worker from intercepting requests on the
 project dashboard.

* [misc] add per-request feature flag for enabling PDF stream caching

* [misc] expose compile stats and timings to the frontend

* [misc] serviceWorker: support clsiServerId and compileGroup url params

* [misc] serviceWorker: polish header maps

* [misc] serviceWorker: drop TODO for p-limit -- the browser has a queue

* [misc] serviceWorker: drop verbose log message on every fetch

* [misc] cut down size of diff in backend code

* [misc] add test case for forwarding of pdf caching and metrics details

* [misc] serviceWorker: drop all the log lines

* [misc] serviceWorker: add boundary guards to the compile request regex

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
GitOrigin-RevId: 4b291b4a4f2866cf07bccf8ec9068f33bbfdc916
2021-05-18 02:07:57 +00:00
..
.github
.storybook
.vscode
app [misc] merge pdf caching into main (#4033) 2021-05-18 02:07:57 +00:00
bin
config [misc] merge pdf caching into main (#4033) 2021-05-18 02:07:57 +00:00
data
frontend [misc] merge pdf caching into main (#4033) 2021-05-18 02:07:57 +00:00
locales Add Symbol Palette component, data and stories (#4027) 2021-05-14 02:09:50 +00:00
modules
public
scripts fix mongo sort argument (#4021) 2021-05-14 02:09:55 +00:00
test [misc] merge pdf caching into main (#4033) 2021-05-18 02:07:57 +00:00
transform/o-error
.dockerignore
.eastrc
.eslintignore
.eslintrc
.gitignore
.nvmrc
.prettierignore
.prettierrc
app.js
babel.config.json
docker-compose.ci.yml
docker-compose.yml
Dockerfile
Dockerfile.frontend
Dockerfile.frontend.ci
i18next-scanner.config.js
install_deps.sh
karma.conf.js
LICENSE
Makefile Merge pull request #4016 from overleaf/jpa-faster-ci-4288 2021-05-18 02:07:44 +00:00
Makefile.module Merge pull request #4016 from overleaf/jpa-faster-ci-4288 2021-05-18 02:07:44 +00:00
nodemon.json
package-lock.json Add Symbol Palette component, data and stories (#4027) 2021-05-14 02:09:50 +00:00
package.json Add Symbol Palette component, data and stories (#4027) 2021-05-14 02:09:50 +00:00
README.md
webpack.config.dev.js
webpack.config.js [misc] merge pdf caching into main (#4033) 2021-05-18 02:07:57 +00:00
webpack.config.prod.js
webpack.config.test.js

overleaf/web

overleaf/web is the front-end web service of the open-source web-based collaborative LaTeX editor, Overleaf. It serves all the HTML pages, CSS and javascript to the client. overleaf/web also contains a lot of logic around creating and editing projects, and account management.

The rest of the Overleaf stack, along with information about contributing can be found in the overleaf/overleaf repository.

Build process

overleaf/web uses Grunt to build its front-end related assets.

Image processing tasks are commented out in the gruntfile and the needed packages aren't presently in the project's package.json. If the images need to be processed again (minified and sprited), start by fetching the packages (npm install grunt-contrib-imagemin grunt-sprity), then decomment the tasks in Gruntfile.coffee. After this, the tasks can be called (explicitly, via grunt imagemin and grunt sprity).

New Docker-based build process

Note that the Grunt workflow from above should still work, but we are transitioning to a Docker based testing workflow, which is documented below:

Running the app

The app runs natively using npm and Node on the local system:

$ npm install
$ npm run start

Ideally the app would run in Docker like the tests below, but with host networking not supported in OS X, we need to run it natively until all services are Dockerised.

Running Tests

To run all tests run:

make test

To run both unit and acceptance tests for a module run:

make test_module MODULE=overleaf-integration

Unit Tests

The test suites run in Docker.

Unit tests can be run in the test_unit container defined in docker-compose.tests.yml.

The makefile contains a short cut to run these:

make test_unit

During development it is often useful to only run a subset of tests, which can be configured with arguments to the mocha CLI:

make test_unit MOCHA_GREP='AuthorizationManager'

To run only the unit tests for a single module do:

make test_unit_module MODULE=overleaf-integration

Module tests can also use a MOCHA_GREP argument:

make test_unit_module MODULE=overleaf-integration MOCHA_GREP=SSO

Acceptance Tests

Acceptance tests are run against a live service, which runs in the acceptance_test container defined in docker-compose.tests.yml.

To run the tests out-of-the-box, the makefile defines:

make test_acceptance

However, during development it is often useful to leave the service running for rapid iteration on the acceptance tests. This can be done with:

make test_acceptance_app_start_service
make test_acceptance_app_run # Run as many times as needed during development
make test_acceptance_app_stop_service

make test_acceptance just runs these three commands in sequence and then runs make test_acceptance_modules which performs the tests for each module in the modules directory. (Note that there is not currently an equivalent to the -start / -run x n / -stop series for modules.)

During development it is often useful to only run a subset of tests, which can be configured with arguments to the mocha CLI:

make test_acceptance_run MOCHA_GREP='AuthorizationManager'

To run only the acceptance tests for a single module do:

make test_acceptance_module MODULE=overleaf-integration

Module tests can also use a MOCHA_GREP argument:

make test_acceptance_module MODULE=overleaf-integration MOCHA_GREP=SSO

Routes

Run bin/routes to print out all routes in the project.

License and Credits

This project is licensed under the AGPLv3 license

Stylesheets

Overleaf is based on Bootstrap, which is licensed under the MIT license. All modifications (*.less files in public/stylesheets) are also licensed under the MIT license.

Artwork

Silk icon set 1.3

We gratefully acknowledge Mark James for releasing his Silk icon set under the Creative Commons Attribution 2.5 license. Some of these icons are used within Overleaf inside the public/img/silk and public/brand/icons directories.

IconShock icons

We gratefully acknowledge IconShock for use of the icons in the public/img/iconshock directory found via findicons.com