Commit graph

88 commits

Author SHA1 Message Date
James Allen
662122bb1c CI and local dev environment improvements
The need for this became very noticeable due to the slowness of filesystem access in docker-in-mac, with a full compile taking over a minute for me in docker. Using make to introduce incremental compile makes this near instantaneous outside of docker (if only a few files have changed), and quick enough inside docker.

With incremental compile via make, it compiles quickly enough that re-compiling and restarting the web service automatically when backend files change is quick enough now. This is how the service is run via docker-compose in https://github.com/sharelatex/sharelatex-dev-environment, so it shouldn't be necessary to manually restart the container each time a coffee file changes.

At the moment Jenkins pull web modules in via the GitSCM plugin, but I believe this is creating a dependency in Jenkins, where any commits to any of the modules causes all of the web branches to rebuild. By doing it via our own scripts we can hopefully avoid this. It also creates a build process which is reproducible locally.

**Note that at the moment in this PR all modules pull from `ja-dockerize-dev` branches, but these should be merged first, and this PR updated to point to the master branches before merging**. This is necessary for other changes to build process/docker-compose workflow.

As well as a Makefile for web, there is now a `Makefile.module`. This is copied into each module directory by the top-level Makefile, and is written in a way to be flexible and support unit tests, acceptance tests, front-end js for the ide and main, and the modules `app/coffee` directory, while allowing modules to have some of these missing (not all modules have e.g. acceptance tests, or front-end JS). This will allows us to refine the build process in future, without needing to update the Makefile in each module repo separately (I found this to be a painful part of this development).

This makes web compatible with the docker-compose workflow at https://github.com/sharelatex/sharelatex-dev-environment, where each service is running in its own docker container, with networking managed by docker.

Previously the Makefile was set up to run unit tests in docker with `make unit_tests`. This now just runs them natively. In the CI, they are run in docker anyway (all steps in Jenkins are), and locally, they run fine natively with `npm run test:unit`, or can be run in docker via https://github.com/sharelatex/sharelatex-dev-environment with `bin/run web_sl npm run test:unit`.

Previously we did a lot of juggling with only mounting source files (coffee, less, etc) into the docker container for acceptance tests. This was to avoid creating root owned files if the whole directory was mounted. Now instead the whole web directory is mounted read-only, with the compilation step done outside of the container before running the tests.

This allows the host and container to share the `node_modules` folder as well, which avoids needing to `npm install` twice on the CI box, and should speed up the build by a few minutes.

On macOS, this would cause a problem with compiled modules if you tried to use the same `node_modules` to run the app natively. However, if running via docker-compose in https://github.com/sharelatex/sharelatex-dev-environment, this is no longer a problem.
2018-01-15 09:36:41 +00:00
Tim Alby
08b74c95f5 proxy requests to analytics service 2018-01-10 15:03:14 +02:00
Henry Oswald
bd7e2d956d add mandrill as email option 2017-12-11 11:05:04 +00:00
Paulo Reis
7c5f6539ae Remove translations from shrinkwrap. 2017-10-24 16:24:27 +01:00
Brian Gough
d6d76f132e replace unzip with yauzl 2017-10-16 14:15:38 +01:00
James Allen
38b8450cee Add jsonwebtoken package 2017-10-03 10:29:49 +01:00
Shane Kilkelly
54070c7734 pin chai 2017-09-12 11:21:34 +01:00
Tim Alby
d6834ff417 add security headers using Helmet
- use all Helmet's default headers except `X-DNS-Prefetch-Control`
- use `Referrer-Policy`
- use cache headers when:
  - a user is logged in, OR
  - a project is displayed
2017-09-12 11:17:59 +02:00
Paulo Reis
98b380f70a Update translations commit in shrinkwrap. 2017-09-08 12:14:05 +01:00
James Allen
b4254f06ef Update Project schema for overleaf imports 2017-08-21 17:49:08 +02:00
James Allen
eac0ce8353 Initial spike of Overleaf based logins 2017-08-08 14:00:21 +02:00
Paulo Reis
b5486155c0 Update shrinkwrap to include the HTML encoder. 2017-07-31 14:01:22 +01:00
Joe Green
10683cbd8c upgraded mongoose package 2017-07-27 10:58:20 +01:00
James Allen
e19f5a1a5e Update redis-sharelatex and shrinkwrap 2017-05-09 17:18:04 +01:00
Shane Kilkelly
1e46a5bf7a update shrinkwrap 2017-04-13 10:12:20 +01:00
Shane Kilkelly
061d2e604f re-add npm-shrinkwrap.json 2017-04-05 10:17:32 +01:00
Shane Kilkelly
98d7dc6f88 Make work on both 0.10 and 6.9 2017-02-27 13:54:57 +00:00
Shane Kilkelly
8d7c39ff42 Merge branch 'master' into node-6.9 2017-02-27 11:25:39 +00:00
Shane Kilkelly
bd997fc574 pin sinon js to 1.17, version 2 betas cause breakage 2017-02-27 11:01:25 +00:00
Shane Kilkelly
621a07aff2 Merge branch 'master' into node-6.9 2017-02-14 11:01:14 +00:00
Shane Kilkelly
4e9426e6bf Merge branch 'master' into sk-pug 2017-01-30 14:36:10 +00:00
Shane Kilkelly
3ed85dad42 Merge branch 'master' into sk-upgrade-uuid 2017-01-30 11:09:50 +00:00
Shane Kilkelly
60500bc4ea update shrinkwrap 2017-01-25 10:52:08 +00:00
Shane Kilkelly
f7b4e00466 Restore npm-shrinkwrap.json 2017-01-25 10:30:53 +00:00
Shane Kilkelly
abbffb8e4b Remove deprecated node-uuid package, replace with uuid. 2017-01-23 10:03:16 +00:00
Shane Kilkelly
57cd54bf55 WIP: migrate from jade to pug 2017-01-20 12:03:02 +00:00
Shane Kilkelly
64f69069b2 Experimental: upgrade to node 6.9.2 (latest LTS release) 2016-12-21 10:23:42 +00:00
Shane Kilkelly
ef0a5801d5 Create a RedisWrapper, and use it for rate limiting. 2016-12-19 12:17:02 +00:00
Shane Kilkelly
803209e668 update shrinkwrap 2016-11-14 10:52:46 +00:00
Shane Kilkelly
bfa0e7cf89 WIP: start moving web sessions to cluster 2016-11-08 15:32:36 +00:00
Shane Kilkelly
b16a533610 add npm-shrinkwrap.json 2016-11-04 10:04:12 +00:00
Shane Kilkelly
41cc9e68b5 update npm shrink-wrap 2016-09-21 15:15:44 +01:00
Paulo Reis
64d0b8bc7c Update shrinkwrap. 2016-08-10 17:46:22 +01:00
Paulo Reis
8eae4e4044 Revert "Use custom shrinkwrap to ignore the translations package."
This reverts commit 25a8a7cd9a59f738277ce893318c69db539384d6.
2016-07-12 13:42:46 +01:00
Paulo Reis
112a64ce84 Use custom shrinkwrap to ignore the translations package. 2016-07-12 13:37:26 +01:00
Henry Oswald
a9f3a29b72 moved translations to dev dependency so its not included in shrinkwrap. 2016-07-12 10:17:58 +01:00
Brian Gough
e0d262a6d8 fix shrinkwrap file 2016-06-28 11:18:20 +01:00
Brian Gough
6e187e3996 added npm shrinkwrap file 2016-06-28 10:42:36 +01:00