Commit graph

95 commits

Author SHA1 Message Date
Jakob Ackermann
94c208311c Merge pull request #5351 from overleaf/jpa-clsi-drop-sqlite
[clsi] goodbye sqlite db

GitOrigin-RevId: e7f16920c44e74a425b92884b48a60272dfb015b
2021-10-07 08:03:50 +00:00
Jakob Ackermann
02918e7483 Merge pull request #4649 from overleaf/jpa-fs-based-caching
[perf] UrlCache: pure fs based cache state for downloads

GitOrigin-RevId: d19afc396324d4c3318b31620c8ad0c04e0544ce
2021-10-07 08:03:18 +00:00
Eric Mc Sween
9ee92daea3 Merge pull request #4893 from overleaf/em-synctex
Use the synctex distributed with TeX Live

GitOrigin-RevId: 5a133f21f48fd1e217ab463e8cb2a5cdec8be1af
2021-09-07 16:26:17 +00:00
Jakob Ackermann
f285e503b4 [misc] run format_fix and lint:fix 2021-07-13 12:04:48 +01:00
Jakob Ackermann
64551f0198 [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:21 +01:00
Jakob Ackermann
294088fb27 [ContentCacheManager] use PDF.js Xref table instead of stream detection (#242)
* make the content cache manager tests configurable

* extend stream content in unit tests

* [ContentCacheManagerTests] prepare for full object caching

* filesystem stream for pdfjs

* working??

* cleaning up

* handle overflow

* [misc] install pdfjs-dist

* [misc] move pdfjs code into app/lib/ and scripts/, also use CamelCase

* [misc] abstract the file loading and parsing of xRef tables into helper

* [misc] pdfjsTests: add snapshot based tests for the Xref table parser

* [misc] FSStream: throw proper error and drop commented code

* [misc] FSStream: integrate throwing of MissingDataException into getter

* [misc] pdfjs: fix eslint errors

* [misc] pdfjs: run format_fix

* [misc] pdfjs: allocate very small non empty dummy buffers explicitly

* [misc] install @overleaf/o-error

* [ContentCacheManager] use PDF.js Xref table instead of stream detection

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>

* [pdfjs] parseXrefTable: handle empty PDF files gracefully

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
2021-05-31 09:20:25 +01:00
Jakob Ackermann
d56a490121 [misc] AllowedImageNamesTests: add missing done callbacks 2021-05-25 10:24:49 +01:00
Alf Eaton
6507d668ae Update references to .coffee files 2021-05-19 18:57:35 +01:00
Alf Eaton
1b1c4aa4a0 Fix formatting 2021-05-19 18:47:51 +01:00
Alf Eaton
f31abe7cb3 Tidy and format 2021-05-19 18:36:27 +01:00
decaffeinate
85920b3120 decaffeinate: Run post-processing cleanups on settings.test.coffee 2021-05-19 16:50:46 +01:00
decaffeinate
8c984a77c4 decaffeinate: Convert settings.test.coffee to JS 2021-05-19 16:50:45 +01:00
decaffeinate
5af8f57dd0 decaffeinate: Rename settings.test.coffee from .coffee to .js 2021-05-19 16:50:45 +01:00
Jakob Ackermann
b456ea726d [misc] merge pdf caching into main (#226)
* wip generate directory for hash content

* cleanup, remove console logging

* add content caching module

* Return PDF stream ranges with compile response

* Return the PDF file size in the compile response

* PDF range endpoint

* [misc] WIP: pdf caching: preserve the m-time on static content files

* [misc] WIP: pdf caching: improve browser caching, emit caching headers

* [misc] WIP: pdf caching: do not emit very small chunks <1kB

* [misc] keep up with moving output files into a separate directory

* [OutputCacheManager] add global feature flag for enabling pdf caching

* [misc] add contentId into the URL for protecting PDF stream contents

* [misc] support PDF stream caching for anonymous users

* [misc] add per-request feature flag for enabling PDF stream caching

* [misc] enable pdf caching in CI and emit metrics at the end of run

* [misc] expose compile stats and timings to the frontend

* [misc] log an error in case saving output files fails

* [misc] add metrics for pdf bandwidth and pdf caching performance

* [misc] add a dark mode to the pdf caching for computing ranges only

* [misc] move pdf caching metrics into ContentCacheMetrics

* [misc] add a config option for the min chunk size of pdf ranges

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
2021-05-13 14:07:54 +01:00
Jakob Ackermann
6cb4246eb1 [misc] consume and validate a custom imageName for synctex requests 2021-03-30 13:57:40 +01:00
Eric Mc Sween
9f2d219102 Add a global test setup file
Set up generally useful stuff:

* chai.should()
* logger stubs
* globals in SandboxedModule, including Buffer and process, which are
  now required in Node 12
2021-03-12 16:27:33 -05:00
Jakob Ackermann
1d1b9ebebc [misc] bump the dev-env to 3.3.2 2020-08-10 17:01:11 +01:00
Jakob Ackermann
0f9d0d9790 Merge pull request #183 from overleaf/jpa-clsi-allowed-image-names
[misc] RequestParser: restrict imageName to an allow list and add tests
2020-07-15 10:58:36 +02:00
Jakob Ackermann
cd87d3018e [misc] fix formatting 2020-07-03 11:47:53 +01:00
Jakob Ackermann
a5c3bad7f1 [ExampleDocumentTests] drop out in case of an error during compilation
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-07-03 11:38:12 +01:00
Jakob Ackermann
c857371fed [misc] wordcount: restrict image to an allow list and add tests 2020-06-26 13:28:12 +01:00
Jakob Ackermann
8846efe7ce [misc] RequestParser: restrict imageName to an allow list and add tests 2020-06-26 13:28:09 +01:00
Brian Gough
262ea01911 add acceptance test for synctex when project/file does not exist 2020-06-15 10:55:01 +01:00
Brian Gough
f99023320d use json parsing in request 2020-06-12 15:15:51 +01:00
Henry Oswald
77e8ba74a7 add pipeUrlToFileWithRetry function to retry file downloads 3 times 2020-05-14 13:24:58 +01:00
Jakob Ackermann
101c4ea4dc [misc] use a directory in /tmp for temporary data 2020-04-03 12:36:32 +02:00
mserranom
778f2a8420 prettier: convert test/acceptance decaffeinated files to Prettier format 2020-02-19 12:16:14 +01:00
mserranom
c5a6496b71 decaffeinate: rename test/acceptance/coffee to test/acceptance/js 2020-02-19 12:16:11 +01:00
decaffeinate
d1da978e39 decaffeinate: Run post-processing cleanups on BrokenLatexFileTests.coffee and 9 other files 2020-02-19 12:16:07 +01:00
decaffeinate
3d3861cb24 decaffeinate: Convert BrokenLatexFileTests.coffee and 9 other files to JS 2020-02-19 12:16:00 +01:00
decaffeinate
035786b204 decaffeinate: Rename BrokenLatexFileTests.coffee and 9 other files from .coffee to .js 2020-02-19 12:15:54 +01:00
Eric Mc Sween
dab6e9aa8e Send output files on timeout
The unconventional use of callbacks to return both an error and data
after compilation created a subtle bug where the output files were
dropped by the LockManager in case of an error such as a timeout.

This prevented the frontend to show error logs when a timeout occurs,
creating confusion among users.

We now attach the output files to the error so that they reach the
controller and are sent back to the web service.
2019-10-22 16:31:37 -04:00
Brian Gough
bc6e560448 increase timeout for long-running acceptance tests 2019-04-01 09:42:54 +01:00
Christopher Hoskin
7a3f1c81bd Switch to node 10 2019-01-16 15:11:49 +00:00
Henry Oswald
3c12f60eeb change timeout test latex code 2018-09-11 11:34:25 +01:00
Henry Oswald
4c96abd6c5 grep works with command
updated build scripts
acceptence tests break, files are written as root when user is node
2018-03-29 17:07:22 +01:00
Henry Oswald
9a519f0d3d added docker runner into core codebase
supports both local command runner and docker runner

added docker files for tex live

also fixed tests so they exit correctly & removed debug lines
2018-03-14 15:44:49 +00:00
Henry Oswald
2168f20715 uncomment tests 2018-03-02 17:59:37 +00:00
Henry Oswald
f9b6b3dda8 make timeout latex more complex(slower) 2018-03-02 17:58:34 +00:00
Henry Oswald
f39d14bf1b unit tests pass, acceptence fail
uncomment tests
2018-03-02 17:34:41 +00:00
Henry Oswald
3399f55153 wip, docker container is correctly created 2018-03-02 17:14:23 +00:00
Henry Oswald
fdac655cd4 acceptence tests pass inside docker container (apart from sync) 2018-03-02 17:14:23 +00:00
Henry Oswald
5cf3c904cb mvp
needs hacked pacth in docker runner

wip

most tests pass
2018-03-02 17:14:20 +00:00
James Allen
df21b9de53 Provide hosts and siblings container as environment settings and add npm run start script
wip acceptence tests run, but don't all pass

wip

removed npm-debug from git
2018-03-02 17:14:18 +00:00
Brian Gough
f1c59150b7 exit if mock server fails to start 2017-10-20 15:16:35 +01:00
Joe Green
4ff6cd3006 Jg jenkinsfile cleanup (#75)
* Update Jenkinsfile

make sure we don't ship unneeded build files

* Update ExampleDocumentTests.coffee

* use node 6.11.2 in jenkins file
2017-09-28 11:50:33 +01:00
Joe Green
41e442d403 Add jenkinsfile (#72)
* create Jenkinsfile

* allow textlive image to be set with env vars

* log error message in test

* use sandboxed compiles variables

* Add SANDBOXED_COMPILES_HOST_DIR var to test config

* add SIBLING_CONTAINER_USER env var
2017-09-08 14:06:04 +01:00
Brian Gough
4a4f3bb24c update acceptance test images for texlive 2017 2017-07-13 13:15:51 +01:00
Brian Gough
0c7a89dfa5 update acceptance tests settings to 2017 image 2017-07-12 16:59:33 +01:00
Brian Gough
f7c3565281 use pdfinfo on output to ensure pdfs are optimised
needed to check that qpdf runs correctly inside the docker container
2017-04-07 11:11:27 +01:00