* make the content cache manager tests configurable
* extend stream content in unit tests
* [ContentCacheManagerTests] prepare for full object caching
* filesystem stream for pdfjs
* working??
* cleaning up
* handle overflow
* [misc] install pdfjs-dist
* [misc] move pdfjs code into app/lib/ and scripts/, also use CamelCase
* [misc] abstract the file loading and parsing of xRef tables into helper
* [misc] pdfjsTests: add snapshot based tests for the Xref table parser
* [misc] FSStream: throw proper error and drop commented code
* [misc] FSStream: integrate throwing of MissingDataException into getter
* [misc] pdfjs: fix eslint errors
* [misc] pdfjs: run format_fix
* [misc] pdfjs: allocate very small non empty dummy buffers explicitly
* [misc] install @overleaf/o-error
* [ContentCacheManager] use PDF.js Xref table instead of stream detection
Co-Authored-By: Brian Gough <brian.gough@overleaf.com>
* [pdfjs] parseXrefTable: handle empty PDF files gracefully
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
* wip generate directory for hash content
* cleanup, remove console logging
* add content caching module
* Return PDF stream ranges with compile response
* Return the PDF file size in the compile response
* PDF range endpoint
* [misc] WIP: pdf caching: preserve the m-time on static content files
* [misc] WIP: pdf caching: improve browser caching, emit caching headers
* [misc] WIP: pdf caching: do not emit very small chunks <1kB
* [misc] keep up with moving output files into a separate directory
* [OutputCacheManager] add global feature flag for enabling pdf caching
* [misc] add contentId into the URL for protecting PDF stream contents
* [misc] support PDF stream caching for anonymous users
* [misc] add per-request feature flag for enabling PDF stream caching
* [misc] enable pdf caching in CI and emit metrics at the end of run
* [misc] expose compile stats and timings to the frontend
* [misc] log an error in case saving output files fails
* [misc] add metrics for pdf bandwidth and pdf caching performance
* [misc] add a dark mode to the pdf caching for computing ranges only
* [misc] move pdf caching metrics into ContentCacheMetrics
* [misc] add a config option for the min chunk size of pdf ranges
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
Set up generally useful stuff:
* chai.should()
* logger stubs
* globals in SandboxedModule, including Buffer and process, which are
now required in Node 12
The unconventional use of callbacks to return both an error and data
after compilation created a subtle bug where the output files were
dropped by the LockManager in case of an error such as a timeout.
This prevented the frontend to show error logs when a timeout occurs,
creating confusion among users.
We now attach the output files to the error so that they reach the
controller and are sent back to the web service.
* create Jenkinsfile
* allow textlive image to be set with env vars
* log error message in test
* use sandboxed compiles variables
* Add SANDBOXED_COMPILES_HOST_DIR var to test config
* add SIBLING_CONTAINER_USER env var
