github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-07 20:31:06 -05:00
Code Issues Projects Releases Packages Wiki Activity
19882 commits 3 branches 7 tags 201 MiB
Commit graph

190 commits

Author SHA1 Message Date
Jakob Ackermann
956cacaef7 Merge pull request #10139 from overleaf/jpa-split-test-min-chunk-size 
[misc] add split test for a per request pdfCachingMinChunkSize

GitOrigin-RevId: 6a8a3c6267501789f2047a67b03db6ac6df427c3
 2022-10-26 08:03:39 +00:00
Jakob Ackermann
e96f7df704 Merge pull request #9475 from overleaf/jpa-ta-clsi-root-doc-path 
[clsi] remove stripping of characters from the root doc path

GitOrigin-RevId: 18a8a5c66eecc1cf21aa202180720707d27671ea
 2022-09-02 08:05:02 +00:00
Jakob Ackermann
18768d3f1b Merge pull request #8950 from overleaf/bg-use-qpdf-xref-table 
[clsi] parse xref table from qpdf output

GitOrigin-RevId: cc400c65082c7c8c28c21037b97b4fde66a85835
 2022-08-03 08:04:10 +00:00
Brian Gough
9ea1226460 Merge pull request #8969 from overleaf/bg-issue8967 
Apply draft mode to graphics package as well (in addition to graphicx)

GitOrigin-RevId: 6137f90ef9d579f0e6554801e8f2e0f5dee2aa83
 2022-07-25 08:03:57 +00:00
Jakob Ackermann
dd5128c1d7 Merge pull request #8952 from overleaf/jpa-pdf-caching-tweaks 
[clsi] server side pdf caching tweaks

GitOrigin-RevId: 758cbcc45b5a7ca0fe3dbf31bc43d9b0ef36e599
 2022-07-21 08:04:41 +00:00
Eric Mc Sween
9199669c0f Merge pull request #8921 from overleaf/em-draft-mode 
Change draft mode implementation

GitOrigin-RevId: 3de81a3643cc024c410b7b49e77cd41c7fec8294
 2022-07-20 08:04:06 +00:00
Eric Mc Sween
7f6e2c971e Merge pull request #8757 from overleaf/em-clear-cache-on-timeout 
Clear project after a compile has been interrupted

GitOrigin-RevId: 1b8e85c39843de3f63b7119f3a8a433567f57bcb
 2022-07-19 08:03:56 +00:00
Jakob Ackermann
97624d0c6c Merge pull request #8847 from overleaf/jpa-emit-start-of-xref-table 
[clsi] try to emit the start of the xref table

GitOrigin-RevId: 6d8348a349572cc997ac5924664428228c00fed1
 2022-07-18 08:04:17 +00:00
Jakob Ackermann
b4402c41e8 Merge pull request #8808 from overleaf/jpa-clsi-linearize-examples 
[clsi] linearize example pdfs and update xref table snapshots

GitOrigin-RevId: 5cfad6fdc4f6aef3366b62ca1c171f92e5f392fd
 2022-07-12 08:03:30 +00:00
Eric Mc Sween
77aa2baa9d Merge pull request #8714 from overleaf/em-promisify-compile-manager 
Promisify CompileManager

GitOrigin-RevId: 644ed061ae139d6196b24f8ead38579de6b844a3
 2022-07-08 08:04:22 +00:00
Eric Mc Sween
baaf4d4240 Merge pull request #8286 from overleaf/em-halt-on-error-backend 
Stop on first error backend implementation

GitOrigin-RevId: 497b1ed2c13f544760d8ad8d029359db75275389
 2022-06-07 08:02:21 +00:00
Eric Mc Sween
bda307fbb5 Merge pull request #8267 from overleaf/em-decaf-clsi 
Decaf cleanup some CLSI files

GitOrigin-RevId: afb4cd6e9eb9e95703efa955b93d4fedada10e3c
 2022-06-07 08:02:18 +00:00
Jakob Ackermann
a78bcee15f Merge pull request #8135 from overleaf/jpa-refactor-zonal-download 
[misc] refactor handling of zone prefix in compile response

GitOrigin-RevId: f1f33d7d257854176f383bb5d786710f6b09f737
 2022-05-26 08:03:53 +00:00
Jakob Ackermann
9a71372c36 Merge pull request #8071 from overleaf/jpa-clsi-zonal 
[misc] prefix output file downloads with /zone/X

GitOrigin-RevId: ba59e97ae0284c68ba551dd49dc5d3daa4d61aa9
 2022-05-25 08:08:39 +00:00
Jakob Ackermann
f0bd6dda23 Merge pull request #7986 from overleaf/jpa-eslint-8 
[misc] upgrade eslint packages to the latest version everywhere

GitOrigin-RevId: f1480d4a171acef82fb26c4aa54be3a6088b0ab3
 2022-05-17 08:05:59 +00:00
Jakob Ackermann
e1b07cd40a Merge pull request #7766 from overleaf/jpa-em-clsi-node-16 
[clsi] upgrade node docker image to upstream version 16

GitOrigin-RevId: 1f7a7ef67c94fd83a2df1061350ba52b8d01e640
 2022-04-27 08:04:26 +00:00
Eric Mc Sween
2401909919 Merge pull request #7381 from overleaf/em-revert-node-16 
Revert part of the Node 16 upgrade

GitOrigin-RevId: 0b1f7c1af6d692510a206bb73dae605ae8095023
 2022-04-05 12:20:57 +00:00
Eric Mc Sween
3235119302 Merge pull request #7228 from overleaf/em-node-16 
Upgrade to Node 16

GitOrigin-RevId: 3db1ae57ffb02f8a2b9012ffbb3efecfc01d2b04
 2022-04-05 12:20:52 +00:00
Jakob Ackermann
264b107e39 Merge pull request #6927 from overleaf/jpa-split-clsi-metrics 
[misc] split clsi metrics into multiple categories

GitOrigin-RevId: 964ab57b344b92383a2937beff50139bae273a0e
 2022-03-02 09:02:52 +00:00
Eric Mc Sween
f3f0be5c56 Merge pull request #6211 from overleaf/em-code-sharing 
Change directory layout in service containers

GitOrigin-RevId: 09ff19db2a123cbf7691d51e9ce9be6eee264287
 2022-01-07 09:03:22 +00:00
Jakob Ackermann
9c9beef305 Merge pull request #5737 from overleaf/jpa-clsi-cleanup-in-memory 
[clsi] separate clearing cache and clearing output cache

GitOrigin-RevId: 9cd903eaf8d291d4821ba8c9ac6c1a9f8d3d9632
 2021-11-16 09:02:36 +00:00
Jakob Ackermann
669606f797 Merge pull request #5568 from overleaf/jpa-node-no-callback-literal 
[misc] fix eslint violations for node/no-callback-literal

GitOrigin-RevId: af5aace52e1476b1b0ee48cc8be2aabbe04efac6
 2021-10-28 08:03:33 +00:00
Jakob Ackermann
7f9fd00bda Merge pull request #5367 from overleaf/jpa-node-handle-callback-err 
[misc] fix eslint violations for node/handle-callback-err

GitOrigin-RevId: 83a4900e8861010df1917bff49382bd9c93375bd
 2021-10-28 08:03:26 +00:00
Jakob Ackermann
94c208311c Merge pull request #5351 from overleaf/jpa-clsi-drop-sqlite 
[clsi] goodbye sqlite db

GitOrigin-RevId: e7f16920c44e74a425b92884b48a60272dfb015b
 2021-10-07 08:03:50 +00:00
Jakob Ackermann
02918e7483 Merge pull request #4649 from overleaf/jpa-fs-based-caching 
[perf] UrlCache: pure fs based cache state for downloads

GitOrigin-RevId: d19afc396324d4c3318b31620c8ad0c04e0544ce
 2021-10-07 08:03:18 +00:00
Brian Gough
cc1b73336a Merge pull request #4944 from overleaf/bg-use-file-line-errors 
use file line errors in log output

GitOrigin-RevId: 6732b19552fe15431a70fbefbc572253c389c64e
 2021-09-22 08:03:25 +00:00
Eric Mc Sween
9ee92daea3 Merge pull request #4893 from overleaf/em-synctex 
Use the synctex distributed with TeX Live

GitOrigin-RevId: 5a133f21f48fd1e217ab463e8cb2a5cdec8be1af
 2021-09-07 16:26:17 +00:00
Eric Mc Sween
2feea2592d Merge pull request #4887 from overleaf/em-decaf 
Decaf cleanup for the CLSI CompileManager

GitOrigin-RevId: 06bba5c8af8808d0fa04187b10c8f31e08cd8754
 2021-08-31 08:03:32 +00:00
Jakob Ackermann
5122558469 Merge pull request #4842 from overleaf/jpa-drop-unused-packages 
[misc] drop unused debug packages

GitOrigin-RevId: 83b1d96ee5a62c14c4a2dc51cefd179306a75a84
 2021-08-27 08:03:17 +00:00
Jakob Ackermann
76e749777d [misc] make build scripts happy again 
- move pdf.js tests into test/unit/js
- add env override to build script config file
- update build scripts
 2021-07-26 12:20:04 +01:00
Jakob Ackermann
f285e503b4 [misc] run format_fix and lint:fix 2021-07-13 12:04:48 +01:00
Jakob Ackermann
64551f0198 [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:21 +01:00
Jakob Ackermann
516525126b [UrlFetcher] do not override domain for clsi-perf requests 2021-07-02 09:17:29 +01:00
Jakob Ackermann
b09e52510f [misc] bail out from pdf caching processing after 10s or earlier 
...for fast compiles.
 2021-06-23 14:20:04 +01:00
Jakob Ackermann
ffaff1bd72 [CompileController] emit status=failure for an empty output.pdf file 2021-06-22 12:15:12 +01:00
Jakob Ackermann
fb3966ef35 [misc] CompileController: simplify composing of outputFiles 2021-06-22 12:15:12 +01:00
Jakob Ackermann
7380b54900 [ProjectPersistenceManager] check all user content dirs for full disk 2021-06-01 16:13:48 +01:00
Jakob Ackermann
294088fb27 [ContentCacheManager] use PDF.js Xref table instead of stream detection (#242) 
* make the content cache manager tests configurable

* extend stream content in unit tests

* [ContentCacheManagerTests] prepare for full object caching

* filesystem stream for pdfjs

* working??

* cleaning up

* handle overflow

* [misc] install pdfjs-dist

* [misc] move pdfjs code into app/lib/ and scripts/, also use CamelCase

* [misc] abstract the file loading and parsing of xRef tables into helper

* [misc] pdfjsTests: add snapshot based tests for the Xref table parser

* [misc] FSStream: throw proper error and drop commented code

* [misc] FSStream: integrate throwing of MissingDataException into getter

* [misc] pdfjs: fix eslint errors

* [misc] pdfjs: run format_fix

* [misc] pdfjs: allocate very small non empty dummy buffers explicitly

* [misc] install @overleaf/o-error

* [ContentCacheManager] use PDF.js Xref table instead of stream detection

Co-Authored-By: Brian Gough <brian.gough@overleaf.com>

* [pdfjs] parseXrefTable: handle empty PDF files gracefully

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
 2021-05-31 09:20:25 +01:00
Jakob Ackermann
f820d32221 Merge pull request #240 from overleaf/bg-jpa-hash-tracker 
[ContentCacheManager] track hashes and expire unused ones
 2021-05-21 11:35:05 +02:00
Jakob Ackermann
b53a105681 [LatexRunner] do not emit empty cpu/sys timings 2021-05-19 11:17:08 +01:00
Jakob Ackermann
7aeeb5a5a9 [ContentCacheManager] finish tracking of ranges across builds 2021-05-18 18:06:15 +01:00
Jakob Ackermann
567d02881d [misc] fix unit tests following the merge of atomic writes 2021-05-18 11:09:30 +01:00
Jakob Ackermann
9b8763aed4 Merge pull request #234 from overleaf/jpa-stream-detection-across-chunks 
[ContentCacheManager] add support for stream detection across chunks
 2021-05-18 11:44:09 +02:00
Jakob Ackermann
224ae0c254 [ContentCacheManager] add support for stream detection across chunks 
Retain a small part (6 or 9 bytes) of each chunk in memory for providing
 the next iteration with enough context for finding the start/end marker
 of a range.
 2021-05-17 17:36:05 +01:00
Brian Gough
371de76a4a use fs.copyFile instead of fse.copy in UrlCache module 2021-05-17 10:54:11 +01:00
Brian Gough
44e0742aa3 use fse.copy for performance 
it uses the native fs.copyFile method
 2021-05-14 15:49:20 +01:00
Jakob Ackermann
b456ea726d [misc] merge pdf caching into main (#226) 
* wip generate directory for hash content

* cleanup, remove console logging

* add content caching module

* Return PDF stream ranges with compile response

* Return the PDF file size in the compile response

* PDF range endpoint

* [misc] WIP: pdf caching: preserve the m-time on static content files

* [misc] WIP: pdf caching: improve browser caching, emit caching headers

* [misc] WIP: pdf caching: do not emit very small chunks <1kB

* [misc] keep up with moving output files into a separate directory

* [OutputCacheManager] add global feature flag for enabling pdf caching

* [misc] add contentId into the URL for protecting PDF stream contents

* [misc] support PDF stream caching for anonymous users

* [misc] add per-request feature flag for enabling PDF stream caching

* [misc] enable pdf caching in CI and emit metrics at the end of run

* [misc] expose compile stats and timings to the frontend

* [misc] log an error in case saving output files fails

* [misc] add metrics for pdf bandwidth and pdf caching performance

* [misc] add a dark mode to the pdf caching for computing ranges only

* [misc] move pdf caching metrics into ContentCacheMetrics

* [misc] add a config option for the min chunk size of pdf ranges

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
 2021-05-13 14:07:54 +01:00
Jakob Ackermann
6cb4246eb1 [misc] consume and validate a custom imageName for synctex requests 2021-03-30 13:57:40 +01:00
Eric Mc Sween
9f2d219102 Add a global test setup file 
Set up generally useful stuff:

* chai.should()
* logger stubs
* globals in SandboxedModule, including Buffer and process, which are
  now required in Node 12
 2021-03-12 16:27:33 -05:00
Brian Gough
a86e521ac0 add unit test for non-existent state file 2021-01-26 11:03:18 +00:00