Commit graph

122 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
3cdf19e9b7
Implement Page bundling and image handling
This commit is not the smallest in Hugo's history.

Some hightlights include:

* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).

A site building  benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:

```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"

benchmark                                                                                                         old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      101785785     78067944      -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     185481057     149159919     -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      103149918     85679409      -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     203515478     169208775     -16.86%

benchmark                                                                                                         old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      532464         391539         -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1056549        772702         -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      555974         406630         -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1086545        789922         -27.30%

benchmark                                                                                                         old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      53243246      43598155      -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     105811617     86087116      -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      54558852      44545097      -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     106903858     86978413      -18.64%
```

Fixes #3651
Closes #3158
Fixes #1014
Closes #2021
Fixes #1240
Updates #3757
2017-12-27 18:44:47 +01:00
Brian Chen
e69da7a4cb Add Pandoc support, refactor external helpers
Recognize the Pandoc format under the file extension .pandoc or .pdc,
and shell out to pandoc as an external helper to format Pandoc content.

Refactor out repeated code with external helpers. Change the error
output formatting. I did not see any of the external helpers print the
string "<input>" to represent stdin as a file; just prepending the file
name to error output is more general and doesn't sacrifice that much in
terms of readability.

Closes #234
2017-11-30 12:15:52 +01:00
Bjørn Erik Pedersen
b3daa1f4bf hugolib: Fix .IsTranslated with identical filenames
This commit refines the key used to map translations:

* Use `translationKey` set in front matter
* Fall back to path + base filename (i.e. the filename without extension and language code)

Note that the Page Kinde will be prepended to both cases above. It does not make sense to have a section as translation for the home page.

Fixes #2699
2017-11-17 19:46:32 +01:00
Bjørn Erik Pedersen
6a30874f19 Make sure Date and PublishDate is always set to a value if one is available
Fixes #3854
2017-10-13 13:18:23 +02:00
Bjørn Erik Pedersen
71ae9b4533 hugolib: Rewrite replaceDivider to reduce memory allocation
```bash
name              old time/op    new time/op    delta
ReplaceDivider-4   9.76µs ±105%    7.96µs ±24%     ~     (p=0.690 n=5+5)

name              old alloc/op   new alloc/op   delta
ReplaceDivider-4    3.46kB ± 0%    1.54kB ± 0%  -55.56%  (p=0.008 n=5+5)

name              old allocs/op  new allocs/op  delta
ReplaceDivider-4      6.00 ± 0%      1.00 ± 0%  -83.33%  (p=0.008 n=5+5)
```
2017-08-14 09:42:55 +02:00
Bjørn Erik Pedersen
199816fddd hugolib: Extract replaceDivider logic
To its own function and add a test and a benchmark for it.
2017-08-14 09:42:55 +02:00
Vasyl Solovei
b60aa1a504 helpers: Add --trace to asciidoctor args
This will help to understand and fix errors by
seeing stacktrace of an error.

See #3714
2017-07-21 03:07:56 -06:00
Bjørn Erik Pedersen
72fd871710 hugolib: Add more summary test
Closes #2973
2017-06-28 20:30:26 +02:00
Bjørn Erik Pedersen
3183b9a29d parser: Fix handling of quoted brackets in JSON front matter
Also remove a broken JSON test.

Fixes #3511
2017-06-19 16:45:52 +02:00
Bjørn Erik Pedersen
873a6f1885 Run gofmt to get imports in line vs gohugoio/hugo 2017-06-13 19:12:10 +02:00
Bjørn Erik Pedersen
d8717cd4c7 all: Update import paths to gohugoio/hugo 2017-06-13 18:42:45 +02:00
Bjørn Erik Pedersen
5be0448635 hugolib: Fix disablePathToLower regression
Fixes #3374
2017-06-06 09:15:42 +02:00
Bjørn Erik Pedersen
ee75e2999b Remove the now superflous defaultExtension
And some other unsed fields and methods.
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
5761b93c96 hugolib, output: Fix RSSLink vs output formats
And remove the now superflous setPageURLs method.
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
c7c6b47ba8 hubolib: Pick layout per output format 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
ad8cea87f3 Refactor layout resolve to a descriptor/adapter pattern 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
6bf010fed4 hugolib: Refactor/-work the permalink/target path logic
This is a pretty fundamental change in Hugo, but absolutely needed if we should have any hope of getting "multiple outputs" done.

This commit's goal is to say:

* Every file target path is created by `createTargetPath`, i.e. one function for all.
* That function takes every page and site parameter into account, to avoid fragile string parsing to uglify etc. later on.
* The path creation logic has full test coverage.
* All permalinks, paginator URLs etc. are then built on top of that same logic.

Fixes #1252
Fixes #2110
Closes #2374
Fixes #1885
Fixes #3102
Fixes #3179
Fixes #1641
Fixes #1989
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
c8fff9501d Implement the first generic JSON output testcase 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
3ec5fc3504 hugolib, output: Incorporate suffix and type in layout resolve
And remove some now superflous and hard to maintain tests.
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
10de077164 hugolib: Use the new layout logic in Page 2017-03-27 15:43:56 +02:00
Albert Nigmatzianov
48120ccfd2 all: Fix some govet complaints 2017-03-09 14:18:12 +01:00
Bjørn Erik Pedersen
2ea242d5fe hugolib: Minor Go vet fix 2017-02-19 10:33:28 +01:00
Anton Staaf
99fbc75e7a hugolib: Fix regression of .Truncated evaluation in manual summaries
This fixes the behavior of .Truncated that was introduced with commit
bef496b97e which was later broken.  The
desired behavior is that .Truncated would evaluate to false when there
was nothing after the user defined summary marker.

This also adds a simple unit test to ensure that this feature isn't
broken again.  The check for content after the user defined summary
marker is done on the raw content instead of the working copy because
some of the markup renderers add elements after the marker, making it
difficult to determine if there is actually any content.

The behavior (evaluating to false when there is no content, just
summary) is also now documented.
2017-02-19 10:11:49 +01:00
John Feminella
b2e3748a4e hugolib: Enhance .Param to permit arbitrarily nested parameter references
The Param method currently assumes that its argument is a single,
distinct, top-level key to look up in the Params map. This enhances the
Param method; it will now also attempt to see if the key can be
interpreted as a nested chain of keys to look up in Params.

Fixes #2598
2017-02-19 08:50:08 +01:00
Bjørn Erik Pedersen
93ca7c9e95 all: Refactor to nonglobal Viper, i18n etc.
This is a final rewrite that removes all the global state in Hugo, which also enables
the use if `t.Parallel` in tests.

Updates #2701
Fixes #3016
2017-02-17 17:15:26 +01:00
Bjørn Erik Pedersen
19aff029b1 hugolib: Skip rendering for the page tests 2017-02-04 23:15:07 +07:00
Bjørn Erik Pedersen
c71e1b106e all: Refactor to nonglobal file systems
Updates #2701
Fixes #2951
2017-02-04 11:37:25 +07:00
Bjørn Erik Pedersen
45e3ed517a all: Refactor to non-global logger
Note that this looks like overkill for just the logger, and that is correct,
but this will make sense once we start with the template handling etc.

Updates #2701
2017-01-07 17:06:35 +01:00
Cameron Moore
3d058a936f hugolib: Allow arrays of arrays in frontmatter Params
Fixes #2752
2017-01-06 11:56:22 +01:00
Cameron Moore
cb63e87f8d hugolib: Clean up a benchmark 2017-01-03 17:08:59 +01:00
Bjørn Erik Pedersen
2f026ab3f3 hugolib: Make RawContent raw again
This was a regression introduced in Hugo 0.17.

Fixes #2601
2016-12-01 17:29:49 +01:00
Bjørn Erik Pedersen
f1ed89fec4 Revise the deprecation strategy
Git users and theme authors two Hugo releases to fix:

1. With a visible warning
2. Then with an ERROR that exits with -1

Fixes #2726
2016-11-23 17:26:13 +01:00
Bjørn Erik Pedersen
99d11386a7 node to page: Fix the Page copy()
By embeding the init sync.Once var in a pointer so we can reset it when we copy the page.

Updates #2297
2016-11-22 09:57:03 +01:00
Bjørn Erik Pedersen
c8d3124dde node to page: Remove Node
And misc. TODO-fixes

Updates #2297
2016-11-22 09:57:03 +01:00
Bjørn Erik Pedersen
9347084d61 node to page: Make Kind a string
Having a custom string type isn't worth it when it doesn't work
with `where`, `eq` etc.

Fixes #2689
Updates #2297
2016-11-22 09:57:03 +01:00
Bjørn Erik Pedersen
9fba2a30a9 node to page: Rename PageType to Kind
And embed that on Page.

Updates #2297
2016-11-22 09:57:03 +01:00
Bjørn Erik Pedersen
c80308e6b3 node to page: Add Pages to Page
As an alias to .Data.Pages for home page etc.

Also renamte NodeType to PageType and make it a string so it can be used in `where`.

Updates #2297
2016-11-22 09:57:03 +01:00
Bjørn Erik Pedersen
063b78d2ec node to page: Make Nodes into Pages
* `.Site.Pages` now contains all page types, including sections etc.
* `.Data.Pages` will also contain "node type" pages where relevant.

Updates #2297
2016-11-22 09:57:03 +01:00
Bjørn Erik Pedersen
e371ac0b6f node to page: Basic outline
Updates #2297
2016-11-22 09:57:03 +01:00
Bjørn Erik Pedersen
342b6fe8a5 Make it clear that Path is relative to content root
Fixes #2290
2016-10-28 10:35:19 +02:00
Bjørn Erik Pedersen
4ad39445ef Refactor Page tests
* To handle config in one place
* To use the real builder chain
2016-10-28 10:02:46 +02:00
Albert Nigmatzianov
f21e2f25c9 all: Unify case of config variable names
All config variables starts with low-case and uses camelCase.

If there is abbreviation at the beginning of the name, the whole
abbreviation will be written in low-case.
If there is abbreviation at the end of the name, the
whole abbreviation will be written in upper-case.
For example, rssURI.
2016-10-24 20:56:00 +02:00
Bjørn Erik Pedersen
a10b2cd372 Avoid reading from Viper for path and URL funcs
The gain, given the "real sites benchmark" below, is obvious:

```
benchmark           old ns/op       new ns/op       delta
BenchmarkHugo-4     14497594101     13084156335     -9.75%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     57404335       48282002       -15.89%

benchmark           old bytes       new bytes      delta
BenchmarkHugo-4     9933505624      9721984424     -2.13%
```

Fixes #2495
2016-10-24 13:45:30 +02:00
Bjørn Erik Pedersen
67df33d83f Fix a more summary corner case
Also refactor the rendering pages test to accept more than one page source per test run, which wasn't really needed for this issue, but may be in the future.

Closes #2586
Fixes #2538
2016-10-18 08:43:44 +02:00
Bjørn Erik Pedersen
d2bc64bee3 Add draft test
See #2549
2016-10-11 18:12:28 +02:00
Bjørn Erik Pedersen
dd45e6d7e5 Lazy calculate WordCount, ReadingTime and FuzzyWordCount
This avoids having to execute these expensive operations for sites not using these values.

This commit sums up a set of wordcounting and autosummary related performance improvements.

The effect of these kind of depends on what features your site use, but a benchmark from 4 Hugo sites in the wild shows promise:

```
benchmark           old ns/op       new ns/op       delta
BenchmarkHugo-4     21293005843     20032857342     -5.92%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     65290922       65186032       -0.16%

benchmark           old bytes      new bytes      delta
BenchmarkHugo-4     9771213416     9681866464     -0.91%
```

Closes #2378
2016-09-14 10:57:39 +02:00
Bjørn Erik Pedersen
0a7d1d0ddc Fix some corner cases in revised summary handling
And clean up the test.

See #2309
2016-09-06 18:32:18 +03:00
Bjørn Erik Pedersen
2079a23dd8 Make it possible to configure Blackfroday per language
See #2309
2016-09-06 18:32:17 +03:00
Bjørn Erik Pedersen
ed0985404d Render the shortcodes as late as possible
This is needed to make shortcode users happy with the new multilanguage support,
but it will also solve many other related posts about "stuff not available in the shortcode".

We will have to revisit this re the handler chain at some point, but that will be easier
now as the integration test story has improved so much.

As part of this commit, the site-building tests in page_test.go is refreshed, they now
tests for all the rendering engines (when available), and all of them now uses the
same code-path as used in production.

Fixes #1229
Fixes #2323
Fixes ##1076
2016-09-06 18:32:16 +03:00
Bjørn Erik Pedersen
708bc78770 Optimize the multilanguage build process
Work In Progress!

This commit makes a rework of the build and rebuild process to better suit a multi-site setup.

This also includes a complete overhaul of the site tests. Previous these were a messy mix that
were testing just small parts of the build chain, some of it testing code-paths not even used in
"real life". Now all tests that depends on a built site follows the same and real production code path.

See #2309
Closes #2211
Closes #477
Closes #1744
2016-09-06 18:32:16 +03:00