Commit graph

39 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
ce8a09a4c0 resources: Move resource interfaces into its own package 2019-01-02 14:25:37 +01:00
Bjørn Erik Pedersen
5df2b79dd2 Fix Permalink for resource, baseURL with path and canonifyURLs set
Fixes #5226
2018-11-15 16:37:11 +01:00
Bjørn Erik Pedersen
b874a1ba7a media: Allow multiple file suffixes per media type
Before this commit, `Suffix` on `MediaType` was used both to set a custom file suffix and as a way to augment the mediatype definition (what you see after the "+", e.g. "image/svg+xml").

This had its limitations. For one, it was only possible with one file extension per MIME type.

Now you can specify multiple file suffixes using "suffixes", but you need to specify the full MIME type
identifier:

[mediaTypes]
[mediaTypes."image/svg+xml"]
suffixes = ["svg", "abc ]

In most cases, it will be enough to just change:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffix = "txt"

To:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffixes = ["txt"]

Hugo will still respect values set in "suffix" if no value for "suffixes" is provided, but this will be removed in a future release.

Note that you can still get the Media Type's suffix from a template: {{ $mediaType.Suffix }}. But this will now map to the MIME type filename.

Fixes #4920
2018-07-10 22:13:52 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
128f14efad hugolib: Reset the global pages cache on server rebuilds
In Hugo 0.42, this could lead to errors of type `runtime error: invalid memory address or nil pointer dereference` in some rare situations.

Note that this was also an issue before 0.42, but the symptom was then potentially stale list content on rebuilds on content changes.

This commit also improves the above error message.

Fixes #4845
2018-06-13 12:00:30 +02:00
Bjørn Erik Pedersen
35ccf06dae
Fix some recently broken embedded templates
And add tests for them.

Fixes #4757
2018-05-23 10:03:11 +02:00
Bjørn Erik Pedersen
086ae81a98
hugolib: Fix possible .Content cut
There have been one report of a site with truncated `.Content` after the Hugo `0.40.1` release.

This commit fixes this so that race should not be possible anymore. It also adds a stress test with focus on content rendering and multiple output formats.

Fixes #4706
2018-05-08 16:52:51 +02:00
Bjørn Erik Pedersen
4edb4bcc65
hugolib: Improve error message in .Render 2018-02-13 09:01:47 +01:00
Bjørn Erik Pedersen
51dd462c39 layout: Respect Type and Layout for list template selection
This commit also has some other nice side-effects:

* The layout logic is unified for all page types, which should make it less surprising
* Page.Render now supports all types
* The legacy "indexes" type is removed from the template lookup order. This is an undocumented type from early Hugo days. This means that having a template in, say, `/layouts/indexes/list.html` will no longer work.
* The theme override logic is improved. As an example, an `index.html` in theme will now wn over a `_default/list.html` in the project, which most will expect.

Fixes #3005
Fixes #3245
2018-01-14 19:59:51 +01:00
Bjørn Erik Pedersen
3cdf19e9b7
Implement Page bundling and image handling
This commit is not the smallest in Hugo's history.

Some hightlights include:

* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).

A site building  benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:

```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"

benchmark                                                                                                         old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      101785785     78067944      -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     185481057     149159919     -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      103149918     85679409      -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     203515478     169208775     -16.86%

benchmark                                                                                                         old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      532464         391539         -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1056549        772702         -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      555974         406630         -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1086545        789922         -27.30%

benchmark                                                                                                         old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      53243246      43598155      -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     105811617     86087116      -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      54558852      44545097      -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     106903858     86978413      -18.64%
```

Fixes #3651
Closes #3158
Fixes #1014
Closes #2021
Fixes #1240
Updates #3757
2017-12-27 18:44:47 +01:00
Bjørn Erik Pedersen
60dfb9a6e0 Add support for multiple staticDirs
This commit adds support for multiple statDirs both on the global and language level.

A simple `config.toml` example:

```bash
staticDir = ["static1", "static2"]
[languages]
[languages.no]
staticDir = ["staticDir_override", "static_no"]
baseURL = "https://example.no"
languageName = "Norsk"
weight = 1
title = "På norsk"

[languages.en]
staticDir2 = "static_en"
baseURL = "https://example.com"
languageName = "English"
weight = 2
title = "In English"
```

In the above, with no theme used:

the English site will get its static files as a union of "static1", "static2" and "static_en". On file duplicates, the right-most version will win.
the Norwegian site will get its static files as a union of "staticDir_override" and "static_no".

This commit also concludes the Multihost support in #4027.

Fixes #36
Closes #4027
2017-11-17 11:01:46 +01:00
Jorin Vogel
81c13171a9 Add some missing doc comments
As pointed out by the linter, some exported functions and types are
missing doc comments.
The linter warnings have been reduced from 194 to 116.
Not all missing comments have been added in this commit though.
2017-08-03 15:57:51 +02:00
Bjørn Erik Pedersen
d8717cd4c7 all: Update import paths to gohugoio/hugo 2017-06-13 18:42:45 +02:00
Bjørn Erik Pedersen
45c7452668 hugolib: Must recreate Paginator on live-reload
The structure may potentially have changed, and then it fails.

Fixes #3315
2017-04-12 21:15:51 +02:00
Bjørn Erik Pedersen
e765b43e2b hugolib: Avoid recreating the Paginator on copy
See #3315
2017-04-12 21:15:51 +02:00
Bjørn Erik Pedersen
2874fc75ce hugolib, output: Add NotAlternative to OutputFormat
To make sure CSS and similar does not appear in the AlternativeOutputFormats list.
2017-04-08 11:21:12 +02:00
Bjørn Erik Pedersen
1425587193 hugolib: Add optional outputFormat to Ref/RelRef
Fixes #3224
2017-04-04 23:09:01 +02:00
Bjørn Erik Pedersen
8b5b558bb5 tpl: Rework to handle both text and HTML templates
Before this commit, Hugo used `html/template` for all Go templates.

While this is a fine choice for HTML and maybe also RSS feeds, it is painful for plain text formats such as CSV, JSON etc.

This commit fixes that by using the `IsPlainText` attribute on the output format to decide what to use.

A couple of notes:

* The above requires a nonambiguous template name to type mapping. I.e. `/layouts/_default/list.json` will only work if there is only one JSON output format, `/layouts/_default/list.mytype.json` will always work.
* Ambiguous types will fall back to HTML.
* Partials inherits the text vs HTML identificator of the container template. This also means that plain text templates can only include plain text partials.
* Shortcode templates are, by definition, currently HTML templates only.

Fixes #3221
2017-04-02 23:13:10 +02:00
Bjørn Erik Pedersen
7eb71ee064 Revert "tpl: Rework to handle both text and HTML templates"
Will have to take another stab at this ...

This reverts commit 5c5efa03d2.

Closes #3260
2017-04-02 14:20:34 +02:00
Bjørn Erik Pedersen
5c5efa03d2 tpl: Rework to handle both text and HTML templates
Before this commit, Hugo used `html/template` for all Go templates.

While this is a fine choice for HTML and maybe also RSS feeds, it is painful for plain text formats such as CSV, JSON etc.

This commit fixes that by using the `IsPlainText` attribute on the output format to decide what to use.

A couple of notes:

* The above requires a nonambiguous template name to type mapping. I.e. `/layouts/_default/list.json` will only work if there is only one JSON output format, `/layouts/_default/list.mytype.json` will always work.
* Ambiguous types will fall back to HTML.
* Partials inherits the text vs HTML identificator of the container template. This also means that plain text templates can only include plain text partials.
* Shortcode templates are, by definition, currently HTML templates only.

Fixes #3221
2017-04-02 11:37:30 +02:00
Bjørn Erik Pedersen
87b3cd4655 hugolib, helpers: Reduce log level to WARN on .Render for non-regular pages
We will eventually support all types in the Render method.
2017-03-29 08:08:45 +02:00
Bjørn Erik Pedersen
930a3df1b7 hugolib, output: Restrict Render to regular Pages
Using it for list pages doesn't work and has potential weird side-effects.

The user probably meant to range over .Site.ReqularPages, and that is now marked clearly in the log.
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
e49a2b83ad hugolib: Fix Render layouts for list pages 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
9a0aa5fdbe hugolib: Wrap pageOutput create in sync.Once 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
b7ed67d425 hugolib: More TODO fixes 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
3cd97951f1 hugolib, layout: Consolidate RSS template handling 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
87188496fb hugolib, output: Handle aliases for all HTML formats 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
d851d6b98f Add custom protocol support in Permalink 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
dbb83f925a hugolib: Read default output formats from site config 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
4aaed87dd9 hugolib, media: Make the MediaType available to the templates 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
c7dbee2321 hugolib, output: Add Rel to the output format
To make it super-easy to create rel-links.
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
1b2be5e878 hugolib: Add OutputFormats with permalinks to Page 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
c7c6b47ba8 hubolib: Pick layout per output format 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
df95383914 hugolib: Speed up URL handling 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
4c2abe0015 Rename OutputType to OutputFormat 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
6bf010fed4 hugolib: Refactor/-work the permalink/target path logic
This is a pretty fundamental change in Hugo, but absolutely needed if we should have any hope of getting "multiple outputs" done.

This commit's goal is to say:

* Every file target path is created by `createTargetPath`, i.e. one function for all.
* That function takes every page and site parameter into account, to avoid fragile string parsing to uglify etc. later on.
* The path creation logic has full test coverage.
* All permalinks, paginator URLs etc. are then built on top of that same logic.

Fixes #1252
Fixes #2110
Closes #2374
Fixes #1885
Fixes #3102
Fixes #3179
Fixes #1641
Fixes #1989
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
c8fff9501d Implement the first generic JSON output testcase 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
03122e51fa hugolib: Revert to using Page as the render chan type
Changing it to PageOutput was a mistake. You may think that the increased parallelism should be a good thing.

But not so much with the increased lock contention and more complex concurrency model.
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
348834d017 hugolib: Add a PageOutput wrapper for rendering 2017-03-27 15:43:56 +02:00