Commit graph

105 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
6b76841b05 output: Fix permalink in sitemap etc. when multiple permalinkable output formats
In Hugo 0.55.0 we made AMP `permalinkable`. We also render the output formats in their natural sort order, meaning `AMP` will be rendered before `HTML`. References in the sitemap would then point to the AMP version, and this is normally not what you'd want.

This commit fixes that by making `HTML` by default sort before the others.

If this is not you want, you can set `weight` on the output format configuration.

Fixes #5910
2019-05-02 14:23:16 +02:00
Bjørn Erik Pedersen
49d0a82641 hugolib: Fix bundle resource publishing when multiple output formats
The faulty logic published the bundled resources for the "first output" format.

This worked most of the time, but since the output formats list is sorted,
any output format only used for some of the pages (e.g. CSS) would not work properly.

Fixes #5858
2019-04-15 17:01:39 +02:00
Bjørn Erik Pedersen
f7375c4972
Fix paginator refresh on server change
Fixes #5838
2019-04-12 09:18:59 +02:00
Bjørn Erik Pedersen
612a06f067 Misc paginator adjustments
* Rewind paginator for server mode
* Add some more related tests.
* Replace the clumsy scratch constructs in internal paginator template with variables

See #5825
2019-04-08 11:02:12 +02:00
Bjørn Erik Pedersen
87b16abd93
Add HUGO_NUMWORKERMULTIPLIER
And use that to calculate number of workers, if set, else fall back to number of logical CPUs.

Also tweak the relevant related settings to match the new setup.

Also remove the setting of `runtime.GOMAXPROCS` as this has been the default behaviour since Go 1.5.

Fixes #5814
2019-04-05 10:21:25 +02:00
Bjørn Erik Pedersen
950295516d
hugolib: Buffer the render pages chan 2019-04-04 16:50:10 +02:00
Bjørn Erik Pedersen
5185fb065b hugolib: Allow relative URLs in front matter
Before this commit you would have to do this in multilingual setups:

```
---
title: "Custom!"
url: "/jp/custom/foo"
---
```

This commit allows for relative URLs, e.g:

```
---
title: "Custom!"
url: "custom/foo"
---
```

Which is obviously easier and more portable.

The meaning of relative may change to include more in the future (e.g. role based access).

Fixes #5704
2019-04-02 10:33:15 +02:00
Bjørn Erik Pedersen
92baa14fd3 hugolib: Allow page-relative aliases
Fixes #5757
2019-04-02 10:32:47 +02:00
Bjørn Erik Pedersen
f9d6feca08
hugolib: Fix alias path for AMP and similar
Fixes #5760
2019-03-30 16:46:51 +01:00
Bjørn Erik Pedersen
597e418cb0
Make Page an interface
The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct.
This is all a preparation step for issue  #5074, "pages from other data sources".

But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes.

Most notable changes:

* The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday.
  This means that any markdown will partake in the global ToC and footnote context etc.
* The Custom Output formats are now "fully virtualized". This removes many of the current limitations.
* The taxonomy list type now has a reference to the `Page` object.
  This improves the taxonomy template `.Title` situation and make common template constructs much simpler.

See #5074
Fixes #5763
Fixes #5758
Fixes #5090
Fixes #5204
Fixes #4695
Fixes #5607
Fixes #5707
Fixes #5719
Fixes #3113
Fixes #5706
Fixes #5767
Fixes #5723
Fixes #5769
Fixes #5770
Fixes #5771
Fixes #5759
Fixes #5776
Fixes #5777
Fixes #5778
2019-03-23 18:51:22 +01:00
Bjørn Erik Pedersen
d5a0b6bbbc hugolib: Remove "double layout" lookup
Fixes #5390
2018-12-17 11:34:32 +01:00
Bjørn Erik Pedersen
6636cf1bea
Resolve error handling/parser related TODOs
See #5324
2018-10-23 19:41:22 +02:00
Bjørn Erik Pedersen
4a366fcfee Prevent stale content in Fast Render Mode
We do that by re-render visited pages that is not already in the stack. This may potentially do some double work, but that small penalty should be well worth it.

Fixes #5281
2018-10-17 10:15:22 +02:00
Bjørn Erik Pedersen
35fbfb19a1
commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes #5284
Fixes #5290
See #5325
See #5324
2018-10-16 22:10:56 +02:00
Bjørn Erik Pedersen
555a5612b2 hugolib: Do not set RSS as Kind in RSS output
Before Hugo this commit  we set the pseudo page kind RSS on the page when output to RSS. This had some unintended side effects, esp. when the only output format for that page  was RSS.

For the page kinds that can have multiple output formats, the Kind should be one of the standard home, page etc.

Fixes #5138
2018-09-14 11:33:02 +02:00
Bjørn Erik Pedersen
789ef8c639
Add support for minification of final output
Hugo Pipes added minification support for resources fetched via ´resources.Get` and similar.

This also adds support for minification of the final output for supported output formats: HTML, XML, SVG, CSS, JavaScript, JSON.

To enable, run Hugo with the `--minify` flag:

```bash
hugo --minify
```

This commit is also a major spring cleaning of the `transform` package to allow the new minification step fit into that processing chain.

Fixes #1251
2018-08-06 19:58:41 +02:00
Bjørn Erik Pedersen
b874a1ba7a media: Allow multiple file suffixes per media type
Before this commit, `Suffix` on `MediaType` was used both to set a custom file suffix and as a way to augment the mediatype definition (what you see after the "+", e.g. "image/svg+xml").

This had its limitations. For one, it was only possible with one file extension per MIME type.

Now you can specify multiple file suffixes using "suffixes", but you need to specify the full MIME type
identifier:

[mediaTypes]
[mediaTypes."image/svg+xml"]
suffixes = ["svg", "abc ]

In most cases, it will be enough to just change:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffix = "txt"

To:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffixes = ["txt"]

Hugo will still respect values set in "suffix" if no value for "suffixes" is provided, but this will be removed in a future release.

Note that you can still get the Media Type's suffix from a template: {{ $mediaType.Suffix }}. But this will now map to the MIME type filename.

Fixes #4920
2018-07-10 22:13:52 +02:00
Bjørn Erik Pedersen
43338c3a99 hugolib: Do not create paginator pages for the other output formats
This is a recent regression in Hugo, where we have started to produce `/page/30/index.json` when the main output format (usually `HTML`) is set up with pagination.

For JSON this is potentially lot of superflous work and hurts performance.

This commit reinstates the earlier behaviour: We only create paginators if in use in the main output format.

And add a test for it to prevent this from happening again.

Fixes #4890
2018-07-06 14:01:27 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
086ae81a98
hugolib: Fix possible .Content cut
There have been one report of a site with truncated `.Content` after the Hugo `0.40.1` release.

This commit fixes this so that race should not be possible anymore. It also adds a stress test with focus on content rendering and multiple output formats.

Fixes #4706
2018-05-08 16:52:51 +02:00
Bjørn Erik Pedersen
4d26ab33dc
Make .Content (almost) always available in shortcodes
This resolves some surprising behaviour when reading other pages' content from shortcodes. Before this commit, that behaviour was undefined. Note that this has never been an issue from regular templates.

It will still not be possible to get **the current shortcode's  page's rendered content**. That would have impressed Einstein.

The new and well defined rules are:

* `.Page.Content` from a shortcode will be empty. The related `.Page.Truncated` `.Page.Summary`, `.Page.WordCount`, `.Page.ReadingTime`, `.Page.Plain` and `.Page.PlainWords` will also have empty values.
* For _other pages_ (retrieved via `.Page.Site.GetPage`, `.Site.Pages` etc.) the `.Content` is there to use as you please as long as you don't have infinite content recursion in your shortcode/content setup. See below.
* `.Page.TableOfContents` is good to go (but does not support shortcodes in headlines; this is unchanged)

If you get into a situation of infinite recursion, the `.Content` will be empty. Run `hugo -v` for more information.

Fixes #4632
Fixes #4653
Fixes #4655
2018-04-21 22:02:56 +02:00
Bjørn Erik Pedersen
94e736c5e1
hugolib: Extract the Fast Render Mode logic into a method
This also improves on the previous commit as it takes pages without content files into account.

Closes #4339
2018-01-30 10:49:24 +01:00
Alexey Grachov
1707dae8d3
hugolib: Handle newly created files in Fast Render Mode
Updates #4339
2018-01-30 10:49:24 +01:00
Bjørn Erik Pedersen
4d912e2aad
hugolib, output: Fix robots.txt in multihost mode
Fixes #4193
2018-01-25 10:36:53 +01:00
Bjørn Erik Pedersen
d418c2c2ea
Remove and update deprecation status 2018-01-25 10:22:11 +01:00
Bjørn Erik Pedersen
0432c64dd2 Add headless bundle support
This commit adds  support for `headless bundles` for the `index` bundle type.

So:

```toml
headless = true
```

In front matter means that

* It will have no `Permalink` and no rendered HTML in /public
* It will not be part of `.Site.RegularPages` etc.

But you can get it by:

* `.Site.GetPage ...`

The use cases are many:

* Shared media galleries
* Reusable page content "snippets"
* ...

Fixes #4311
2018-01-24 09:00:21 +01:00
Bjørn Erik Pedersen
20c9b6ec81
resource: Add front matter metadata to Resource
This commit expands the Resource interface with 3 new methods:

* Name
* Title
* Params

All of these can be set in the Page front matter. `Name` will get its default value from the base filename, and is the value used in the ByPrefix and GetByPrefix lookup methods.

Fixes #4244
2018-01-17 16:22:33 +01:00
Bjørn Erik Pedersen
51dd462c39 layout: Respect Type and Layout for list template selection
This commit also has some other nice side-effects:

* The layout logic is unified for all page types, which should make it less surprising
* Page.Render now supports all types
* The legacy "indexes" type is removed from the template lookup order. This is an undocumented type from early Hugo days. This means that having a template in, say, `/layouts/indexes/list.html` will no longer work.
* The theme override logic is improved. As an example, an `index.html` in theme will now wn over a `_default/list.html` in the project, which most will expect.

Fixes #3005
Fixes #3245
2018-01-14 19:59:51 +01:00
Bjørn Erik Pedersen
3cdf19e9b7
Implement Page bundling and image handling
This commit is not the smallest in Hugo's history.

Some hightlights include:

* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).

A site building  benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:

```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"

benchmark                                                                                                         old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      101785785     78067944      -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     185481057     149159919     -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      103149918     85679409      -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     203515478     169208775     -16.86%

benchmark                                                                                                         old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      532464         391539         -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1056549        772702         -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      555974         406630         -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1086545        789922         -27.30%

benchmark                                                                                                         old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      53243246      43598155      -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     105811617     86087116      -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      54558852      44545097      -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     106903858     86978413      -18.64%
```

Fixes #3651
Closes #3158
Fixes #1014
Closes #2021
Fixes #1240
Updates #3757
2017-12-27 18:44:47 +01:00
Bjørn Erik Pedersen
089fe49309 hugolib: Make sure everything ends up in its lang root in multihost mode
Fixes #4105
2017-11-20 11:17:46 +01:00
Bjørn Erik Pedersen
60dfb9a6e0 Add support for multiple staticDirs
This commit adds support for multiple statDirs both on the global and language level.

A simple `config.toml` example:

```bash
staticDir = ["static1", "static2"]
[languages]
[languages.no]
staticDir = ["staticDir_override", "static_no"]
baseURL = "https://example.no"
languageName = "Norsk"
weight = 1
title = "På norsk"

[languages.en]
staticDir2 = "static_en"
baseURL = "https://example.com"
languageName = "English"
weight = 2
title = "In English"
```

In the above, with no theme used:

the English site will get its static files as a union of "static1", "static2" and "static_en". On file duplicates, the right-most version will win.
the Norwegian site will get its static files as a union of "staticDir_override" and "static_no".

This commit also concludes the Multihost support in #4027.

Fixes #36
Closes #4027
2017-11-17 11:01:46 +01:00
Bjørn Erik Pedersen
2e0465764b Add multilingual multihost support
This commit adds multihost support when more than one language is configured and `baseURL` is set per language.

Updates #4027
2017-11-17 11:01:46 +01:00
Bjørn Erik Pedersen
60bd332c1f Only re-render the view(s) you're working on
Hugo already, in its server mode,  support partial rebuilds. To put it simply: If you change `about.md`, only that content page is read and processed, then Hugo does some processing (taxonomies etc.) and the full site is rendered.
This commit covers the rendering part: We now only re-render the pages you work on, i.e. the last n pages you watched in the browser (which obviously also includes the  page in the example above).

To be more specific: When you are running the hugo server in watch (aka. livereload) mode, and change a template or a content file, then we do a partial re-rendering of the following:

* The current content page (if it is a content change)
* The home page
* Up to the last 10 pages you visited on the site.

This should in most cases be enough, but if you navigate to something completely different, you may see stale content. Doing an edit will then refresh that page.

Note that this feature is enabled by default. To turn it off, run `hugo server --disableFastRender`.

Fixes #3962
See  #1643
2017-10-14 13:40:43 +02:00
Mitchell Cohen
41805dca9e hugolib: Render 404.html for all languages
Fixes #3598
2017-07-01 22:58:52 +02:00
Bjørn Erik Pedersen
d8717cd4c7 all: Update import paths to gohugoio/hugo 2017-06-13 18:42:45 +02:00
Bjørn Erik Pedersen
f1da5a15a3 hugolib: Make the RSS feed use the date for the node it represents
Closes #2708
2017-06-09 01:10:16 +02:00
Bjørn Erik Pedersen
b39689393c hugolib: Enable nested sections
Fixes #465
2017-06-08 11:21:34 +02:00
Bjørn Erik Pedersen
af72db806f hugolib: Handle shortcode per output format
This commit allows shortcode per output format, a typical use case would be the special AMP media tags.

Note that this will only re-render the "overridden" shortcodes and only  in pages where these are used, so performance in the normal case should not suffer.

Closes #3220
2017-05-13 22:44:15 +03:00
Bjørn Erik Pedersen
1e4d082cf5 hubolib: Refactor site rendering with an "output format context"
Fixes #3397
2017-05-13 22:44:15 +03:00
Mitchell Cohen
154e18ddb9 Render 404 in default language only
This prevents 404.html from being re-rendered for each site.

Fixes #3075
2017-04-30 10:40:37 +02:00
Bjørn Erik Pedersen
0e87b18b66 hugolib: Fix handling of zero-length files
This was a regression in Hugo 0.20. This commit makes sure that zeron-length files are not rendered to file.

Fixes #3355
2017-04-23 22:46:01 +02:00
Bjørn Erik Pedersen
45c7452668 hugolib: Must recreate Paginator on live-reload
The structure may potentially have changed, and then it fails.

Fixes #3315
2017-04-12 21:15:51 +02:00
Bjørn Erik Pedersen
2874fc75ce hugolib, output: Add NotAlternative to OutputFormat
To make sure CSS and similar does not appear in the AlternativeOutputFormats list.
2017-04-08 11:21:12 +02:00
Bjørn Erik Pedersen
9e69a92e85 hugolib: Deprecate rssURI 2017-04-08 10:46:13 +02:00
Bjørn Erik Pedersen
c4a1165587 all: Handle all errors
As reported by `errcheck`.
2017-04-06 20:35:26 +02:00
Bjørn Erik Pedersen
868f89d5c6 hugolib: Improve render error handling
Catch and return the "template not found" error earlier.
2017-04-04 15:12:30 +02:00
Bjørn Erik Pedersen
c97dae40d9 hugolib: Use Page Kind in template errors to prevent log spam
Having the content page name in the log key for the distinct error logger isnt't very usable when you have an error in a commonly used partial.

Using the Page Kind reduces the amount of log entries. Here is an example from an error in the partial menu.html, used in all the page templates:

```
Started building sites ...
ERROR 2017/04/02 12:19:43 Error while rendering "page": template: /Users/bep/sites/bepsays.com/layouts/_default/single.html:17:7: executing "/Users/bep/sites/bepsays.com/layouts/_default/single.html" at <partial "menu.html" ...>: error calling partial: template: partials/menu.html:9:11: executing "partials/menu.html" at <.DoesNotExist>: can't evaluate field DoesNotExist in type *hugolib.PageOutput
ERROR 2017/04/02 12:19:43 Error while rendering "section": template: /Users/bep/sites/bepsays.com/layouts/_default/section.html:17:7: executing "/Users/bep/sites/bepsays.com/layouts/_default/section.html" at <partial "menu.html" ...>: error calling partial: template: partials/menu.html:9:11: executing "partials/menu.html" at <.DoesNotExist>: can't evaluate field DoesNotExist in type *hugolib.PageOutput
ERROR 2017/04/02 12:19:43 Error while rendering "taxonomy": template: /Users/bep/sites/bepsays.com/layouts/_default/list.html:17:7: executing "/Users/bep/sites/bepsays.com/layouts/_default/list.html" at <partial "menu.html" ...>: error calling partial: template: partials/menu.html:9:11: executing "partials/menu.html" at <.DoesNotExist>: can't evaluate field DoesNotExist in type *hugolib.PageOutput
ERROR 2017/04/02 12:19:43 Error while rendering "home": template: /Users/bep/sites/bepsays.com/layouts/index.html:17:7: executing "/Users/bep/sites/bepsays.com/layouts/index.html" at <partial "menu.html" ...>: error calling partial: template: partials/menu.html:9:11: executing "partials/menu.html" at <.DoesNotExist>: can't evaluate field DoesNotExist in type *hugolib.PageOutput
ERROR 2017/04/02 12:19:43 Error while rendering "404": template: 404.html:2:3: executing "404.html" at <partial "menu.html" ...>: error calling partial: template: partials/menu.html:9:11: executing "partials/menu.html" at <.DoesNotExist>: can't evaluate field DoesNotExist in type *hugolib.PageOutput
Built site for language nn:
```
Which is pretty good.
2017-04-02 12:22:54 +02:00
Bjørn Erik Pedersen
930a3df1b7 hugolib, output: Restrict Render to regular Pages
Using it for list pages doesn't work and has potential weird side-effects.

The user probably meant to range over .Site.ReqularPages, and that is now marked clearly in the log.
2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
9a0aa5fdbe hugolib: Wrap pageOutput create in sync.Once 2017-03-27 15:43:56 +02:00
Bjørn Erik Pedersen
0aaf3c56a5 hugolib: Fix panic for Permalink in 404 etc. templates 2017-03-27 15:43:56 +02:00