The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct.
This is all a preparation step for issue #5074, "pages from other data sources".
But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes.
Most notable changes:
* The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday.
This means that any markdown will partake in the global ToC and footnote context etc.
* The Custom Output formats are now "fully virtualized". This removes many of the current limitations.
* The taxonomy list type now has a reference to the `Page` object.
This improves the taxonomy template `.Title` situation and make common template constructs much simpler.
See #5074Fixes#5763Fixes#5758Fixes#5090Fixes#5204Fixes#4695Fixes#5607Fixes#5707Fixes#5719Fixes#3113Fixes#5706Fixes#5767Fixes#5723Fixes#5769Fixes#5770Fixes#5771Fixes#5759Fixes#5776Fixes#5777Fixes#5778
Before this commit, due to a bug in Go's `text/template` package, this would print different output for typed nil interface values:
```
{{ if .AuthenticatedUser }}User is authenticated!{{ else }}{{ end }}
{{ if not .AuthenticatedUser }}{{ else }}}User is authenticated!{{ end }}
```
This commit works around this by wrapping every `if` and `with` with a custom `getif` template func with truth logic that matches `not`, `and` and `or`.
Those 3 template funcs from Go's stdlib are now pulled into Hugo's source tree and adjusted to support custom zero values, e.g. types that implement `IsZero`.
This means that you can now do:
```
{{ with .Date }}{{ . }}{{ end }}
```
And it would work as expected.
Fixes#5738
Before this commit `where` would produce an error and bail building the
site. Now, `where` simply skips an element of a collection and does not
add it to the final result.
Closes#5637Closes#5416
This means that the current `.Site` and ´.Hugo` is available as a globals, so you can do `site.IsServer`, `hugo.Version` etc.
Fixes#5470Fixes#5467Fixes#5503
In the newly consolidated file cache implementation, we forgot that we also look in the theme(s) for assets (SCSS transformations etc.), which is not good for Netlify and the demo sites.
Fixes#5460
This commits reworks how file caching is performed in Hugo. Now there is only one way, and it can be configured.
This is the default configuration:
```toml
[caches]
[caches.getjson]
dir = ":cacheDir"
maxAge = -1
[caches.getcsv]
dir = ":cacheDir"
maxAge = -1
[caches.images]
dir = ":resourceDir/_gen"
maxAge = -1
[caches.assets]
dir = ":resourceDir/_gen"
maxAge = -1
```
You can override any of these cache setting in your own `config.toml`.
The placeholders explained:
`:cacheDir`: This is the value of the `cacheDir` config option if set (can also be set via OS env variable `HUGO_CACHEDIR`). It will fall back to `/opt/build/cache/hugo_cache/` on Netlify, or a `hugo_cache` directory below the OS temp dir for the others.
`:resourceDir`: This is the value of the `resourceDir` config option.
`maxAge` is the time in seconds before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off.
This means that if you run your builds on Netlify, all caches configured with `:cacheDir` will be saved and restored on the next build. For other CI vendors, please read their documentation. For an CircleCI example, see 6c3960a8f4/.circleci/config.ymlFixes#5404
When run under gccgo, the test looks for the name that gccgo gives to
a thunk method. This name is not normally visible, but can be seen
when using reflect.FuncForPC as this code does. That name changed in
https://golang.org/cl/89555. Change the test to work with both the
old name "$thunk0" and the new name "thunk0".
Fixesgolang/go#28669
This commit also pulls down the log level for a set of WARN statements to INFO. There should be no ERRORs or WARNINGs in a regular Hugo build. That is the story about the Boy Who Cried Wolf.
Since the WARN log is now more visible, this commit also improves on some of them, most notable the "layout not found", which now would look something like this:
```bash
WARN 2018/11/02 09:02:18 Found no layout for "home", language "en", output format "CSS": create a template below /layouts with one of these filenames: index.en.css.css, home.en.css.css, list.en.css.css, index.css.css, home.css.css, list.css.css, index.en.css, home.en.css, list.en.css, index.css, home.css, list.css, _default/index.en.css.css, _default/home.en.css.css, _default/list.en.css.css, _default/index.css.css, _default/home.css.css, _default/list.css.css, _default/index.en.css, _default/home.en.css, _default/list.en.css, _default/index.css, _default/home.css, _default/list.css
```
Fixes#5203
Long identifiers will give errors on the format:
```bash
_default/single.html:5:14: executing "main" at <.ThisIsAVeryLongTitl...>: can't evaluate field ThisIsAVeryLongTitle
```
Hugo use this value to match the "base template or not", so we need to strip the "...".
Fixes#5346
`*json.UnmarshalTypeError` and `*json.SyntaxError` has a byte `Offset`, so use that.
This commit also reworks/simplifies the errror line matching logic. This also makes the file reading unbuffered, but that should be fine in this error case.
See #5324
The main item in this commit is showing of errors with a file context when running `hugo server`.
This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).
But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.
Fixes#5284Fixes#5290
See #5325
See #5324
This commit consolidates the reflective collections handling in `.Scratch` vs the `tpl` package so they use the same code paths.
This commit also adds support for a corner case where a typed slice is appended to a nil or empty `[]interface{}`.
Fixes#5275
In Hugo `0.49` we improved type support in `slice`. This has an unfortunate side effect in that `resources.Concat` now expects something that can resolve to `resource.Resources`.
This worked for most situations, but when you try to `slice` different `Resource` objects, you would be getting `[]interface {}` and not `resource.Resources`. And `concat` would fail:
```bash
error calling Concat: slice []interface {} not supported in concat.
```
This commit fixes that by simplifying the type checking logic in `Slice`:
* If the first item implements the `Slicer` interface, we try that
* If the above fails or the first item does not implement `Slicer`, we just return the `[]interface {}`
Fixes#5269
The original implementation of NumFmt did not take into account that the
options delimiter (a space) could be a valid option. Adding a delim
parameter seems like the simplest, safest, and most flexible way to
solve this oversight in a backwards-compatible way.
Fixes#5260
Before this commit you would typically use `.Scratch.Add` to manually create slices in a loop.
With variable overwrite in Go 1.11, we can do better. This commit adds the `append` template func.
A made-up example:
```bash
{{ $p1 := index .Site.RegularPages 0 }}{{ $p2 := index .Site.RegularPages 1 }}
{{ $pages := slice }}
{{ if true }}
{{ $pages = $pages | append $p2 $p1 }}
{{ end }}
```
Note that with 2 slices as arguments, the two examples below will give the same result:
```bash
{{ $s1 := slice "a" "b" | append (slice "c" "d") }}
{{ $s2 := slice "a" "b" | append "c" "d" }}
```
Both of the above will give `[]string{a, b, c, d}`.
This commit also improves the type handling in the `slice` template function. Now `slice "a" "b"` will give a `[]string` slice. The old behaviour was to return a `[]interface{}`.
Fixes#5190
This extends the page grouping in Hugo with a template function that allows for ad-hoc grouping.
A made-up example:
```
{{ $cool := where .Site.RegularPages "Params.cool" true | group "cool" }}
{{ $blue := where .Site.RegularPages "Params.blue" true | group "blue" }}
{{ $paginator := .Paginate (slice $cool $blue) }}
```
Closes#4865
All `.Params` are stored lowercase, but it should work to access them `.Page.camelCase` etc. There was, however, some holes in the logic with the old transformer.
This commit fixes that by applying a blacklist instead of the old whitelist logic. `.Param` is a very distinct key. The original case will be kept in `.Data.Params.myParam`, but other than that it will be lowercased.
Fixes#5068
Hugo Pipes added minification support for resources fetched via ´resources.Get` and similar.
This also adds support for minification of the final output for supported output formats: HTML, XML, SVG, CSS, JavaScript, JSON.
To enable, run Hugo with the `--minify` flag:
```bash
hugo --minify
```
This commit is also a major spring cleaning of the `transform` package to allow the new minification step fit into that processing chain.
Fixes#1251
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.
This commit adds
* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.
This means that you can now do this in your templates (or shortcodes):
```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```
This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:
```
HUGO_BUILD_TAGS=extended mage install
```
Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.
The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:
```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```
The transformation funcs above have aliases, so it can be shortened to:
```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```
A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.
Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test
New functions to create `Resource` objects:
* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.
New `Resource` transformation funcs:
* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.
Fixes#4381Fixes#4903Fixes#4858
Go developers have undone the breaking API changes
in the following commit:
commit bedfa4e1c37bd08063865da628f242d27ca06ec4
Author: Daniel Theophanes <kardianos@gmail.com>
Date: Thu Jun 21 10:41:26 2018 -0700
text/template/parse: undo breaking API changes
golang.org/cl/84480 altered the API for the parse package for
clarity and consistency. However, the changes also broke the
API for consumers of the package. This CL reverts the API
to the previous spelling, adding only a single new exported
symbol.
Fixes#25968
Change-Id: Ieb81054b61eeac7df3bc3864ef446df43c26b80f
Reviewed-on: https://go-review.googlesource.com/120355
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
See https://github.com/golang/go/issues/25968
This reverts commit 9f27091e10.
Closes#4784Fixes#4873
This commit adds support for theme composition and inheritance in Hugo.
With this, it helps thinking about a theme as a set of ordered components:
```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```
The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.
So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.
Hugo uses two different algorithms to merge the filesystems, depending on the file type:
* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.
The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are plans to improve on this and get a URL scheme so this can be resolved automatically.
Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:
* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`
The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.
A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.
Fixes#4460Fixes#4450
The current full set of options for GA is now:
```toml
[privacy]
[privacy.googleAnalytics]
disable = false
respectDoNotTrack = true
anonymizeIP = true
useSessionStorage = true
```
Fixes#4775
Having them in separate files should make maintainance easier.
When adding new or making changes to the templates:
```bash
mage generate
```
This will get the Go code in sync.
Fixes#4457
A sample config:
```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true
[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"
[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```
The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.
The content files will be assigned a language by
1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.
The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.
Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.
If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.
Fixes#4523Fixes#4552Fixes#4553
As an example:
```html
{{ $pages := .Site.RegularPages | lang.Merge $frSite.RegularPages | lang.Merge $enSite.RegularPages }}
```
Will "fill in the gaps" in the current site with, from left to right, content from the French site, and lastly the English.
Fixes#4463
This means that you can do something ala:
```html
{{ if ge .Hugo.Version "0.36" }}Reasonable new Hugo version!{{ end }}
```
The intented use is feature toggling, but please note that it will take some time and Hugo versions until this can be trusted. It does not work in older Hugo versions.
Fixes#4443
Also:
- Remove unnecessary space from `figure` tag if no class is specified.
- Update related tests.
- Add test cases for the changes made to the figure shortcode.
- Document the newly added target and rel parameters
- Add more detail to the documentation of all figure shortcode parameters.
The tplimpl package was misusing the TemplateLookupDescriptor.WorkingDir
field from the output package. By incorrectly setting it to the theme
directory instead of the site root, the user is unable to override theme
templates in some situations.
Fixes#3505
The new lookup order:
1) Page.Params.images if set
2) Image resources: images with name "feature" (priority), "cover", "thumbnail"
3) Site.Params.images if set
Fixes#4349
This is in heavy use in rendering, so this makes a difference:
```bash
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 124551144 107743429 -13.49%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 528684 435118 -17.70%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 53306848 45147832 -15.31%
```
This commit adds support for multiple statDirs both on the global and language level.
A simple `config.toml` example:
```bash
staticDir = ["static1", "static2"]
[languages]
[languages.no]
staticDir = ["staticDir_override", "static_no"]
baseURL = "https://example.no"
languageName = "Norsk"
weight = 1
title = "På norsk"
[languages.en]
staticDir2 = "static_en"
baseURL = "https://example.com"
languageName = "English"
weight = 2
title = "In English"
```
In the above, with no theme used:
the English site will get its static files as a union of "static1", "static2" and "static_en". On file duplicates, the right-most version will win.
the Norwegian site will get its static files as a union of "staticDir_override" and "static_no".
This commit also concludes the Multihost support in #4027.
Fixes#36Closes#4027
This commit adds a "cache potential" column when running `hugo --templateMetrics --templateMetricsHints`.
This is only calculated when `--templateMetricsHints` is set, as these calculations has an negative effect on the other timings.
This gives a value for partials only, and is a number between 0-100 that indicates if `partial` can be replaced with `partialCached`.
100 means that all execution of the same partial resulted in the same output.
You should do some manual research before going "all cache".
Add template function that will build a string from the given format
string and arguments, then log it to ERROR. This has an intended
side-effect of causing the build to fail, when executed.
Resolves#3817
Changes fall into one of the following:
- gofmt -s
- receiver name is inconsistent
- omit unused 2nd value from range
- godoc comment formed incorrectly
- err assigned and not used
- if block ends with a return statement followed by else
Add a template function that allows conversion to float. This is
useful, for example, when passing aspect ratios into templates,
which tend to not be integers.
Fixes#3307
Ceil and Floor are frontends for the stdlib math functions. The Round
implementation is essentially the same thing except that the Go stdlib
doesn't include a Round implementation in a stable release yet. I've
included the Round function slated for Go 1.10.
Fixes#3883
This closes#98, even if this commit does not do full content text search.
We may revisit that problem in the future, but that deserves its own issue.
Fixes#98
These funcs were added during the move to namespaces but were
undocumented. This commit fixes the order of the arguments and adds the
funcs to the method mapping.
And use it in `eq` and `ne` so `Page` values can be compared directly in the templates without thinking about it being a `Page` or a `PageOutput` wrapper.
Fixes#3807
As pointed out by the linter, some exported functions and types are
missing doc comments.
The linter warnings have been reduced from 194 to 116.
Not all missing comments have been added in this commit though.
This works for the `title` func and the other places where Hugo makes title case.
* AP style (new default)
* Chicago style
* Go style (what we have today)
Fixes#989
This enables `AND` (`intersect`) and `OR` (`union`) filters when combined with `where`.
Example:
```go
{{ $pages := where .Site.RegularPages "Type" "not in" (slice "page" "about") }}
{{ $pages := $pages | union (where .Site.RegularPages "Params.pinned" true) }}
{{ $pages := $pages | intersect (where .Site.RegularPages "Params.images" "!=" nil) }}
```
The above fetches regular pages not of `page` or `about` type unless they are pinned. And finally, we exclude all pages with no `images` set in Page params.
Fixes#3174
* Improve the built-in Disqus template
Set `disqus_identifier`, `disqus_title`, and `disqus_url`
only if the user has explicitly provided them.
Do not load Disqus when the website is previewed locally,
otherwise it is very confusing.
* Use disqus_config instead of three global variables
https://help.disqus.com/customer/portal/articles/472098-javascript-configuration-variables
Unsupported types are currently silently ignored by IsSet. An earlier
attempt was made to solve the issue by returning an error. That attempt
was reverted since it broke some existing themes.
So instead, we'll log an error. Hopefully, people will stop using IsSet
in this way, and we can eventually return an error outright.
Updates #3092
This commit allows shortcode per output format, a typical use case would be the special AMP media tags.
Note that this will only re-render the "overridden" shortcodes and only in pages where these are used, so performance in the normal case should not suffer.
Closes#3220
- Move the main GetCSV and GetJSON into data.go.
- Add error returns to GetCSV and GetJSON.
- Add http client to Namespace for test mocking.
- Send accept headers on remote requests. Fixes#3395
- Return an error on non-2XX HTTP response codes and don't retry.
- Move cache tests to cache_test.go.
As a first step to remove the hard ties between `tplimpl` and the different namespace packages.
The `lang` package is used as the first example use case.
See #3042
This commit moves almost all of the template functions into separate
packages under tpl/ and adds a namespace framework. All changes should
be backward compatible for end users, as all existing function names in
the template funcMap are left intact.
Seq and DoArithmatic have been moved out of the helpers package and into
template namespaces.
Most of the tests involved have been refactored, and many new tests have
been written. There's still work to do, but this is a big improvement.
I got a little overzealous and added some new functions along the way:
- strings.Contains
- strings.ContainsAny
- strings.HasSuffix
- strings.TrimPrefix
- strings.TrimSuffix
Documentation is forthcoming.
Fixes#3042
For pages with translations, add links with hreflang attributes to the
default sitemap template. This helps Google to show the correct
language page in its search results. The syntax used is based on
Google's example at [1].
Also update the sitemap template docs to reflect the changes in the
default template.
[1]
https://support.google.com/webmasters/answer/2620865?hl=en&topic=2370587&ctx=topicFixes#2569
Most obvius benefit of this is to include CSS partials with css file suffix into HTML templates.
A valid workaround would be to rename the file `mystyles.html`, but that doesn't work too good for external editors etc.
The css partial is a method used in some themes before Hugo 0.20, but then it stopped working.
This commit reintroduces that behaviour.
Note that the regular layout lookups for text templates, i.e. "single.json" will be
prefixed with "_text/" on lookup and will only match in the text collection.
Fixes#3273
Before this commit, Hugo used `html/template` for all Go templates.
While this is a fine choice for HTML and maybe also RSS feeds, it is painful for plain text formats such as CSV, JSON etc.
This commit fixes that by using the `IsPlainText` attribute on the output format to decide what to use.
A couple of notes:
* The above requires a nonambiguous template name to type mapping. I.e. `/layouts/_default/list.json` will only work if there is only one JSON output format, `/layouts/_default/list.mytype.json` will always work.
* Ambiguous types will fall back to HTML.
* Partials inherits the text vs HTML identificator of the container template. This also means that plain text templates can only include plain text partials.
* Shortcode templates are, by definition, currently HTML templates only.
Fixes#3221
Before this commit, Hugo used `html/template` for all Go templates.
While this is a fine choice for HTML and maybe also RSS feeds, it is painful for plain text formats such as CSV, JSON etc.
This commit fixes that by using the `IsPlainText` attribute on the output format to decide what to use.
A couple of notes:
* The above requires a nonambiguous template name to type mapping. I.e. `/layouts/_default/list.json` will only work if there is only one JSON output format, `/layouts/_default/list.mytype.json` will always work.
* Ambiguous types will fall back to HTML.
* Partials inherits the text vs HTML identificator of the container template. This also means that plain text templates can only include plain text partials.
* Shortcode templates are, by definition, currently HTML templates only.
Fixes#3221
Add a new rssLimit site configuration option with default of 15. Prior
to this fix, you could create your own RSS feed to override the default
limit of 15, but we still had a hardcoded limit of 50 items set in
`hugolib.renderRSS()`.
With this option in place, the `range first 15 .Data.Pages` logic is no
longer hardcoded into the embedded RSS template.
Because the size of the slice passed to the template is now limited to
rssLimit instead of 50, this commit is a breaking change for sites
with a custom RSS template that expects more than 15 items.
Fixes#3035
Allow all nodes/pages to use date-related OpenGraph meta tags, not
only sites which are using the as-of-yet unfinished .Site.Authors
functionality.
Improve compliance of tags with Facebook's OpenGraph docs for the
"website" and "article" types[0][1]. Also, use the proper tag for
modification date (og:update_time vs article:modified_time).
Generate date published using either .PublishDate or .Date, and use
.Lastmod for modification date, which can use the new enableGitInfo
functionality from Hugo 0.18, but seamlessly falls back to .Date if
the site does not have this enabled/supported.
[0] https://developers.facebook.com/docs/reference/opengraph/object-type/website/
[1] https://developers.facebook.com/docs/reference/opengraph/object-type/article/
