This ommmit contains some security hardening measures for the Hugo build runtime.
There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers".
For `asciidoctor` and some others we use Go's `os/exec` package to start a new process.
These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off.
You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do.
The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all.
```toml
[security]
enableInlineShortcodes = false
[security.exec]
allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']
[security.funcs]
getenv = ['^HUGO_']
[security.http]
methods = ['(?i)GET|POST']
urls = ['.*']
```
In Hugo 0.90.0 we introduced remote support in `resources.Get`.
But with remote resources comes with a higher chance of failing a build (network issues, remote server down etc.).
Before this commit we always failed the build on any unexpected error.
This commit allows the user to check for any error (and potentially fall back to a default local resource):
```htmlbars
{{ $result := resources.Get "https://gohugo.io/img/hugo-logo.png" }}
{{ with $result }}
{{ if .Err }}
{{/* log the error, insert a default image etc. *}}
{{ else }}
<img src="{{ .RelPermalink }}" width="{{ .Width }}" height="{{ .Height }}" alt="">
{{ end }}
{{ end }}
```
Note that the default behaviour is still to fail the build, but we will delay that error until you start using the `Resource`.
Fixes#9529
The existing endpoint will be retired and removed on November 23, 2021.
References:
- https://twittercommunity.com/t/consolidating-the-oembed-functionality/154690
- https://developer.twitter.com/en/docs/twitter-for-websites/oembed-api#Embedded
This is a backward compatible change.
The existing endpoint requires a single parameter: the id of the tweet.
The new endpoint requires two parameters: the id of the tweet, and the
user with whom it is associated. For the moment, if you supply the wrong
user, the request will be redirected (with a small delay) to the correct
user/id pair. This behavior is undocumented, but we will take advantage
of it as Hugo site authors transition to the new syntax.
{{< tweet 1453110110599868418 >}} --> works, throws warning, deprecate at some point
{{< tweet user="SanDiegoZoo" id="1453110110599868418" >}} --> new syntax
Fixes#8130
The old implementation had some issues, mostly related to the context (e.g. name, file paths) passed to the template.
This new implementation is using the exact same code path for evaluating the pages as in a regular build.
This also makes it more robust and easier to reason about in a multilingual setup.
Now, if you are explicit about the target path, Hugo will now always pick the correct mount and language:
```bash
hugo new content/en/posts/my-first-post.md
```
Fixes#9032Fixes#7589Fixes#9043Fixes#9046Fixes#9047
Previously, *minifyTransformation.Transform suppressed the
error returned by t.m.Minify. This meant that when minification
returned an error, the error would not reach the user. Instead,
minification would silently fail. For example, if a JavaScript
file included a call to the Date constructor with:
new Date(2020, 04, 02)
The package that the minification library uses to parse JS files,
github.com/tdewolff/parse would return an error, since "04" would
be parsed as a legacy octal. However, the JS file would remain
un-minified with no error.
Fixing this is not as simple as replacing "_" with an "err" in
*minifyTransformation.Transform, however (though this is
necessary). If we only returned this error from Transform,
then hugolib.TestResourceMinifyDisabled would fail. Instead of
being a no-op, as TestResourceMinifyDisabled expects, using the
"minify" template function with a "disableXML=true" config
setting instead returns the error, "minifier does not exist for
mimetype."
The "minifier does not exist" error is returned because of the
way minifiers.New works. If the user's config disables
minification for a particular MIME type, minifiers.New does
not add it to the resulting Client's *minify.M. However, this
also means that when the "minify" template function is executed,
a *resourceAdapter's transformations still add a minification.
When it comes time to call the minify.Minifier for a specific
MIME type via *M.MinifyMimetype, the github.com/tdewolff/minify
library throws the "does not exist" error for the missing MIME
type.
The solution was to change minifiers.New so, instead of skipping
a minifier for each disabled MIME type, it adds a NoOpMinifier,
which simply copies the source to the destination without
minification. This means that when the "minify" template
function is used for a particular resource, and that resource's
MIME type has minification disabled, minification is genuinely
skipped, and does not result in an error.
In order to add this, I've fixed a possibly unwanted interaction
between minifiers.TestConfigureMinify and
hugolib.TestResourceMinifyDisabled. The latter disables
minification and expects minification to be a no-op. The former
disables minification and expects it to result in an error. The
only reason hugolib.TestResourceMinifyDisabled passes in the
original code is that the "does not exist" error is suppressed.
However, we shouldn't suppress minification errors, since they
can leave users perplexed. I've changed the test assertion in
minifiers.TestConfigureMinify to expect no errors and a no-op
if minification is disabled for a particular MIME type.
Fixes#8954
Unless the merge strategy is set up to do so.
For `disableKinds` the current workaround is to make sure the project config has an entry, even if is empty:
```
disableKinds = []
```
Note that this issue only touches root, non-map config-values that either is not set in project config or in Hugo's defaults.
Fixes#8866
We have been using `go-toml` for language files only. This commit makes it the only TOML library.
It's spec compliant and very fast.
A benchark building a site with 200 pages with TOML front matter:
```bash
name old time/op new time/op delta
SiteNew/Regular_TOML_front_matter-16 48.5ms ± 1% 47.1ms ± 1% -2.85% (p=0.029 n=4+4)
name old alloc/op new alloc/op delta
SiteNew/Regular_TOML_front_matter-16 16.9MB ± 0% 16.7MB ± 0% -1.56% (p=0.029 n=4+4)
name old allocs/op new allocs/op delta
SiteNew/Regular_TOML_front_matter-16 302k ± 0% 296k ± 0% -2.20% (p=0.029 n=4+4)
```
Note that the front matter unmarshaling is only a small part of building a site, so the above is very good.
Fixes#8801
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.
This is easier to reason about, and it's more effective:
```
name old time/op new time/op delta
SiteNew/Regular_Deep_content_tree-16 71.5ms ± 3% 69.4ms ± 5% ~ (p=0.200 n=4+4)
name old alloc/op new alloc/op delta
SiteNew/Regular_Deep_content_tree-16 29.7MB ± 0% 27.9MB ± 0% -5.82% (p=0.029 n=4+4)
name old allocs/op new allocs/op delta
SiteNew/Regular_Deep_content_tree-16 313k ± 0% 303k ± 0% -3.35% (p=0.029 n=4+4)
```
See #8749
Currently the generated `<pre>` element isn't fully accessible as it can't be focused by keyboard users.
To make this fully accessible, the attribute `tabindex="0"` should be added to the `<pre>` tag.
Closes#7194
This commit adds support for using the `cascade` keyword in your configuration file(s), e.g. `config.toml`.
Note that
* Every feature of `cascade` is available, e.g. `_target` to target specific page sets.
* Pages, e.g. the home page, can overwrite the cascade defined in config.
Fixes#8741
We have some commands that will continue even if the config loading fails (e.g. because a module can not be found). In Hugo 0.84.0 we introduced a new `_merge` flag that we removed once the configuration was loaded. But we did not do that in error situations, leading to failures in some situations.
This commit fixes that by making sure the configuration is always cleaned before return, even in error situations.
Fixes#8697
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster:
```
BenchmarkDefaultConfigProvider/Viper-16 252418 4546 ns/op 2720 B/op 30 allocs/op
BenchmarkDefaultConfigProvider/Custom-16 450756 2651 ns/op 1008 B/op 6 allocs/op
```
Fixes#8633Fixes#8618Fixes#8630
Updates #8591Closes#6680Closes#5192
Currently, no name shortcodes (`{{< >}}`) enter unexpected branch and
throw `BUG: template info not set`. This patch checks if shortcode has
name or not earlier and throws specific error.
Closes#6866
.Site.Menus is mutated when it is sorted for now and this causes concurrency problem (#7594)
In this patch, each related sort function copies Menu before sorting to prevent
race condition.
Pages already have such a sort and cache logic and this patch is identical to it.
Closes#7594
- Reorder code blocks
- Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information
- Expand benchmark to test for minified and unminified content
Fixes#8396Fixes#8417
The recent fix for #8346 had an unfortunate side-effect, as it prevented overrides of config options used in the module collector, e.g. `HUGO_MODULE_REPLACEMENTS`.
This commit fixes that by applying the overrides twice.
So we can use it and output.Format as map key etc.
This commit also fixes the media.Type implementation so it does not need to mutate itself to handle different suffixes for the same MIME type, e.g. jpg vs. jpeg.
This means that there are no Suffix or FullSuffix on media.Type anymore.
Fixes#8317Fixes#8324
