Commit graph

7008 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
44954497bc
Always use content to resolve content type in resources.GetRemote
This is a security hardening measure; don't trust the URL extension or any `Content-Type`/`Content-Disposition` header on its own, always look at the file content using Go's `http.DetectContentType`.

This commit also adds ttf and otf media type definitions to Hugo.

Fixes #9302
Fixes #9301
2021-12-17 09:50:28 +01:00
Bjørn Erik Pedersen
22ef5da20d
Add resources.GetRemote
In Hugo 0.89 we added remote support to `resources.Get`.

In hindsight that was not a great idea, as a poll from many Hugo users showed. See Issue #9285 for more details.

After this commit `resources.Get` only supports local resource lookups. If you want to support both, you need to use a construct similar to:

Also improve some option case handling.

```
{{ resource := "" }}
{{ if (urls.Parse $url).IsAbs }}
{{ $resource = resources.GetRemote $url }}
{{ else }}
{{ $resource = resources.Get $url }}
{{ end }}
```

Fixes #9285
Fixes #9296
2021-12-17 09:33:51 +01:00
Paul Gottschling
5758c370ea
Allow for return partials with falsy arguments (#9298)
Partials with returns values are parsed, then inserted into a
partial return wrapper via wrapInPartialReturnWrapper in order
to assign the return value via *contextWrapper.Set. The
predefined wrapper template for partials inserts a partial's nodes
into a "with" template action in order to set dot to a
*contextWrapper within the partial. However, because "with" is
skipped if its argument is falsy, partials with falsy arguments
were not being evaluated.

This replaces the "with" action in the partial wrapper with a
"range" action that isn't skipped if .Arg is falsy.

Fixes #7528
2021-12-17 08:35:21 +01:00
Bjørn Erik Pedersen
8ee6de6d96 deps: Upgrade github.com/evanw/esbuild v0.14.2 => v0.14.5 2021-12-17 08:26:45 +01:00
Joachim Ansorg
a4b9f1a92c don't use path.Join, because it cleans the final path 2021-12-16 21:14:33 +01:00
Bjørn Erik Pedersen
f4389e48ce
Add some basic security policies with sensible defaults
This ommmit contains some security hardening measures for the Hugo build runtime.

There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers".

For `asciidoctor` and some others we use Go's `os/exec` package to start a new process.

These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off.

You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do.

The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all.

```toml
[security]
  enableInlineShortcodes = false
  [security.exec]
    allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
    osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

  [security.funcs]
    getenv = ['^HUGO_']

  [security.http]
    methods = ['(?i)GET|POST']
    urls = ['.*']
```
2021-12-16 09:40:22 +01:00
Bjørn Erik Pedersen
803f572e66
Simplify Babel test assertions
To make the tests pass on my grand old Windows 8 VM.
2021-12-14 07:26:53 +01:00
Bjørn Erik Pedersen
6183184b96
Merge commit '45e6fdb315d113ba13e20a633ed0c67e3f25170d' 2021-12-13 21:05:10 +01:00
Bjørn Erik Pedersen
45e6fdb315 Squashed 'docs/' changes from 316cec249..4eb10c1a9
4eb10c1a9 Set a sensible maxAge on the image cache
2c37f4f89 Update lists.md
8690c018f Update introduction.md
0458c3d88 Update index.md
150982f5e Update index.md
50dc0b6b9 Update multilingual.md
648598090 Configure image cache to use :cacheDir/images
4dfb523c3 Update introduction.md
b94d42e6f Add text to social images
d3253d2aa Add resources/_gen/images/ to .gitignore
8a56e1dfd Delete resources/_gen/images
1521e79b6 Update introduction.md
c7110a597 Update introduction.md
f7c4d220f Update introduction.md
c14ecf1ff Update introduction.md
c4b388c20 Fix shortcode in hugo-pipes/introduction (#1605)
d9876bd68 Update introduction.md
a39a532d5 Update introduction.md
fa8758122 netlify: Hugo 0.90.1
50cc40570 Update Contribute to Hugo Development
fc98fc82c Add .vscode dir to .gitignore (#1602)
8f8bcdefc Add frontmatter param to code-toggle shortcode (#1601)
f3b432c7c Add markdownlint configuration file (#1597)
7ea3aea92 netlify: Bump to 0.90.0
e2b6f990b docs: Regenerate docs helper
c02259e33 Merge commit '8d9511a08f14260cbfb73119e4afae50e5a9966d'
2e3573899 Add custom font support to images.Text
3186e6458 images: Text filter that draws text with the given options (#9239)
424f219eb tpl/transform: Optional options for highlight func
7993f17d0 Implement XML data support
1eb9d34cf Make resources.Get use a file cache for remote resources
8a0fffd56 Add remote support to resources.Get

git-subtree-dir: docs
git-subtree-split: 4eb10c1a9689aebf652011a6260a14bf8a3dcef4
2021-12-13 21:04:12 +01:00
Joe Mooring
a037be774d
Improve handling of remote image/jpeg resources (#9278)
Add jpe, jif, and jfif to image/jpeg extensions.
For remote image/jpeg without extension, always use jpg extension.

Closes #9275
2021-12-13 08:55:15 +01:00
hitzhangjie
8a005538db
Fix Dockerfile
Mage uses git, so we should install git before run mage.

Closes #9261
2021-12-12 14:45:08 +01:00
Joe Mooring
657d0272e3 Remove debug statement 2021-12-11 22:59:36 +01:00
Bjørn Erik Pedersen
159120cddd
Fix deprecation notice
Asterisks needs to be quoted on command line.
2021-12-11 11:54:28 +01:00
hugoreleaser
3f0d49e502 releaser: Prepare repository for 0.91.0-DEV
[ci skip]
2021-12-10 11:42:18 +00:00
hugoreleaser
4890788974 releaser: Bump versions for release of 0.90.1
[ci skip]
2021-12-10 10:56:40 +00:00
hugoreleaser
3075eaa3b0 releaser: Add release notes for 0.90.1
[ci skip]
2021-12-10 10:56:40 +00:00
Bjørn Erik Pedersen
3bc6830411 Remove the retries on error in remote resources.Get
Fixes #9271
See  #9259
2021-12-10 11:10:41 +01:00
Bjørn Erik Pedersen
e4d6ec94b5 Allow user to handle/ignore errors in resources.Get
In Hugo 0.90.0 we introduced remote support in `resources.Get`.

But with remote resources comes with a higher chance of failing a build (network issues, remote server down etc.).

Before this commit we always failed the build on any unexpected error.

This commit allows the user to check for any error (and potentially fall back to a default local resource):

```htmlbars
{{ $result := resources.Get "https://gohugo.io/img/hugo-logo.png" }}
{{ with $result }}
        {{ if .Err }}
        {{/* log the error, insert a default image etc. *}}
        {{ else }}
        <img src="{{ .RelPermalink }}" width="{{ .Width }}" height="{{ .Height }}" alt="">
        {{ end }}
{{ end }}
```

Note that the default behaviour is still to fail the build, but we will delay that error until you start using the `Resource`.

Fixes #9529
2021-12-10 11:10:41 +01:00
Bjørn Erik Pedersen
6260455ba7 Make resource.Get return nil on 404 not found
This is in line with the interface declaration and also how local lookups work.

Fixes #9267
2021-12-10 11:10:41 +01:00
Bjørn Erik Pedersen
c397975af8
Update to Go 1.17.5
Fixes #9269
2021-12-10 10:36:02 +01:00
Bjørn Erik Pedersen
965a6cbff9 Update to Go 1.17.4 and remove timeout in resources.Get
Fixes #9265
2021-12-09 13:33:30 +01:00
hugoreleaser
34a96290fa releaser: Prepare repository for 0.91.0-DEV
[ci skip]
2021-12-08 10:11:31 +00:00
hugoreleaser
dd0d3fdbbe releaser: Bump versions for release of 0.90.0
[ci skip]
2021-12-08 09:33:46 +00:00
hugoreleaser
b921752331 releaser: Add release notes for 0.90.0
[ci skip]
2021-12-08 09:33:46 +00:00
Bjørn Erik Pedersen
0fa40ce58f
releaser: Simplify the release process
Make it into a one step even for major releases.
2021-12-08 09:38:15 +01:00
Bjørn Erik Pedersen
bf537f1c6d
releaser: Remove unused code 2021-12-08 09:02:45 +01:00
Bjørn Erik Pedersen
e86b331138
docs: Regenerate docs helper 2021-12-08 08:56:16 +01:00
Bjørn Erik Pedersen
6c841a691e
Merge commit '8d9511a08f14260cbfb73119e4afae50e5a9966d' 2021-12-08 08:54:25 +01:00
Bjørn Erik Pedersen
8d9511a08f Squashed 'docs/' changes from 39a7fac34..316cec249
316cec249 Update future events template example (#1595)
3bde7d489 Install mage outside module (#1592)
762e27eff Clarify ignoreFiles regex matching
4d0032051 Add id attribute to h2 elements (#1590)
8262b077c Improve inline resource examples (#1587)
2eae7c7ec fix disqus example name (#1588)
a772f4804 Added install instructions for openSUSE Tumbleweed (#1459)
7ad1c301b Remove screen capture from Hosting on GitHub page (#1586)
a58541f49 add more details on about gh-pages and baseURL on hosting-on-github.md (#1346)
3bd0b46dc Update configuration page (#1585)
4cf1f013e Update OS functions
2c45a95c2 Remove getting-started/code-toggle/
40fdff598 Describe artificial language private use subtags (#1577)
91011d210 Remove google_news from list of internal templates (#1576)
36c7879e4 Update the .Unix function
731063488 Remove a showcase
818c371a0 Update index.md
3136d39d9 netlify: Hugo 0.89.4
092bc9278 Merge branch 'tempv0.89.4'
18e01f105 releaser: Add release notes to /docs for release of 0.89.4
79135281f Correct and sort list of target image formats (#1574)
af4170c7e netlify: Hugo 0.89.3
7f5444251 Merge branch 'tempv0.89.3'
a32e4a6c2 releaser: Add release notes to /docs for release of 0.89.3
6dd3dc3f9 Update configuration.md
5fbe741d7 Update index.md (#1570)
37a69496f netlify: Bump to Hugo 0.89.2
3b293f1f4 Merge branch 'tempv0.89.2'
64c934e7a releaser: Add release notes to /docs for release of 0.89.2
919c51c7d Update index.md
13dd463b1 netlify: Hugo 0.89.1
d8cda1474 releaser: Add release notes to /docs for release of 0.89.1
a2adf7742 releaser: Add release notes to /docs for release of 0.89.1
c3088c4fc Add code toggle to menus page (#1568)
2d0f38978 Remove blank lines from code-toggle output (#1564)
7cf058bfd Add localization examples (#1563)
cf8627c2e Fixing typos, fixing incomplete link (#1561)
c78cc014b Document the removePathAccents setting
70beddaf4 Make corrections to 0.89.0 release notes (#1560)
1917195f0 Update index.md
7fb8e070c Run hugo --gc
1772d45fb Release 0.89.0
d9006179b Merge branch 'tempv0.89.0'
8db86b61e releaser: Add release notes to /docs for release of 0.89.0
abf268571 docs: Regen CLI docs
fbbdb0ab1 Update the timeout default
9cbd1c15a Fix description of lang.FormatNumberCustom
6043b54cc Remove "render" keyword from Host on Render page
f8ea8e84f Clarify description of front matter url (#1557)
91a0c9954 Update Twitter shortcode oEmbed endpoint
79a7405b8 Merge commit 'aa5ac36a3eb68b86c803caec703869efefc8447e'
57667bae6 hugofs: Add includeFiles and excludeFiles to mount configuration
0c9ee0a04 Allow multiple plugins in the PostCSS options map
155799e6b docs: Create path.Clean documentation

git-subtree-dir: docs
git-subtree-split: 316cec2494dc5f908283289371d74f36a73d3d8d
2021-12-08 08:42:31 +01:00
Bjørn Erik Pedersen
e71d715b9b Add custom font support to images.Text
Fixes #9253
2021-12-07 16:53:02 +01:00
Bjørn Erik Pedersen
e61cdf335f images: Fix cache busting of image text filter
See #9238
2021-12-07 16:53:02 +01:00
dependabot[bot]
6c3bc5ebad build(deps): bump github.com/getkin/kin-openapi from 0.80.0 to 0.85.0
Bumps [github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi) from 0.80.0 to 0.85.0.
- [Release notes](https://github.com/getkin/kin-openapi/releases)
- [Commits](https://github.com/getkin/kin-openapi/compare/v0.80.0...v0.85.0)

---
updated-dependencies:
- dependency-name: github.com/getkin/kin-openapi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-07 13:12:42 +01:00
Paul van Brouwershaven
283394a4fd
images: Text filter that draws text with the given options (#9239)
Fixes #9238
2021-12-07 11:29:55 +01:00
Joe Mooring
5538507e90 tpl/transform: Optional options for highlight func
Closes #9249
Fixes gohugoio/hugoDocs#63
2021-12-07 11:26:56 +01:00
Bjørn Erik Pedersen
b4f27ef8e7
deps: Upgrade github.com/evanw/esbuild v0.13.12 => v0.14.2 (note)
Note: ESBuild v0.14.0 contains backwards-incompatible changes, see https://github.com/evanw/esbuild/releases/tag/v0.14.0

Fixes #9244
2021-12-04 16:28:50 +01:00
Bjørn Erik Pedersen
3473e31eb1
releaser: Add "note" to Note regexp 2021-12-04 16:27:56 +01:00
dependabot[bot]
fa0da004a9 build(deps): bump github.com/mitchellh/mapstructure from 1.4.2 to 1.4.3
Bumps [github.com/mitchellh/mapstructure](https://github.com/mitchellh/mapstructure) from 1.4.2 to 1.4.3.
- [Release notes](https://github.com/mitchellh/mapstructure/releases)
- [Changelog](https://github.com/mitchellh/mapstructure/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitchellh/mapstructure/compare/v1.4.2...v1.4.3)

---
updated-dependencies:
- dependency-name: github.com/mitchellh/mapstructure
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-03 18:16:50 +01:00
Bjørn Erik Pedersen
24a893cf83 releaser: Rework and simplify to use GitHub only for release notes 2021-12-03 17:02:20 +01:00
dependabot[bot]
bf1564bd2c build(deps): bump google.golang.org/api from 0.51.0 to 0.61.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.51.0 to 0.61.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.51.0...v0.61.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-12-03 17:01:37 +01:00
Paul van Brouwershaven
cd44d409ba
media: Add rss suffix for application/rss+xml
Closes #9233
2021-12-03 12:45:49 +01:00
Bjørn Erik Pedersen
9a326d5636
parser: Add a test case in format resolution
See #9233
2021-12-03 10:34:34 +01:00
Bjørn Erik Pedersen
b10381fbe0 lazy: Reset error in Reset
To prevent sticky errors on server rebuilds.

Fixes #7043
Closes #9194
2021-12-02 18:06:19 +01:00
Paul van Brouwershaven
0eaaa8fee3
Implement XML data support
Example:

```
{{ with resources.Get "https://example.com/rss.xml" | transform.Unmarshal }}
    {{ range .channel.item }}
        <strong>{{ .title | plainify | htmlUnescape }}</strong><br />
        <p>{{ .description | plainify | htmlUnescape }}</p>
        {{ $link := .link | plainify | htmlUnescape }}
        <a href="{{ $link }}">{{ $link }}</a><br />
        <hr>
    {{ end }}
{{ end }}
```

Closes #4470
2021-12-02 17:30:36 +01:00
Joe Mooring
58adbeef88 Validate private use language tags
Fixes #9119
2021-12-02 16:30:53 +01:00
Bjørn Erik Pedersen
93572e5318 resources: Add timeout to the HTTP request in Get
Workaround for https://github.com/golang/go/issues/49366
2021-12-02 16:11:14 +01:00
Bjørn Erik Pedersen
94f149b21e Add a remote retry for resources.Get 2021-12-02 16:11:14 +01:00
Paul van Brouwershaven
66753416b5
Make resources.Get use a file cache for remote resources
Closes #9228
2021-12-02 12:56:25 +01:00
Joe Mooring
133e4bfbee Remove empty href element from pagination template
Closes #9149
2021-12-02 09:14:30 +01:00
Joe Mooring
f122771fb1 Check for empty deployment targets and matchers
Fixes #9220
2021-12-01 10:17:42 +01:00
Bjørn Erik Pedersen
08a863e1e8
resources: Adjust the remote Get cache so it does not get evicted on restarts
We fill improve this, soon.
2021-11-30 16:54:01 +01:00