Commit graph

889 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
ea6bcd6944
tpl: Remove TODO comment 2022-02-15 23:39:54 +01:00
Bjørn Erik Pedersen
f2e7b49acf Add --printUnusedTemplates
Fixes #9502
2022-02-15 20:01:57 +01:00
Bjørn Erik Pedersen
b0a9cf0a75 tpl: Use go:embed to load internal templates
Fixes #8297
2022-02-14 09:39:46 +01:00
Bjørn Erik Pedersen
da4866c2be Simplify some integration tests 2022-02-10 20:43:19 +01:00
Bjørn Erik Pedersen
6a238a7276 tpl/templates: Fix templates.Exist issue with base templates
Fixes #9477
2022-02-09 22:33:07 +01:00
Bjørn Erik Pedersen
39f69ca7f0 openapi3: Port integration test into correct package 2022-02-09 15:41:32 +01:00
Joe Mooring
9262719092 Validate comparison operator argument count
Fixes #9462
2022-02-05 17:41:43 +01:00
Bjørn Erik Pedersen
2655739940 tpl/collections: Fix apply with namespaced template funcs
We changed the signature to `func(...interface{}) (interface{}, error)` some time ago, but sadly we had no test for this for `apply`. Now we do.

Fixes #9393
2022-01-17 09:50:48 +01:00
Bjørn Erik Pedersen
b84745d497
tpl/resources: Add empty method mapping for GetRemote
To get auto generated documentation for it.
2021-12-17 10:30:10 +01:00
Bjørn Erik Pedersen
44954497bc
Always use content to resolve content type in resources.GetRemote
This is a security hardening measure; don't trust the URL extension or any `Content-Type`/`Content-Disposition` header on its own, always look at the file content using Go's `http.DetectContentType`.

This commit also adds ttf and otf media type definitions to Hugo.

Fixes #9302
Fixes #9301
2021-12-17 09:50:28 +01:00
Bjørn Erik Pedersen
22ef5da20d
Add resources.GetRemote
In Hugo 0.89 we added remote support to `resources.Get`.

In hindsight that was not a great idea, as a poll from many Hugo users showed. See Issue #9285 for more details.

After this commit `resources.Get` only supports local resource lookups. If you want to support both, you need to use a construct similar to:

Also improve some option case handling.

```
{{ resource := "" }}
{{ if (urls.Parse $url).IsAbs }}
{{ $resource = resources.GetRemote $url }}
{{ else }}
{{ $resource = resources.Get $url }}
{{ end }}
```

Fixes #9285
Fixes #9296
2021-12-17 09:33:51 +01:00
Paul Gottschling
5758c370ea
Allow for return partials with falsy arguments (#9298)
Partials with returns values are parsed, then inserted into a
partial return wrapper via wrapInPartialReturnWrapper in order
to assign the return value via *contextWrapper.Set. The
predefined wrapper template for partials inserts a partial's nodes
into a "with" template action in order to set dot to a
*contextWrapper within the partial. However, because "with" is
skipped if its argument is falsy, partials with falsy arguments
were not being evaluated.

This replaces the "with" action in the partial wrapper with a
"range" action that isn't skipped if .Arg is falsy.

Fixes #7528
2021-12-17 08:35:21 +01:00
Bjørn Erik Pedersen
f4389e48ce
Add some basic security policies with sensible defaults
This ommmit contains some security hardening measures for the Hugo build runtime.

There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers".

For `asciidoctor` and some others we use Go's `os/exec` package to start a new process.

These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off.

You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do.

The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all.

```toml
[security]
  enableInlineShortcodes = false
  [security.exec]
    allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
    osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

  [security.funcs]
    getenv = ['^HUGO_']

  [security.http]
    methods = ['(?i)GET|POST']
    urls = ['.*']
```
2021-12-16 09:40:22 +01:00
Bjørn Erik Pedersen
3bc6830411 Remove the retries on error in remote resources.Get
Fixes #9271
See  #9259
2021-12-10 11:10:41 +01:00
Bjørn Erik Pedersen
e4d6ec94b5 Allow user to handle/ignore errors in resources.Get
In Hugo 0.90.0 we introduced remote support in `resources.Get`.

But with remote resources comes with a higher chance of failing a build (network issues, remote server down etc.).

Before this commit we always failed the build on any unexpected error.

This commit allows the user to check for any error (and potentially fall back to a default local resource):

```htmlbars
{{ $result := resources.Get "https://gohugo.io/img/hugo-logo.png" }}
{{ with $result }}
        {{ if .Err }}
        {{/* log the error, insert a default image etc. *}}
        {{ else }}
        <img src="{{ .RelPermalink }}" width="{{ .Width }}" height="{{ .Height }}" alt="">
        {{ end }}
{{ end }}
```

Note that the default behaviour is still to fail the build, but we will delay that error until you start using the `Resource`.

Fixes #9529
2021-12-10 11:10:41 +01:00
Joe Mooring
5538507e90 tpl/transform: Optional options for highlight func
Closes #9249
Fixes gohugoio/hugoDocs#63
2021-12-07 11:26:56 +01:00
Paul van Brouwershaven
0eaaa8fee3
Implement XML data support
Example:

```
{{ with resources.Get "https://example.com/rss.xml" | transform.Unmarshal }}
    {{ range .channel.item }}
        <strong>{{ .title | plainify | htmlUnescape }}</strong><br />
        <p>{{ .description | plainify | htmlUnescape }}</p>
        {{ $link := .link | plainify | htmlUnescape }}
        <a href="{{ $link }}">{{ $link }}</a><br />
        <hr>
    {{ end }}
{{ end }}
```

Closes #4470
2021-12-02 17:30:36 +01:00
Bjørn Erik Pedersen
93572e5318 resources: Add timeout to the HTTP request in Get
Workaround for https://github.com/golang/go/issues/49366
2021-12-02 16:11:14 +01:00
Joe Mooring
133e4bfbee Remove empty href element from pagination template
Closes #9149
2021-12-02 09:14:30 +01:00
Paul van Brouwershaven
8aa7257f65
Add remote support to resources.Get
Closes #5255
Supports #9044
2021-11-30 11:49:51 +01:00
Joe Mooring
75a823a36a Add deprecation warning to google_news template
Closes #9172
2021-11-21 11:25:17 +01:00
Joe Mooring
04a3b45db4 Fix description of lang.FormatNumberCustom
It currently refers to itself as a simple alternative, when it should
refer to lang.FormatNumber.
2021-11-01 18:54:43 +01:00
Joe Mooring
0cc39af682 Update Twitter shortcode oEmbed endpoint
The existing endpoint will be retired and removed on November 23, 2021.
References:

- https://twittercommunity.com/t/consolidating-the-oembed-functionality/154690
- https://developer.twitter.com/en/docs/twitter-for-websites/oembed-api#Embedded

This is a backward compatible change.

The existing endpoint requires a single parameter: the id of the tweet.

The new endpoint requires two parameters: the id of the tweet, and the
user with whom it is associated. For the moment, if you supply the wrong
user, the request will be redirected (with a small delay) to the correct
user/id pair. This behavior is undocumented, but we will take advantage
of it as Hugo site authors transition to the new syntax.

{{< tweet 1453110110599868418 >}} --> works, throws warning, deprecate at some point

{{< tweet user="SanDiegoZoo" id="1453110110599868418" >}} --> new syntax

Fixes #8130
2021-11-01 15:51:00 +01:00
Bjørn Erik Pedersen
e82cbd746f
tpl/time: Use configured location when date passed to Format is string
Updates #9084
2021-10-30 16:09:38 +02:00
Brad
e55466ce70
tpl/path: Add path.Clean
Fixes #8885
2021-10-05 16:15:10 +02:00
Paul Gottschling
a864ffe9ac Clarify "precision" in currency format functions
The documentation of the FormatAccounting and FormatCurrency
functions could be clearer in terms of how the precision param
works. This commit makes it more explicit that adding a precision
of < 2 will not format the return values to include fewer decimals.

Resolves #8858
2021-09-22 20:00:30 +02:00
Bjørn Erik Pedersen
abd969a670
Revert "tpl/time: Handle nil values in time.AsTime"
This reverts commit 3e11072892.
2021-08-10 13:21:49 +02:00
Bjørn Erik Pedersen
3e11072892
tpl/time: Handle nil values in time.AsTime
Closes #8865
2021-08-09 18:44:35 +02:00
Bjørn Erik Pedersen
9bba9a3a98
parser: Indent TOML tables
Fixes #8850
2021-08-04 11:39:16 +02:00
Salim B
d6c8cd7718 Fix lang.FormatPercent description
Successor PR for https://github.com/gohugoio/hugoDocs/pull/1504
2021-08-03 16:56:06 +02:00
Bjørn Erik Pedersen
9ff17c3324
tpl/time: Adjust tests to handle matching local time zones
Closes #8843
2021-08-03 08:54:45 +02:00
Bjørn Erik Pedersen
3a96655592 Reduce binary size vs locale, update to CLDR v36.1
Test building with `go build -ldflags="-s -w"`

Hugo 0.86.2: 46MB
Before this commit: 77MB
After this commit: 54MB

Fixes #8839
Fixes #8841
2021-08-02 19:40:32 +02:00
Bjørn Erik Pedersen
6c70e1f22f Fix error handling for the time func alias
Fixes #8835
2021-08-01 13:39:30 +02:00
Bjørn Erik Pedersen
7907d24ba1
tpl/lang: Add new localized versions of lang.FormatNumber etc.
Fixes #8820
2021-07-29 16:40:06 +02:00
Bjørn Erik Pedersen
b5de37ee79
Handle toml.LocalDate and toml.LocalDateTime in front matter
See #8801
2021-07-28 18:02:42 +02:00
Bjørn Erik Pedersen
a3701e0931 Switch to go-toml v2
We have been using `go-toml` for language files only. This commit makes it the only TOML library.

It's spec compliant and very fast.

A benchark building a site with 200 pages with TOML front matter:

```bash
name                                  old time/op    new time/op    delta
SiteNew/Regular_TOML_front_matter-16    48.5ms ± 1%    47.1ms ± 1%  -2.85%  (p=0.029 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_TOML_front_matter-16    16.9MB ± 0%    16.7MB ± 0%  -1.56%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_TOML_front_matter-16      302k ± 0%      296k ± 0%  -2.20%  (p=0.029 n=4+4)
```

Note that the front matter unmarshaling is only a small part of building a site, so the above is very good.

Fixes #8801
2021-07-28 11:51:13 +02:00
Bjørn Erik Pedersen
efa5760db5 Add timezone support for front matter dates without one
Fixes #8810
2021-07-27 19:02:48 +02:00
Bjørn Erik Pedersen
a57dda854b Localize time.Format
Fixes #8797
2021-07-27 19:02:48 +02:00
Bjørn Erik Pedersen
022c479551
hugofs: Make FileMeta a struct
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See #8749
2021-07-15 17:14:26 +02:00
Raoul
f75f90079a Fix tab selection of disabled items in internal pagination template 2021-07-05 10:30:59 +02:00
Joe Mooring
34e4742f0c Fix date format in schema and opengraph templates
Fixes #8671
2021-06-29 20:20:38 +02:00
Bjørn Erik Pedersen
9096842b04
tpl: Rename err-missing-instagram-accesstoken => error-missing-instagram-accesstoken
To get it in line with the other.
2021-06-18 12:27:37 +02:00
Bjørn Erik Pedersen
d392893cd7
Misc config loading fixes
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster:

```
BenchmarkDefaultConfigProvider/Viper-16         	  252418	      4546 ns/op	    2720 B/op	      30 allocs/op
BenchmarkDefaultConfigProvider/Custom-16        	  450756	      2651 ns/op	    1008 B/op	       6 allocs/op
```

Fixes #8633
Fixes #8618
Fixes #8630
Updates #8591
Closes #6680
Closes #5192
2021-06-14 17:00:32 +02:00
Bjørn Erik Pedersen
a91cd7652f
docs: Regenerate docs helper 2021-06-08 18:52:38 +02:00
Joe Mooring
73483d0f9e
tpl: Add a terse pagination template variant to improve performance
These calls are equivalent:

{{ template "_internal/pagination.html" . }}
{{ template "_internal/pagination.html" (dict "page" .) }}
{{ template "_internal/pagination.html" (dict "page" . "format" "default") }}

To use an alternate format:

{{ template "_internal/pagination.html" (dict "page" . "format" "terse") }}

Fixes #8599
2021-06-08 10:41:21 +02:00
Bjørn Erik Pedersen
9b5debe4b8 Upgrade Instagram shortcode
Fixes #7879
2021-06-08 10:30:45 +02:00
Bjørn Erik Pedersen
f55d2f4376
tpl/fmt: Add erroridf template func
Fixes #8613
2021-06-07 19:11:03 +02:00
Bjørn Erik Pedersen
282f1aa3db
tpl/data: Print response body on HTTP errors
Which makes it easier to debug.
2021-06-07 12:06:27 +02:00
Bjørn Erik Pedersen
fcd63de3a5 tpl/data: Misc header improvements, tests, allow multiple headers of same key
Closes #5617
2021-06-06 13:32:12 +02:00
Paul Chamberlain
150d75738b tpl/data: Allows user-defined HTTP headers with getJSON and getCSV
Updates #5617
2021-06-06 13:32:12 +02:00
Joe Mooring
01758f99b9 Add math.Max and math.Min
Closes #8583
2021-05-28 20:38:45 +02:00
Ujjwal Goyal
c46fc838a9 tpl: Allow 'Querify' to take lone slice/interface argument
Querify can now take a lone string/interface slice (with string
keys) as a parameter, or multiple string parameters, to build
URL queries.

Querify earlier used 'Dictionary' to add key/value pairs to a
map to build URL queries. Changed to dynamically generate ordered
key/value pairs. Cannot take string slice as key (earlier
possible due to Dictionary).

Added tests and benchmarks for querify.

Closes #6735
2021-05-09 13:14:14 +02:00
Julien Midedji
7a2c10ae60
tpl: Fix countwords to handle special chars
Fixes #8479
2021-05-03 09:10:06 +02:00
Bjørn Erik Pedersen
3cc4fdd6f3 deps: Update getkin/kin-openapi v0.60.0 => v0.61. 2021-04-30 14:27:21 +02:00
Joe Mooring
f6745ad358 Remove .Site.Authors from embedded templates
Closes #4458
2021-04-29 17:07:05 +02:00
Bjørn Erik Pedersen
e4dc9a82b5 tpl/collections: Fix where on type mismatches
Fixes #8353
2021-04-23 18:06:49 +02:00
Bjørn Erik Pedersen
0d86a32d8f Make the shortcode template lookup for output formats stable
Fixes #7774
2021-04-23 15:04:35 +02:00
Bjørn Erik Pedersen
bca40cf0c9 Fix Params case handling in where with slices of structs (e.g. Pages)
Fixes #7009
2021-04-21 19:28:18 +02:00
Joe Mooring
9b34d42bb2 Remove extraneous space from figure shortcode
Fixes #8401
2021-04-15 14:43:32 +02:00
Bjørn Erik Pedersen
5656a908d8
tpl: Remove the FuzzMarkdownify func for now
It has gotten us nothing but "build fails" mail and work we don't have time to follow up on.
2021-03-30 20:57:31 +02:00
Bjørn Erik Pedersen
5e2f128911
Try to fix the fuzz build 2021-03-28 21:42:44 +02:00
Bjørn Erik Pedersen
ba1d0051b4 media: Make Type comparable
So we can use it and output.Format as map key etc.

This commit also fixes the media.Type implementation so it does not need to mutate itself to handle different suffixes for the same MIME type, e.g. jpg vs. jpeg.

This means that there are no Suffix or FullSuffix on media.Type anymore.

Fixes #8317
Fixes #8324
2021-03-14 15:21:54 +01:00
Daniel Atwood
ba16a14c6e
Add support for Google Analytics v4 2021-03-03 13:30:06 +01:00
Bjørn Erik Pedersen
7f8530039a
tpl: Add method mappings for strings.Contains, strings.ContainsAny
Just to confirm/document the function signature.
2021-02-23 09:36:43 +01:00
Bjørn Erik Pedersen
fe77f7434b
tpl: Make the build green again 2021-02-18 18:17:43 +01:00
Bjørn Erik Pedersen
c60806550a
tpl: Regenerate internal templates 2021-02-18 17:52:32 +01:00
Daniel Atwood
ffd9dac421
tpl: Update date logic of opengraph and schema internal templates
* Fix: updated date logic in opengraph template

* Updated date logic in schema template

* Reformatted opengraph and schema

* Wrapped PublishDate and Lastmod in with
2021-02-18 17:51:32 +01:00
Bjørn Erik Pedersen
9650e56841 tpl: Add temporary patch to fix template data race
Keep this as a separate commit so we can reapply it if needed.

Fixes #7293
2021-02-18 14:11:48 +01:00
Bjørn Erik Pedersen
b5485aeae7 Add breaking tests for "map read and map write in templates"
The fix upstream in Go was reverted, so we apply a temporary patch for this in Hugo.

Updates #7293
2021-02-18 14:11:48 +01:00
Bjørn Erik Pedersen
ccb822eb5a Pull in latest Go template source 2021-02-18 14:11:48 +01:00
Bjørn Erik Pedersen
cf3e077da3 tpl/internal: Synch Go templates fork with Go 1.16dev 2021-02-18 14:11:48 +01:00
susiwen8
bf55afd71f
Fix some humanize issues
Fixes #7912
2021-02-14 18:30:59 +01:00
Jeremy Epstein
4867cd1dea
tpl/embedded: Exclude pages without Permalink from sitemap 2021-02-06 18:30:09 +01:00
Baris Ceviz
35def0ae45
tpl/data: Add default user-agent header for getJSON requests 2021-02-01 09:31:02 +01:00
Aleksandr Demakin
ee9c136763 tpl/os: remove 1mb limit for readFile. 2021-01-23 00:39:14 +01:00
Cameron Moore
241b7483ea tpl: Fix race condition in text template baseof
Copy most of the htmltemplate cloning to the textemplate implementation
in the same function.
2021-01-22 19:25:45 +01:00
Cameron Moore
0004a733c8 tpl: Fix metrics hint tracking
When tracking for cache hints, track the same template name as the call
to MeasureSince in Execute.  When referencing a partial "foo", the value
of `n` does not match `templ.Name()` (`partials/foo` versus
`partials/foo.html`).  This was causing hints to go untracked since
there was no existing metric to append the hint to.

Fixes #8125
2021-01-08 20:01:11 +01:00
Cameron Moore
8a26ab0bc5 tpl: Do not return errors in substr for out-of-bounds cases
Most other substr implementations don't error out in out-of-bounds cases
but simply return an empty string (or a value that's printed as an empty
string). We'll follow their lead and not exit template execution.  Allow
the user decide what to do with the empty result.

Fixes #8113
2021-01-08 10:13:10 +01:00
Cameron Moore
788e50ad3a tpl: Add missing test scenario for strings.Substr 2021-01-08 10:13:10 +01:00
Bjørn Erik Pedersen
cea1574023
Add Dart Sass support
But note that the Dart Sass Embedded Protocol is still in beta (beta 5), a main release scheduled for Q1 2021.

Fixes #7380
Fixes #8102
2020-12-30 17:32:25 +01:00
Bjørn Erik Pedersen
10ae7c3210
Improve LookPath 2020-12-19 17:03:07 +01:00
Bjørn Erik Pedersen
a2d146ec32
tpl: Regenerate templates 2020-12-16 12:20:02 +01:00
Maciej Sawicki
d2d493ab5d
tpl: Fix series detection in opengraph
When inside front matter you specified series with spaces,
then the opengraph template wouldn't detect other articles,
because in `.Site.Taxonomies.series` they are stored by 
urlized key.

Example:

```yaml
# in front matter
series:
    - My Series
```

```gohtml
{{/* in a template */}}
{{- $series := index .Site.Taxonomies.series$name }}

{{/* was resolved to */}}
{{- $series := index {'my-series': ...} "MySeries" }}
```
2020-12-16 12:13:30 +01:00
Phil Davis
04b89857e1
all: Fix minor typos 2020-12-16 12:11:32 +01:00
Bjørn Erik Pedersen
718e09ed4b
tpl/internal/go_templates: Revert formatting
Should make future fork synch easier.
2020-12-03 13:47:43 +01:00
Bjørn Erik Pedersen
d90e37e0c6 all: Format code with gofumpt
See https://github.com/mvdan/gofumpt
2020-12-03 13:12:58 +01:00
Andrew Zenk
4fc918e02c
tpl: Add title parameter to YouTube shortcode 2020-12-02 12:53:53 +01:00
Cameron Moore
5862fd2a60 tpl: Fix substr when length parameter is zero
When length parameter is zero, always return an empty string.

Updates #7993
2020-12-01 23:20:04 +01:00
Cameron Moore
64789fb5dc tpl: Refactor and fix substr logic
Fix miscalculations when start is negative.  Results should now match
PHP substr.

Fixes #7993
2020-11-28 11:53:46 +01:00
Bjørn Erik Pedersen
b298c06e05
deps: Update to Chroma v0.8.2
Closes #7970
2020-11-20 09:30:05 +01:00
AdamKorcz
4f20bf29eb Updated year in header 2020-11-06 13:19:18 +01:00
AdamKorcz
4c613d5d5d Added first fuzzer 2020-11-06 13:19:18 +01:00
Cameron Moore
6d95dc9d74 tpl: Fix reflection bug in merge
Value.Type().Key() must only be called on map values.

Fixes #7899
2020-10-29 19:19:41 +01:00
Bjørn Erik Pedersen
fdfa4a5fe6 Allow getJSON errors to be ignored
This change is mostly motivated to get a more stable CI build (we're building the Hugo site there, with Instagram and Twitter shortcodes sometimes failing).

Fixes #7866
2020-10-22 09:09:29 +02:00
Cameron Moore
807db97af8 tpl: Refactor time.AsTime location implementation 2020-10-21 09:49:25 +02:00
Mark Johnson
26eeb29147 tpl: Update Hugo time to support optional [LOCATION] parameter 2020-10-21 09:49:25 +02:00
Cameron Moore
08e4f9ff9c embedded: Always show page number when 5 pages or less
Fixes #7523
2020-10-16 16:16:49 +02:00
Josh Gerdes
edc5c4741c tpl: Add Do Not Track (dnt) option to Vimeo shortcode
Added a Vimeo EnableDNT privacy option to the Hugo config. This will enable the Vimeo 'Do Not Track' flag when either Vimeo shortcode tempalte options are used. When enabled, it will force the Vimeo player to be blocked from tracking any session data, including all cookies and stats.

Fixes #7700
2020-10-02 23:02:38 +02:00
Bjørn Erik Pedersen
97987e5c02 langs/i18n: Upgrade to go-i18n v2
Fixes #5242
2020-09-29 17:48:07 +02:00
Evgeny Kuznetsov
cd830bb027 tpl: Fix grammar in the new 'requires non-zero' error message 2020-09-14 19:54:48 +02:00