Commit graph

2295 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
29ccb36069 Fix /static performance regression from Hugo 0.103.0
In `v0.103.0` we added support for `resources.PostProcess` for all file types, not just HTML. We had benchmarks that said we were fine in that department, but those did not consider the static file syncing.

This fixes that by:

* Making sure that the /static syncer always gets its own file system without any checks for the post process token.
* For dynamic files (e.g. rendered HTML files) we add an additional check to make sure that we skip binary files (e.g. images)

Fixes #10328
2022-09-26 19:02:25 +02:00
Bjørn Erik Pedersen
8e77bcc930
Filter out any duplicate files to post process
Updates #10269
2022-09-14 19:42:55 +02:00
Bjørn Erik Pedersen
74daca6b30 Support PostProcess for all file types
Not just HTML.

Fixes #10269
2022-09-14 19:09:20 +02:00
satotake
7d40da876c Add --force to hugo new
Closes #9243
2022-09-08 15:35:11 +02:00
Bjørn Erik Pedersen
8e5044d7f5 Fix shortcode parser regression with quoted param values
This issue was introduced in `v0.102.0`.

In 223bf28004 we removed the byte source from the parsed page result, which
meant we had to preserve exact positioning for all elements. This introduced some new `TypeIgnore` tokens
which we, wrongly, assumed didn't matter where we put in the result slice (they should be ignored anyway).

But it seems that this broke the logic where we determine if it's positional or named params in the case
where the paramater value contains escaped quoutes.

This commit makes sure that these ignore tokens (the back slashes) are never sent back to the client, which is how it was before `v0.102.0`.

This commit also fixes some lost error information in that same commit.

Fixes #10236
2022-09-01 12:13:23 +02:00
Bjørn Erik Pedersen
223bf28004 parser/pageparser: Don't store the byte slices
On its own this change doesn't do any magic, but this is part of a bigger picture about making Hugo leaner in the
memory usage department.
2022-07-09 16:03:11 +02:00
Bjørn Erik Pedersen
72b0ccdb01
Make the baseline benchmark's test files stable
And add some assertions.
2022-06-30 12:36:55 +02:00
Bjørn Erik Pedersen
d1278f696a
Extract the baseline benchmark to a test 2022-06-26 09:56:01 +02:00
Bjørn Erik Pedersen
92f31ae63b
Add a baseline benchmark 2022-06-25 18:26:40 +02:00
Bjørn Erik Pedersen
35fa192838 deps: Udpate to github.com/alecthomas/chroma/v2
Fixes #9932
Fixes #9931
2022-06-14 14:08:40 +02:00
Bjørn Erik Pedersen
a5a4422aae Fix relURL with leading slash when baseURL includes a subdirectory
Fixes #9994
2022-06-13 11:40:25 +02:00
Bjørn Erik Pedersen
0566bbf7c7 Fix raw TOML dates in where/eq
Note that this has only been a problem with "raw dates" in TOML files in /data and similar. The predefined front matter
dates `.Date` etc. are converted to a Go Time and has worked fine even after upgrading to v2 of the go-toml lib.

Fixes #9979
2022-06-07 13:02:58 +02:00
Bjørn Erik Pedersen
311b8008bf
helpers: Fix panic with invalid defaultMarkdownHandler
Fixes #9968
2022-06-04 17:47:20 +02:00
Bjørn Erik Pedersen
212d9e3017 Fix panic with markdownify/RenderString with shortcode on Page with no content file
Fixes #9959
2022-06-01 12:04:55 +02:00
Bjørn Erik Pedersen
6f7bf3f2d7 Fix indentation in highlight shortcode
This commit adds a new `.InnerDeindent` method to the shortcode context, which is `.Inner` with any
indendation removed. This is then used in the built-in `highlight` shortcode to prevent the extra
whitespace getting hightlighted.

Fixes #4717
2022-05-31 09:05:54 +02:00
Bjørn Erik Pedersen
9e904d756b
Make .RenderString render shortcodes
Fixes #6703
2022-05-30 11:32:55 +02:00
Bjørn Erik Pedersen
d2cfaede5b
Improve shortcode indentation handling
* Record the leading whitespace (tabs, spaces) before the shortcode when parsing the page.
* Apply that indentation to the rendered result of shortcodes without inner content (where the user will apply indentation).

Fixes #9946
2022-05-30 11:32:55 +02:00
Bjørn Erik Pedersen
322d19a81f
Add Markdown as an output format
The motivation behind this is not to make it easier to publish Markdown files, as that sounds unusual.

This is mainly meant for shortcodes that produces Markdown to be inlined.

You would do this by creating shortcodes with `*.md` suffix (e.g. `layouts/shortcodes/myshortcode.md`).

This output format is defined as plain text, and will use Go's much more lenient text template parser.

Updates #9821
2022-05-30 11:32:55 +02:00
Bjørn Erik Pedersen
c1a83076bf
Add a shortcode benchmark 2022-05-29 15:21:08 +02:00
Bjørn Erik Pedersen
0f8dc47037 Remove Blackfriday markdown engine
It has been deprecated for a long time, its v1 version is not maintained anymore, and there are many known issues. Goldmark should be
a mature replacement by now.

Closes #9934
2022-05-29 11:50:58 +02:00
Bjørn Erik Pedersen
3b478f50b7 Fix HasMenuCurrent and IsDescendant/IsAncestor when comparing to itself
There may be sites in the wild that depends on the faulty behaviour of IsDescendant/IsAncestor when comparing to itself, but

* The documentation and common sense says that a thing cannot be descendant or ancestor to itself.
* The bug introduced in `HasMenuCurrent` comes directly from that confusion.

Fixes #9846
2022-05-28 10:56:54 +02:00
Bjørn Erik Pedersen
dd9eaf19fd Don't use the baseURL /path as part of the resource cache key
As that prevents Hugo projects with sub paths in their `baseURL` to use themes with cached resources.

Fixes #9787
2022-05-27 19:11:16 +02:00
Bjørn Erik Pedersen
3854a6fa6c Fix Plainify edge cases
This commit replaces the main part of `helpers.StripHTML` with Go's implementation in its html/template package.

It's a little slower, but correctness is more important:

```bash
BenchmarkStripHTMLOld-10    	  680316	      1764 ns/op	     728 B/op	       4 allocs/op
BenchmarkStripHTMLNew-10    	  384520	      3099 ns/op	    2089 B/op	      10 allocs/op
```

Fixes #9199
Fixes #9909
Closes #9410
2022-05-25 17:55:23 +02:00
satotake
6f7fbe03b1 basefs: add noBuildLock flag
If the flag is enabled, `.hugo_build.lock` will not be created.
This ensures safe running on read-only filesystem etc.
Close #9780
2022-05-24 13:57:23 +02:00
Bjørn Erik Pedersen
87a22eb6d6 server: Fix SIGINT handling after loading bad configuration
Also fix the config error messages.

Fixes #9664
2022-05-15 22:58:05 +02:00
Bjørn Erik Pedersen
fc9f315d86 Improve SASS errors
Fixes #9897
2022-05-15 20:25:25 +02:00
Bjørn Erik Pedersen
4b189d8fd9 postcss: Fix import error handling
Note that we will now fail if `inlineImports` is enabled and we cannot resolve an import.

You can work around this by either:

* Use url imports or imports with media queries.
* Set `skipInlineImportsNotFound=true` in the options

Also get the argument order in the different NewFileError* funcs in line.

Fixes #9895
2022-05-15 20:25:25 +02:00
Bjørn Erik Pedersen
5c96bda70a
errors: Misc improvements
* Redo the server error template
* Always add the content file context if relevant
* Remove some now superflous error string matching
* Move the server error template to _server/error.html
* Add file context (with position) to codeblock render blocks
* Improve JS build errors

Fixes #9892
Fixes #9891
Fixes #9893
2022-05-14 13:40:56 +02:00
Bjørn Erik Pedersen
9d7f166244
hugolib: Check for nil in shouldRender 2022-05-09 15:53:25 +02:00
Bjørn Erik Pedersen
35c88a7f90 Use configured timeZone for the clock
And some other related adjustments.

Updates #8787
2022-05-08 16:56:26 +02:00
satotake
e77ca3c105 Add clock cli flag
Close #8787
2022-05-08 16:56:26 +02:00
Bjørn Erik Pedersen
f2946da9e8 Improve error messages, esp. when the server is running
* Add file context to minifier errors when publishing
* Misc fixes (see issues)
* Allow custom server error template in layouts/server/error.html

To get to this, this commit also cleans up and simplifies the code surrounding errors and files. This also removes the usage of `github.com/pkg/errors`, mostly because of https://github.com/pkg/errors/issues/223 -- but also because most of this is now built-in to Go.

Fixes #9852
Fixes #9857
Fixes #9863
2022-05-06 19:43:22 +02:00
Bjørn Erik Pedersen
fa80fe3c8a Some godoc adjustments and image struct renames 2022-04-27 23:53:56 +02:00
Joe Mooring
a022ca271b deps: Update github.com/yuin/goldmark v1.4.11 => v1.4.12
Fixes #9054
Fixes #9756
Fixes #9757
2022-04-27 10:04:00 +02:00
Bjørn Erik Pedersen
05b45c35c8 tpl/lang: Handle nil values in lang.Merge 2022-04-23 22:22:50 +02:00
Bjørn Erik Pedersen
097fd588ca Deprecate page.Author and page.Authors
These are not documented, and they don't belong on Page. We should consider having author a first class citizen of Hugo, but as it is not it's better modelled as a taxonomy.
2022-04-23 22:22:50 +02:00
Bjørn Erik Pedersen
397fce5603
Fix MediaType when reading images from cache
Fixes #8931
2022-04-16 16:36:15 +02:00
Bjørn Erik Pedersen
627eed1d62
Make string sorting (e.g. ByTitle, ByLinkTitle and ByParam) language aware
Fixes #2180
2022-04-12 13:24:16 +02:00
Bjørn Erik Pedersen
30c2e54c25 Replace all usage of CopyOnWriteFs with OverlayFs
Fixes #9761
2022-04-10 13:49:31 +02:00
Bjørn Erik Pedersen
d070bdf10f
Rework the Destination filesystem to make --renderStaticToDisk work
See #9626
2022-04-08 13:26:17 +02:00
Bjørn Erik Pedersen
0a56f2af4e
Revert "Revert "Allow rendering static files to disk and dynamic to memory in server mode""
This reverts commit 64b7b7a897.
2022-04-08 13:26:16 +02:00
Bjørn Erik Pedersen
e58a540895 resources: Create a common ResourceFinder interface
And make both .Resources and resources implement it.

This gets us 2 new methods/functions, so you can now also do:

* .Resources.Get
* resources.ByType

Note that GetRemote is not covered by this interface, as that is only available as a global template function.

Fixes #8653
2022-04-05 18:00:44 +02:00
Bjørn Erik Pedersen
658e11ebaa Localize all the GroupBy*Date methods
Fixes #9745
2022-04-05 16:11:11 +02:00
Cathrine Paulsen
da00e7714e
Add environment as a new filter to _cascade.target
Fixes #9612
2022-04-05 09:41:24 +02:00
Sara Op den Orth
94459680ba Deprecate .File.Extension
Fixes #9352
2022-03-26 10:03:36 +01:00
Bjørn Erik Pedersen
9202117ba0 resources: Add more details to .Err
This commit adds a .Data object (a map with `Body`, `StatusCode` etc.) to the .Err returned from `resources.GetRemote`, which means you can now do:

```
{{ with .Err }}
{{ range $k, $v := .Data }}
{{ end }}
{{ end }}
```

Fixes #9708
2022-03-25 16:40:36 +01:00
cuishuang
48c98a8d24 Fix some typos
Signed-off-by: cuishuang <imcusg@gmail.com>
[foka@debian.org: Resolve merge conflict and squash 2 commits]
Signed-off-by: Anthony Fok <foka@debian.org>
2022-03-22 00:38:23 -06:00
Bjørn Erik Pedersen
1c0e7c1ae1 Make sure file mounts higher up wins
Fixes #9693
2022-03-20 20:23:03 +01:00
Bjørn Erik Pedersen
b80853de90
all: gofmt -w -r 'interface{} -> any' .
Updates #9687
2022-03-17 22:03:27 +01:00
Bjørn Erik Pedersen
423594e03a dartsass: Enable deprecation, @warn and @debug logging
* @warn and Sass deprecations are printed as WARN
* @debug is currently logged as INFO (needs the `--verbose` flag). We may adjust this if it gets too chatty.

Fixes #9683
2022-03-17 21:45:43 +01:00
Cathrine Paulsen
61cf3c9f63 Fix and refactor typos 2022-03-14 19:39:07 +01:00
Bjørn Erik Pedersen
1a796d723c deps: Fix Goldmark regression with HTML comments
Fixes #9650
2022-03-11 14:44:01 +01:00
Bjørn Erik Pedersen
64b7b7a897 Revert "Allow rendering static files to disk and dynamic to memory in server mode"
This reverts commit 7d8011ed63.

Updates #9647
2022-03-11 13:10:47 +01:00
Bjørn Erik Pedersen
4e14cf7607
Fail with error when double-rendering text in markdownify/RenderString
This commit prevents the most commons case of infinite recursion in link render hooks when the `linkify` option is enabled (see below). This is always a user error, but getting a `stack overflow` (the current stack limit in Go is 1 GB on 64-bit, 250 MB on 32-bit) error isn't very helpful. This fix will not prevent all such errors, though, but we may do better once #9570 is in place.

So, these will fail:

```
<a href="{{ .Destination | safeURL }}" >{{ .Text | markdownify }}</a>
<a href="{{ .Destination | safeURL }}" >{{ .Text | .Page.RenderString }}</a>
```

`.Text` is already rendered to `HTML`. The above needs to be rewritten to:

```
<a href="{{ .Destination | safeURL }}" >{{ .Text | safeHTML }}</a>
<a href="{{ .Destination | safeURL }}" >{{ .Text | safeHTML }}</a>
```

Fixes #8959
2022-03-10 08:19:03 +01:00
Joe Mooring
f98e570b17 Add lang attribute to internal alias template
Closes #9586
2022-03-09 19:06:59 +01:00
Bjørn Erik Pedersen
53a6210d82 markup/goldmark/codeblocks: Fix slice bounds out of range
For the Position in code blocks we try to match the .Inner with the original source. This isn't always possible.

This commits avoids panics in these situations.

Fixes #9627
2022-03-08 21:50:21 +01:00
SatowTakeshi
7d8011ed63 Allow rendering static files to disk and dynamic to memory in server mode
Updates #9625
2022-03-08 19:27:54 +01:00
Bjørn Erik Pedersen
0e0d672bcb Remove the decorator from the fs used in ReadDir
There have been a site breakage reported in the wild after 0.93.1. With this commit we shoudl be back to how it behaved in 0.92.2.

Closes #9609
2022-03-08 09:51:59 +01:00
Bjørn Erik Pedersen
673cde1eb1 tpl/os: Revert readDir in theme behaviour
Fixes #9599
2022-03-04 08:43:47 +01:00
Bjørn Erik Pedersen
cff14144a6
Rename Codeowners() to CodeOwners() 2022-02-27 19:40:07 +01:00
Bjørn Erik Pedersen
f7109771a0 CodeblockContext method renames
Fixes #9577
2022-02-27 17:59:36 +01:00
Bjørn Erik Pedersen
579ff9b652 markup/goldmark: Improve attributes vs options
Fixes #9571
2022-02-26 21:54:36 +01:00
Bjørn Erik Pedersen
928a896962 markup/goldmark: Add Position to CodeblockContext
But note that this is not particulary fast and the recommendad usage is error logging only.

Updates #9574
2022-02-26 21:54:36 +01:00
Bjørn Erik Pedersen
10928a4f78 Remove the trailing new line in .Code
Fixes #9572
2022-02-26 21:54:36 +01:00
RheingoldRiver
0df096b868 Update error message about failed menus in config.toml 2022-02-24 23:04:06 +01:00
Bjørn Erik Pedersen
08fdca9d93 Add Markdown diagrams and render hooks for code blocks
You can now create custom hook templates for code blocks, either one for all (`render-codeblock.html`) or for a given code language (e.g. `render-codeblock-go.html`).

We also used this new hook to add support for diagrams in Hugo:

* Goat (Go ASCII Tool) is built-in and enabled by default; just create a fenced code block with the language `goat` and start draw your Ascii diagrams.
* Another popular alternative for diagrams in Markdown, Mermaid (supported by GitHub), can also be implemented with a simple template. See the Hugo documentation for more information.

Updates #7765
Closes #9538
Fixes #9553
Fixes #8520
Fixes #6702
Fixes #9558
2022-02-24 18:59:50 +01:00
Marshall Cottrell
06bac57ab0 Add support for CODEOWNERS
Fixes #9474
2022-02-23 22:43:19 +01:00
Bjørn Erik Pedersen
e97d3c640f Add page.Store
Fixes #9546
2022-02-23 10:02:16 +01:00
Bjørn Erik Pedersen
aebde49b88 commands: Fix server panic regression
And now with a proper server test.

Fixes #9518
Fixes #9530
Fixes #9539
2022-02-21 21:43:26 +01:00
Bjørn Erik Pedersen
f4c90bd6ba
Fix BenchmarkCascadeTarget 2022-02-16 13:23:43 +01:00
Joe Mooring
923419d7fd deps: Update github.com/tdewolff/minify/v2 v2.9.22 => v2.9.29
Fixes #9244
Fixes #9132
Fixes https://discourse.gohugo.io/t/36523
2022-02-15 17:36:29 +01:00
Bjørn Erik Pedersen
ea54a99ca5 deps: Update github.com/alecthomas/chroma v0.9.4 => v0.10.0 2022-02-15 10:37:14 +01:00
Bjørn Erik Pedersen
9563c7d13b Finally remove deprecated Page methods
They have been deprecated for a very long time, first with a warning, then with an ERROR. Now they are removed.

Closes #4117
2022-02-14 13:24:36 +01:00
Bjørn Erik Pedersen
a7d182cea1
Add another cascade benchmark 2022-02-10 20:40:49 +01:00
Bjørn Erik Pedersen
f60714b5a1
Add a migration test helper 2022-02-09 16:38:12 +01:00
Bjørn Erik Pedersen
215a715ddd babel: Port integration tests to their own package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
d128d260b5 js: Port integration tests to its own package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
c4aaf1d516 postcss: Move integration test to its own package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
94f10cf4f9 minifier: Port integration tests to its package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
b06c2103b8 templates: Port integration test to its package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
d22f7795c7 tocss: Port integration tests to their package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
39f69ca7f0 openapi3: Port integration test into correct package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
64f75adcf6 Add a new integration test framework
I have had this living in a separate branch for now, but we need this in the main branch sooner rather than later.

One big advantage of this is that integration tests can live in any package, not just hugolib.
2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
ef7d14a241 Fix erroneous warning with .Page.RenderString on a page without a backing file
Fixes #9433
2022-01-28 17:35:49 +01:00
Bjørn Erik Pedersen
f22c4aba04 Make the RenderString content provider fix more general
Updates #9383
2022-01-27 11:51:13 +01:00
Paul Gottschling
85d31f7bfb Fix .RenderString issue in .Translations
Fixes #9383
2022-01-27 11:51:13 +01:00
Bjørn Erik Pedersen
7a080b624e Fix duplicate mount sources
Fixes #9426
2022-01-26 20:32:13 +01:00
Bjørn Erik Pedersen
2655739940 tpl/collections: Fix apply with namespaced template funcs
We changed the signature to `func(...interface{}) (interface{}, error)` some time ago, but sadly we had no test for this for `apply`. Now we do.

Fixes #9393
2022-01-17 09:50:48 +01:00
Andreas Deininger
fcbbbef22d
Fix typo in warning message 2022-01-13 12:21:13 +01:00
Bjørn Erik Pedersen
cdcd15b6c2
Only create LazyContentProvider for the non-rendering Site
Which saves a fair amound of allocations:

```
gobench --package ./hugolib --bench "SiteNew/Regular_D" --base master
```

Before:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-10    40.7ms ± 3%    41.2ms ± 1%    ~     (p=0.343 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-10    27.7MB ± 0%    28.8MB ± 0%  +3.76%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-10      304k ± 0%      329k ± 0%  +8.07%  (p=0.029 n=4+4)
```

After:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-10    34.2ms ± 1%    34.7ms ± 1%    ~     (p=0.114 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-10    27.7MB ± 0%    28.1MB ± 0%  +1.38%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-10      304k ± 0%      314k ± 0%  +3.03%  (p=0.029 n=4+4)
```

Updates #8919
2022-01-12 07:45:53 +01:00
Paul Gottschling
25d645f47a
Fix missing page data for alternative formats
When a template calls the .Translations function and a
Hugo environment is using multiple output formats,
a template that calls methods like .Summary and .Len on
each translation will unexpectedly show empty return
values for these methods.

This is because each pageOutput's ContentProvider is
assigned to a page.NopPage in newPageOutput. When
*HugoSites.render assigns pageContentOutputs to
pageOutputs in *pageState.shiftToOutputFormat, it
reuses pageContentOutputs from other pageOutputs,
leaving some pageContentOutputs as NopPages. While this
approach conserves resources, sometimes it means that
a template will unexpectedly call a method on a
pageContentOutput that is actually a NopPage.

In the case of ContentProvider methods called on
translations for alternative output formats, the methods
were called on NopPages.

This change introduces LazyContentProvider, which
performs late initialization when one of its methods is
called. This way, we can reuse content in "normal" cases
but ensure that ContentProvider methods work as expected
when a pageOutput is not assigned a pageContentOutput
during the initial pre-render phase.

Fixes #8919
2022-01-12 07:45:53 +01:00
Bjørn Erik Pedersen
7396aa945a Add hugo.Deps
Fixes #8949
2022-01-11 18:06:23 +01:00
roointan
d82cef5c53
hugolib: Fix livereload problem with files including NFC characters in MacOs
Fixes #8429
2022-01-08 11:35:30 +01:00
Bjørn Erik Pedersen
1651beb2c1 Remove mmark
Closes #9350
2022-01-04 17:10:39 +01:00
Bjørn Erik Pedersen
2b6063c3e3 Misc depreation updates
* Deprecate .Page.Path when backed by a file
* site.Permalinks
* --ignoreVendor (use --ignoreVendorPaths)

Closes #9348
Closes #9349
2022-01-04 17:10:39 +01:00
Paul Gottschling
d3c4fdb8ff Fix surprise OutputFormat.Rel overwriting
In page.NewOutputFormat, we take an output.Format f and use it to
create a page.OutputFormat. If the format is canonical, we assign
the final OutputFormat's Rel to "canonical" rather than using
f.Rel. However, this leads to unexpected behavior for custom
output formats, where a user can define a "rel" for a format
via the config file.

For example, the standard for "humans.txt" files requires using
rel="author" in HTML "link" elements. Meanwhile, humans.txt is
usually the only format used for its content. As a result, for
Hugo configurations that define a humans.txt custom output format,
Hugo will render "link" elements to content in this format with
rel="canonical," rather than "author" as required by the standard.

This commit changes page.NewOutputFormat to check whether a given
format is user defined and, if so, skips assigning Rel to
"canonical," even if isCanonical is true.

Fixes #8030
2022-01-04 10:38:38 +01:00
Bjørn Erik Pedersen
d632dd7d74
hugolib: Make an RST test optional
If no RST installed.
2022-01-01 12:20:07 +01:00
Bjørn Erik Pedersen
aee9e11a40 Make sure we always create the /public folder
* Before this commit, when you had static files in the root of /content and no /public folder, that folder would not be created unless the /static syncer had already run.
* So, with a common pattern doing `rm -rf public && hugo` would the fail now and then because /static and /content are processed in parallel (unless you have cleanDestinationDir=true)
* This was even worse before commit 0b918e131f – a frozen build.

Closes #8166
2021-12-23 15:09:27 +01:00
Bjørn Erik Pedersen
bd63c1aa56 Fix "stuck on build" in error situations in content processing
Updates #8166
2021-12-23 15:09:27 +01:00
Bjørn Erik Pedersen
6779117f72 media: Also consider extension in FromContent
As used in `resources.GetRemote`.

This will now reject image files with text and text files with images.
2021-12-22 11:35:53 +01:00
Bjørn Erik Pedersen
22ef5da20d
Add resources.GetRemote
In Hugo 0.89 we added remote support to `resources.Get`.

In hindsight that was not a great idea, as a poll from many Hugo users showed. See Issue #9285 for more details.

After this commit `resources.Get` only supports local resource lookups. If you want to support both, you need to use a construct similar to:

Also improve some option case handling.

```
{{ resource := "" }}
{{ if (urls.Parse $url).IsAbs }}
{{ $resource = resources.GetRemote $url }}
{{ else }}
{{ $resource = resources.Get $url }}
{{ end }}
```

Fixes #9285
Fixes #9296
2021-12-17 09:33:51 +01:00
Paul Gottschling
5758c370ea
Allow for return partials with falsy arguments (#9298)
Partials with returns values are parsed, then inserted into a
partial return wrapper via wrapInPartialReturnWrapper in order
to assign the return value via *contextWrapper.Set. The
predefined wrapper template for partials inserts a partial's nodes
into a "with" template action in order to set dot to a
*contextWrapper within the partial. However, because "with" is
skipped if its argument is falsy, partials with falsy arguments
were not being evaluated.

This replaces the "with" action in the partial wrapper with a
"range" action that isn't skipped if .Arg is falsy.

Fixes #7528
2021-12-17 08:35:21 +01:00
Bjørn Erik Pedersen
8ee6de6d96 deps: Upgrade github.com/evanw/esbuild v0.14.2 => v0.14.5 2021-12-17 08:26:45 +01:00
Bjørn Erik Pedersen
f4389e48ce
Add some basic security policies with sensible defaults
This ommmit contains some security hardening measures for the Hugo build runtime.

There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers".

For `asciidoctor` and some others we use Go's `os/exec` package to start a new process.

These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off.

You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do.

The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all.

```toml
[security]
  enableInlineShortcodes = false
  [security.exec]
    allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
    osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

  [security.funcs]
    getenv = ['^HUGO_']

  [security.http]
    methods = ['(?i)GET|POST']
    urls = ['.*']
```
2021-12-16 09:40:22 +01:00
Bjørn Erik Pedersen
803f572e66
Simplify Babel test assertions
To make the tests pass on my grand old Windows 8 VM.
2021-12-14 07:26:53 +01:00
Bjørn Erik Pedersen
159120cddd
Fix deprecation notice
Asterisks needs to be quoted on command line.
2021-12-11 11:54:28 +01:00
Bjørn Erik Pedersen
e4d6ec94b5 Allow user to handle/ignore errors in resources.Get
In Hugo 0.90.0 we introduced remote support in `resources.Get`.

But with remote resources comes with a higher chance of failing a build (network issues, remote server down etc.).

Before this commit we always failed the build on any unexpected error.

This commit allows the user to check for any error (and potentially fall back to a default local resource):

```htmlbars
{{ $result := resources.Get "https://gohugo.io/img/hugo-logo.png" }}
{{ with $result }}
        {{ if .Err }}
        {{/* log the error, insert a default image etc. *}}
        {{ else }}
        <img src="{{ .RelPermalink }}" width="{{ .Width }}" height="{{ .Height }}" alt="">
        {{ end }}
{{ end }}
```

Note that the default behaviour is still to fail the build, but we will delay that error until you start using the `Resource`.

Fixes #9529
2021-12-10 11:10:41 +01:00
Bjørn Erik Pedersen
6260455ba7 Make resource.Get return nil on 404 not found
This is in line with the interface declaration and also how local lookups work.

Fixes #9267
2021-12-10 11:10:41 +01:00
Paul van Brouwershaven
0eaaa8fee3
Implement XML data support
Example:

```
{{ with resources.Get "https://example.com/rss.xml" | transform.Unmarshal }}
    {{ range .channel.item }}
        <strong>{{ .title | plainify | htmlUnescape }}</strong><br />
        <p>{{ .description | plainify | htmlUnescape }}</p>
        {{ $link := .link | plainify | htmlUnescape }}
        <a href="{{ $link }}">{{ $link }}</a><br />
        <hr>
    {{ end }}
{{ end }}
```

Closes #4470
2021-12-02 17:30:36 +01:00
Joe Mooring
58adbeef88 Validate private use language tags
Fixes #9119
2021-12-02 16:30:53 +01:00
Joe Mooring
133e4bfbee Remove empty href element from pagination template
Closes #9149
2021-12-02 09:14:30 +01:00
Paul van Brouwershaven
8aa7257f65
Add remote support to resources.Get
Closes #5255
Supports #9044
2021-11-30 11:49:51 +01:00
Bjørn Erik Pedersen
2e70f61fb0 Fix content dir resolution when main project is a Hugo Module
Fixes #9177
2021-11-16 20:42:34 +01:00
Bjørn Erik Pedersen
b8155452ac Improve error when we cannot determine content directory in "hugo new"
See #9166
2021-11-15 11:21:39 +01:00
Bjørn Erik Pedersen
2b01c85d14 Fix path resolution in hugo new
With theme and project with content directories and command on the form `hugo new posts/test.md`.

Fixes #9129
2021-11-08 14:10:46 +01:00
Joe Mooring
0cc39af682 Update Twitter shortcode oEmbed endpoint
The existing endpoint will be retired and removed on November 23, 2021.
References:

- https://twittercommunity.com/t/consolidating-the-oembed-functionality/154690
- https://developer.twitter.com/en/docs/twitter-for-websites/oembed-api#Embedded

This is a backward compatible change.

The existing endpoint requires a single parameter: the id of the tweet.

The new endpoint requires two parameters: the id of the tweet, and the
user with whom it is associated. For the moment, if you supply the wrong
user, the request will be redirected (with a small delay) to the correct
user/id pair. This behavior is undocumented, but we will take advantage
of it as Hugo site authors transition to the new syntax.

{{< tweet 1453110110599868418 >}} --> works, throws warning, deprecate at some point

{{< tweet user="SanDiegoZoo" id="1453110110599868418" >}} --> new syntax

Fixes #8130
2021-11-01 15:51:00 +01:00
Bjørn Erik Pedersen
64e1613fb3 Fix panic when specifying multiple excludeFiles directives
Fixes #9076
2021-10-25 19:50:18 +02:00
Anthony Fok
355ff83e74 config: Set HUGO_ENABLEGITINFO=false override in Set_in_string
This allows TestLoadConfigWithOsEnvOverrides/Set_in_string to PASS
even if there is no .git directory, e.g. during Debian package build.
2021-10-22 19:40:18 +02:00
Bjørn Erik Pedersen
471ed91c60 hugofs: Add includeFiles and excludeFiles to mount configuration
Fixes #9042
2021-10-20 05:00:17 +02:00
Bjørn Erik Pedersen
e02e0727e5
Fix file permissions in new archetype implementation
Closes #9057
2021-10-18 19:08:19 +02:00
Bjørn Erik Pedersen
ba35e69856 Add a cross process build lock and use it in the archetype content builder
Fixes #9048
2021-10-18 12:13:13 +02:00
Bjørn Erik Pedersen
9185e11eff Reimplement archetypes
The old implementation had some issues, mostly related to the context (e.g. name, file paths) passed to the template.

This new implementation is using the exact same code path for evaluating the pages as in a regular build.

This also makes it more robust and easier to reason about in a multilingual setup.

Now, if you are explicit about the target path, Hugo will now always pick the correct mount and language:

```bash
hugo new content/en/posts/my-first-post.md
```

Fixes #9032
Fixes #7589
Fixes #9043
Fixes #9046
Fixes #9047
2021-10-16 15:22:03 +02:00
Joe Mooring
e6e44b7c41 Fix value of useResourceCacheWhen in TestResourceChainPostCSS 2021-10-10 11:12:01 +02:00
Bjørn Erik Pedersen
26f1919ae0 Skip a test assertion on CI
Need to investigate later, but got to get the tests green.
2021-10-09 20:36:32 +02:00
Paul Gottschling
e03f82eef2
Pass minification errors to the user
Previously, *minifyTransformation.Transform suppressed the
error returned by t.m.Minify. This meant that when minification
returned an error, the error would not reach the user. Instead,
minification would silently fail. For example, if a JavaScript
file included a call to the Date constructor with:

new Date(2020, 04, 02)

The package that the minification library uses to parse JS files,
github.com/tdewolff/parse would return an error, since "04" would
be parsed as a legacy octal. However, the JS file would remain
un-minified with no error.

Fixing this is not as simple as replacing "_" with an "err" in
*minifyTransformation.Transform, however (though this is
necessary). If we only returned this error from Transform,
then hugolib.TestResourceMinifyDisabled would fail. Instead of
being a no-op, as TestResourceMinifyDisabled expects, using the
"minify" template function with a "disableXML=true" config
setting instead returns the error, "minifier does not exist for
mimetype."

The "minifier does not exist" error is returned because of the
way minifiers.New works. If the user's config disables
minification for a particular MIME type, minifiers.New does
not add it to the resulting Client's *minify.M. However, this
also means that when the "minify" template function is executed,
 a *resourceAdapter's transformations still add a minification.
When it comes time to call the minify.Minifier for a specific
MIME type via *M.MinifyMimetype, the github.com/tdewolff/minify
library throws the "does not exist" error for the missing MIME
type.

The solution was to change minifiers.New so, instead of skipping
a minifier for each disabled MIME type, it adds  a NoOpMinifier,
which simply copies the source to the destination without
minification. This means that when the "minify" template
function is used for a particular resource, and that resource's
MIME type has minification disabled, minification is genuinely
skipped, and does not result in an error.

In order to add this, I've fixed a possibly unwanted interaction
between minifiers.TestConfigureMinify and
hugolib.TestResourceMinifyDisabled. The latter disables
minification and expects minification to be a no-op. The former
disables minification and expects it to result in an error. The
only reason hugolib.TestResourceMinifyDisabled passes in the
original code is that the "does not exist" error is suppressed.
However, we shouldn't suppress minification errors, since they
can leave users perplexed. I've changed the test assertion in
minifiers.TestConfigureMinify to expect no errors and a no-op
if minification is disabled for a particular MIME type.

Fixes #8954
2021-09-22 20:54:40 +02:00
Bjørn Erik Pedersen
7d1f806ecb commands: Don't fail on template errors on go mod graph etc.
Fixes #8942
2021-08-31 22:16:12 +02:00
Bjørn Erik Pedersen
a0489c2dfd
Avoid failing with "module not found" for hugo mod init and similar
Fixes #8940
2021-08-31 12:08:11 +02:00
Bjørn Erik Pedersen
f4ffeea71d Fix it so disableKinds etc. does not get merged in from theme
Unless the merge strategy is set up to do so.

For `disableKinds` the current workaround is to make sure the project config has an entry, even if is empty:

```
disableKinds = []
```

Note that this issue only touches root, non-map config-values that either is not set in project config or in Hugo's defaults.

Fixes #8866
2021-08-22 13:25:20 +02:00
Bjørn Erik Pedersen
bc0743ed8e Prevent minifier from removing quoutes around post-processed attributes
Fixes #8884
2021-08-22 12:47:47 +02:00
Helder Pereira
d966f5d08d highlight: Remove some pygments references 2021-08-21 15:50:49 +02:00
Bjørn Erik Pedersen
d70c485707
Make sure module config loading errors have file positioning info
Fixes #8845
2021-08-03 09:57:14 +02:00
Bjørn Erik Pedersen
3a96655592 Reduce binary size vs locale, update to CLDR v36.1
Test building with `go build -ldflags="-s -w"`

Hugo 0.86.2: 46MB
Before this commit: 77MB
After this commit: 54MB

Fixes #8839
Fixes #8841
2021-08-02 19:40:32 +02:00
Bjørn Erik Pedersen
6c70e1f22f Fix error handling for the time func alias
Fixes #8835
2021-08-01 13:39:30 +02:00
Bjørn Erik Pedersen
4d221ce468 Fail on invalid time zone
Fixes #8832
2021-08-01 13:39:30 +02:00
Bjørn Erik Pedersen
e3dc5240f0 Improve handling of <nil> Params
Fixes #8825
2021-07-30 21:07:52 +02:00
Bjørn Erik Pedersen
7907d24ba1
tpl/lang: Add new localized versions of lang.FormatNumber etc.
Fixes #8820
2021-07-29 16:40:06 +02:00
Bjørn Erik Pedersen
b5de37ee79
Handle toml.LocalDate and toml.LocalDateTime in front matter
See #8801
2021-07-28 18:02:42 +02:00
Bjørn Erik Pedersen
a3701e0931 Switch to go-toml v2
We have been using `go-toml` for language files only. This commit makes it the only TOML library.

It's spec compliant and very fast.

A benchark building a site with 200 pages with TOML front matter:

```bash
name                                  old time/op    new time/op    delta
SiteNew/Regular_TOML_front_matter-16    48.5ms ± 1%    47.1ms ± 1%  -2.85%  (p=0.029 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_TOML_front_matter-16    16.9MB ± 0%    16.7MB ± 0%  -1.56%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_TOML_front_matter-16      302k ± 0%      296k ± 0%  -2.20%  (p=0.029 n=4+4)
```

Note that the front matter unmarshaling is only a small part of building a site, so the above is very good.

Fixes #8801
2021-07-28 11:51:13 +02:00
Bjørn Erik Pedersen
7e1305710f
Add a TOML front matter benchmark 2021-07-27 19:27:54 +02:00
Bjørn Erik Pedersen
efa5760db5 Add timezone support for front matter dates without one
Fixes #8810
2021-07-27 19:02:48 +02:00
Bjørn Erik Pedersen
a57dda854b Localize time.Format
Fixes #8797
2021-07-27 19:02:48 +02:00
Bjørn Erik Pedersen
aeb1935d44 deps: Update github.com/evanw/esbuild v0.11.16 => v0.12.16 2021-07-26 20:23:00 +02:00
Bjørn Erik Pedersen
c7252224c4 Deprecate Blackfriday and fix a potential deadlock in config
Note that the deadlock has not been seen earlier, in tests on in real Hugo sites.

Fixes #8792
Fixes #8791
2021-07-26 16:28:16 +02:00
Bjørn Erik Pedersen
91cbb96302 Bump all long-living deprecations to ERRORs 2021-07-25 17:17:21 +02:00
Bjørn Erik Pedersen
d831d2fce8 Simplify "active menu" logic for section menus
Fixes #8776
2021-07-20 17:50:59 +02:00
Bjørn Erik Pedersen
634481ba8c Fix Params case handling for menu items defined in site config
Fixes #8775
2021-07-20 17:50:59 +02:00
Bjørn Erik Pedersen
c19f65f956 minifiers: Make keepWhitespace = true default for HTML
Fixes #8771
2021-07-19 23:35:20 +02:00
Bjørn Erik Pedersen
022c479551
hugofs: Make FileMeta a struct
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See #8749
2021-07-15 17:14:26 +02:00
rhymes
f27e542442
markup: Add tabindex="0" to default <pre> wrapper
Currently the generated `<pre>` element isn't fully accessible as it can't be focused by keyboard users.
To make this fully accessible, the attribute `tabindex="0"` should be added to the `<pre>` tag.

Closes #7194
2021-07-15 16:48:39 +02:00
Bjørn Erik Pedersen
ae6cf93c84 Fix default values when loading from config dir
By waiting until we've loaded the config dir config before applying the default values.

Fixes #8763
2021-07-15 16:15:32 +02:00
Bjørn Erik Pedersen
ee3d2bb1d3 markup/goldmark: Support auto links in render hook
Fixes #8755
2021-07-15 10:14:52 +02:00
Bjørn Erik Pedersen
eb2a500367
Adjust a test helper 2021-07-14 21:47:26 +02:00
Bjørn Erik Pedersen
5cb52c2315 Add config.cascade
This commit adds support for using the `cascade` keyword in your configuration file(s), e.g. `config.toml`.

Note that

* Every feature of `cascade` is available, e.g. `_target` to target specific page sets.
* Pages, e.g. the home page, can overwrite the cascade defined in config.

Fixes #8741
2021-07-10 11:13:41 +02:00
Raoul
f75f90079a Fix tab selection of disabled items in internal pagination template 2021-07-05 10:30:59 +02:00
Bjørn Erik Pedersen
e31b1d1946
commands: Make the --poll flag a duration
So you can do:

```
hugo server --poll 700ms
```

See #8720
2021-07-05 10:23:29 +02:00
Bjørn Erik Pedersen
e451b984cf Fix panic when theme has permalinks config
Fixes #8724
2021-07-03 15:53:21 +02:00
Joe Mooring
34e4742f0c Fix date format in schema and opengraph templates
Fixes #8671
2021-06-29 20:20:38 +02:00
Bjørn Erik Pedersen
49fedbc51c
config: Fix handling of invalid OS env config overrides
Fixes #8709
2021-06-29 09:38:05 +02:00
Bjørn Erik Pedersen
923dd9d1c1
Fix config loading for "hugo mod init"
We have some commands that will continue even if the config loading fails (e.g. because a module can not be found). In Hugo 0.84.0 we introduced a new `_merge` flag that we removed once the configuration was loaded. But we did not do that in error situations, leading to failures in some situations.

This commit fixes that by making sure the configuration is always cleaned before return, even in error situations.

Fixes #8697
2021-06-27 12:03:43 +02:00
Bjørn Erik Pedersen
093dacab29 Fix language menu config regression
Fixes #8672
2021-06-24 13:03:09 +02:00
Bjørn Erik Pedersen
4a9d408fe0 config: Fix merge of config with map[string]string values.
Fixes #8679
2021-06-22 21:38:28 +02:00
Bjørn Erik Pedersen
bb2aa08709 Implement configuration in a directory for modules
Fixes #8654
2021-06-18 12:54:30 +02:00
Bjørn Erik Pedersen
93aad3c543
Split out the puthe path/filepath functions into common/paths
So they can be used from the config package without cyclic troubles.

Updates #8654
2021-06-18 10:55:00 +02:00
Bjørn Erik Pedersen
31fb29fb3f
Do not read config from os.Environ when running tests
Fixes #8655
2021-06-16 18:58:40 +02:00
Bjørn Erik Pedersen
d392893cd7
Misc config loading fixes
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster:

```
BenchmarkDefaultConfigProvider/Viper-16         	  252418	      4546 ns/op	    2720 B/op	      30 allocs/op
BenchmarkDefaultConfigProvider/Custom-16        	  450756	      2651 ns/op	    1008 B/op	       6 allocs/op
```

Fixes #8633
Fixes #8618
Fixes #8630
Updates #8591
Closes #6680
Closes #5192
2021-06-14 17:00:32 +02:00
Joe Mooring
73483d0f9e
tpl: Add a terse pagination template variant to improve performance
These calls are equivalent:

{{ template "_internal/pagination.html" . }}
{{ template "_internal/pagination.html" (dict "page" .) }}
{{ template "_internal/pagination.html" (dict "page" . "format" "default") }}

To use an alternate format:

{{ template "_internal/pagination.html" (dict "page" . "format" "terse") }}

Fixes #8599
2021-06-08 10:41:21 +02:00
Bjørn Erik Pedersen
9b5debe4b8 Upgrade Instagram shortcode
Fixes #7879
2021-06-08 10:30:45 +02:00
Bjørn Erik Pedersen
12530519d8
Fix nested OS env config override when parent does not exist
Fixes #8618
2021-06-08 09:57:18 +02:00
Bjørn Erik Pedersen
f55d2f4376
tpl/fmt: Add erroridf template func
Fixes #8613
2021-06-07 19:11:03 +02:00
卜木
ee733085b7
config: Fix env split to allow = character in values
Fixes #8589
2021-05-30 20:25:37 +02:00
satotake
845a7ba4fc
Catch incomplete shortcode error
Currently, no name shortcodes (`{{< >}}`) enter unexpected branch and
throw `BUG: template info not set`. This patch checks if shortcode has
name or not earlier and throws specific error.

Closes #6866
2021-05-24 14:59:02 +02:00
satotake
785a31b5b8
navigation: Cache and copy Menu for sorting
.Site.Menus is mutated when it is sorted for now and this causes concurrency problem (#7594)
In this patch, each related sort function copies Menu before sorting to prevent
race condition.

Pages already have such a sort and cache logic and this patch is identical to it.

Closes #7594
2021-05-23 10:42:01 +02:00
Shohei Ueda
504c78da4b
modules/npm: Change SetEscapeHTML to false
Closes #8512
2021-05-09 00:20:28 +02:00
Bjørn Erik Pedersen
b660ea8d54
Add a benchmark 2021-05-04 17:59:04 +02:00
Bjørn Erik Pedersen
0d86a32d8f Make the shortcode template lookup for output formats stable
Fixes #7774
2021-04-23 15:04:35 +02:00
Dirk Olbrich
bc80022e03
publisher: Exclude comment and doctype elements from writeStats
- Reorder code blocks
- Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information
- Expand benchmark to test for minified and unminified content

Fixes #8396
Fixes #8417
2021-04-20 17:24:17 +02:00
Joe Mooring
3ddffd064d build(deps): bump github.com/yuin/goldmark from 1.3.2 to 1.3.5
Updated test per <https://github.com/yuin/goldmark/pull/205>.

Fixes #8377
2021-04-16 09:12:28 +02:00
Bjørn Erik Pedersen
33d5f80592
Add webp image encoding support
Fixes #5924
2021-04-15 17:22:55 +02:00
Joe Mooring
9b34d42bb2 Remove extraneous space from figure shortcode
Fixes #8401
2021-04-15 14:43:32 +02:00
Bjørn Erik Pedersen
df8bb8812f
Simplify some config loading code 2021-03-21 13:29:11 +01:00
Bjørn Erik Pedersen
fc06e85082
Apply OS env overrides twice
The recent fix for #8346 had an unfortunate side-effect, as it prevented overrides of config options used in the module collector, e.g. `HUGO_MODULE_REPLACEMENTS`.

This commit fixes that by applying the overrides twice.
2021-03-21 10:54:30 +01:00
Bjørn Erik Pedersen
7ed56c6941 Fix OS env override for nested config param only available in theme
Fixes #8346
2021-03-20 13:34:51 +01:00
Bjørn Erik Pedersen
ba1d0051b4 media: Make Type comparable
So we can use it and output.Format as map key etc.

This commit also fixes the media.Type implementation so it does not need to mutate itself to handle different suffixes for the same MIME type, e.g. jpg vs. jpeg.

This means that there are no Suffix or FullSuffix on media.Type anymore.

Fixes #8317
Fixes #8324
2021-03-14 15:21:54 +01:00
Bjørn Erik Pedersen
18074d0c23 Fix output format handling for render hooks
Fixes #8176
2021-03-09 13:26:39 +01:00
Bjørn Erik Pedersen
35bfb66222
Rename a test 2021-03-04 13:46:31 +01:00
Bjørn Erik Pedersen
6d21559fb5
Add a debug helper 2021-03-03 20:16:58 +01:00
Daniel Atwood
ba16a14c6e
Add support for Google Analytics v4 2021-03-03 13:30:06 +01:00
Bjørn Erik Pedersen
cd0c5d7ef3 Allow markdown attribute lists to be used in title render hooks
Fixes #8270
2021-02-23 18:08:39 +01:00
Bjørn Erik Pedersen
b5485aeae7 Add breaking tests for "map read and map write in templates"
The fix upstream in Go was reverted, so we apply a temporary patch for this in Hugo.

Updates #7293
2021-02-18 14:11:48 +01:00
Bjørn Erik Pedersen
21e9eb18ac Expand template newline testcase to commands 2021-02-18 14:11:48 +01:00
Bjørn Erik Pedersen
ae57ba6a9d Add a test case for Go 1.16 template action newlines 2021-02-18 14:11:48 +01:00
Bjørn Erik Pedersen
66beac99c6 deps: Update github.com/tdewolff/minify/v2 v2.6.2 => v2.9.13
Fixes #8258
2021-02-18 10:19:22 +01:00
susiwen8
bf55afd71f
Fix some humanize issues
Fixes #7912
2021-02-14 18:30:59 +01:00
Jeremy Epstein
4867cd1dea
tpl/embedded: Exclude pages without Permalink from sitemap 2021-02-06 18:30:09 +01:00
Bjørn Erik Pedersen
144943798c github: Enable NPM tests on Windows
Fixes #8196
2021-02-01 11:32:17 +01:00
Bjørn Erik Pedersen
32b86076ee js: Add Inject config option
Fixes #8164
2021-01-22 23:43:44 +01:00
Bjørn Erik Pedersen
e19a046c4b js: Add Shims option
This commit adds a new `shims` option to `js.Build` that allows swapping out a component with another.

Fixes #8165
2021-01-22 09:03:24 +01:00
Andreas Richter
2c8b5d9165
pipes: Add external source map support to js.Build and Babel
Fixes #8132
2021-01-18 10:38:09 +01:00
Bjørn Erik Pedersen
cea1574023
Add Dart Sass support
But note that the Dart Sass Embedded Protocol is still in beta (beta 5), a main release scheduled for Q1 2021.

Fixes #7380
Fixes #8102
2020-12-30 17:32:25 +01:00
Bjørn Erik Pedersen
a9718f44cd
para: Skip para test when not on CI
Fixes #6963
2020-12-23 19:47:20 +01:00
Bjørn Erik Pedersen
10ae7c3210
Improve LookPath 2020-12-19 17:03:07 +01:00
Bjørn Erik Pedersen
81975f847d Fix Resource.ResourceType so it always returns MIME's main type
The one exception being for the Page, which does not have a MIME type, in which you will get the value `page`.

Fixes #8052
2020-12-17 22:20:12 +01:00
真夜
ce96895deb
hugolib/paths: Fix typo 2020-12-16 12:12:14 +01:00
Phil Davis
04b89857e1
all: Fix minor typos 2020-12-16 12:11:32 +01:00
Bjørn Erik Pedersen
21fa1e86f2
Fix BenchmarkMergeByLanguage
Fixes #7914
2020-12-09 17:47:10 +01:00
Bjørn Erik Pedersen
d90e37e0c6 all: Format code with gofumpt
See https://github.com/mvdan/gofumpt
2020-12-03 13:12:58 +01:00
Andrew Zenk
4fc918e02c
tpl: Add title parameter to YouTube shortcode 2020-12-02 12:53:53 +01:00
sth
0ad378b09c
Use --baseURL path for live-reload URL
Fixes #6595
2020-12-02 12:52:26 +01:00
Bjørn Erik Pedersen
e442cf30a2 Fix server rebuild issue with partials referenced from render hooks
Fixes #7990
2020-11-26 20:41:54 +01:00
Bjørn Erik Pedersen
7e223b3baa Allow setting the delimiter used for setting config via OS env, e.g. HUGO_
Fixes #7829
2020-11-25 20:34:34 +01:00
Bjørn Erik Pedersen
8a6e706053
deps: Update to github.com/evanw/esbuild 0.8.11 to 0.8.14
Closes #7986
2020-11-25 17:39:51 +01:00
David Jones
8f5c9a747f
Add menu params
Fixes #7951
2020-11-22 22:09:59 +01:00
Bjørn Erik Pedersen
78f227b664
js: Let ESBuild handle all imports from node_modules
This commit fixes some issues where modules in /assets share the same name as in node_modules.

This was not intended, and with this commit the node_modules-components should be isolated. If you want to redefine something inside node_modules, use the `defines` option.

Fixes #7948
2020-11-13 08:54:29 +01:00
Bjørn Erik Pedersen
3437174c3a
Disable NPM test on Travis on Windows
For now.
2020-11-05 09:56:22 +01:00
Bjørn Erik Pedersen
bf2837a314
js: Misc fixes
* Fix resolve of package.json deps in submodules
* Fix directory logic for writing assets/jsconfig.json

Fixes #7924
Fixes #7923
2020-11-04 19:21:43 +01:00
Bjørn Erik Pedersen
85e4dd7370 Make js.Build fully support modules
Fixes #7816
Fixes #7777
Fixes #7916
2020-11-03 13:04:37 +01:00
Andreas Richter
3089fc0ba1 js.Build: Generate tsconfig files
Updates #7777

Added support to allow SourceMap files to be external to the build.
In addition added more information when the js compilation has an error.
Correctly append sourceMappingURL to output file.
Fix merge conflict.
2020-11-03 13:04:37 +01:00
Bjørn Erik Pedersen
8a1c637c44
Fix setting HUGO_MODULE_PROXY etc. via env vars
Fixes #7903
2020-10-30 09:41:04 +01:00
Gareth Watts
3400aff258 Allow cascade _target to work with non toml fm
The TOML lib unmarshals slices of string maps to []map[string]interface{}
whereas YAML and JSON decode to []interface{}

The existing tests only check for TOML working correctly, and _target
with cascade did not work at all for frontmatter defined in other formats.

Add a function to normalize those slices

Fixes #7874
2020-10-22 23:00:19 +02:00
Bjørn Erik Pedersen
fdfa4a5fe6 Allow getJSON errors to be ignored
This change is mostly motivated to get a more stable CI build (we're building the Hugo site there, with Instagram and Twitter shortcodes sometimes failing).

Fixes #7866
2020-10-22 09:09:29 +02:00
Bjørn Erik Pedersen
ead5799f7e Render aliases even if render=link
Fixes #7832
2020-10-14 16:59:37 +02:00
Bjørn Erik Pedersen
506820435c lang/i18n: Fix for language code case issue with pt-br etc.
Fixes #7804
2020-10-09 18:02:43 +02:00
Bjørn Erik Pedersen
6dd60fca73
Revert "deps: Update to github.com/tdewolff/minify v2.9.4"
Closes #7792

This reverts commit b254532b52.
2020-10-07 10:46:19 +02:00
Bjørn Erik Pedersen
634938908e pagemeta: Make BuildConfig.Render an enum
Allowing links on pages without rendering them.

Fixes #7783
2020-10-06 14:06:10 +02:00
Bjørn Erik Pedersen
c63db7f1f6 Allow cascade to be a slice with a _target discriminator
Fixes #7782
2020-10-06 14:06:10 +02:00
Bjørn Erik Pedersen
3acde9ae04
Make sure CSS is rebuilt when postcss.config.js or tailwind.config.js changes
Fixes #7715
2020-09-20 13:35:41 +02:00
Bjørn Erik Pedersen
b254532b52 deps: Update to github.com/tdewolff/minify v2.9.4 2020-09-19 21:17:02 +02:00
Bjørn Erik Pedersen
214afe4c1b modules/npm: Preserve the original package.json if it exists
Fixes #7690
2020-09-15 01:04:31 +02:00
Bjørn Erik Pedersen
85ba9bfffb Add "hugo mod npm pack"
This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into:

```
assets/_jsconfig
´``

These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename.

But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific.

This commit also adds adds `workDir/node_modules` to `NODE_PATH` and `HUGO_WORKDIR` to the env when running the JS tools above.

Fixes #7644
Fixes #7656
Fixes #7675
2020-09-13 20:55:29 +02:00
Andreas Richter
9df60b62f9
Print layout name if it was specified when showing missing layout file error
Fixes #7617
2020-09-13 20:36:37 +02:00
Bjørn Erik Pedersen
d4611c4322 modules: Add noVendor to module config
Fixes #7647
2020-09-10 08:47:05 +02:00
Bjørn Erik Pedersen
20af9a0781 modules: Add ignoreImports to module imports config
Fixes #7646
2020-09-10 08:47:05 +02:00
Bjørn Erik Pedersen
9a1e6d15a3 modules: Make ignoreVendor a glob pattern
Fixes #7642
2020-09-10 08:47:05 +02:00
Bjørn Erik Pedersen
4055c12184 Fix some change detection issues on server reloads
* Fix change detection when .GetPage/site.GetPage is used from shortcode
* Fix stale content for GetPage results with short name lookups on server reloads

Fixes #7623
Fixes #7624
Fixes #7625
2020-09-07 21:06:44 +02:00
Aurken Bilbao
ad01aea3f4 Fixed misspelled words 2020-08-20 22:34:35 +02:00
Bjørn Erik Pedersen
ec37420468 Improve stderr logging for PostCSS and simlilar
Fixes #7584
2020-08-20 22:32:20 +02:00
Bjørn Erik Pedersen
ae63c2b5c9
Fail on partials with return when given none or a zero argument
We need to make a proper fix for this, but it is better with an error than just silently continue.

Fixes #7572
Updates #7528
2020-08-19 10:01:16 +02:00
Bjørn Erik Pedersen
c91dbe4ce9 Fix baseof block regression
From Hugo 0.74.0.

Fixes #7478
2020-07-13 20:45:09 +02:00
Bjørn Erik Pedersen
9df98ec49c Add proper Media Type handling in js.Build
See #732
2020-07-13 10:56:23 +02:00
Remko Tronçon
2fc3380707 Add js.Build asset bundling
Fixes #7321
2020-07-13 10:56:23 +02:00
Bjørn Erik Pedersen
12a65e76df Add openapi3.Unmarshal
Fixes #7442
Fixes #7443
2020-07-06 20:03:36 +02:00
Bjørn Erik Pedersen
42e150fbfa Fix server reload when non-HTML shortcode changes
Fixes #7448
2020-07-03 23:31:51 +02:00
Bjørn Erik Pedersen
4a3efea7ef Add support for inline partials
Fixes #7444
2020-07-01 23:10:21 +02:00
Bjørn Erik Pedersen
ccfaeb678b
hugolib: Add missing zero check on file 2020-06-26 13:27:01 +02:00
Derk Muenchhausen
f0266e2ef3
Rework external asciidoctor integration
This commit solves the relative path problem with asciidoctor tooling. An include will resolve relatively, so you can refer easily to files in the same folder.

Also `asciidoctor-diagram` and PlantUML rendering works now, because the created temporary files will be placed in the correct folder.

This patch covers just the Ruby version of asciidoctor. The old AsciiDoc CLI EOLs in Jan 2020, so this variant is removed from code.

The configuration is completely rewritten and now available in `config.toml` under the key `[markup.asciidocext]`:

```toml
[markup.asciidocext]
    extensions = ["asciidoctor-html5s", "asciidoctor-diagram"]
    workingFolderCurrent = true
    trace = true
    [markup.asciidocext.attributes]
        my-base-url = "https://example.com/"
        my-attribute-name = "my value"
```

- backends, safe-modes, and extensions are now whitelisted to the popular (ruby) extensions and valid values.
- the default for extensions is to not enable any, because they're all external dependencies so the build would break if the user didn't install them beforehand.
- the default backend is html5 because html5s is an external gem dependency.
- the default safe-mode is safe, explanations of the modes: https://asciidoctor.org/man/asciidoctor/
- the config is namespaced under asciidocext_config and the parser looks at asciidocext to allow a future native Go asciidoc.
- `uglyUrls=true` option and `--source` flag are supported
- `--destination` flag is required

Follow the updated documentation under `docs/content/en/content-management/formats.md`. 
  
This patch would be a breaking change, because you need to correct all your absolute include pathes to relative paths, so using relative paths must be configured explicitly by setting `workingFolderCurrent = true`.
2020-06-25 09:51:33 +02:00
Bjørn Erik Pedersen
77aa385b84
Enable the embedded template test when race detector is off
See #5926
2020-06-25 09:16:28 +02:00
Nico
3466884e36 Create robots.txt in the domain root directory
Before a robots.txt is created in every Site. So in public/robots.txt if there are no languages (was correct). But if there are multiple languages in every language directory, too (was wrong). If defaultContentLanguageInSubdir is true, no language is created into the root directory, so no robots.txt is in the root directory (was wrong). If multihosts are configured for each language, that is the only case where one robots.txt must be created in each language directory (was correct).

I've changed the behaviour, that only in the multihost case the robots.txt is created in the language directories. In any other case it is created in public/robots.txt. I've also added tests that files are not created in the wrong directories.

Fixes #5160
See also #4193
2020-06-20 17:13:26 +02:00
Bjørn Erik Pedersen
fc045e12a9 Rename taxonomy kinds from taxonomy to term, taxonomyTerm to taxonomy
And we have taken great measures to limit potential site breakage:

* For `disableKinds` and `outputs` we try to map from old to new values if possible, if not we print an ERROR that can be toggled off if not relevant.
* The layout lookup is mostly compatible with more options for the new `term` kind.

That leaves:

* Where queries in site.Pages using taxonomy/taxonomyTerm Kind values as filter.
* Other places where these kind value are used in the templates (classes etc.)

Fixes #6911
Fixes #7395
2020-06-18 09:09:56 +02:00
Helder Pereira
9679023f2b Fix aliases with path in baseURL 2020-06-17 10:52:35 +02:00
Bjørn Erik Pedersen
6408c1cbc8 Fix server data race/nil pointer in withMaps
Fixes #7392
2020-06-16 11:48:10 +02:00
Bjørn Erik Pedersen
522ba1cd98 Fix order of GetTerms
Preserve the order from front matter, which would be behaviour when doing this manually (before GetTerms).

Fixes #7213
2020-06-15 22:23:02 +02:00
Helder Pereira
f720fe56db Fix aliases with uglyURLs 2020-06-15 12:19:59 +02:00
Bjørn Erik Pedersen
d6ed17c60f Fix crash for closing shortcode with no .Inner set
Fixes #6857
Closes #7330
2020-06-14 20:19:08 +02:00
Helder Pereira
145b3fcce3 Fix aliases with relativeURLs 2020-06-14 14:08:41 +02:00
Bjørn Erik Pedersen
e3e627e6b7
Add a nested data dir test
Ref. a discussion.
2020-06-12 09:14:20 +02:00
Bjørn Erik Pedersen
f8c67f93e1 Allow hook template per section/type
Fixes #7349
2020-06-02 19:02:10 +02:00
Bjørn Erik Pedersen
c950c86b4e publisher: Fix tag collector for nested table elements
Fixes #7318
2020-05-27 11:10:48 +02:00
Bjørn Erik Pedersen
81f5633245
Add some more date test cases
See #7310
2020-05-24 14:35:49 +02:00
Bjørn Erik Pedersen
9698b0dab1
Fix RenderString vs render hooks
Fixes #7265
2020-05-24 13:20:57 +02:00
Bjørn Erik Pedersen
32344fe3db
Prevent WARNINGs in RenderString 2020-05-24 12:49:45 +02:00
Bjørn Erik Pedersen
4d7fa9f114 Fix IsAncestor/IsDescendant for taxonomies
Fixes #7305
2020-05-24 12:35:45 +02:00
Bjørn Erik Pedersen
a985efcecf Fix GetPage on section/bundle name overlaps
In the internal Radix we stored the directory based nodes without a traling slash, e.g. `/blog`.

The original motivation was probably to make it easy to do prefix searching: Give me all ancestors.

This, however have lead to some ambigouty with overlapping directory names.

This particular problem was, however, not possible to work around in an easy way, so from now we store these as `/blog/`.

Fixes #7301
2020-05-24 12:35:45 +02:00
Bjørn Erik Pedersen
6c3c6686f5 Fix Go template script escaping
Fixes #6695
2020-05-23 22:00:34 +02:00
Bjørn Erik Pedersen
c34bf48560
Add a test helper 2020-05-22 10:57:25 +02:00
Cameron Little
6e051c053e Add test for headings render hook 2020-05-15 21:12:43 +02:00
Eli W. Hunter
423b8f2fb8 Add render template hooks for headings
This commit also

* Renames previous types to be non-specific. (e.g. hookedRenderer rather
  than linkRenderer)

Resolves #6713
2020-05-15 21:12:43 +02:00
Bjørn Erik Pedersen
6add6d77b4 Rename transpileJS to babel
And add a test.

Updates #5764
2020-04-29 10:51:33 +02:00
Bjørn Erik Pedersen
67f920419a
Disable a test locally
It has started to fail in LiteIDE for some reason (works from the shell and on CI).
2020-04-28 12:24:02 +02:00
Takayama Fumihiko
8d5766d417 Fix IsAncestor and IsDescendant when the same page is passed 2020-04-23 20:23:42 +02:00
Takayama Fumihiko
27a4c4410c Fix IsAncestor and IsDescendant under subsection 2020-04-23 18:12:13 +02:00
Panagiotis Koursaris
49e6c8cb4e
hugolib/filesystems: Fix typo in test suite 2020-04-21 23:16:36 +02:00
Bjørn Erik Pedersen
f37e77f2d3 Fix class collector when running with --minify
Also add a related stresstest.

Fixes #7161
2020-04-21 21:00:52 +02:00
Bjørn Erik Pedersen
b3c825756f
Fix broken test 2020-04-21 16:44:35 +02:00
Boqin Qin
736f84b2d5
hugolib: Add Unlock before panic 2020-04-20 17:47:34 +02:00
Bjørn Erik Pedersen
095bf64c99
Collect HTML elements during the build to use in PurgeCSS etc.
The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss.

You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites.

Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes.

You can enable this in your site config:

```toml
[build]
  writeStats = true
```

It will then write a `hugo_stats.json` file to the project root as part of the build.

If you're only using this for the production build, you should consider putting it below `config/production`.

You can then set it up with PostCSS like this:

```js
const purgecss = require('@fullhuman/postcss-purgecss')({
    content: [ './hugo_stats.json' ],
    defaultExtractor: (content) => {
        let els = JSON.parse(content).htmlElements;
        return els.tags.concat(els.classes, els.ids);
    }
});

module.exports = {
    plugins: [
        require('tailwindcss'),
        require('autoprefixer'),
        ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : [])
    ]
};
```

Fixes #6999
2020-04-09 22:57:26 +02:00
Bjørn Erik Pedersen
2f721f8ec6
Add basic "post resource publish support"
Fixes #7146
2020-04-07 21:59:20 +02:00
Bjørn Erik Pedersen
4a39564efe Fix IsDescendant/IsAncestor for overlapping section names
Fixes #7096
2020-03-26 16:11:36 +01:00
Bjørn Erik Pedersen
523d51948f Fix _build.list.local logic
Fixes #7089
2020-03-24 12:47:42 +01:00
Bjørn Erik Pedersen
cfa73050a4 Fix cache reset for a page's collections on server live reload
Fixes #7085
2020-03-23 23:14:39 +01:00
Bjørn Erik Pedersen
7204b354a9 Some minify configuration adjustments 2020-03-20 20:35:57 +01:00
SatowTakeshi
574c2959b8 Add minify config
Fixes #6750
Updates #6892
2020-03-20 20:35:57 +01:00
Bjørn Erik Pedersen
99958f90fe Allow headless bundles to list pages via $page.Pages and $page.RegularPages
Fixes #7075
2020-03-20 18:28:55 +01:00
Carl Johnson
95f492114e
Fix GetTerms nil pointer
Fixes #7061
2020-03-20 14:35:41 +01:00
Bjørn Erik Pedersen
1a8af7d4f0 Add workaround for regular CSS imports in SCSS
Fixes #7059
2020-03-16 23:21:44 +01:00
Bjørn Erik Pedersen
03b93bb988 Add .RegularPagesRecursive
Fixes #6411
2020-03-16 17:37:36 +01:00
Bjørn Erik Pedersen
df298558a5
Improve Tailwind/PostCSS error messages
Fixes #7041
Fixes #7042
2020-03-11 14:13:03 +01:00
Bjørn Erik Pedersen
6cceef65c2 Fix ambigous error on site.GetPage
Fixes #7016
2020-03-09 16:11:50 +01:00
Bjørn Erik Pedersen
ffcb4aeb8e Fix handling of HTML files without front matter
This means that any HTML file inside /content will be treated as a regular file.

If you want it processes with shortcodes and a layout, add front matter.

The defintion of an HTML file here is:

* File with extension .htm or .html
* With first non-whitespace character "<" that isn't a HTML comment.

This is in line with the documentation.

Fixes #7030
Fixes #7028
See #6789
2020-03-09 15:01:55 +01:00
Bjørn Erik Pedersen
6a34f88dcc
Skip some tests on CircleCI 2020-03-03 15:45:13 +01:00
Bjørn Erik Pedersen
3d3fa5c3fe Add build.UseResourceCacheWhen
Fixes #6993
2020-03-03 13:29:58 +01:00
Bjørn Erik Pedersen
8947c3fa0b Fix ref/relref short lookup for pages in sub-folder
Fixes #6952
2020-03-02 21:08:28 +01:00
Bjørn Erik Pedersen
3e9db2ad95
hugolib: Fix error handling in page collector
Fixes #6988
2020-03-02 11:07:09 +01:00
Bjørn Erik Pedersen
1746e8a9b2 Fix ref/relRef regression for relative refs from bundles
Fixes #6952
2020-02-29 18:32:06 +01:00
Bjørn Erik Pedersen
b0d850321e Fix rebuild logic when editing template using a base template
We have been doing too much work in that case.

Fixes #6968
2020-02-28 14:14:55 +01:00
Bjørn Erik Pedersen
b66d38c419 resources: Add basic @import support to resources.PostCSS
This commit also makes the HUGO_ENVIRONMENT environment variable available to Node.

Fixes #6957
Fixes #6961
2020-02-27 11:47:24 +01:00
Bjørn Erik Pedersen
0bd6356c6d Fix panic when home page is drafted
Fixes #6927
2020-02-23 00:11:10 +01:00
satotake
ca68abf0bc
Fix goldmark toc rendering
Previously gordmark-based TOC renderes only `KindText` and `KindString`

This commit expands target node with Goldmark's renderer

I am not sure of what are expected results as TOC contents in some (rare) cases
but Blackfriday's behaviours are fundamentally respected.

For example,
- image `[image text](link)` is rendered as `<img>` tag
- GFM AutoLink `gohugo.io` is rendered as text

* Render AutoLink as <a> tag as Blackfriday does

Fixes #6736
Fixes #6809
2020-02-22 18:06:30 +01:00
Bjørn Erik Pedersen
a524124beb Fix crashes for 404 in IsAncestor etc.
Fixes #6931
2020-02-22 13:36:04 +01:00
Bjørn Erik Pedersen
f460530347
Fix panic on no output formats
A page needs its output formats even if it should not be rendered or its resources should not be published.

Fixes #6924
2020-02-21 09:41:39 +01:00
Bjørn Erik Pedersen
4c2a0de412
Fix panic in 404.Parent
Fixes #6924
2020-02-21 08:14:08 +01:00
Bjørn Erik Pedersen
7ef5a4c83e
hugolib: Fix 2 Paginator.Pages taxonomy regressions
Fixes #6921
Fixes #6918
2020-02-20 14:13:04 +01:00
Bjørn Erik Pedersen
a70bbd0696
hugolib: Fix deletion of orphaned sections
Avoid deleting inside the recursive walk.

Fixes #6920
2020-02-20 13:40:16 +01:00
Bjørn Erik Pedersen
9bdedb251c
Fix lazy publishing with publishResources=false
Fixes #6914
2020-02-19 18:22:41 +01:00
Bjørn Erik Pedersen
fa520a2d98 Add Page.GetTerms
Fixes #6905
2020-02-19 14:52:23 +01:00
Bjørn Erik Pedersen
7489a86459
Add a list terms benchmark
See #6905
2020-02-19 09:08:32 +01:00
Bjørn Erik Pedersen
da54787cfa
Handle disabled RSS even if it's defined in outputs
See https://github.com/gohugoio/hugo/issues/6897#issuecomment-587947078
2020-02-19 09:08:31 +01:00
Bjørn Erik Pedersen
b2dcd53e3c
Use the tree for taxonomy.Pages() 2020-02-18 19:47:45 +01:00
Bjørn Erik Pedersen
36983e6189
Add some cagegories to the site collections benchmarks 2020-02-18 18:52:35 +01:00
Bjørn Erik Pedersen
aa3e183056
tpl: Fix RSS template for the terms listing
Fixes #6909
2020-02-18 16:55:30 +01:00
Bjørn Erik Pedersen
1b7acfe763
Fix taxonomy
Recently introduced in master.

See https://github.com/gohugoio/hugo/issues/6897#issuecomment-587499907
2020-02-18 16:43:36 +01:00
Bjørn Erik Pedersen
19e12caf8c
Fix RenderString for pages without content
Fixes #6882
2020-02-18 14:08:59 +01:00
Bjørn Erik Pedersen
eada236f87
Introduce a tree map for all content
This commit introduces a new data structure to store pages and their resources.

This data structure is backed by radix trees.

This simplies tree operations, makes all pages a bundle,  and paves the way for #6310.

It also solves a set of annoying issues (see list below).

Not a motivation behind this, but this commit also makes Hugo in general a little bit faster and more memory effective (see benchmarks). Especially for partial rebuilds on content edits, but also when taxonomies is in use.

```
name                                   old time/op    new time/op    delta
SiteNew/Bundle_with_image/Edit-16        1.32ms ± 8%    1.00ms ± 9%  -24.42%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16    1.28ms ± 0%    0.94ms ± 0%  -26.26%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16      33.9ms ± 2%    21.8ms ± 1%  -35.67%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16            40.6ms ± 1%    37.7ms ± 3%   -7.20%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16        56.7ms ± 0%    51.7ms ± 1%   -8.82%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16      19.9ms ± 2%    18.3ms ± 3%   -7.64%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16         37.9ms ± 4%    34.0ms ± 2%  -10.28%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16             10.7ms ± 0%    10.6ms ± 0%   -1.15%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16         10.8ms ± 0%    10.7ms ± 0%   -1.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16           43.2ms ± 1%    39.6ms ± 1%   -8.35%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                 47.6ms ± 1%    47.3ms ± 0%     ~     (p=0.057 n=4+4)
SiteNew/Deep_content_tree-16             73.0ms ± 1%    74.2ms ± 1%     ~     (p=0.114 n=4+4)
SiteNew/Many_HTML_templates-16           37.9ms ± 0%    38.1ms ± 1%     ~     (p=0.114 n=4+4)
SiteNew/Page_collections-16              53.6ms ± 1%    54.7ms ± 1%   +2.09%  (p=0.029 n=4+4)

name                                   old alloc/op   new alloc/op   delta
SiteNew/Bundle_with_image/Edit-16         486kB ± 0%     430kB ± 0%  -11.47%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16     265kB ± 0%     209kB ± 0%  -21.06%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16      13.6MB ± 0%     8.8MB ± 0%  -34.93%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16            66.5MB ± 0%    63.9MB ± 0%   -3.95%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16        28.8MB ± 0%    25.8MB ± 0%  -10.55%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16      6.16MB ± 0%    5.56MB ± 0%   -9.86%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16         16.9MB ± 0%    16.0MB ± 0%   -5.19%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16             2.28MB ± 0%    2.29MB ± 0%   +0.35%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16         2.07MB ± 0%    2.07MB ± 0%     ~     (p=0.114 n=4+4)
SiteNew/Tags_and_categories-16           14.3MB ± 0%    13.2MB ± 0%   -7.30%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                 69.1MB ± 0%    69.0MB ± 0%     ~     (p=0.343 n=4+4)
SiteNew/Deep_content_tree-16             31.3MB ± 0%    31.8MB ± 0%   +1.49%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16           10.8MB ± 0%    10.9MB ± 0%   +1.11%  (p=0.029 n=4+4)
SiteNew/Page_collections-16              21.4MB ± 0%    21.6MB ± 0%   +1.15%  (p=0.029 n=4+4)

name                                   old allocs/op  new allocs/op  delta
SiteNew/Bundle_with_image/Edit-16         4.74k ± 0%     3.86k ± 0%  -18.57%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16     4.73k ± 0%     3.85k ± 0%  -18.58%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16        301k ± 0%      198k ± 0%  -34.14%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16              389k ± 0%      373k ± 0%   -4.07%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16          338k ± 0%      262k ± 0%  -22.63%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16        102k ± 0%       88k ± 0%  -13.81%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16           176k ± 0%      152k ± 0%  -13.32%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16              26.8k ± 0%     26.8k ± 0%   +0.05%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16          26.8k ± 0%     26.8k ± 0%   +0.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16             273k ± 0%      245k ± 0%  -10.36%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                   396k ± 0%      398k ± 0%   +0.39%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16               317k ± 0%      325k ± 0%   +2.53%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16             146k ± 0%      147k ± 0%   +0.98%  (p=0.029 n=4+4)
SiteNew/Page_collections-16                210k ± 0%      215k ± 0%   +2.44%  (p=0.029 n=4+4)
```

Fixes #6312
Fixes #6087
Fixes #6738
Fixes #6412
Fixes #6743
Fixes #6875
Fixes #6034
Fixes #6902
Fixes #6173
Fixes #6590
2020-02-18 09:49:42 +01:00
Bjørn Erik Pedersen
e5329f13c0
Another benchmark rename 2020-02-18 09:48:36 +01:00
Bjørn Erik Pedersen
5b145ddc4c
Rename the Edit benchmarks
To make them easier to filter out.
2020-02-18 09:45:21 +01:00
Bjørn Erik Pedersen
54bdcaacae
Refactor a benchmark to make it runnable as test 2020-02-17 09:24:15 +01:00
Bjørn Erik Pedersen
1622510a5c
Add benchmark for content edits 2020-02-17 09:05:37 +01:00
Bjørn Erik Pedersen
3b721110d5
Shuffle test files before insertion 2020-02-10 12:03:41 +01:00
Bjørn Erik Pedersen
18888e09bb Fix bundle resource ordering regression
Introduced in Hugo 0.64.0

Fixes #6851
2020-02-09 21:35:39 +01:00
Bjørn Erik Pedersen
80dd6ddde2 Fix module mount in sub folder
This addresses a specific issue, but is a also a major simplification of the filesystem file mounts.

Fixes #6730
2020-02-04 00:17:10 +01:00
Bjørn Erik Pedersen
5859586453
hubolig: Add a render hook whitespace test
See #6832
2020-01-31 07:51:05 +01:00
Bjørn Erik Pedersen
2d159e9cc7 Do not render alias paginator pages for non-HTML outputs
Update #6797
2020-01-30 23:45:19 +01:00
Bjørn Erik Pedersen
f45cb31728 Fix base template handling with preceding comments
Fixes #6816
2020-01-30 23:44:42 +01:00
Bjørn Erik Pedersen
49ef647203 modules: Fix "hugo mod get -u" with no arguments
Fixes #6826
Closes #6825
2020-01-30 17:15:35 +01:00
Bjørn Erik Pedersen
e8831a056b
hubolib: Revert to .Type = "page" when empty
This was changed with good intentions in 0.63.0.

This behaviour was not documented, but it was of course in use.

This commit rolls back to how it behaved before:

For `Page.Type` you will get:

* `type` from front matter if set.
* `.Section`
* If none of the above returns anything, return "page"

Fixes #6805
2020-01-27 12:33:33 +01:00
Bjørn Erik Pedersen
74b6c4e5ff
And now finally fix the 404 templates
Fixes #6795
2020-01-27 12:30:31 +01:00
Bjørn Erik Pedersen
8df5d76e70
Fix 404 with base template regression
Fixes #6795
2020-01-26 13:14:08 +01:00
Bjørn Erik Pedersen
f441f67512 Fix baseof with regular define regression
Fixes #6790
2020-01-23 19:54:06 +01:00
Bjørn Erik Pedersen
17af79a03e Fix 0.62.1 server rebuild slowdown regression
Fixes #6784
2020-01-23 11:50:02 +01:00
Bjørn Erik Pedersen
836c24261f
hugolib: Disable a test assertion on ARM
See #6655
2020-01-22 11:44:29 +01:00
Bjørn Erik Pedersen
c6d650c8c8
tpl/tplimpl: Rework template management to get rid of concurrency issues
This more or less completes the simplification of the template handling code in Hugo started in v0.62.

The main motivation was to fix a long lasting issue about a crash in HTML content files  without front matter.

But this commit also comes with a big functional improvement.

As we now have moved the base template evaluation to the build stage we now use the same lookup rules for `baseof` as for `list` etc. type of templates.

This means that in this simple example you can have a `baseof` template for the `blog` section without having to duplicate the others:

```
layouts
├── _default
│   ├── baseof.html
│   ├── list.html
│   └── single.html
└── blog
    └── baseof.html
```

Also, when simplifying code, you often get rid of some double work, as shown in the "site building" benchmarks below.

These benchmarks looks suspiciously good, but I have repeated the below with ca. the same result. Compared to master:

```
name                              old time/op    new time/op    delta
SiteNew/Bundle_with_image-16        13.1ms ± 1%    10.5ms ± 1%  -19.34%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16    13.0ms ± 0%    10.7ms ± 1%  -18.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16      46.4ms ± 2%    43.1ms ± 1%   -7.15%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16            52.2ms ± 2%    47.8ms ± 1%   -8.30%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16        77.9ms ± 1%    70.9ms ± 1%   -9.01%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16      43.0ms ± 0%    37.2ms ± 1%  -13.54%  (p=0.029 n=4+4)
SiteNew/Page_collections-16         58.2ms ± 1%    52.4ms ± 1%   -9.95%  (p=0.029 n=4+4)

name                              old alloc/op   new alloc/op   delta
SiteNew/Bundle_with_image-16        3.81MB ± 0%    2.22MB ± 0%  -41.70%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16    3.60MB ± 0%    2.01MB ± 0%  -44.20%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16      19.3MB ± 1%    14.1MB ± 0%  -26.91%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16            70.7MB ± 0%    69.0MB ± 0%   -2.40%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16        37.1MB ± 0%    31.2MB ± 0%  -15.94%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16      17.6MB ± 0%    10.6MB ± 0%  -39.92%  (p=0.029 n=4+4)
SiteNew/Page_collections-16         25.9MB ± 0%    21.2MB ± 0%  -17.99%  (p=0.029 n=4+4)

name                              old allocs/op  new allocs/op  delta
SiteNew/Bundle_with_image-16         52.3k ± 0%     26.1k ± 0%  -50.18%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16     52.3k ± 0%     26.1k ± 0%  -50.16%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16        336k ± 1%      269k ± 0%  -19.90%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16              422k ± 0%      395k ± 0%   -6.43%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16          401k ± 0%      313k ± 0%  -21.79%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16        247k ± 0%      143k ± 0%  -42.17%  (p=0.029 n=4+4)
SiteNew/Page_collections-16           282k ± 0%      207k ± 0%  -26.55%  (p=0.029 n=4+4)
```

Fixes #6716
Fixes #6760
Fixes #6768
Fixes #6778
2020-01-22 09:39:49 +01:00
Matt Riggott
d3e8ab2e39 deps: Update Goldmark to v1.1.21
This is the first version of Goldmark that supports all the
Smartypants-style typographic punctuation transformations. Now, a
straight single quote in the middle of a word is translated into a curly
quote (e.g. "that's" becomes "that&rsquo;s"). Earlier versions leave
them untouched. This brings Goldmark in line with Blackfriday.

Fixes #6571.
2020-01-15 12:07:25 +01:00
Bjørn Erik Pedersen
ddd75f2121
hugolib: Some more benchmark adjustments 2020-01-13 16:47:10 +01:00
Bjørn Erik Pedersen
4ed6ebef4c
hugolib: Adjust site benchmarks 2020-01-13 16:39:33 +01:00
Bjørn Erik Pedersen
ea05c0e845
hugolib: Add a benchmark with lots of templates 2020-01-07 10:48:17 +01:00
Bjørn Erik Pedersen
196a9df585 hugolib: Fix relative .Page.GetPage from bundle
Fixes #6705
2020-01-05 19:41:53 +01:00
Bjørn Erik Pedersen
ff6253bc7c Support files in content mounts
This commit is a general improvement of handling if single file mounts.

Fixes #6684
Fixes #6696
2020-01-01 18:19:49 +01:00
Bjørn Erik Pedersen
5509954c7e hugolib: Fix inline shortcode regression
Fixes #6677
2019-12-28 14:31:51 +01:00
Bjørn Erik Pedersen
1fb17be9a0 deps: Update Goldmark to v1.1.18
Fixes #6649
2019-12-22 18:14:04 +01:00
Bjørn Erik Pedersen
8a58ebb311 hugolib: Improve error and reload handling of hook templates in server mode
Fixes #6635
2019-12-20 11:38:44 +01:00
Bjørn Erik Pedersen
a67d95fe1a Preserve HTML Text for image render hooks
Fixes #6639
2019-12-19 19:02:03 +01:00
Bjørn Erik Pedersen
ad6504e6b5 Fix abs path handling in module mounts
Fixes #6622
2019-12-19 18:51:59 +01:00
John Zaitseff
158e7ec204 Fix incorrect MIME type from image/jpg to image/jpeg
According to multiple sources, both official ([IANA] and [RFC2046]) and
otherwise (eg, [Mozilla] and [Wikipedia]), the official MIME type for
JPEG images is `image/jpeg`, not `image/jpg`.  Change Hugo to match.

[IANA]: https://www.iana.org/assignments/media-types/media-types.xhtml#image
[RFC2046]: https://tools.ietf.org/html/rfc2046
[Mozilla]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types
[Wikipedia]: https://en.wikipedia.org/wiki/JPEG
2019-12-19 18:02:11 +01:00
Bjørn Erik Pedersen
00954c5d1f Preserve HTML Text for link render hooks
The context now have two text methods:

* Text - rendered
* PlainText

Fixes #6629
2019-12-18 22:55:19 +01:00
Bjørn Erik Pedersen
d20ca37005 tpl: Get rid of the custom template truth logic
Fixes #6615
2019-12-18 20:05:22 +01:00
Bjørn Erik Pedersen
e625088ef5
Add render template hooks for links and images
This commit also

* revises the change detection for templates used by content files in server mode.
* Adds a Page.RenderString method

Fixes #6545
Fixes #4663
Closes #6043
2019-12-18 11:44:40 +01:00
Bjørn Erik Pedersen
3c24ae030f
hugolib: Fix test 2019-12-12 14:48:59 +01:00
Bjørn Erik Pedersen
a03c631c42
Rework template handling for function and map lookups
This is a big commit, but it deletes lots of code and simplifies a lot.

* Resolving the template funcs at execution time means we don't have to create template clones per site
* Having a custom map resolver means that we can remove the AST lower case transformation for the special lower case Params map

Not only is the above easier to reason about, it's also faster, especially if you have more than one language, as in the benchmark below:

```
name                          old time/op    new time/op    delta
SiteNew/Deep_content_tree-16    53.7ms ± 0%    48.1ms ± 2%  -10.38%  (p=0.029 n=4+4)

name                          old alloc/op   new alloc/op   delta
SiteNew/Deep_content_tree-16    41.0MB ± 0%    36.8MB ± 0%  -10.26%  (p=0.029 n=4+4)

name                          old allocs/op  new allocs/op  delta
SiteNew/Deep_content_tree-16      481k ± 0%      410k ± 0%  -14.66%  (p=0.029 n=4+4)
```

This should be even better if you also have lots of templates.

Closes #6594
2019-12-12 10:04:35 +01:00
Bjørn Erik Pedersen
167c01530b
Create lightweight forks of text/template and html/template
This commit also removes support for Ace and Amber templates.

Updates #6594
2019-12-12 09:59:34 +01:00
Gavin D. Howard
4c804319f6 markup/tableofcontents: Add config option for ordered list 2019-12-12 07:48:40 +01:00
Bjørn Erik Pedersen
3cc217a650 deps: Update Goldmark 2019-12-11 08:41:57 +01:00