Commit graph

2050 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
52ea07d2eb Fix taxonomy weight sort regression
Fixes #10406
2022-11-01 18:45:34 +01:00
Bjørn Erik Pedersen
631d768be9 Revise the fix for shortcode vs output format nilpointer
We do lazy initialization and (potentially) reuse of an output format's rendered content. We do this evaluation when we
start a new rendering a new output format. There are, however, situation where these borders gets crossed (e.g.
accessing content from another output format). We have a check for this in place for most cases, but not the content
rendering of inner markdown blocks inside shortcodes. This patch applies that same logic to the newly introduced
RenderContent method (which is not available from the templates).

Fixes #10391
2022-10-26 13:00:21 +02:00
davidejones
e5d2a8f6a3 Avoid nilpointer when shortcode page content output nil
Updates #10391
2022-10-26 13:00:21 +02:00
Bjørn Erik Pedersen
2171e3c9a5
Revert "Adjust a test"
Committed by mistake.
This reverts commit cac773aeff.
2022-09-27 11:42:25 +02:00
Bjørn Erik Pedersen
cac773aeff
Adjust a test 2022-09-27 11:41:15 +02:00
Bjørn Erik Pedersen
29ccb36069 Fix /static performance regression from Hugo 0.103.0
In `v0.103.0` we added support for `resources.PostProcess` for all file types, not just HTML. We had benchmarks that said we were fine in that department, but those did not consider the static file syncing.

This fixes that by:

* Making sure that the /static syncer always gets its own file system without any checks for the post process token.
* For dynamic files (e.g. rendered HTML files) we add an additional check to make sure that we skip binary files (e.g. images)

Fixes #10328
2022-09-26 19:02:25 +02:00
Bjørn Erik Pedersen
8e77bcc930
Filter out any duplicate files to post process
Updates #10269
2022-09-14 19:42:55 +02:00
Bjørn Erik Pedersen
74daca6b30 Support PostProcess for all file types
Not just HTML.

Fixes #10269
2022-09-14 19:09:20 +02:00
satotake
7d40da876c Add --force to hugo new
Closes #9243
2022-09-08 15:35:11 +02:00
Bjørn Erik Pedersen
8e5044d7f5 Fix shortcode parser regression with quoted param values
This issue was introduced in `v0.102.0`.

In 223bf28004 we removed the byte source from the parsed page result, which
meant we had to preserve exact positioning for all elements. This introduced some new `TypeIgnore` tokens
which we, wrongly, assumed didn't matter where we put in the result slice (they should be ignored anyway).

But it seems that this broke the logic where we determine if it's positional or named params in the case
where the paramater value contains escaped quoutes.

This commit makes sure that these ignore tokens (the back slashes) are never sent back to the client, which is how it was before `v0.102.0`.

This commit also fixes some lost error information in that same commit.

Fixes #10236
2022-09-01 12:13:23 +02:00
Bjørn Erik Pedersen
223bf28004 parser/pageparser: Don't store the byte slices
On its own this change doesn't do any magic, but this is part of a bigger picture about making Hugo leaner in the
memory usage department.
2022-07-09 16:03:11 +02:00
Bjørn Erik Pedersen
72b0ccdb01
Make the baseline benchmark's test files stable
And add some assertions.
2022-06-30 12:36:55 +02:00
Bjørn Erik Pedersen
d1278f696a
Extract the baseline benchmark to a test 2022-06-26 09:56:01 +02:00
Bjørn Erik Pedersen
92f31ae63b
Add a baseline benchmark 2022-06-25 18:26:40 +02:00
Bjørn Erik Pedersen
35fa192838 deps: Udpate to github.com/alecthomas/chroma/v2
Fixes #9932
Fixes #9931
2022-06-14 14:08:40 +02:00
Bjørn Erik Pedersen
a5a4422aae Fix relURL with leading slash when baseURL includes a subdirectory
Fixes #9994
2022-06-13 11:40:25 +02:00
Bjørn Erik Pedersen
0566bbf7c7 Fix raw TOML dates in where/eq
Note that this has only been a problem with "raw dates" in TOML files in /data and similar. The predefined front matter
dates `.Date` etc. are converted to a Go Time and has worked fine even after upgrading to v2 of the go-toml lib.

Fixes #9979
2022-06-07 13:02:58 +02:00
Bjørn Erik Pedersen
311b8008bf
helpers: Fix panic with invalid defaultMarkdownHandler
Fixes #9968
2022-06-04 17:47:20 +02:00
Bjørn Erik Pedersen
212d9e3017 Fix panic with markdownify/RenderString with shortcode on Page with no content file
Fixes #9959
2022-06-01 12:04:55 +02:00
Bjørn Erik Pedersen
6f7bf3f2d7 Fix indentation in highlight shortcode
This commit adds a new `.InnerDeindent` method to the shortcode context, which is `.Inner` with any
indendation removed. This is then used in the built-in `highlight` shortcode to prevent the extra
whitespace getting hightlighted.

Fixes #4717
2022-05-31 09:05:54 +02:00
Bjørn Erik Pedersen
9e904d756b
Make .RenderString render shortcodes
Fixes #6703
2022-05-30 11:32:55 +02:00
Bjørn Erik Pedersen
d2cfaede5b
Improve shortcode indentation handling
* Record the leading whitespace (tabs, spaces) before the shortcode when parsing the page.
* Apply that indentation to the rendered result of shortcodes without inner content (where the user will apply indentation).

Fixes #9946
2022-05-30 11:32:55 +02:00
Bjørn Erik Pedersen
322d19a81f
Add Markdown as an output format
The motivation behind this is not to make it easier to publish Markdown files, as that sounds unusual.

This is mainly meant for shortcodes that produces Markdown to be inlined.

You would do this by creating shortcodes with `*.md` suffix (e.g. `layouts/shortcodes/myshortcode.md`).

This output format is defined as plain text, and will use Go's much more lenient text template parser.

Updates #9821
2022-05-30 11:32:55 +02:00
Bjørn Erik Pedersen
c1a83076bf
Add a shortcode benchmark 2022-05-29 15:21:08 +02:00
Bjørn Erik Pedersen
0f8dc47037 Remove Blackfriday markdown engine
It has been deprecated for a long time, its v1 version is not maintained anymore, and there are many known issues. Goldmark should be
a mature replacement by now.

Closes #9934
2022-05-29 11:50:58 +02:00
Bjørn Erik Pedersen
3b478f50b7 Fix HasMenuCurrent and IsDescendant/IsAncestor when comparing to itself
There may be sites in the wild that depends on the faulty behaviour of IsDescendant/IsAncestor when comparing to itself, but

* The documentation and common sense says that a thing cannot be descendant or ancestor to itself.
* The bug introduced in `HasMenuCurrent` comes directly from that confusion.

Fixes #9846
2022-05-28 10:56:54 +02:00
Bjørn Erik Pedersen
dd9eaf19fd Don't use the baseURL /path as part of the resource cache key
As that prevents Hugo projects with sub paths in their `baseURL` to use themes with cached resources.

Fixes #9787
2022-05-27 19:11:16 +02:00
Bjørn Erik Pedersen
3854a6fa6c Fix Plainify edge cases
This commit replaces the main part of `helpers.StripHTML` with Go's implementation in its html/template package.

It's a little slower, but correctness is more important:

```bash
BenchmarkStripHTMLOld-10    	  680316	      1764 ns/op	     728 B/op	       4 allocs/op
BenchmarkStripHTMLNew-10    	  384520	      3099 ns/op	    2089 B/op	      10 allocs/op
```

Fixes #9199
Fixes #9909
Closes #9410
2022-05-25 17:55:23 +02:00
satotake
6f7fbe03b1 basefs: add noBuildLock flag
If the flag is enabled, `.hugo_build.lock` will not be created.
This ensures safe running on read-only filesystem etc.
Close #9780
2022-05-24 13:57:23 +02:00
Bjørn Erik Pedersen
87a22eb6d6 server: Fix SIGINT handling after loading bad configuration
Also fix the config error messages.

Fixes #9664
2022-05-15 22:58:05 +02:00
Bjørn Erik Pedersen
fc9f315d86 Improve SASS errors
Fixes #9897
2022-05-15 20:25:25 +02:00
Bjørn Erik Pedersen
4b189d8fd9 postcss: Fix import error handling
Note that we will now fail if `inlineImports` is enabled and we cannot resolve an import.

You can work around this by either:

* Use url imports or imports with media queries.
* Set `skipInlineImportsNotFound=true` in the options

Also get the argument order in the different NewFileError* funcs in line.

Fixes #9895
2022-05-15 20:25:25 +02:00
Bjørn Erik Pedersen
5c96bda70a
errors: Misc improvements
* Redo the server error template
* Always add the content file context if relevant
* Remove some now superflous error string matching
* Move the server error template to _server/error.html
* Add file context (with position) to codeblock render blocks
* Improve JS build errors

Fixes #9892
Fixes #9891
Fixes #9893
2022-05-14 13:40:56 +02:00
Bjørn Erik Pedersen
9d7f166244
hugolib: Check for nil in shouldRender 2022-05-09 15:53:25 +02:00
Bjørn Erik Pedersen
35c88a7f90 Use configured timeZone for the clock
And some other related adjustments.

Updates #8787
2022-05-08 16:56:26 +02:00
satotake
e77ca3c105 Add clock cli flag
Close #8787
2022-05-08 16:56:26 +02:00
Bjørn Erik Pedersen
f2946da9e8 Improve error messages, esp. when the server is running
* Add file context to minifier errors when publishing
* Misc fixes (see issues)
* Allow custom server error template in layouts/server/error.html

To get to this, this commit also cleans up and simplifies the code surrounding errors and files. This also removes the usage of `github.com/pkg/errors`, mostly because of https://github.com/pkg/errors/issues/223 -- but also because most of this is now built-in to Go.

Fixes #9852
Fixes #9857
Fixes #9863
2022-05-06 19:43:22 +02:00
Bjørn Erik Pedersen
fa80fe3c8a Some godoc adjustments and image struct renames 2022-04-27 23:53:56 +02:00
Joe Mooring
a022ca271b deps: Update github.com/yuin/goldmark v1.4.11 => v1.4.12
Fixes #9054
Fixes #9756
Fixes #9757
2022-04-27 10:04:00 +02:00
Bjørn Erik Pedersen
05b45c35c8 tpl/lang: Handle nil values in lang.Merge 2022-04-23 22:22:50 +02:00
Bjørn Erik Pedersen
097fd588ca Deprecate page.Author and page.Authors
These are not documented, and they don't belong on Page. We should consider having author a first class citizen of Hugo, but as it is not it's better modelled as a taxonomy.
2022-04-23 22:22:50 +02:00
Bjørn Erik Pedersen
397fce5603
Fix MediaType when reading images from cache
Fixes #8931
2022-04-16 16:36:15 +02:00
Bjørn Erik Pedersen
627eed1d62
Make string sorting (e.g. ByTitle, ByLinkTitle and ByParam) language aware
Fixes #2180
2022-04-12 13:24:16 +02:00
Bjørn Erik Pedersen
30c2e54c25 Replace all usage of CopyOnWriteFs with OverlayFs
Fixes #9761
2022-04-10 13:49:31 +02:00
Bjørn Erik Pedersen
d070bdf10f
Rework the Destination filesystem to make --renderStaticToDisk work
See #9626
2022-04-08 13:26:17 +02:00
Bjørn Erik Pedersen
0a56f2af4e
Revert "Revert "Allow rendering static files to disk and dynamic to memory in server mode""
This reverts commit 64b7b7a897.
2022-04-08 13:26:16 +02:00
Bjørn Erik Pedersen
e58a540895 resources: Create a common ResourceFinder interface
And make both .Resources and resources implement it.

This gets us 2 new methods/functions, so you can now also do:

* .Resources.Get
* resources.ByType

Note that GetRemote is not covered by this interface, as that is only available as a global template function.

Fixes #8653
2022-04-05 18:00:44 +02:00
Bjørn Erik Pedersen
658e11ebaa Localize all the GroupBy*Date methods
Fixes #9745
2022-04-05 16:11:11 +02:00
Cathrine Paulsen
da00e7714e
Add environment as a new filter to _cascade.target
Fixes #9612
2022-04-05 09:41:24 +02:00
Sara Op den Orth
94459680ba Deprecate .File.Extension
Fixes #9352
2022-03-26 10:03:36 +01:00
Bjørn Erik Pedersen
9202117ba0 resources: Add more details to .Err
This commit adds a .Data object (a map with `Body`, `StatusCode` etc.) to the .Err returned from `resources.GetRemote`, which means you can now do:

```
{{ with .Err }}
{{ range $k, $v := .Data }}
{{ end }}
{{ end }}
```

Fixes #9708
2022-03-25 16:40:36 +01:00
cuishuang
48c98a8d24 Fix some typos
Signed-off-by: cuishuang <imcusg@gmail.com>
[foka@debian.org: Resolve merge conflict and squash 2 commits]
Signed-off-by: Anthony Fok <foka@debian.org>
2022-03-22 00:38:23 -06:00
Bjørn Erik Pedersen
1c0e7c1ae1 Make sure file mounts higher up wins
Fixes #9693
2022-03-20 20:23:03 +01:00
Bjørn Erik Pedersen
b80853de90
all: gofmt -w -r 'interface{} -> any' .
Updates #9687
2022-03-17 22:03:27 +01:00
Bjørn Erik Pedersen
423594e03a dartsass: Enable deprecation, @warn and @debug logging
* @warn and Sass deprecations are printed as WARN
* @debug is currently logged as INFO (needs the `--verbose` flag). We may adjust this if it gets too chatty.

Fixes #9683
2022-03-17 21:45:43 +01:00
Cathrine Paulsen
61cf3c9f63 Fix and refactor typos 2022-03-14 19:39:07 +01:00
Bjørn Erik Pedersen
1a796d723c deps: Fix Goldmark regression with HTML comments
Fixes #9650
2022-03-11 14:44:01 +01:00
Bjørn Erik Pedersen
64b7b7a897 Revert "Allow rendering static files to disk and dynamic to memory in server mode"
This reverts commit 7d8011ed63.

Updates #9647
2022-03-11 13:10:47 +01:00
Bjørn Erik Pedersen
4e14cf7607
Fail with error when double-rendering text in markdownify/RenderString
This commit prevents the most commons case of infinite recursion in link render hooks when the `linkify` option is enabled (see below). This is always a user error, but getting a `stack overflow` (the current stack limit in Go is 1 GB on 64-bit, 250 MB on 32-bit) error isn't very helpful. This fix will not prevent all such errors, though, but we may do better once #9570 is in place.

So, these will fail:

```
<a href="{{ .Destination | safeURL }}" >{{ .Text | markdownify }}</a>
<a href="{{ .Destination | safeURL }}" >{{ .Text | .Page.RenderString }}</a>
```

`.Text` is already rendered to `HTML`. The above needs to be rewritten to:

```
<a href="{{ .Destination | safeURL }}" >{{ .Text | safeHTML }}</a>
<a href="{{ .Destination | safeURL }}" >{{ .Text | safeHTML }}</a>
```

Fixes #8959
2022-03-10 08:19:03 +01:00
Joe Mooring
f98e570b17 Add lang attribute to internal alias template
Closes #9586
2022-03-09 19:06:59 +01:00
Bjørn Erik Pedersen
53a6210d82 markup/goldmark/codeblocks: Fix slice bounds out of range
For the Position in code blocks we try to match the .Inner with the original source. This isn't always possible.

This commits avoids panics in these situations.

Fixes #9627
2022-03-08 21:50:21 +01:00
SatowTakeshi
7d8011ed63 Allow rendering static files to disk and dynamic to memory in server mode
Updates #9625
2022-03-08 19:27:54 +01:00
Bjørn Erik Pedersen
0e0d672bcb Remove the decorator from the fs used in ReadDir
There have been a site breakage reported in the wild after 0.93.1. With this commit we shoudl be back to how it behaved in 0.92.2.

Closes #9609
2022-03-08 09:51:59 +01:00
Bjørn Erik Pedersen
673cde1eb1 tpl/os: Revert readDir in theme behaviour
Fixes #9599
2022-03-04 08:43:47 +01:00
Bjørn Erik Pedersen
cff14144a6
Rename Codeowners() to CodeOwners() 2022-02-27 19:40:07 +01:00
Bjørn Erik Pedersen
f7109771a0 CodeblockContext method renames
Fixes #9577
2022-02-27 17:59:36 +01:00
Bjørn Erik Pedersen
579ff9b652 markup/goldmark: Improve attributes vs options
Fixes #9571
2022-02-26 21:54:36 +01:00
Bjørn Erik Pedersen
928a896962 markup/goldmark: Add Position to CodeblockContext
But note that this is not particulary fast and the recommendad usage is error logging only.

Updates #9574
2022-02-26 21:54:36 +01:00
Bjørn Erik Pedersen
10928a4f78 Remove the trailing new line in .Code
Fixes #9572
2022-02-26 21:54:36 +01:00
RheingoldRiver
0df096b868 Update error message about failed menus in config.toml 2022-02-24 23:04:06 +01:00
Bjørn Erik Pedersen
08fdca9d93 Add Markdown diagrams and render hooks for code blocks
You can now create custom hook templates for code blocks, either one for all (`render-codeblock.html`) or for a given code language (e.g. `render-codeblock-go.html`).

We also used this new hook to add support for diagrams in Hugo:

* Goat (Go ASCII Tool) is built-in and enabled by default; just create a fenced code block with the language `goat` and start draw your Ascii diagrams.
* Another popular alternative for diagrams in Markdown, Mermaid (supported by GitHub), can also be implemented with a simple template. See the Hugo documentation for more information.

Updates #7765
Closes #9538
Fixes #9553
Fixes #8520
Fixes #6702
Fixes #9558
2022-02-24 18:59:50 +01:00
Marshall Cottrell
06bac57ab0 Add support for CODEOWNERS
Fixes #9474
2022-02-23 22:43:19 +01:00
Bjørn Erik Pedersen
e97d3c640f Add page.Store
Fixes #9546
2022-02-23 10:02:16 +01:00
Bjørn Erik Pedersen
aebde49b88 commands: Fix server panic regression
And now with a proper server test.

Fixes #9518
Fixes #9530
Fixes #9539
2022-02-21 21:43:26 +01:00
Bjørn Erik Pedersen
f4c90bd6ba
Fix BenchmarkCascadeTarget 2022-02-16 13:23:43 +01:00
Joe Mooring
923419d7fd deps: Update github.com/tdewolff/minify/v2 v2.9.22 => v2.9.29
Fixes #9244
Fixes #9132
Fixes https://discourse.gohugo.io/t/36523
2022-02-15 17:36:29 +01:00
Bjørn Erik Pedersen
ea54a99ca5 deps: Update github.com/alecthomas/chroma v0.9.4 => v0.10.0 2022-02-15 10:37:14 +01:00
Bjørn Erik Pedersen
9563c7d13b Finally remove deprecated Page methods
They have been deprecated for a very long time, first with a warning, then with an ERROR. Now they are removed.

Closes #4117
2022-02-14 13:24:36 +01:00
Bjørn Erik Pedersen
a7d182cea1
Add another cascade benchmark 2022-02-10 20:40:49 +01:00
Bjørn Erik Pedersen
f60714b5a1
Add a migration test helper 2022-02-09 16:38:12 +01:00
Bjørn Erik Pedersen
215a715ddd babel: Port integration tests to their own package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
d128d260b5 js: Port integration tests to its own package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
c4aaf1d516 postcss: Move integration test to its own package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
94f10cf4f9 minifier: Port integration tests to its package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
b06c2103b8 templates: Port integration test to its package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
d22f7795c7 tocss: Port integration tests to their package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
39f69ca7f0 openapi3: Port integration test into correct package 2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
64f75adcf6 Add a new integration test framework
I have had this living in a separate branch for now, but we need this in the main branch sooner rather than later.

One big advantage of this is that integration tests can live in any package, not just hugolib.
2022-02-09 15:41:32 +01:00
Bjørn Erik Pedersen
ef7d14a241 Fix erroneous warning with .Page.RenderString on a page without a backing file
Fixes #9433
2022-01-28 17:35:49 +01:00
Bjørn Erik Pedersen
f22c4aba04 Make the RenderString content provider fix more general
Updates #9383
2022-01-27 11:51:13 +01:00
Paul Gottschling
85d31f7bfb Fix .RenderString issue in .Translations
Fixes #9383
2022-01-27 11:51:13 +01:00
Bjørn Erik Pedersen
7a080b624e Fix duplicate mount sources
Fixes #9426
2022-01-26 20:32:13 +01:00
Bjørn Erik Pedersen
2655739940 tpl/collections: Fix apply with namespaced template funcs
We changed the signature to `func(...interface{}) (interface{}, error)` some time ago, but sadly we had no test for this for `apply`. Now we do.

Fixes #9393
2022-01-17 09:50:48 +01:00
Andreas Deininger
fcbbbef22d
Fix typo in warning message 2022-01-13 12:21:13 +01:00
Bjørn Erik Pedersen
cdcd15b6c2
Only create LazyContentProvider for the non-rendering Site
Which saves a fair amound of allocations:

```
gobench --package ./hugolib --bench "SiteNew/Regular_D" --base master
```

Before:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-10    40.7ms ± 3%    41.2ms ± 1%    ~     (p=0.343 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-10    27.7MB ± 0%    28.8MB ± 0%  +3.76%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-10      304k ± 0%      329k ± 0%  +8.07%  (p=0.029 n=4+4)
```

After:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-10    34.2ms ± 1%    34.7ms ± 1%    ~     (p=0.114 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-10    27.7MB ± 0%    28.1MB ± 0%  +1.38%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-10      304k ± 0%      314k ± 0%  +3.03%  (p=0.029 n=4+4)
```

Updates #8919
2022-01-12 07:45:53 +01:00
Paul Gottschling
25d645f47a
Fix missing page data for alternative formats
When a template calls the .Translations function and a
Hugo environment is using multiple output formats,
a template that calls methods like .Summary and .Len on
each translation will unexpectedly show empty return
values for these methods.

This is because each pageOutput's ContentProvider is
assigned to a page.NopPage in newPageOutput. When
*HugoSites.render assigns pageContentOutputs to
pageOutputs in *pageState.shiftToOutputFormat, it
reuses pageContentOutputs from other pageOutputs,
leaving some pageContentOutputs as NopPages. While this
approach conserves resources, sometimes it means that
a template will unexpectedly call a method on a
pageContentOutput that is actually a NopPage.

In the case of ContentProvider methods called on
translations for alternative output formats, the methods
were called on NopPages.

This change introduces LazyContentProvider, which
performs late initialization when one of its methods is
called. This way, we can reuse content in "normal" cases
but ensure that ContentProvider methods work as expected
when a pageOutput is not assigned a pageContentOutput
during the initial pre-render phase.

Fixes #8919
2022-01-12 07:45:53 +01:00
Bjørn Erik Pedersen
7396aa945a Add hugo.Deps
Fixes #8949
2022-01-11 18:06:23 +01:00
roointan
d82cef5c53
hugolib: Fix livereload problem with files including NFC characters in MacOs
Fixes #8429
2022-01-08 11:35:30 +01:00
Bjørn Erik Pedersen
1651beb2c1 Remove mmark
Closes #9350
2022-01-04 17:10:39 +01:00
Bjørn Erik Pedersen
2b6063c3e3 Misc depreation updates
* Deprecate .Page.Path when backed by a file
* site.Permalinks
* --ignoreVendor (use --ignoreVendorPaths)

Closes #9348
Closes #9349
2022-01-04 17:10:39 +01:00