Commit graph

581 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
7c9fada778 Replace the old log setup, with structured logging etc.
Fixes #11124
2023-06-18 13:03:04 +02:00
Erica Mays
258884f44f cache: Set default cache path based on $USER
Change the default cache directory to `$TMPDIR/hugo_cache_$USER`, so
that multi-user systems do not have caches that interfere with each
other.  The other cache-choosing logic (e.g. Netlify exceptions,
configuration options) are not affected.

Fixes #7391
2023-06-12 21:33:35 +02:00
Bjørn Erik Pedersen
ed7e250068 helpers: Avoid url.Parse in RelURL and AbsURL if we can
```
RelURL-10            159ns ± 5%      18ns ± 4%   -88.89%  (p=0.029 n=4+4)
AbsURL/relurl-10     532ns ± 0%     537ns ± 0%    +1.09%  (p=0.029 n=4+4)
AbsURL/absurl-10     142ns ± 0%       4ns ± 3%   -96.91%  (p=0.029 n=4+4)

name              old alloc/op   new alloc/op   delta
RelURL-10             144B ± 0%        0B       -100.00%  (p=0.029 n=4+4)
AbsURL/relurl-10      544B ± 0%      544B ± 0%      ~     (all equal)
AbsURL/absurl-10      144B ± 0%        0B       -100.00%  (p=0.029 n=4+4)

name              old allocs/op  new allocs/op  delta
RelURL-10             1.00 ± 0%      0.00       -100.00%  (p=0.029 n=4+4)
AbsURL/relurl-10      10.0 ± 0%      10.0 ± 0%      ~     (all equal)
AbsURL/absurl-10      1.00 ± 0%      0.00       -100.00%  (p=0.029 n=4+4)
```
2023-06-12 16:15:59 +02:00
Bjørn Erik Pedersen
6a09e7f28e
Adjust benchmarks 2023-06-12 16:06:11 +02:00
Bjørn Erik Pedersen
29e5cbb699
Adjust benchmark 2023-06-12 15:18:41 +02:00
Bjørn Erik Pedersen
ded6866001
Add BenchmarkAbsURL 2023-06-12 15:15:28 +02:00
Joe Mooring
e1d43021ca helpers: Improve schema detection when creating relative URLs
Fixes #11080
2023-06-12 15:01:30 +02:00
Bjørn Erik Pedersen
5db215d4d6
helpers: Add a basic benchmark for RelURL 2023-06-12 14:44:07 +02:00
Oleksandr Redko
610cedaa61 all: Fix comments for exported functions and packages 2023-05-18 21:25:27 +02:00
Oleksandr Redko
3d90871e9e helpers: simplify path tests with T.TempDir 2023-05-16 18:35:19 +02:00
Bjørn Erik Pedersen
241b21b0fd Create a struct with all of Hugo's config options
Primary motivation is documentation, but it will also hopefully simplify the code.

Also,

* Lower case the default output format names; this is in line with the custom ones (map keys) and how
it's treated all the places. This avoids doing `stringds.EqualFold` everywhere.

Closes #10896
Closes #10620
2023-05-16 18:01:29 +02:00
Bjørn Erik Pedersen
0fbab7cbc5
commands: Fix data race in test
Note that this is a test fix only.
2023-03-14 12:18:42 +01:00
Oleksandr Redko
36ce3a4a9d Correct typos in Go comments 2023-03-02 16:32:32 +01:00
Oleksandr Redko
d453c12742 Replace deprecated ioutil with io and os
https://pkg.go.dev/io/ioutil is deprecated since Go 1.16.
2023-03-01 16:28:43 +01:00
Bjørn Erik Pedersen
6abd15e781 Adjust tests for GO 1.20
Updates #10691
2023-02-22 11:26:52 +01:00
Bjørn Erik Pedersen
4ef9baf5bd Only invoke a given cached partial once
Note that this is backed by a LRU cache (which we soon shall see more usage of), so if you're a heavy user of cached partials it may be evicted and
refreshed if needed. But in most cases every partial is only invoked once.

This commit also adds a timeout (the global `timeout` config option) to make infinite recursion in partials
easier to reason about.

```
name              old time/op    new time/op    delta
IncludeCached-10    8.92ms ± 0%    8.48ms ± 1%   -4.87%  (p=0.016 n=4+5)

name              old alloc/op   new alloc/op   delta
IncludeCached-10    6.65MB ± 0%    5.17MB ± 0%  -22.32%  (p=0.002 n=6+6)

name              old allocs/op  new allocs/op  delta
IncludeCached-10      117k ± 0%       71k ± 0%  -39.44%  (p=0.002 n=6+6)
```

Closes #4086
Updates #9588
2023-01-25 17:35:23 +01:00
Joe Mooring
2d217cba51
helpers: Allow at signs in UnicodeSanitize (note)
Closes #10548
2022-12-19 09:58:56 +01:00
Bjørn Erik Pedersen
ad2059878a Also consider wrapped errors when checking for file IsNotExist errors
Fixes #10534
2022-12-14 13:51:06 +01:00
Bjørn Erik Pedersen
35fa192838 deps: Udpate to github.com/alecthomas/chroma/v2
Fixes #9932
Fixes #9931
2022-06-14 14:08:40 +02:00
Bjørn Erik Pedersen
a5a4422aae Fix relURL with leading slash when baseURL includes a subdirectory
Fixes #9994
2022-06-13 11:40:25 +02:00
Bjørn Erik Pedersen
3854a6fa6c Fix Plainify edge cases
This commit replaces the main part of `helpers.StripHTML` with Go's implementation in its html/template package.

It's a little slower, but correctness is more important:

```bash
BenchmarkStripHTMLOld-10    	  680316	      1764 ns/op	     728 B/op	       4 allocs/op
BenchmarkStripHTMLNew-10    	  384520	      3099 ns/op	    2089 B/op	      10 allocs/op
```

Fixes #9199
Fixes #9909
Closes #9410
2022-05-25 17:55:23 +02:00
Bjørn Erik Pedersen
cd0112a05a Add resources.Copy
Implemented by most Resource objects, but not Page (for now).

Fixes #9313
2022-05-25 10:35:31 +02:00
Bjørn Erik Pedersen
f2946da9e8 Improve error messages, esp. when the server is running
* Add file context to minifier errors when publishing
* Misc fixes (see issues)
* Allow custom server error template in layouts/server/error.html

To get to this, this commit also cleans up and simplifies the code surrounding errors and files. This also removes the usage of `github.com/pkg/errors`, mostly because of https://github.com/pkg/errors/issues/223 -- but also because most of this is now built-in to Go.

Fixes #9852
Fixes #9857
Fixes #9863
2022-05-06 19:43:22 +02:00
Bjørn Erik Pedersen
d070bdf10f
Rework the Destination filesystem to make --renderStaticToDisk work
See #9626
2022-04-08 13:26:17 +02:00
Bjørn Erik Pedersen
9539069f5e commands: Improve server startup to make tests less flaky
Do this by announcing/listen on the local address before we start the server.
2022-03-21 09:32:35 +01:00
Bjørn Erik Pedersen
b80853de90
all: gofmt -w -r 'interface{} -> any' .
Updates #9687
2022-03-17 22:03:27 +01:00
Bjørn Erik Pedersen
08fdca9d93 Add Markdown diagrams and render hooks for code blocks
You can now create custom hook templates for code blocks, either one for all (`render-codeblock.html`) or for a given code language (e.g. `render-codeblock-go.html`).

We also used this new hook to add support for diagrams in Hugo:

* Goat (Go ASCII Tool) is built-in and enabled by default; just create a fenced code block with the language `goat` and start draw your Ascii diagrams.
* Another popular alternative for diagrams in Markdown, Mermaid (supported by GitHub), can also be implemented with a simple template. See the Hugo documentation for more information.

Updates #7765
Closes #9538
Fixes #9553
Fixes #8520
Fixes #6702
Fixes #9558
2022-02-24 18:59:50 +01:00
Cameron Moore
6407b2cd01 helpers: Allow hyphens in UnicodeSanitize
Improve handling of existing hyphens in input to UnicodeSanitize.
This commit accomplishes three things:

1. Explicitly allow hyphens
2. Avoid appending a hyphen if a preceeding hyphen is found
3. Avoid prepending a hyphen if a trailing hyphen is found

Fixes #7288
2022-02-23 21:21:46 +01:00
Ephex2
22055176d2 general: Fix issue causing log threads to hang indefinitely when print() panics
The function printIfNotPrinted() defined for DistinctLogger unlocked the mutex within
the logger only after the print() function ran. If print panics, the mutex would stay
locked and future attempts to read or write from the logger mutex would cause the goroutine
to hang indefinitely.

Deferred the unlocking of the mutex to prevent this. Also, put l.m[key] before the print()
call since this will prevent another bug where the same warning potentially gets logged
multiple times if the print() call panics.

Fixes #9380
2022-01-27 10:38:50 +01:00
Bjørn Erik Pedersen
55a9bc1e70 helpers: Remove unused code 2022-01-16 17:12:56 +01:00
Bjørn Erik Pedersen
c8b5ab75b7 Add --panicOnWarning flag
Fixes #9357
Fixes #9359
2022-01-06 12:27:04 +01:00
Bjørn Erik Pedersen
1651beb2c1 Remove mmark
Closes #9350
2022-01-04 17:10:39 +01:00
Bjørn Erik Pedersen
f4389e48ce
Add some basic security policies with sensible defaults
This ommmit contains some security hardening measures for the Hugo build runtime.

There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers".

For `asciidoctor` and some others we use Go's `os/exec` package to start a new process.

These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off.

You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do.

The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all.

```toml
[security]
  enableInlineShortcodes = false
  [security.exec]
    allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
    osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

  [security.funcs]
    getenv = ['^HUGO_']

  [security.http]
    methods = ['(?i)GET|POST']
    urls = ['.*']
```
2021-12-16 09:40:22 +01:00
Bjørn Erik Pedersen
5e0947c5b5 helpers: Make UniqueStringsReuse allocation free
This should be faster for all of our use cases (small string slices), but more important, it makes UniqueStringsReuse allocation free.

```bash
name                                 old time/op    new time/op    delta
UniqueStrings/Safe-16                   776ns ± 0%     305ns ± 0%   -60.68%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice-16            621ns ± 3%     174ns ± 0%   -72.01%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice_sorted-16     319ns ± 4%     313ns ± 0%    -2.01%  (p=0.029 n=4+4)

name                                 old alloc/op   new alloc/op   delta
UniqueStrings/Safe-16                    224B ± 0%      128B ± 0%   -42.86%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice-16            96.0B ± 0%      0.0B       -100.00%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice_sorted-16     24.0B ± 0%     24.0B ± 0%      ~     (all equal)

name                                 old allocs/op  new allocs/op  delta
UniqueStrings/Safe-16                    7.00 ± 0%      1.00 ± 0%   -85.71%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice-16             6.00 ± 0%      0.00       -100.00%  (p=0.029 n=4+4)
UniqueStrings/Reuse_slice_sorted-16      1.00 ± 0%      1.00 ± 0%      ~     (all equal)
```
2021-11-17 10:44:03 +01:00
Bjørn Erik Pedersen
91cbb96302 Bump all long-living deprecations to ERRORs 2021-07-25 17:17:21 +02:00
Bjørn Erik Pedersen
022c479551
hugofs: Make FileMeta a struct
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See #8749
2021-07-15 17:14:26 +02:00
Bjørn Erik Pedersen
a70da2b74a
Fix the deprecation error/warn log levels 2021-07-15 12:24:20 +02:00
Bjørn Erik Pedersen
4c8552b114
Fix Cloudflare vs Netlify cache dir issue
Re-add the additional environment checks to determine if its Netlify. Seems that Cloudflare also sets `NETLIFY=true`.

This makes it look, basically, like a variant of the conditional we had before we started fixing this, but I have checked this logic on Netlify now and it should work.

Fixes #8714
2021-06-30 12:58:43 +02:00
Bjørn Erik Pedersen
6c8c0c8b6a
config: Fix Netlify default cache dir logic
Fixes #8710
2021-06-29 09:51:09 +02:00
Bjørn Erik Pedersen
93aad3c543
Split out the puthe path/filepath functions into common/paths
So they can be used from the config package without cyclic troubles.

Updates #8654
2021-06-18 10:55:00 +02:00
Bjørn Erik Pedersen
d392893cd7
Misc config loading fixes
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster:

```
BenchmarkDefaultConfigProvider/Viper-16         	  252418	      4546 ns/op	    2720 B/op	      30 allocs/op
BenchmarkDefaultConfigProvider/Custom-16        	  450756	      2651 ns/op	    1008 B/op	       6 allocs/op
```

Fixes #8633
Fixes #8618
Fixes #8630
Updates #8591
Closes #6680
Closes #5192
2021-06-14 17:00:32 +02:00
Bjørn Erik Pedersen
f55d2f4376
tpl/fmt: Add erroridf template func
Fixes #8613
2021-06-07 19:11:03 +02:00
Cameron Moore
1f7e9f7333 Update emoji import paths and version
Since emoji is v2, we must import it with the `/v2` suffix.  Doing so
allows the Go tools to detect the latest release.
2020-12-30 22:46:19 +01:00
Phil Davis
04b89857e1
all: Fix minor typos 2020-12-16 12:11:32 +01:00
Bjørn Erik Pedersen
d90e37e0c6 all: Format code with gofumpt
See https://github.com/mvdan/gofumpt
2020-12-03 13:12:58 +01:00
ivan-meridianbanc-com
aebfe156fb
Fix RelURL and AbsURL when path starts with language 2020-12-02 12:47:01 +01:00
Bjørn Erik Pedersen
fdfa4a5fe6 Allow getJSON errors to be ignored
This change is mostly motivated to get a more stable CI build (we're building the Hugo site there, with Instagram and Twitter shortcodes sometimes failing).

Fixes #7866
2020-10-22 09:09:29 +02:00
Derk Muenchhausen
f0266e2ef3
Rework external asciidoctor integration
This commit solves the relative path problem with asciidoctor tooling. An include will resolve relatively, so you can refer easily to files in the same folder.

Also `asciidoctor-diagram` and PlantUML rendering works now, because the created temporary files will be placed in the correct folder.

This patch covers just the Ruby version of asciidoctor. The old AsciiDoc CLI EOLs in Jan 2020, so this variant is removed from code.

The configuration is completely rewritten and now available in `config.toml` under the key `[markup.asciidocext]`:

```toml
[markup.asciidocext]
    extensions = ["asciidoctor-html5s", "asciidoctor-diagram"]
    workingFolderCurrent = true
    trace = true
    [markup.asciidocext.attributes]
        my-base-url = "https://example.com/"
        my-attribute-name = "my value"
```

- backends, safe-modes, and extensions are now whitelisted to the popular (ruby) extensions and valid values.
- the default for extensions is to not enable any, because they're all external dependencies so the build would break if the user didn't install them beforehand.
- the default backend is html5 because html5s is an external gem dependency.
- the default safe-mode is safe, explanations of the modes: https://asciidoctor.org/man/asciidoctor/
- the config is namespaced under asciidocext_config and the parser looks at asciidocext to allow a future native Go asciidoc.
- `uglyUrls=true` option and `--source` flag are supported
- `--destination` flag is required

Follow the updated documentation under `docs/content/en/content-management/formats.md`. 
  
This patch would be a breaking change, because you need to correct all your absolute include pathes to relative paths, so using relative paths must be configured explicitly by setting `workingFolderCurrent = true`.
2020-06-25 09:51:33 +02:00
Bjørn Erik Pedersen
a985efcecf Fix GetPage on section/bundle name overlaps
In the internal Radix we stored the directory based nodes without a traling slash, e.g. `/blog`.

The original motivation was probably to make it easy to do prefix searching: Give me all ancestors.

This, however have lead to some ambigouty with overlapping directory names.

This particular problem was, however, not possible to work around in an easy way, so from now we store these as `/blog/`.

Fixes #7301
2020-05-24 12:35:45 +02:00
SatowTakeshi
9c9987535f helpers: Fix TrimShortHTML
Where some tags are siblings of p tag.

Fixes #7081
2020-03-28 11:10:25 +01:00