Commit graph

24 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
241b21b0fd Create a struct with all of Hugo's config options
Primary motivation is documentation, but it will also hopefully simplify the code.

Also,

* Lower case the default output format names; this is in line with the custom ones (map keys) and how
it's treated all the places. This avoids doing `stringds.EqualFold` everywhere.

Closes #10896
Closes #10620
2023-05-16 18:01:29 +02:00
Anton Sattarov
a49e51fd0b
resources: Increase timeout for http.Client
Increase timeout for http.Client in resources.GetRemote  from 10 second to 1 minute

Fixes #10478
2022-12-03 15:47:47 +01:00
Bjørn Erik Pedersen
5c41653364 Consolidate the glob case logic
Looking at the code as a whole, we ended up with a little to much "buttons". It turns out that doing case insensitive matching (lower both pattern and strings to match) performs just fine. Or at least, it
gives the penalty to the people who uses mixed case filenames.

```
GetGlob/Default_cache-10                          10.6ns ± 2%    10.6ns ± 1%   ~     (p=0.657 n=4+4)
GetGlob/Filenames_cache,_lowercase_searchs-10     10.6ns ± 2%    10.6ns ± 0%   ~     (p=1.000 n=4+4)
GetGlob/Filenames_cache,_mixed_case_searchs-10    29.7ns ± 1%    29.6ns ± 1%   ~     (p=0.886 n=4+4)
GetGlob/GetGlob-10                                13.7ns ± 1%    13.7ns ± 0%   ~     (p=0.429 n=4+4)

name                                            old alloc/op   new alloc/op   delta
GetGlob/Default_cache-10                           0.00B          0.00B        ~     (all equal)
GetGlob/Filenames_cache,_lowercase_searchs-10      0.00B          0.00B        ~     (all equal)
GetGlob/Filenames_cache,_mixed_case_searchs-10     5.00B ± 0%     5.00B ± 0%   ~     (all equal)
GetGlob/GetGlob-10                                 0.00B          0.00B        ~     (all equal)

name                                            old allocs/op  new allocs/op  delta
GetGlob/Default_cache-10                            0.00           0.00        ~     (all equal)
GetGlob/Filenames_cache,_lowercase_searchs-10       0.00           0.00        ~     (all equal)
GetGlob/Filenames_cache,_mixed_case_searchs-10      1.00 ± 0%      1.00 ± 0%   ~     (all equal)
GetGlob/GetGlob-10
```
2022-09-23 16:19:21 +02:00
satotake
281554ee97
hugofs: Fix glob case-sensitivity bug
On Linux, `hugofs.Glob` does not hit any directories which includes
uppercase letters. (This does not happen on macOS.)

Since `resources.GetMatch/Match` uses `Glob`,

```
{{ resources.GetMatch "Foo/bar.css" }}
```

this does not match `assets/Foo/bar.css` .

On the other hand, you can get it with

```
{{ resources.Get "Foo/bar.css" }}
```
2022-09-23 13:12:57 +02:00
Bjørn Erik Pedersen
cd0112a05a Add resources.Copy
Implemented by most Resource objects, but not Page (for now).

Fixes #9313
2022-05-25 10:35:31 +02:00
Bjørn Erik Pedersen
e58a540895 resources: Create a common ResourceFinder interface
And make both .Resources and resources implement it.

This gets us 2 new methods/functions, so you can now also do:

* .Resources.Get
* resources.ByType

Note that GetRemote is not covered by this interface, as that is only available as a global template function.

Fixes #8653
2022-04-05 18:00:44 +02:00
Bjørn Erik Pedersen
22ef5da20d
Add resources.GetRemote
In Hugo 0.89 we added remote support to `resources.Get`.

In hindsight that was not a great idea, as a poll from many Hugo users showed. See Issue #9285 for more details.

After this commit `resources.Get` only supports local resource lookups. If you want to support both, you need to use a construct similar to:

Also improve some option case handling.

```
{{ resource := "" }}
{{ if (urls.Parse $url).IsAbs }}
{{ $resource = resources.GetRemote $url }}
{{ else }}
{{ $resource = resources.Get $url }}
{{ end }}
```

Fixes #9285
Fixes #9296
2021-12-17 09:33:51 +01:00
Bjørn Erik Pedersen
f4389e48ce
Add some basic security policies with sensible defaults
This ommmit contains some security hardening measures for the Hugo build runtime.

There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers".

For `asciidoctor` and some others we use Go's `os/exec` package to start a new process.

These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off.

You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do.

The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all.

```toml
[security]
  enableInlineShortcodes = false
  [security.exec]
    allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
    osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

  [security.funcs]
    getenv = ['^HUGO_']

  [security.http]
    methods = ['(?i)GET|POST']
    urls = ['.*']
```
2021-12-16 09:40:22 +01:00
Joe Mooring
a037be774d
Improve handling of remote image/jpeg resources (#9278)
Add jpe, jif, and jfif to image/jpeg extensions.
For remote image/jpeg without extension, always use jpg extension.

Closes #9275
2021-12-13 08:55:15 +01:00
Bjørn Erik Pedersen
e4d6ec94b5 Allow user to handle/ignore errors in resources.Get
In Hugo 0.90.0 we introduced remote support in `resources.Get`.

But with remote resources comes with a higher chance of failing a build (network issues, remote server down etc.).

Before this commit we always failed the build on any unexpected error.

This commit allows the user to check for any error (and potentially fall back to a default local resource):

```htmlbars
{{ $result := resources.Get "https://gohugo.io/img/hugo-logo.png" }}
{{ with $result }}
        {{ if .Err }}
        {{/* log the error, insert a default image etc. *}}
        {{ else }}
        <img src="{{ .RelPermalink }}" width="{{ .Width }}" height="{{ .Height }}" alt="">
        {{ end }}
{{ end }}
```

Note that the default behaviour is still to fail the build, but we will delay that error until you start using the `Resource`.

Fixes #9529
2021-12-10 11:10:41 +01:00
Bjørn Erik Pedersen
6260455ba7 Make resource.Get return nil on 404 not found
This is in line with the interface declaration and also how local lookups work.

Fixes #9267
2021-12-10 11:10:41 +01:00
Bjørn Erik Pedersen
965a6cbff9 Update to Go 1.17.4 and remove timeout in resources.Get
Fixes #9265
2021-12-09 13:33:30 +01:00
Bjørn Erik Pedersen
93572e5318 resources: Add timeout to the HTTP request in Get
Workaround for https://github.com/golang/go/issues/49366
2021-12-02 16:11:14 +01:00
Bjørn Erik Pedersen
94f149b21e Add a remote retry for resources.Get 2021-12-02 16:11:14 +01:00
Paul van Brouwershaven
66753416b5
Make resources.Get use a file cache for remote resources
Closes #9228
2021-12-02 12:56:25 +01:00
Bjørn Erik Pedersen
08a863e1e8
resources: Adjust the remote Get cache so it does not get evicted on restarts
We fill improve this, soon.
2021-11-30 16:54:01 +01:00
Paul van Brouwershaven
8aa7257f65
Add remote support to resources.Get
Closes #5255
Supports #9044
2021-11-30 11:49:51 +01:00
Bjørn Erik Pedersen
022c479551
hugofs: Make FileMeta a struct
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See #8749
2021-07-15 17:14:26 +02:00
Bjørn Erik Pedersen
d90e37e0c6 all: Format code with gofumpt
See https://github.com/mvdan/gofumpt
2020-12-03 13:12:58 +01:00
Bjørn Erik Pedersen
1089cfe4e1
Add FileInfo to resources created with resources.Match etc.
Without it, image resize fails.

See #6190
2019-08-13 18:45:08 +02:00
Bjørn Erik Pedersen
cd575023af Improve the server assets cache invalidation logic
Fixes #6199
2019-08-13 18:09:46 +02:00
Bjørn Erik Pedersen
b64617fe4f
Add resources.Match and resources.GetMatch
Fix #6190
2019-08-13 11:44:20 +02:00
Bjørn Erik Pedersen
9f5a92078a
Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092
2019-07-24 09:35:53 +02:00
Bjørn Erik Pedersen
ce8a09a4c0 resources: Move resource interfaces into its own package 2019-01-02 14:25:37 +01:00
Renamed from resource/resource_factories/create/create.go (Browse further)