Commit graph

16 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
f7aeaa6129 Add a consolidated file cache
This commits reworks how file caching is performed in Hugo. Now there is only one way, and it can be configured.

This is the default configuration:

```toml
[caches]
[caches.getjson]
dir = ":cacheDir"
maxAge = -1
[caches.getcsv]
dir = ":cacheDir"
maxAge = -1
[caches.images]
dir = ":resourceDir/_gen"
maxAge = -1
[caches.assets]
dir = ":resourceDir/_gen"
maxAge = -1
```

You can override any of these cache setting in your own `config.toml`.

The placeholders explained:

`:cacheDir`: This is the value of the `cacheDir` config option if set (can also be set via OS env variable `HUGO_CACHEDIR`). It will fall back to `/opt/build/cache/hugo_cache/` on Netlify, or a `hugo_cache` directory below the OS temp dir for the others.
`:resourceDir`: This is the value of the `resourceDir` config option.

`maxAge` is the time in seconds before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off.

This means that if you run your builds on Netlify, all caches configured with `:cacheDir` will be saved and restored on the next build. For other CI vendors, please read their documentation. For an CircleCI example, see 6c3960a8f4/.circleci/config.yml

Fixes #5404
2018-11-13 14:19:42 +01:00
Bjørn Erik Pedersen
b27ccf34bf resource: Allow .Data.Integrity to be accessed on its own
Fixes #5296
2018-10-27 11:34:15 +02:00
Bjørn Erik Pedersen
35fbfb19a1
commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes #5284
Fixes #5290
See #5325
See #5324
2018-10-16 22:10:56 +02:00
Bjørn Erik Pedersen
e27fd4c1b8 tpl/collections: Add collections.Append
Before this commit you would typically use `.Scratch.Add` to manually create slices in a loop.

With variable overwrite in Go 1.11, we can do better. This commit adds the `append` template func.

A made-up example:

```bash
{{ $p1 := index .Site.RegularPages 0 }}{{ $p2 := index .Site.RegularPages 1 }}
{{ $pages := slice }}
{{ if true }}
  {{ $pages = $pages | append $p2 $p1 }}
{{ end }}
```

Note that with 2 slices as arguments, the two examples below will give the same result:

```bash
{{ $s1 := slice "a" "b" | append (slice "c" "d") }}
{{ $s2 := slice "a" "b" | append "c" "d" }}
```

Both of the above will give `[]string{a, b, c, d}`.

This commit also improves the type handling in the `slice` template function. Now `slice "a" "b"` will give a `[]string` slice. The old behaviour was to return a `[]interface{}`.

Fixes #5190
2018-09-14 10:12:08 +02:00
Cameron Moore
b8b91f5506 common: Fix golint errors
common/errors/errors.go:21:1: comment on exported var FeatureNotAvailableErr should be of the form "FeatureNotAvailableErr ..."
common/errors/errors.go:23:5: error var FeatureNotAvailableErr should have name of the form ErrFoo
common/maps/scratch.go:76:1: comment on exported method Scratch.Delete should be of the form "Delete ..."
common/maps/scratch.go:133:1: exported function NewScratch should have comment or be unexported
common/types/types.go:44:1: exported function NewKeyValuesStrings should have comment or be unexported
2018-09-07 08:25:51 +02:00
Bjørn Erik Pedersen
c15c7da42a
resource: Remove some duplicate code 2018-09-02 23:57:42 +02:00
Anthony Fok
08d14113b6 resource: Add instruction to install PostCSS when missing
Fixes #5111
2018-08-26 15:22:58 +02:00
Bjørn Erik Pedersen
d32ff16fd6 Update Travis to Go 1.11 and Go 1.10.4
See #5115
2018-08-25 17:37:42 +02:00
Bjørn Erik Pedersen
78f8475a05 Fix Resource output in multihost setups
In Hugo 0.46 we made the output of what you get from resources.Get and similar static, i.e. language agnostic. This makes total sense, as it is wasteful and time-consuming to do SASS/SCSS/PostCSS processing for lots of languages when the output is lots of duplicates with different filenames.

But since we now output the result once only, this had a negative side effect for multihost setups: We publish the resource once only to the root folder (i.e. not to the language "domain folder").

This commit removes the language code from the processed image keys. This creates less duplication in the file cache, but it means that you should do a `hugo --gc` to clean up stale files.

Fixes #5058
2018-08-13 19:00:51 +02:00
Bjørn Erik Pedersen
789ef8c639
Add support for minification of final output
Hugo Pipes added minification support for resources fetched via ´resources.Get` and similar.

This also adds support for minification of the final output for supported output formats: HTML, XML, SVG, CSS, JavaScript, JSON.

To enable, run Hugo with the `--minify` flag:

```bash
hugo --minify
```

This commit is also a major spring cleaning of the `transform` package to allow the new minification step fit into that processing chain.

Fixes #1251
2018-08-06 19:58:41 +02:00
Bjørn Erik Pedersen
47d38628ec resource: Clean up the in-memory Resource reader usage
Turns out `strings.Reader` implements both `io.Reader` and `io.Seeker`, so we don't need anything special.

Updates #4936
2018-07-12 21:32:52 +02:00
Bjørn Erik Pedersen
0024dcfe3e resource: Move opening of the transformed resources after cache check
The old version should be functionally the same, but opening up these readers (potentially files) every time is wasteful if we don't read from them.
2018-07-12 19:36:12 +02:00
Bjørn Erik Pedersen
beec1fc98e resource: Fix resources.Concat for transformed resources
Fixes #4936
2018-07-12 19:36:12 +02:00
Bjørn Erik Pedersen
d96f2a460f
resource: Include the transformation step in the error message
Fixes #4924
2018-07-11 11:03:15 +02:00
Yang Li
3cea2932e1 Fix typos 2018-07-07 12:29:56 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00