Commit graph

69 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
7203a95a60 Fix rebuilds when running hugo -w
This was partly broken in Hugo 0.123.0.

We have two internal config options that gets set from the CLI:

* Running; a web server is running
* Watching; either set via `hugo -w`  or `hugo server --watch=false`

Part of the change detection code wrongly used the `Running` as a flag when `Watching` would be the correct.

Fixes #12296
2024-04-25 14:35:49 +02:00
Christian Oliff
78178d0c2a
all: Typo fixes 2024-03-15 17:25:52 +01:00
Bjørn Erik Pedersen
7285e74090
all: Rework page store, add a dynacache, improve partial rebuilds, and some general spring cleaning
There are some breaking changes in this commit, see #11455.

Closes #11455
Closes #11549

This fixes a set of bugs (see issue list) and it is also paying some technical debt accumulated over the years. We now build with Staticcheck enabled in the CI build.

The performance should be about the same as before for regular sized Hugo sites, but it should perform and scale much better to larger data sets, as objects that uses lots of memory (e.g. rendered Markdown, big JSON files read into maps with transform.Unmarshal etc.) will now get automatically garbage collected if needed. Performance on partial rebuilds when running the server in fast render mode should be the same, but the change detection should be much more accurate.

A list of the notable new features:

* A new dependency tracker that covers (almost) all of Hugo's API and is used to do fine grained partial rebuilds when running the server.
* A new and simpler tree document store which allows fast lookups and prefix-walking in all dimensions (e.g. language) concurrently.
* You can now configure an upper memory limit allowing for much larger data sets and/or running on lower specced PCs.
We have lifted the "no resources in sub folders" restriction for branch bundles (e.g. sections).
Memory Limit
* Hugos will, by default, set aside a quarter of the total system memory, but you can set this via the OS environment variable HUGO_MEMORYLIMIT (in gigabytes). This is backed by a partitioned LRU cache used throughout Hugo. A cache that gets dynamically resized in low memory situations, allowing Go's Garbage Collector to free the memory.

New Dependency Tracker: Hugo has had a rule based coarse grained approach to server rebuilds that has worked mostly pretty well, but there have been some surprises (e.g. stale content). This is now revamped with a new dependency tracker that can quickly calculate the delta given a changed resource (e.g. a content file, template, JS file etc.). This handles transitive relations, e.g. $page -> js.Build -> JS import, or $page1.Content -> render hook -> site.GetPage -> $page2.Title, or $page1.Content -> shortcode -> partial -> site.RegularPages -> $page2.Content -> shortcode ..., and should also handle changes to aggregated values (e.g. site.Lastmod) effectively.

This covers all of Hugo's API with 2 known exceptions (a list that may not be fully exhaustive):

Changes to files loaded with template func os.ReadFile may not be handled correctly. We recommend loading resources with resources.Get
Changes to Hugo objects (e.g. Page) passed in the template context to lang.Translate may not be detected correctly. We recommend having simple i18n templates without too much data context passed in other than simple types such as strings and numbers.
Note that the cachebuster configuration (when A changes then rebuild B) works well with the above, but we recommend that you revise that configuration, as it in most situations should not be needed. One example where it is still needed is with TailwindCSS and using changes to hugo_stats.json to trigger new CSS rebuilds.

Document Store: Previously, a little simplified, we split the document store (where we store pages and resources) in a tree per language. This worked pretty well, but the structure made some operations harder than they needed to be. We have now restructured it into one Radix tree for all languages. Internally the language is considered to be a dimension of that tree, and the tree can be viewed in all dimensions concurrently. This makes some operations re. language simpler (e.g. finding translations is just a slice range), but the idea is that it should also be relatively inexpensive to add more dimensions if needed (e.g. role).

Fixes #10169
Fixes #10364
Fixes #10482
Fixes #10630
Fixes #10656
Fixes #10694
Fixes #10918
Fixes #11262
Fixes #11439
Fixes #11453
Fixes #11457
Fixes #11466
Fixes #11540
Fixes #11551
Fixes #11556
Fixes #11654
Fixes #11661
Fixes #11663
Fixes #11664
Fixes #11669
Fixes #11671
Fixes #11807
Fixes #11808
Fixes #11809
Fixes #11815
Fixes #11840
Fixes #11853
Fixes #11860
Fixes #11883
Fixes #11904
Fixes #7388
Fixes #7425
Fixes #7436
Fixes #7544
Fixes #7882
Fixes #7960
Fixes #8255
Fixes #8307
Fixes #8863
Fixes #8927
Fixes #9192
Fixes #9324
2024-01-27 16:28:14 +01:00
Bjørn Erik Pedersen
5160c7efa5 tpl/debug: Add debug.Timer
Closes #11580
2023-10-20 09:46:45 +02:00
Bjørn Erik Pedersen
7c9fada778 Replace the old log setup, with structured logging etc.
Fixes #11124
2023-06-18 13:03:04 +02:00
Bjørn Erik Pedersen
fd099331ec Fix Processed images count regression for multiple languages
Fixes #11002
2023-05-28 12:55:44 +02:00
Oleksandr Redko
610cedaa61 all: Fix comments for exported functions and packages 2023-05-18 21:25:27 +02:00
Bjørn Erik Pedersen
241b21b0fd Create a struct with all of Hugo's config options
Primary motivation is documentation, but it will also hopefully simplify the code.

Also,

* Lower case the default output format names; this is in line with the custom ones (map keys) and how
it's treated all the places. This avoids doing `stringds.EqualFold` everywhere.

Closes #10896
Closes #10620
2023-05-16 18:01:29 +02:00
Bjørn Erik Pedersen
9906c1ae52
Prevent the global error collector to panic when sending on closed channel 2023-04-13 11:44:22 +02:00
Bjørn Erik Pedersen
6c798eba60 Page context handling in i18n
This is a workaround. We need to improve on this, but not today.

Fixes #10782
2023-03-04 17:19:14 +01:00
Bjørn Erik Pedersen
4ef9baf5bd Only invoke a given cached partial once
Note that this is backed by a LRU cache (which we soon shall see more usage of), so if you're a heavy user of cached partials it may be evicted and
refreshed if needed. But in most cases every partial is only invoked once.

This commit also adds a timeout (the global `timeout` config option) to make infinite recursion in partials
easier to reason about.

```
name              old time/op    new time/op    delta
IncludeCached-10    8.92ms ± 0%    8.48ms ± 1%   -4.87%  (p=0.016 n=4+5)

name              old alloc/op   new alloc/op   delta
IncludeCached-10    6.65MB ± 0%    5.17MB ± 0%  -22.32%  (p=0.002 n=6+6)

name              old allocs/op  new allocs/op  delta
IncludeCached-10      117k ± 0%       71k ± 0%  -39.44%  (p=0.002 n=6+6)
```

Closes #4086
Updates #9588
2023-01-25 17:35:23 +01:00
Bjørn Erik Pedersen
29ccb36069 Fix /static performance regression from Hugo 0.103.0
In `v0.103.0` we added support for `resources.PostProcess` for all file types, not just HTML. We had benchmarks that said we were fine in that department, but those did not consider the static file syncing.

This fixes that by:

* Making sure that the /static syncer always gets its own file system without any checks for the post process token.
* For dynamic files (e.g. rendered HTML files) we add an additional check to make sure that we skip binary files (e.g. images)

Fixes #10328
2022-09-26 19:02:25 +02:00
Bjørn Erik Pedersen
74daca6b30 Support PostProcess for all file types
Not just HTML.

Fixes #10269
2022-09-14 19:09:20 +02:00
Bjørn Erik Pedersen
f2946da9e8 Improve error messages, esp. when the server is running
* Add file context to minifier errors when publishing
* Misc fixes (see issues)
* Allow custom server error template in layouts/server/error.html

To get to this, this commit also cleans up and simplifies the code surrounding errors and files. This also removes the usage of `github.com/pkg/errors`, mostly because of https://github.com/pkg/errors/issues/223 -- but also because most of this is now built-in to Go.

Fixes #9852
Fixes #9857
Fixes #9863
2022-05-06 19:43:22 +02:00
Bjørn Erik Pedersen
b80853de90
all: gofmt -w -r 'interface{} -> any' .
Updates #9687
2022-03-17 22:03:27 +01:00
Bjørn Erik Pedersen
f4389e48ce
Add some basic security policies with sensible defaults
This ommmit contains some security hardening measures for the Hugo build runtime.

There are some rarely used features in Hugo that would be good to have disabled by default. One example would be the "external helpers".

For `asciidoctor` and some others we use Go's `os/exec` package to start a new process.

These are a predefined set of binary names, all loaded from `PATH` and with a predefined set of arguments. Still, if you don't use `asciidoctor` in your project, you might as well have it turned off.

You can configure your own in the new `security` configuration section, but the defaults are configured to create a minimal amount of site breakage. And if that do happen, you will get clear instructions in the loa about what to do.

The default configuration is listed below. Note that almost all of these options are regular expression _whitelists_ (a string or a slice); the value `none` will block all.

```toml
[security]
  enableInlineShortcodes = false
  [security.exec]
    allow = ['^dart-sass-embedded$', '^go$', '^npx$', '^postcss$']
    osEnv = ['(?i)^(PATH|PATHEXT|APPDATA|TMP|TEMP|TERM)$']

  [security.funcs]
    getenv = ['^HUGO_']

  [security.http]
    methods = ['(?i)GET|POST']
    urls = ['.*']
```
2021-12-16 09:40:22 +01:00
Bjørn Erik Pedersen
9185e11eff Reimplement archetypes
The old implementation had some issues, mostly related to the context (e.g. name, file paths) passed to the template.

This new implementation is using the exact same code path for evaluating the pages as in a regular build.

This also makes it more robust and easier to reason about in a multilingual setup.

Now, if you are explicit about the target path, Hugo will now always pick the correct mount and language:

```bash
hugo new content/en/posts/my-first-post.md
```

Fixes #9032
Fixes #7589
Fixes #9043
Fixes #9046
Fixes #9047
2021-10-16 15:22:03 +02:00
Bjørn Erik Pedersen
f55d2f4376
tpl/fmt: Add erroridf template func
Fixes #8613
2021-06-07 19:11:03 +02:00
Bjørn Erik Pedersen
cea1574023
Add Dart Sass support
But note that the Dart Sass Embedded Protocol is still in beta (beta 5), a main release scheduled for Q1 2021.

Fixes #7380
Fixes #8102
2020-12-30 17:32:25 +01:00
Bjørn Erik Pedersen
d90e37e0c6 all: Format code with gofumpt
See https://github.com/mvdan/gofumpt
2020-12-03 13:12:58 +01:00
Bjørn Erik Pedersen
85e4dd7370 Make js.Build fully support modules
Fixes #7816
Fixes #7777
Fixes #7916
2020-11-03 13:04:37 +01:00
Bjørn Erik Pedersen
fdfa4a5fe6 Allow getJSON errors to be ignored
This change is mostly motivated to get a more stable CI build (we're building the Hugo site there, with Instagram and Twitter shortcodes sometimes failing).

Fixes #7866
2020-10-22 09:09:29 +02:00
Bjørn Erik Pedersen
97987e5c02 langs/i18n: Upgrade to go-i18n v2
Fixes #5242
2020-09-29 17:48:07 +02:00
Bjørn Erik Pedersen
4055c12184 Fix some change detection issues on server reloads
* Fix change detection when .GetPage/site.GetPage is used from shortcode
* Fix stale content for GetPage results with short name lookups on server reloads

Fixes #7623
Fixes #7624
Fixes #7625
2020-09-07 21:06:44 +02:00
Bjørn Erik Pedersen
2f721f8ec6
Add basic "post resource publish support"
Fixes #7146
2020-04-07 21:59:20 +02:00
Bjørn Erik Pedersen
df298558a5
Improve Tailwind/PostCSS error messages
Fixes #7041
Fixes #7042
2020-03-11 14:13:03 +01:00
Bjørn Erik Pedersen
ffcb4aeb8e Fix handling of HTML files without front matter
This means that any HTML file inside /content will be treated as a regular file.

If you want it processes with shortcodes and a layout, add front matter.

The defintion of an HTML file here is:

* File with extension .htm or .html
* With first non-whitespace character "<" that isn't a HTML comment.

This is in line with the documentation.

Fixes #7030
Fixes #7028
See #6789
2020-03-09 15:01:55 +01:00
Bjørn Erik Pedersen
0df7bd62df
deps: Make the build flags shared between sites
Updates #6789
2020-01-23 20:11:53 +01:00
Bjørn Erik Pedersen
c6d650c8c8
tpl/tplimpl: Rework template management to get rid of concurrency issues
This more or less completes the simplification of the template handling code in Hugo started in v0.62.

The main motivation was to fix a long lasting issue about a crash in HTML content files  without front matter.

But this commit also comes with a big functional improvement.

As we now have moved the base template evaluation to the build stage we now use the same lookup rules for `baseof` as for `list` etc. type of templates.

This means that in this simple example you can have a `baseof` template for the `blog` section without having to duplicate the others:

```
layouts
├── _default
│   ├── baseof.html
│   ├── list.html
│   └── single.html
└── blog
    └── baseof.html
```

Also, when simplifying code, you often get rid of some double work, as shown in the "site building" benchmarks below.

These benchmarks looks suspiciously good, but I have repeated the below with ca. the same result. Compared to master:

```
name                              old time/op    new time/op    delta
SiteNew/Bundle_with_image-16        13.1ms ± 1%    10.5ms ± 1%  -19.34%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16    13.0ms ± 0%    10.7ms ± 1%  -18.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16      46.4ms ± 2%    43.1ms ± 1%   -7.15%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16            52.2ms ± 2%    47.8ms ± 1%   -8.30%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16        77.9ms ± 1%    70.9ms ± 1%   -9.01%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16      43.0ms ± 0%    37.2ms ± 1%  -13.54%  (p=0.029 n=4+4)
SiteNew/Page_collections-16         58.2ms ± 1%    52.4ms ± 1%   -9.95%  (p=0.029 n=4+4)

name                              old alloc/op   new alloc/op   delta
SiteNew/Bundle_with_image-16        3.81MB ± 0%    2.22MB ± 0%  -41.70%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16    3.60MB ± 0%    2.01MB ± 0%  -44.20%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16      19.3MB ± 1%    14.1MB ± 0%  -26.91%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16            70.7MB ± 0%    69.0MB ± 0%   -2.40%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16        37.1MB ± 0%    31.2MB ± 0%  -15.94%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16      17.6MB ± 0%    10.6MB ± 0%  -39.92%  (p=0.029 n=4+4)
SiteNew/Page_collections-16         25.9MB ± 0%    21.2MB ± 0%  -17.99%  (p=0.029 n=4+4)

name                              old allocs/op  new allocs/op  delta
SiteNew/Bundle_with_image-16         52.3k ± 0%     26.1k ± 0%  -50.18%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16     52.3k ± 0%     26.1k ± 0%  -50.16%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16        336k ± 1%      269k ± 0%  -19.90%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16              422k ± 0%      395k ± 0%   -6.43%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16          401k ± 0%      313k ± 0%  -21.79%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16        247k ± 0%      143k ± 0%  -42.17%  (p=0.029 n=4+4)
SiteNew/Page_collections-16           282k ± 0%      207k ± 0%  -26.55%  (p=0.029 n=4+4)
```

Fixes #6716
Fixes #6760
Fixes #6768
Fixes #6778
2020-01-22 09:39:49 +01:00
Bjørn Erik Pedersen
a03c631c42
Rework template handling for function and map lookups
This is a big commit, but it deletes lots of code and simplifies a lot.

* Resolving the template funcs at execution time means we don't have to create template clones per site
* Having a custom map resolver means that we can remove the AST lower case transformation for the special lower case Params map

Not only is the above easier to reason about, it's also faster, especially if you have more than one language, as in the benchmark below:

```
name                          old time/op    new time/op    delta
SiteNew/Deep_content_tree-16    53.7ms ± 0%    48.1ms ± 2%  -10.38%  (p=0.029 n=4+4)

name                          old alloc/op   new alloc/op   delta
SiteNew/Deep_content_tree-16    41.0MB ± 0%    36.8MB ± 0%  -10.26%  (p=0.029 n=4+4)

name                          old allocs/op  new allocs/op  delta
SiteNew/Deep_content_tree-16      481k ± 0%      410k ± 0%  -14.66%  (p=0.029 n=4+4)
```

This should be even better if you also have lots of templates.

Closes #6594
2019-12-12 10:04:35 +01:00
Bjørn Erik Pedersen
5f6b6ec689
Prepare for Goldmark
This commmit prepares for the addition of Goldmark as the new Markdown renderer in Hugo.

This introduces a new `markup` package with some common interfaces and each implementation in its own package.

See #5963
2019-11-06 19:09:08 +01:00
Bjørn Erik Pedersen
e5f2299741 Block symlink dir traversal for /static
This is in line with how it behaved before, but it was lifted a little for the project mount for Hugo Modules,
but that could create hard-to-detect loops.
2019-07-25 11:27:25 +02:00
Bjørn Erik Pedersen
9f5a92078a
Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092
2019-07-24 09:35:53 +02:00
Bjørn Erik Pedersen
4dae52af68 Avoid nilpointer on no File on Page
Fixes #5781
2019-03-26 10:20:40 +01:00
Bjørn Erik Pedersen
597e418cb0
Make Page an interface
The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct.
This is all a preparation step for issue  #5074, "pages from other data sources".

But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes.

Most notable changes:

* The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday.
  This means that any markdown will partake in the global ToC and footnote context etc.
* The Custom Output formats are now "fully virtualized". This removes many of the current limitations.
* The taxonomy list type now has a reference to the `Page` object.
  This improves the taxonomy template `.Title` situation and make common template constructs much simpler.

See #5074
Fixes #5763
Fixes #5758
Fixes #5090
Fixes #5204
Fixes #4695
Fixes #5607
Fixes #5707
Fixes #5719
Fixes #3113
Fixes #5706
Fixes #5767
Fixes #5723
Fixes #5769
Fixes #5770
Fixes #5771
Fixes #5759
Fixes #5776
Fixes #5777
Fixes #5778
2019-03-23 18:51:22 +01:00
Bjørn Erik Pedersen
ce8a09a4c0 resources: Move resource interfaces into its own package 2019-01-02 14:25:37 +01:00
Bjørn Erik Pedersen
822dc627a1
tpl/transform: Add transform.Unmarshal func
Fixes #5428
2018-12-23 10:02:42 +01:00
Bjørn Erik Pedersen
831d23cb4d Add tpl/site and tpl/hugo
This means that the current `.Site` and ´.Hugo` is available as a globals, so you can do `site.IsServer`, `hugo.Version` etc.

Fixes #5470
Fixes #5467
Fixes #5503
2018-12-06 14:37:25 +01:00
Bjørn Erik Pedersen
f9b4eb4f39 Handle themes in the new file cache (for images, assets)
In the newly consolidated file cache implementation, we forgot that we also look in the theme(s) for assets (SCSS transformations etc.), which is not good for Netlify and the demo sites.

Fixes #5460
2018-11-23 21:09:00 +01:00
Bjørn Erik Pedersen
f7aeaa6129 Add a consolidated file cache
This commits reworks how file caching is performed in Hugo. Now there is only one way, and it can be configured.

This is the default configuration:

```toml
[caches]
[caches.getjson]
dir = ":cacheDir"
maxAge = -1
[caches.getcsv]
dir = ":cacheDir"
maxAge = -1
[caches.images]
dir = ":resourceDir/_gen"
maxAge = -1
[caches.assets]
dir = ":resourceDir/_gen"
maxAge = -1
```

You can override any of these cache setting in your own `config.toml`.

The placeholders explained:

`:cacheDir`: This is the value of the `cacheDir` config option if set (can also be set via OS env variable `HUGO_CACHEDIR`). It will fall back to `/opt/build/cache/hugo_cache/` on Netlify, or a `hugo_cache` directory below the OS temp dir for the others.
`:resourceDir`: This is the value of the `resourceDir` config option.

`maxAge` is the time in seconds before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off.

This means that if you run your builds on Netlify, all caches configured with `:cacheDir` will be saved and restored on the next build. For other CI vendors, please read their documentation. For an CircleCI example, see 6c3960a8f4/.circleci/config.yml

Fixes #5404
2018-11-13 14:19:42 +01:00
Bjørn Erik Pedersen
d1661b823a
hugolib: Continue the file context/line number errors work
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen
35fbfb19a1
commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes #5284
Fixes #5290
See #5325
See #5324
2018-10-16 22:10:56 +02:00
Cameron Moore
a53f962312 deps: Fix most golint issues
deps/deps.go:79:6: exported type Listeners should have comment or be unexported
deps/deps.go:86:1: exported method Listeners.Add should have comment or be unexported
deps/deps.go:92:1: exported method Listeners.Notify should have comment or be unexported
2018-09-07 08:25:51 +02:00
Bjørn Erik Pedersen
789ef8c639
Add support for minification of final output
Hugo Pipes added minification support for resources fetched via ´resources.Get` and similar.

This also adds support for minification of the final output for supported output formats: HTML, XML, SVG, CSS, JavaScript, JSON.

To enable, run Hugo with the `--minify` flag:

```bash
hugo --minify
```

This commit is also a major spring cleaning of the `transform` package to allow the new minification step fit into that processing chain.

Fixes #1251
2018-08-06 19:58:41 +02:00
Bjørn Erik Pedersen
6b02f5c0f4 Make resources fetched via resources.Get and similar language agnostic
With the newly released Hugo Pipes, resources fetched and processed via `resources.Get` and similar was published to the relevant language sub folder when in multilingual mode.

The thought behind that was maximum flexibility with support for `assetDir` per language.

In practice this was a bad idea:

* You get duplication of identical content, with added processing time
* You end up with path issues that seem to be hard to find a way around (`@fa-font-path` is one example)

This commit changes that. Now there is only one `assetDir` and if you, as one example, need to generate a CSS per langugage, you need to set the paths yourself.

Fixes #5017
2018-07-31 23:33:04 +02:00
Bjørn Erik Pedersen
6b6dcb44a0 Flush partialCached cache on rebuilds
Fixes #4931
2018-07-11 20:40:04 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
80230f26a3
Add support for theme composition and inheritance
This commit adds support for theme composition and inheritance in Hugo.

With this, it helps thinking about a theme as a set of ordered components:

```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```

The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.

So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.

Hugo uses two different algorithms to merge the filesystems, depending on the file type:

* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.

The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are  plans to improve on this and get a URL scheme so this can be resolved automatically.

Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:

* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`

The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.

A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.

Fixes #4460
Fixes #4450
2018-06-10 23:55:20 +02:00
Bjørn Erik Pedersen
4256de3392
deps: Go fmt 2018-05-23 16:33:11 +02:00
Bjørn Erik Pedersen
35ccf06dae
Fix some recently broken embedded templates
And add tests for them.

Fixes #4757
2018-05-23 10:03:11 +02:00