Commit graph

508 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
9cd54cab20 Move the emoji parsing to pageparser
This avoids double parsing the page content when `enableEmoji=true`.

This commit also adds some general improvements to the parser, making it in general much faster:

```bash
benchmark                     old ns/op     new ns/op     delta
BenchmarkShortcodeLexer-4     90258         101730        +12.71%
BenchmarkParse-4              148940        15037         -89.90%

benchmark                     old allocs     new allocs     delta
BenchmarkShortcodeLexer-4     456            700            +53.51%
BenchmarkParse-4              28             33             +17.86%

benchmark                     old bytes     new bytes     delta
BenchmarkShortcodeLexer-4     69875         81014         +15.94%
BenchmarkParse-4              8128          8304          +2.17%
```

Running some site benchmarks with Emoji support turned on:

```bash
benchmark                                                                                     old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4     924556797     818115620     -11.51%

benchmark                                                                                     old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4     4112613        4133787        +0.51%

benchmark                                                                                     old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4     426982864     424363832     -0.61%
```

Fixes #5534
2018-12-20 20:08:01 +01:00
Bjørn Erik Pedersen
7829474088
Add /config dir support
This commit adds support for a configuration directory (default `config`). The different pieces in this puzzle are:

* A new `--environment` (or `-e`) flag. This can also be set with the `HUGO_ENVIRONMENT` OS environment variable. The value for `environment` defaults to `production` when running `hugo` and `development` when running `hugo server`. You can set it to any value you want (e.g. `hugo server -e "Sensible Environment"`), but as it is used to load configuration from the file system, the letter case may be important. You can get this value in your templates with `{{ hugo.Environment }}`.
* A new `--configDir` flag (defaults to `config` below your project). This can also be set with `HUGO_CONFIGDIR` OS environment variable.

If the `configDir` exists, the configuration files will be read and merged on top of each other from left to right; the right-most value will win on duplicates.

Given the example tree below:

If `environment` is `production`, the left-most `config.toml` would be the one directly below the project (this can now be omitted if you want), and then `_default/config.toml` and finally `production/config.toml`. And since these will be merged, you can just provide the environment specific configuration setting in you production config, e.g. `enableGitInfo = true`. The order within the directories will be lexical (`config.toml` and then `params.toml`).

```bash
config
├── _default
│   ├── config.toml
│   ├── languages.toml
│   ├── menus
│   │   ├── menus.en.toml
│   │   └── menus.zh.toml
│   └── params.toml
├── development
│   └── params.toml
└── production
    ├── config.toml
    └── params.toml
```

Some configuration maps support the language code in the filename (e.g. `menus.en.toml`): `menus` (`menu` also works) and `params`.

Also note that the only folders with "a meaning" in the above listing is the top level directories below `config`. The `menus` sub folder is just added for better organization.

We use `TOML` in the example above, but Hugo also supports `JSON` and `YAML` as configuration formats. These can be mixed.

Fixes #5422
2018-12-11 13:08:36 +01:00
Bjørn Erik Pedersen
831d23cb4d Add tpl/site and tpl/hugo
This means that the current `.Site` and ´.Hugo` is available as a globals, so you can do `site.IsServer`, `hugo.Version` etc.

Fixes #5470
Fixes #5467
Fixes #5503
2018-12-06 14:37:25 +01:00
hugoreleaser
54ecade02f releaser: Prepare repository for 0.53-DEV
[ci skip]
2018-11-28 14:08:36 +00:00
hugoreleaser
3c909eb3f3 releaser: Bump versions for release of 0.52
[ci skip]
2018-11-28 14:06:31 +00:00
Bjørn Erik Pedersen
5df2b79dd2 Fix Permalink for resource, baseURL with path and canonifyURLs set
Fixes #5226
2018-11-15 16:37:11 +01:00
Bjørn Erik Pedersen
fabf026f49
helpers: Add a comment about file mode for new files
See #5434
2018-11-14 23:14:51 +01:00
Bjørn Erik Pedersen
f7aeaa6129 Add a consolidated file cache
This commits reworks how file caching is performed in Hugo. Now there is only one way, and it can be configured.

This is the default configuration:

```toml
[caches]
[caches.getjson]
dir = ":cacheDir"
maxAge = -1
[caches.getcsv]
dir = ":cacheDir"
maxAge = -1
[caches.images]
dir = ":resourceDir/_gen"
maxAge = -1
[caches.assets]
dir = ":resourceDir/_gen"
maxAge = -1
```

You can override any of these cache setting in your own `config.toml`.

The placeholders explained:

`:cacheDir`: This is the value of the `cacheDir` config option if set (can also be set via OS env variable `HUGO_CACHEDIR`). It will fall back to `/opt/build/cache/hugo_cache/` on Netlify, or a `hugo_cache` directory below the OS temp dir for the others.
`:resourceDir`: This is the value of the `resourceDir` config option.

`maxAge` is the time in seconds before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off.

This means that if you run your builds on Netlify, all caches configured with `:cacheDir` will be saved and restored on the next build. For other CI vendors, please read their documentation. For an CircleCI example, see 6c3960a8f4/.circleci/config.yml

Fixes #5404
2018-11-13 14:19:42 +01:00
hugoreleaser
2900801acf releaser: Prepare repository for 0.52-DEV
[ci skip]
2018-11-07 10:11:48 +00:00
hugoreleaser
1ccc0b79a2 releaser: Bump versions for release of 0.51
[ci skip]
2018-11-07 10:09:43 +00:00
hugoreleaser
f863b6a07e releaser: Prepare repository for 0.51-DEV
[ci skip]
2018-10-29 09:54:42 +00:00
hugoreleaser
b14c11b0e1 releaser: Bump versions for release of 0.50
[ci skip]
2018-10-29 09:51:50 +00:00
Bjørn Erik Pedersen
35fbfb19a1
commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes #5284
Fixes #5290
See #5325
See #5324
2018-10-16 22:10:56 +02:00
Shreyansh Khajanchi
3d4a9882bf helpers: Call rst2html directly on *nix
Initially, rst2html was called via the python interpreter which would
fail if the script was wrapped in a launcher as on NixOS.
Ideally, on *nix, binaries should be invoked directly to ensure that
shebangs work properly as is being done now.
Handle the case of windows as it doesn't do shebangs.
2018-10-11 22:46:10 +02:00
Bjørn Erik Pedersen
e421696d02 helpers: Consolidate MakeSegment vs MakePathSanitized
In short:

* Avoid double tolower in MakeSegment
* Use MakePathSanitized for taxonomies in pageToPermalinkTitle; this matches what MakeSegment does.
* Move the "double hyphen and space" logic into UnicodeSanitize

The last bullet may be slightly breaking for some that now does not get the "--" in some URLs, but we need to reduce the amount of URL logic.

See #4926
2018-10-03 15:55:25 +02:00
hugoreleaser
66b9e2bff2 releaser: Prepare repository for 0.50-DEV
[ci skip]
2018-09-24 10:04:24 +00:00
hugoreleaser
7f005f5e98 releaser: Bump versions for release of 0.49
[ci skip]
2018-09-24 10:01:04 +00:00
Cameron Moore
c8ce65046d helpers: Fix golint issues
helpers/general.go:448:1: comment on exported function DiffStrings should be of the form "DiffStrings ..."
helpers/hugo.go:42:6: exported type HugoVersionString should have comment or be unexported
helpers/hugo.go:48:1: exported method HugoVersion.Version should have comment or be unexported
helpers/hugo.go:56:1: comment on exported method HugoVersionString.Compare should be of the form "Compare ..."
helpers/hugo.go:62:1: comment on exported method HugoVersionString.Eq should be of the form "Eq ..."
helpers/path.go:548:1: comment on exported function OpenFilesForWriting should be of the form "OpenFilesForWriting ..."
helpers/processing_stats.go:24:6: exported type ProcessingStats should have comment or be unexported
helpers/processing_stats.go:55:1: exported function NewProcessingStats should have comment or be unexported
helpers/processing_stats.go:59:1: exported method ProcessingStats.Incr should have comment or be unexported
helpers/processing_stats.go:63:1: exported method ProcessingStats.Add should have comment or be unexported
helpers/processing_stats.go:67:1: exported method ProcessingStats.Table should have comment or be unexported
helpers/processing_stats.go:83:1: exported function ProcessingStatsTable should have comment or be unexported
2018-09-07 08:25:51 +02:00
hugoreleaser
1ff54ab3d9 releaser: Prepare repository for 0.49-DEV
[ci skip]
2018-08-29 06:36:18 +00:00
hugoreleaser
09811d0bee releaser: Bump versions for release of 0.48
[ci skip]
2018-08-29 06:33:03 +00:00
Bjørn Erik Pedersen
f4675fa0f0
helpers: Fix permissions when creating new folders
This was mistakingly changed from 07777 (rwx, rw, r before umask) to 0755 in Hugo 0.37. This commit reverts to the old value.

Fixes #5128
2018-08-28 17:14:56 +02:00
Cameron Moore
fff132537b Fix handling of taxonomy terms containing slashes
Fixes #4090
2018-08-22 07:18:37 +02:00
hugoreleaser
a81c64e43e releaser: Prepare repository for 0.48-DEV
[ci skip]
2018-08-20 08:19:37 +00:00
hugoreleaser
16b691a875 releaser: Bump versions for release of 0.47.1
[ci skip]
2018-08-20 08:16:49 +00:00
hugoreleaser
6746826689 releaser: Prepare repository for 0.48-DEV
[ci skip]
2018-08-17 09:17:44 +00:00
hugoreleaser
6711dacded releaser: Bump versions for release of 0.47
[ci skip]
2018-08-17 09:15:18 +00:00
Bjørn Erik Pedersen
78f8475a05 Fix Resource output in multihost setups
In Hugo 0.46 we made the output of what you get from resources.Get and similar static, i.e. language agnostic. This makes total sense, as it is wasteful and time-consuming to do SASS/SCSS/PostCSS processing for lots of languages when the output is lots of duplicates with different filenames.

But since we now output the result once only, this had a negative side effect for multihost setups: We publish the resource once only to the root folder (i.e. not to the language "domain folder").

This commit removes the language code from the processed image keys. This creates less duplication in the file cache, but it means that you should do a `hugo --gc` to clean up stale files.

Fixes #5058
2018-08-13 19:00:51 +02:00
Bjørn Erik Pedersen
789ef8c639
Add support for minification of final output
Hugo Pipes added minification support for resources fetched via ´resources.Get` and similar.

This also adds support for minification of the final output for supported output formats: HTML, XML, SVG, CSS, JavaScript, JSON.

To enable, run Hugo with the `--minify` flag:

```bash
hugo --minify
```

This commit is also a major spring cleaning of the `transform` package to allow the new minification step fit into that processing chain.

Fixes #1251
2018-08-06 19:58:41 +02:00
hugoreleaser
31faf98ce3 releaser: Prepare repository for 0.47-DEV
[ci skip]
2018-08-01 09:03:22 +00:00
hugoreleaser
3aa838cf2b releaser: Bump versions for release of 0.46
[ci skip]
2018-08-01 09:00:52 +00:00
hugoreleaser
0cae1cf828 releaser: Prepare repository for 0.46-DEV
[ci skip]
2018-07-25 08:58:54 +00:00
hugoreleaser
e38843f4bf releaser: Bump versions for release of 0.45.1
[ci skip]
2018-07-25 08:56:21 +00:00
hugoreleaser
3a6784b162 releaser: Prepare repository for 0.46-DEV
[ci skip]
2018-07-22 12:12:08 +00:00
hugoreleaser
2c5a036def releaser: Bump versions for release of 0.45
[ci skip]
2018-07-22 12:09:26 +00:00
hugoreleaser
f44c9ab9cd releaser: Prepare repository for 0.45-DEV
[ci skip]
2018-07-13 06:05:41 +00:00
hugoreleaser
0831d8ccaf releaser: Bump versions for release of 0.44
[ci skip]
2018-07-13 06:03:09 +00:00
hugoreleaser
6084f0433c releaser: Prepare repository for 0.44-DEV
[ci skip]
2018-07-09 10:03:30 +00:00
hugoreleaser
e6136b36f3 releaser: Bump versions for release of 0.43
[ci skip]
2018-07-09 10:00:04 +00:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
hugoreleaser
187621ae24 releaser: Prepare repository for 0.43-DEV
[ci skip]
2018-06-13 10:19:00 +00:00
hugoreleaser
fc97388968 releaser: Bump versions for release of 0.42.1
[ci skip]
2018-06-13 10:16:33 +00:00
hugoreleaser
0f1fc01ef2
releaser: Prepare repository for 0.43-DEV
[ci skip]
2018-06-12 08:45:06 +02:00
hugoreleaser
19d91abc92 releaser: Bump versions for release of 0.42
[ci skip]
2018-06-12 06:37:19 +00:00
Bjørn Erik Pedersen
80230f26a3
Add support for theme composition and inheritance
This commit adds support for theme composition and inheritance in Hugo.

With this, it helps thinking about a theme as a set of ordered components:

```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```

The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.

So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.

Hugo uses two different algorithms to merge the filesystems, depending on the file type:

* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.

The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are  plans to improve on this and get a URL scheme so this can be resolved automatically.

Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:

* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`

The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.

A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.

Fixes #4460
Fixes #4450
2018-06-10 23:55:20 +02:00
Stefan Neuhaus
20cbc2c785 Add a BlackFriday option for rel="noreferrer" on external links
Add a configuration option "noreferrerLinks". When set to "true" the "HTML_NOREFERRER_LINKS" flag is being passed to Blackfriday. Thereby all *absolute* links will get a "noreferrer" value for their "rel" attribute.

See #4722
2018-05-29 16:54:43 +02:00
Stefan Neuhaus
7a6192647a Add a BlackFriday option for rel="nofollow" on external links
Add a configuration option "nofollowLinks". When set to "true" the "HTML_NOFOLLOW_LINKS" flag is being passed to Blackfriday. Thereby all *absolute* links will get a "nofollow" value for the "rel" attribute.

Fixes #4722
2018-05-28 01:21:36 +02:00
hugoreleaser
5241f0baee
releaser: Prepare repository for 0.42-DEV
[ci skip]
2018-05-27 10:30:06 +02:00
hugoreleaser
0e77f5d4e7 releaser: Bump versions for release of 0.41
[ci skip]
2018-05-25 16:57:17 +00:00
Bjørn Erik Pedersen
35ccf06dae
Fix some recently broken embedded templates
And add tests for them.

Fixes #4757
2018-05-23 10:03:11 +02:00
hugoreleaser
8b03c51eae releaser: Prepare repository for 0.41-DEV
[ci skip]
2018-04-30 06:50:00 +00:00