Commit graph

561 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
b874a1ba7a media: Allow multiple file suffixes per media type
Before this commit, `Suffix` on `MediaType` was used both to set a custom file suffix and as a way to augment the mediatype definition (what you see after the "+", e.g. "image/svg+xml").

This had its limitations. For one, it was only possible with one file extension per MIME type.

Now you can specify multiple file suffixes using "suffixes", but you need to specify the full MIME type
identifier:

[mediaTypes]
[mediaTypes."image/svg+xml"]
suffixes = ["svg", "abc ]

In most cases, it will be enough to just change:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffix = "txt"

To:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffixes = ["txt"]

Hugo will still respect values set in "suffix" if no value for "suffixes" is provided, but this will be removed in a future release.

Note that you can still get the Media Type's suffix from a template: {{ $mediaType.Suffix }}. But this will now map to the MIME type filename.

Fixes #4920
2018-07-10 22:13:52 +02:00
Bjørn Erik Pedersen
2b8d907ab7 Add a newScratch template func
Fixes #4685
2018-07-06 17:51:38 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
128f14efad hugolib: Reset the global pages cache on server rebuilds
In Hugo 0.42, this could lead to errors of type `runtime error: invalid memory address or nil pointer dereference` in some rare situations.

Note that this was also an issue before 0.42, but the symptom was then potentially stale list content on rebuilds on content changes.

This commit also improves the above error message.

Fixes #4845
2018-06-13 12:00:30 +02:00
Bjørn Erik Pedersen
80230f26a3
Add support for theme composition and inheritance
This commit adds support for theme composition and inheritance in Hugo.

With this, it helps thinking about a theme as a set of ordered components:

```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```

The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.

So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.

Hugo uses two different algorithms to merge the filesystems, depending on the file type:

* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.

The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are  plans to improve on this and get a URL scheme so this can be resolved automatically.

Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:

* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`

The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.

A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.

Fixes #4460
Fixes #4450
2018-06-10 23:55:20 +02:00
Bjørn Erik Pedersen
6464981adb
hugolib: Make sure that .Site.Taxonomies is always set on rebuilds
Fixes #4838
2018-06-10 20:55:17 +02:00
Bjørn Erik Pedersen
dc4226a8b2 hugolib: Reset Page's main output on server rebuilds
Not doing so prevents the paginators to be rebuilt on changes.

Fixes #4819
2018-06-05 10:45:24 +03:00
Bjørn Erik Pedersen
35ccf06dae
Fix some recently broken embedded templates
And add tests for them.

Fixes #4757
2018-05-23 10:03:11 +02:00
Bjørn Erik Pedersen
4ddcf52ccc Add an unified .Site.Config with a services section
Fixes #4751
2018-05-22 18:11:03 +02:00
Bjørn Erik Pedersen
353148c2bc Move the privacy config into a parent
See #4751
2018-05-22 18:11:03 +02:00
Bjørn Erik Pedersen
0bbdef986d config: Add the foundation for GDPR privacy configuration
See #4616
2018-05-21 00:41:42 +02:00
Bjørn Erik Pedersen
655983a22d Reset .Page.Scratch on live-reload
Fixes #4683
2018-04-30 08:21:22 +02:00
Bjørn Erik Pedersen
288c396439
hugolib: Fix some shortcode vs .Content corner cases
This is a follow-up to #4632. There were some assumptions in that implementation that did not hold water in all situations.

This commit simplifies the content lazy initalization making it more robust.

Fixes #4664
2018-04-25 08:56:46 +02:00
Bjørn Erik Pedersen
4d26ab33dc
Make .Content (almost) always available in shortcodes
This resolves some surprising behaviour when reading other pages' content from shortcodes. Before this commit, that behaviour was undefined. Note that this has never been an issue from regular templates.

It will still not be possible to get **the current shortcode's  page's rendered content**. That would have impressed Einstein.

The new and well defined rules are:

* `.Page.Content` from a shortcode will be empty. The related `.Page.Truncated` `.Page.Summary`, `.Page.WordCount`, `.Page.ReadingTime`, `.Page.Plain` and `.Page.PlainWords` will also have empty values.
* For _other pages_ (retrieved via `.Page.Site.GetPage`, `.Site.Pages` etc.) the `.Content` is there to use as you please as long as you don't have infinite content recursion in your shortcode/content setup. See below.
* `.Page.TableOfContents` is good to go (but does not support shortcodes in headlines; this is unchanged)

If you get into a situation of infinite recursion, the `.Content` will be empty. Run `hugo -v` for more information.

Fixes #4632
Fixes #4653
Fixes #4655
2018-04-21 22:02:56 +02:00
Bjørn Erik Pedersen
d6a2024e6b
Revert "Improve .Content vs shortcodes"
This reverts commit e590cc26eb.
2018-04-19 17:40:57 +02:00
Bjørn Erik Pedersen
e590cc26eb
Improve .Content vs shortcodes
For the content from other pages in shortcodes there are some chicken and
egg dependencies that is hard to get around. But we can improve on this  by preparing the pages in a certain order:

 1. The headless bundles goes first. These are page typically page and image collections..
 2. Leaf bundles
 3. Regular single pages
 4. Branch bundles

Fixes #4632
2018-04-19 14:46:50 +02:00
Bjørn Erik Pedersen
61d52f1462
hugolib: Do not reset .Page.Scratch on rebuilds
Fixes #4627
2018-04-15 19:31:42 +02:00
Bjørn Erik Pedersen
a4deaeff0c
Fix some livereload content regressions
Introduced in Hugo 0.38.

Fixes #4566
2018-04-05 14:28:45 +02:00
Bjørn Erik Pedersen
eb42774e58
Add support for a content dir set per language
A sample config:

```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true

[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"

[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```

The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.

The content files will be assigned a language by

1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.

The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.

Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.

If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.

Fixes #4523
Fixes #4552
Fixes #4553
2018-04-02 08:06:21 +02:00
Ricardo N Feliciano
1823c053c8 Add .Site.IsServer
Fixes #4478
2018-03-30 20:15:22 +02:00
Bjørn Erik Pedersen
93e24a03ce
hugolib: Fix freeze in invalid front matter error case
Fixes #4526
2018-03-24 10:06:58 +01:00
Bjørn Erik Pedersen
e9c7b6205f
Allow themes to define output formats, media types and params
This allows a `config.toml` (or `yaml`, ´yml`, or `json`)  in the theme to set:

1) `params` (but cannot override params in project. Will also get its own "namespace", i.e. `{{ .Site.Params.mytheme.my_param }}` will be the same as `{{ .Site.Params.my_param }}` providing that the main project does not define a param with that key.
2) `menu` -- but cannot redefine/add menus in the project. Must create its own menus with its own identifiers.
3) `languages` -- only `params` and `menu`. Same rules as above.
4) **new** `outputFormats`
5) **new** `mediaTypes`

This should help with the "theme portability" issue and people having to copy and paste lots of setting into their projects.

Fixes #4490
2018-03-21 09:22:19 +01:00
Bjørn Erik Pedersen
ffaec4ca8c
Add a way to merge pages by language
As an example:

```html
{{ $pages := .Site.RegularPages | lang.Merge $frSite.RegularPages | lang.Merge $enSite.RegularPages }}
```

Will "fill in the gaps" in the current site with, from left to right, content from the French site, and lastly the English.

Fixes #4463
2018-03-16 10:10:01 +01:00
Bjørn Erik Pedersen
68bf1511f2
hugolib: Extract date and slug from filename
This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter.

This should make it easier to move content from Jekyll to Hugo.

To enable, put this in your `config.toml`:

```toml
[frontmatter]
date  = [":filename", ":default"]
```

This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc.

So, if you want to use the `file modification time`, this can be a good configuration:

 ```toml
[frontmatter]
date = [ "date",":fileModTime", ":default"]
lastmod = ["lastmod" ,":fileModTime", ":default"]
```

The current `:default` values for the different dates are

```toml
[frontmatter]
date = ["date","publishDate", "lastmod"]
lastmod = ["lastmod", "date","publishDate"]
publishDate = ["publishDate", "date"]
expiryDate = ["expiryDate"]
```

The above will now be the same as:

```toml
[frontmatter]
date = [":default"]
lastmod = [":default"]
publishDate = [":default"]
expiryDate = [":default"]
```

Note:

* We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate.
* If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved.
* All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`).
* The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values.
* The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list.

Fixes #285
Closes #3310
Closes #3762
Closes #4340
2018-03-11 18:26:18 +01:00
Vas Sudanagunta
1fa2417777 Add support for YAML array data files
* Unmarshaled YAML arrays indistinguishable from JSON arrays.
* Fixes #3890
2018-02-12 17:14:40 +01:00
Vas Sudanagunta
bb549a0d57 Account for array type data in data dir merge/override logic
* Fixes #4366

* Error message to console for unsupported data types
2018-02-12 17:14:40 +01:00
Vas Sudanagunta
4402c07775 Fix JSON array-based data file handling regression
This bug was introduced in Hugo 0.35.

Fixes #4361
2018-02-02 09:14:37 +01:00
Bjørn Erik Pedersen
94e736c5e1
hugolib: Extract the Fast Render Mode logic into a method
This also improves on the previous commit as it takes pages without content files into account.

Closes #4339
2018-01-30 10:49:24 +01:00
Alexey Grachov
1707dae8d3
hugolib: Handle newly created files in Fast Render Mode
Updates #4339
2018-01-30 10:49:24 +01:00
Bjørn Erik Pedersen
016398ffe2
hugolib: Fix --uglyURLs from comand line regression
This bug was introduced in Hugo 0.33.

Fixes #4343
2018-01-28 17:03:10 +01:00
Bjørn Erik Pedersen
6413559f75 Add a way to disable one or more languages
This commit adds a new config setting:

```toml
disableLanguages = ["fr"]
```

If this is a multilingual site:

* No site for the French language will be created
* French content pages will be ignored/not read
* The French language configuration (menus etc.) will also be ignored

This makes it possible to start translating new languages and turn it on when you're happy etc.

Fixes #4297
Fixed #4329
2018-01-26 14:04:14 +01:00
Bjørn Erik Pedersen
ae5a45be6f
hugolib: Remove unused code 2018-01-25 17:14:03 +01:00
Bjørn Erik Pedersen
d418c2c2ea
Remove and update deprecation status 2018-01-25 10:22:11 +01:00
Bjørn Erik Pedersen
20c9b6ec81
resource: Add front matter metadata to Resource
This commit expands the Resource interface with 3 new methods:

* Name
* Title
* Params

All of these can be set in the Page front matter. `Name` will get its default value from the base filename, and is the value used in the ByPrefix and GetByPrefix lookup methods.

Fixes #4244
2018-01-17 16:22:33 +01:00
Bjørn Erik Pedersen
51dd462c39 layout: Respect Type and Layout for list template selection
This commit also has some other nice side-effects:

* The layout logic is unified for all page types, which should make it less surprising
* Page.Render now supports all types
* The legacy "indexes" type is removed from the template lookup order. This is an undocumented type from early Hugo days. This means that having a template in, say, `/layouts/indexes/list.html` will no longer work.
* The theme override logic is improved. As an example, an `index.html` in theme will now wn over a `_default/list.html` in the project, which most will expect.

Fixes #3005
Fixes #3245
2018-01-14 19:59:51 +01:00
Bjørn Erik Pedersen
57e10f174e Support uglyURLs per section
Fixes #4256
2018-01-12 10:39:09 +01:00
Bjørn Erik Pedersen
1c114d539b
hugolib: Do not tolower result from Page.GetParam
We still do lowering of the param strings in some internal use of this, but the exported `GetParam` method is changed to a more sensible default.

This was used for the `disqus_title` etc. in the internal Disqus template, which was obviously not right.

If you really want to lowercase your params, do it with `.GetParam "myparam" | lower` or similar.

Fixes #4187
2017-12-29 08:58:38 +01:00
Bjørn Erik Pedersen
3cdf19e9b7
Implement Page bundling and image handling
This commit is not the smallest in Hugo's history.

Some hightlights include:

* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).

A site building  benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:

```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"

benchmark                                                                                                         old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      101785785     78067944      -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     185481057     149159919     -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      103149918     85679409      -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     203515478     169208775     -16.86%

benchmark                                                                                                         old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      532464         391539         -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1056549        772702         -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      555974         406630         -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1086545        789922         -27.30%

benchmark                                                                                                         old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      53243246      43598155      -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     105811617     86087116      -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      54558852      44545097      -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     106903858     86978413      -18.64%
```

Fixes #3651
Closes #3158
Fixes #1014
Closes #2021
Fixes #1240
Updates #3757
2017-12-27 18:44:47 +01:00
Bjørn Erik Pedersen
c1d325934e helpers: Remove (now) unused GetStaticDir
* In Hugo there is no single static dir
* It was used as a filter in the content dir, which makes no sense since any overlap here is impossible
2017-11-19 14:03:21 +01:00
Bjørn Erik Pedersen
60dfb9a6e0 Add support for multiple staticDirs
This commit adds support for multiple statDirs both on the global and language level.

A simple `config.toml` example:

```bash
staticDir = ["static1", "static2"]
[languages]
[languages.no]
staticDir = ["staticDir_override", "static_no"]
baseURL = "https://example.no"
languageName = "Norsk"
weight = 1
title = "På norsk"

[languages.en]
staticDir2 = "static_en"
baseURL = "https://example.com"
languageName = "English"
weight = 2
title = "In English"
```

In the above, with no theme used:

the English site will get its static files as a union of "static1", "static2" and "static_en". On file duplicates, the right-most version will win.
the Norwegian site will get its static files as a union of "staticDir_override" and "static_no".

This commit also concludes the Multihost support in #4027.

Fixes #36
Closes #4027
2017-11-17 11:01:46 +01:00
Bjørn Erik Pedersen
60bd332c1f Only re-render the view(s) you're working on
Hugo already, in its server mode,  support partial rebuilds. To put it simply: If you change `about.md`, only that content page is read and processed, then Hugo does some processing (taxonomies etc.) and the full site is rendered.
This commit covers the rendering part: We now only re-render the pages you work on, i.e. the last n pages you watched in the browser (which obviously also includes the  page in the example above).

To be more specific: When you are running the hugo server in watch (aka. livereload) mode, and change a template or a content file, then we do a partial re-rendering of the following:

* The current content page (if it is a content change)
* The home page
* Up to the last 10 pages you visited on the site.

This should in most cases be enough, but if you navigate to something completely different, you may see stale content. Doing an edit will then refresh that page.

Note that this feature is enabled by default. To turn it off, run `hugo server --disableFastRender`.

Fixes #3962
See  #1643
2017-10-14 13:40:43 +02:00
Cameron Moore
b277cb33e4 hugolib: Move metrics output to the end of the site build 2017-09-27 21:26:08 +02:00
Cameron Moore
b4a14c25fe metrics: Add simple template metrics feature 2017-09-26 20:03:04 +02:00
Bjørn Erik Pedersen
fb33d8286d Use Chroma as new default syntax highlighter
If you want to use Pygments, set `pygmentsUseClassic=true` in your site config.

Fixes #3888
2017-09-25 08:59:02 +02:00
Bjørn Erik Pedersen
b180477631 Add liveReloadPort flag to server
This makes live reloading behind a HTTPS proxy working, as in the example below using the service from https://ngrok.com:

```
hugo server -b https://ba6sdfe72.ngrok.io --appendPort=false --liveReloadPort=443 --navigateToChanged
```

Fixes #3882
2017-09-23 17:08:48 +02:00
Bjørn Erik Pedersen
3b4f17bbc9 hugolib: Implement "related content"
This closes #98, even if this commit does not do full content text search.

We may revisit that problem in the future, but that deserves its own issue.

Fixes #98
2017-09-06 00:20:02 +02:00
Bjørn Erik Pedersen
9891c0fb0e Remove sourceRelativeLinks
Fixes #3766
2017-08-02 22:04:38 +02:00
Jorin Vogel
09907d36af Switch from fork bep/inflect to markbates/inflect
Original package has received updates the fork hasn't.
Without fork updates are easier to maintain.
2017-08-01 13:05:09 +02:00
Bjørn Erik Pedersen
8fb594bfb0 Make the title case style guide configurable
This works for the `title` func and the other places where Hugo makes title case.

* AP style (new default)
* Chicago style
* Go style (what we have today)

Fixes #989
2017-07-31 22:16:46 +02:00
Bjørn Erik Pedersen
dbe63970e0 hugolib: Support reflinks starting with a slash
Fixes #3703
2017-07-21 13:08:32 +02:00