The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct.
This is all a preparation step for issue #5074, "pages from other data sources".
But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes.
Most notable changes:
* The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday.
This means that any markdown will partake in the global ToC and footnote context etc.
* The Custom Output formats are now "fully virtualized". This removes many of the current limitations.
* The taxonomy list type now has a reference to the `Page` object.
This improves the taxonomy template `.Title` situation and make common template constructs much simpler.
See #5074Fixes#5763Fixes#5758Fixes#5090Fixes#5204Fixes#4695Fixes#5607Fixes#5707Fixes#5719Fixes#3113Fixes#5706Fixes#5767Fixes#5723Fixes#5769Fixes#5770Fixes#5771Fixes#5759Fixes#5776Fixes#5777Fixes#5778
This commit adds support for a configuration directory (default `config`). The different pieces in this puzzle are:
* A new `--environment` (or `-e`) flag. This can also be set with the `HUGO_ENVIRONMENT` OS environment variable. The value for `environment` defaults to `production` when running `hugo` and `development` when running `hugo server`. You can set it to any value you want (e.g. `hugo server -e "Sensible Environment"`), but as it is used to load configuration from the file system, the letter case may be important. You can get this value in your templates with `{{ hugo.Environment }}`.
* A new `--configDir` flag (defaults to `config` below your project). This can also be set with `HUGO_CONFIGDIR` OS environment variable.
If the `configDir` exists, the configuration files will be read and merged on top of each other from left to right; the right-most value will win on duplicates.
Given the example tree below:
If `environment` is `production`, the left-most `config.toml` would be the one directly below the project (this can now be omitted if you want), and then `_default/config.toml` and finally `production/config.toml`. And since these will be merged, you can just provide the environment specific configuration setting in you production config, e.g. `enableGitInfo = true`. The order within the directories will be lexical (`config.toml` and then `params.toml`).
```bash
config
├── _default
│ ├── config.toml
│ ├── languages.toml
│ ├── menus
│ │ ├── menus.en.toml
│ │ └── menus.zh.toml
│ └── params.toml
├── development
│ └── params.toml
└── production
├── config.toml
└── params.toml
```
Some configuration maps support the language code in the filename (e.g. `menus.en.toml`): `menus` (`menu` also works) and `params`.
Also note that the only folders with "a meaning" in the above listing is the top level directories below `config`. The `menus` sub folder is just added for better organization.
We use `TOML` in the example above, but Hugo also supports `JSON` and `YAML` as configuration formats. These can be mixed.
Fixes#5422
This means that the current `.Site` and ´.Hugo` is available as a globals, so you can do `site.IsServer`, `hugo.Version` etc.
Fixes#5470Fixes#5467Fixes#5503
This commit also pulls down the log level for a set of WARN statements to INFO. There should be no ERRORs or WARNINGs in a regular Hugo build. That is the story about the Boy Who Cried Wolf.
Since the WARN log is now more visible, this commit also improves on some of them, most notable the "layout not found", which now would look something like this:
```bash
WARN 2018/11/02 09:02:18 Found no layout for "home", language "en", output format "CSS": create a template below /layouts with one of these filenames: index.en.css.css, home.en.css.css, list.en.css.css, index.css.css, home.css.css, list.css.css, index.en.css, home.en.css, list.en.css, index.css, home.css, list.css, _default/index.en.css.css, _default/home.en.css.css, _default/list.en.css.css, _default/index.css.css, _default/home.css.css, _default/list.css.css, _default/index.en.css, _default/home.en.css, _default/list.en.css, _default/index.css, _default/home.css, _default/list.css
```
Fixes#5203
The main item in this commit is showing of errors with a file context when running `hugo server`.
This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).
But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.
Fixes#5284Fixes#5290
See #5325
See #5324
Introduce new page position variables in order to fix the ordering issue
of `.Next` and `.Prev` while also allowing an upgrade path via
deprecation.
`.NextInSection` becomes `.NextPageInSection`.
`.PrevInSection` becomes `.PrevPageInSection`.
`.Next` becomes a function returning `.PrevPage`.
`.Prev` becomes a function returning `.NextPage`.
Fixes#1061
This extends the page grouping in Hugo with a template function that allows for ad-hoc grouping.
A made-up example:
```
{{ $cool := where .Site.RegularPages "Params.cool" true | group "cool" }}
{{ $blue := where .Site.RegularPages "Params.blue" true | group "blue" }}
{{ $paginator := .Paginate (slice $cool $blue) }}
```
Closes#4865
If a content file contains shortcode(s), we have logic in place to re-render it per output format.
We also have logic in place that avoids making a copy of the content used for this process if we don't need it.
This was before this commit limited to server mode and if the page should be output to multiple formats.
But there is a third case: If a site (language) borrows and renders `.Content` from another language. This would, before this commit, behave oddly for content with shortcodes.
Fixes#4986
Two new settings:
* refLinksErrorLevel: ERROR (default) or WARNING. ERROR will fail the build.
* refLinksNotFoundURL: Used as a placeholder when page references cannot be found.
Fixes#4964
This commit is a follow up to a recent overhaul of the GetPage/ref/relref implemenation.
The most important change in this commit is the update to `.Site.GetPage`:
* To reduce the amount of breakage in the wild to its minimum, I have reworked .Site.GetPage with some rules:
* We cannot support more than 2 arguments, i.e. .Site.GetPage "page" "posts" "mypage.md" will now throw an error. I think this is the most uncommon syntax and should be OK. It is an easy fix to change the above to .Site.GetPage "/posts/mypage.md" or similar.
* .Site.GetPage "home", .Site.GetPage "home" "" and .Site.GetPage "home" "/" will give you the home page. This means that if you have page in root with the name home.md you need to do .Site.GetPage "/home.md" or similar
This commit also fixes some multilingual issues, most notable it is now possible to do cross-language ref/relref lookups by prepending the language code to the path, e.g. `/jp/posts/mypage.md`.
This commit also reverts the site building tests related to this to "Hugo 0.44 state", to get better control of the changes made.
Closes#4147Closes#4727Closes#4728Closes#4728Closes#4726Closes#4652
This commit unifies the core internal page index for all page kinds.
This enables the `ref` and `relref` shortcodes to support all pages kinds, and adds a new page-relative `.GetPage` method with simplified signature.
See #4147
See #4727
See #4728
See #4728
See #4726
See #4652
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.
This commit adds
* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.
This means that you can now do this in your templates (or shortcodes):
```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```
This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:
```
HUGO_BUILD_TAGS=extended mage install
```
Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.
The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:
```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```
The transformation funcs above have aliases, so it can be shortened to:
```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```
A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.
Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test
New functions to create `Resource` objects:
* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.
New `Resource` transformation funcs:
* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.
Fixes#4381Fixes#4903Fixes#4858
This should allow for less duplication of templates. Before this commit it was possible to override the content page of a given page/section, but only one page at a time.
Full "template sets" can now be inherited by setting `type: blog` etc. in the section content page's front matter, and that type will be considered when looking for layouts for all pages in that section.
For nested sections, it will use consider both `type` set in the current section first, then `type` set in the first section below home, e.g. `/docs`.
This commit also adds a new Page method: `FirstSection`. This navigates up to the first section below home (e.g. `/docs`). For the home page it will return itself.
Fixes#4891
This commit adds support for theme composition and inheritance in Hugo.
With this, it helps thinking about a theme as a set of ordered components:
```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```
The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.
So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.
Hugo uses two different algorithms to merge the filesystems, depending on the file type:
* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.
The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are plans to improve on this and get a URL scheme so this can be resolved automatically.
Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:
* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`
The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.
A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.
Fixes#4460Fixes#4450
Sets Page.markup earlier (as early as possible, when the page is
loaded). Sets it once and only once, removing many redundant calls
to determineMarkupType().
This kills a sleeping bug that was avoided by the parts of the
code depending on this value making those redundant calls.
There have been one report of a site with truncated `.Content` after the Hugo `0.40.1` release.
This commit fixes this so that race should not be possible anymore. It also adds a stress test with focus on content rendering and multiple output formats.
Fixes#4706
This bug was introduced in Hugo 0.40. It is when you use the `<!--more-->` summary marker.
Note that this affects the word stats only. The related `PlainWords`, `Plain`, `Content` all return correct values.
Fixes#4675Fixes#4682
This is a follow-up to #4632. There were some assumptions in that implementation that did not hold water in all situations.
This commit simplifies the content lazy initalization making it more robust.
Fixes#4664
I eturn either:
1. leaf
2. branch
3. empty string
The above sits well with constructs like:
```
{{ with .BundleType }}
// Now we know it is a bundle
{{ end }}
```
Fixes#4662
This resolves some surprising behaviour when reading other pages' content from shortcodes. Before this commit, that behaviour was undefined. Note that this has never been an issue from regular templates.
It will still not be possible to get **the current shortcode's page's rendered content**. That would have impressed Einstein.
The new and well defined rules are:
* `.Page.Content` from a shortcode will be empty. The related `.Page.Truncated` `.Page.Summary`, `.Page.WordCount`, `.Page.ReadingTime`, `.Page.Plain` and `.Page.PlainWords` will also have empty values.
* For _other pages_ (retrieved via `.Page.Site.GetPage`, `.Site.Pages` etc.) the `.Content` is there to use as you please as long as you don't have infinite content recursion in your shortcode/content setup. See below.
* `.Page.TableOfContents` is good to go (but does not support shortcodes in headlines; this is unchanged)
If you get into a situation of infinite recursion, the `.Content` will be empty. Run `hugo -v` for more information.
Fixes#4632Fixes#4653Fixes#4655
For the content from other pages in shortcodes there are some chicken and
egg dependencies that is hard to get around. But we can improve on this by preparing the pages in a certain order:
1. The headless bundles goes first. These are page typically page and image collections..
2. Leaf bundles
3. Regular single pages
4. Branch bundles
Fixes#4632
A sample config:
```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true
[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"
[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```
The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.
The content files will be assigned a language by
1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.
The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.
Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.
If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.
Fixes#4523Fixes#4552Fixes#4553
As an example:
```html
{{ $pages := .Site.RegularPages | lang.Merge $frSite.RegularPages | lang.Merge $enSite.RegularPages }}
```
Will "fill in the gaps" in the current site with, from left to right, content from the French site, and lastly the English.
Fixes#4463
This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter.
This should make it easier to move content from Jekyll to Hugo.
To enable, put this in your `config.toml`:
```toml
[frontmatter]
date = [":filename", ":default"]
```
This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc.
So, if you want to use the `file modification time`, this can be a good configuration:
```toml
[frontmatter]
date = [ "date",":fileModTime", ":default"]
lastmod = ["lastmod" ,":fileModTime", ":default"]
```
The current `:default` values for the different dates are
```toml
[frontmatter]
date = ["date","publishDate", "lastmod"]
lastmod = ["lastmod", "date","publishDate"]
publishDate = ["publishDate", "date"]
expiryDate = ["expiryDate"]
```
The above will now be the same as:
```toml
[frontmatter]
date = [":default"]
lastmod = [":default"]
publishDate = [":default"]
expiryDate = [":default"]
```
Note:
* We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate.
* If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved.
* All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`).
* The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values.
* The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list.
Fixes#285Closes#3310Closes#3762Closes#4340
* Page without front matter now treated same as a page with empty front matter.
* Test cases added to cover this and repro issue #4320.
* Type safety of front matter code improved.
Fixes#4320
This commit adds support for `headless bundles` for the `index` bundle type.
So:
```toml
headless = true
```
In front matter means that
* It will have no `Permalink` and no rendered HTML in /public
* It will not be part of `.Site.RegularPages` etc.
But you can get it by:
* `.Site.GetPage ...`
The use cases are many:
* Shared media galleries
* Reusable page content "snippets"
* ...
Fixes#4311
This commit expands the Resource interface with 3 new methods:
* Name
* Title
* Params
All of these can be set in the Page front matter. `Name` will get its default value from the base filename, and is the value used in the ByPrefix and GetByPrefix lookup methods.
Fixes#4244
We still do lowering of the param strings in some internal use of this, but the exported `GetParam` method is changed to a more sensible default.
This was used for the `disqus_title` etc. in the internal Disqus template, which was obviously not right.
If you really want to lowercase your params, do it with `.GetParam "myparam" | lower` or similar.
Fixes#4187
This commit is not the smallest in Hugo's history.
Some hightlights include:
* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).
A site building benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:
```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 101785785 78067944 -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 185481057 149159919 -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 103149918 85679409 -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 203515478 169208775 -16.86%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 532464 391539 -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 1056549 772702 -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 555974 406630 -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 1086545 789922 -27.30%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 53243246 43598155 -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 105811617 86087116 -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 54558852 44545097 -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 106903858 86978413 -18.64%
```
Fixes#3651Closes#3158Fixes#1014Closes#2021Fixes#1240
Updates #3757
This is in heavy use in rendering, so this makes a difference:
```bash
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 124551144 107743429 -13.49%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 528684 435118 -17.70%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 53306848 45147832 -15.31%
```
This commit refines the key used to map translations:
* Use `translationKey` set in front matter
* Fall back to path + base filename (i.e. the filename without extension and language code)
Note that the Page Kinde will be prepended to both cases above. It does not make sense to have a section as translation for the home page.
Fixes#2699
This commit adds support for multiple statDirs both on the global and language level.
A simple `config.toml` example:
```bash
staticDir = ["static1", "static2"]
[languages]
[languages.no]
staticDir = ["staticDir_override", "static_no"]
baseURL = "https://example.no"
languageName = "Norsk"
weight = 1
title = "På norsk"
[languages.en]
staticDir2 = "static_en"
baseURL = "https://example.com"
languageName = "English"
weight = 2
title = "In English"
```
In the above, with no theme used:
the English site will get its static files as a union of "static1", "static2" and "static_en". On file duplicates, the right-most version will win.
the Norwegian site will get its static files as a union of "staticDir_override" and "static_no".
This commit also concludes the Multihost support in #4027.
Fixes#36Closes#4027
Return the correct virtual Section for Taxonomy and TaxonomyTerm.
Restrict permalink expansion to only Pages and Taxonomies, but then
actually use expanded permalinks even for non-Pages.
Fixes#1208.
Changes fall into one of the following:
- gofmt -s
- receiver name is inconsistent
- omit unused 2nd value from range
- godoc comment formed incorrectly
- err assigned and not used
- if block ends with a return statement followed by else
Move SummaryLength into the ContentSpec struct and refactor the
relevant summary functions to be methods of ContentSpec. The new
summaryLength struct member is configurable by the summaryLength config
value, and the default remains 70. Also updates hugolib/page to use the
refactored methods.
Resolves#3734
Also related:
* support "modified" as an optional way to signal "last modified"
* make sure all relevant page dates are also added to params
Fixes#3867
This closes#98, even if this commit does not do full content text search.
We may revisit that problem in the future, but that deserves its own issue.
Fixes#98
And use it in `eq` and `ne` so `Page` values can be compared directly in the templates without thinking about it being a `Page` or a `PageOutput` wrapper.
Fixes#3807
This applies to both regular templates and shortcodes. So, if the site language is French and the output format is AMP, this is the (start) of the lookup order for the home page:
1. index.fr.amp.html
2. index.amp.html
3. index.fr.html
4. index.html
5. ...
Fixes#3360
