This more or less completes the simplification of the template handling code in Hugo started in v0.62.
The main motivation was to fix a long lasting issue about a crash in HTML content files without front matter.
But this commit also comes with a big functional improvement.
As we now have moved the base template evaluation to the build stage we now use the same lookup rules for `baseof` as for `list` etc. type of templates.
This means that in this simple example you can have a `baseof` template for the `blog` section without having to duplicate the others:
```
layouts
├── _default
│ ├── baseof.html
│ ├── list.html
│ └── single.html
└── blog
└── baseof.html
```
Also, when simplifying code, you often get rid of some double work, as shown in the "site building" benchmarks below.
These benchmarks looks suspiciously good, but I have repeated the below with ca. the same result. Compared to master:
```
name old time/op new time/op delta
SiteNew/Bundle_with_image-16 13.1ms ± 1% 10.5ms ± 1% -19.34% (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16 13.0ms ± 0% 10.7ms ± 1% -18.05% (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16 46.4ms ± 2% 43.1ms ± 1% -7.15% (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16 52.2ms ± 2% 47.8ms ± 1% -8.30% (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16 77.9ms ± 1% 70.9ms ± 1% -9.01% (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16 43.0ms ± 0% 37.2ms ± 1% -13.54% (p=0.029 n=4+4)
SiteNew/Page_collections-16 58.2ms ± 1% 52.4ms ± 1% -9.95% (p=0.029 n=4+4)
name old alloc/op new alloc/op delta
SiteNew/Bundle_with_image-16 3.81MB ± 0% 2.22MB ± 0% -41.70% (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16 3.60MB ± 0% 2.01MB ± 0% -44.20% (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16 19.3MB ± 1% 14.1MB ± 0% -26.91% (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16 70.7MB ± 0% 69.0MB ± 0% -2.40% (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16 37.1MB ± 0% 31.2MB ± 0% -15.94% (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16 17.6MB ± 0% 10.6MB ± 0% -39.92% (p=0.029 n=4+4)
SiteNew/Page_collections-16 25.9MB ± 0% 21.2MB ± 0% -17.99% (p=0.029 n=4+4)
name old allocs/op new allocs/op delta
SiteNew/Bundle_with_image-16 52.3k ± 0% 26.1k ± 0% -50.18% (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16 52.3k ± 0% 26.1k ± 0% -50.16% (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16 336k ± 1% 269k ± 0% -19.90% (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16 422k ± 0% 395k ± 0% -6.43% (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16 401k ± 0% 313k ± 0% -21.79% (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16 247k ± 0% 143k ± 0% -42.17% (p=0.029 n=4+4)
SiteNew/Page_collections-16 282k ± 0% 207k ± 0% -26.55% (p=0.029 n=4+4)
```
Fixes#6716Fixes#6760Fixes#6768Fixes#6778
This is the first version of Goldmark that supports all the
Smartypants-style typographic punctuation transformations. Now, a
straight single quote in the middle of a word is translated into a curly
quote (e.g. "that's" becomes "that’s"). Earlier versions leave
them untouched. This brings Goldmark in line with Blackfriday.
Fixes#6571.
This commit also
* revises the change detection for templates used by content files in server mode.
* Adds a Page.RenderString method
Fixes#6545Fixes#4663Closes#6043
This is a big commit, but it deletes lots of code and simplifies a lot.
* Resolving the template funcs at execution time means we don't have to create template clones per site
* Having a custom map resolver means that we can remove the AST lower case transformation for the special lower case Params map
Not only is the above easier to reason about, it's also faster, especially if you have more than one language, as in the benchmark below:
```
name old time/op new time/op delta
SiteNew/Deep_content_tree-16 53.7ms ± 0% 48.1ms ± 2% -10.38% (p=0.029 n=4+4)
name old alloc/op new alloc/op delta
SiteNew/Deep_content_tree-16 41.0MB ± 0% 36.8MB ± 0% -10.26% (p=0.029 n=4+4)
name old allocs/op new allocs/op delta
SiteNew/Deep_content_tree-16 481k ± 0% 410k ± 0% -14.66% (p=0.029 n=4+4)
```
This should be even better if you also have lots of templates.
Closes#6594
This commit adds the fast and CommonMark compliant Goldmark as the new default markdown handler in Hugo.
If you want to continue using BlackFriday as the default for md/markdown extensions, you can use this configuration:
```toml
[markup]
defaultMarkdownHandler="blackfriday"
```
Fixes#5963Fixes#1778Fixes#6355
This commmit prepares for the addition of Goldmark as the new Markdown renderer in Hugo.
This introduces a new `markup` package with some common interfaces and each implementation in its own package.
See #5963
This is preparation for #6041.
For historic reasons, the code for bulding the section tree and the taxonomies were very much separate.
This works, but makes it hard to extend, maintain, and possibly not so fast as it could be.
This simplification also introduces 3 slightly breaking changes, which I suspect most people will be pleased about. See referenced issues:
This commit also switches the radix tree dependency to a mutable implementation: github.com/armon/go-radix.
Fixes#6154Fixes#6153Fixes#6152
This commit implements Hugo Modules.
This is a broad subject, but some keywords include:
* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`, `hugo mod get`, `hugo mod graph`, `hugo mod tidy`, and `hugo mod vendor`.
All of the above is backed by Go Modules.
Fixes#5973Fixes#5996Fixes#6010Fixes#5911Fixes#5940Fixes#6074Fixes#6082Fixes#6092
Sadly, goorgeous has not been updated in over a year and still has a lot of
open issues (e.g. no support for nested lists).
go-org fixes most of those issues and supports a larger subset of Org mode
syntax.
Add the ability to have a `summary` page variable that overrides
the auto-generated summary. Logic for obtaining summary becomes:
* if summary divider is present in content, use the text above it
* if summary variables is present in page metadata, use that
* auto-generate summary from first _x_ words of the content
Fixes#5800
The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct.
This is all a preparation step for issue #5074, "pages from other data sources".
But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes.
Most notable changes:
* The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday.
This means that any markdown will partake in the global ToC and footnote context etc.
* The Custom Output formats are now "fully virtualized". This removes many of the current limitations.
* The taxonomy list type now has a reference to the `Page` object.
This improves the taxonomy template `.Title` situation and make common template constructs much simpler.
See #5074Fixes#5763Fixes#5758Fixes#5090Fixes#5204Fixes#4695Fixes#5607Fixes#5707Fixes#5719Fixes#3113Fixes#5706Fixes#5767Fixes#5723Fixes#5769Fixes#5770Fixes#5771Fixes#5759Fixes#5776Fixes#5777Fixes#5778
This avoids double parsing the page content when `enableEmoji=true`.
This commit also adds some general improvements to the parser, making it in general much faster:
```bash
benchmark old ns/op new ns/op delta
BenchmarkShortcodeLexer-4 90258 101730 +12.71%
BenchmarkParse-4 148940 15037 -89.90%
benchmark old allocs new allocs delta
BenchmarkShortcodeLexer-4 456 700 +53.51%
BenchmarkParse-4 28 33 +17.86%
benchmark old bytes new bytes delta
BenchmarkShortcodeLexer-4 69875 81014 +15.94%
BenchmarkParse-4 8128 8304 +2.17%
```
Running some site benchmarks with Emoji support turned on:
```bash
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 924556797 818115620 -11.51%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 4112613 4133787 +0.51%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 426982864 424363832 -0.61%
```
Fixes#5534
This commit adds support for a configuration directory (default `config`). The different pieces in this puzzle are:
* A new `--environment` (or `-e`) flag. This can also be set with the `HUGO_ENVIRONMENT` OS environment variable. The value for `environment` defaults to `production` when running `hugo` and `development` when running `hugo server`. You can set it to any value you want (e.g. `hugo server -e "Sensible Environment"`), but as it is used to load configuration from the file system, the letter case may be important. You can get this value in your templates with `{{ hugo.Environment }}`.
* A new `--configDir` flag (defaults to `config` below your project). This can also be set with `HUGO_CONFIGDIR` OS environment variable.
If the `configDir` exists, the configuration files will be read and merged on top of each other from left to right; the right-most value will win on duplicates.
Given the example tree below:
If `environment` is `production`, the left-most `config.toml` would be the one directly below the project (this can now be omitted if you want), and then `_default/config.toml` and finally `production/config.toml`. And since these will be merged, you can just provide the environment specific configuration setting in you production config, e.g. `enableGitInfo = true`. The order within the directories will be lexical (`config.toml` and then `params.toml`).
```bash
config
├── _default
│ ├── config.toml
│ ├── languages.toml
│ ├── menus
│ │ ├── menus.en.toml
│ │ └── menus.zh.toml
│ └── params.toml
├── development
│ └── params.toml
└── production
├── config.toml
└── params.toml
```
Some configuration maps support the language code in the filename (e.g. `menus.en.toml`): `menus` (`menu` also works) and `params`.
Also note that the only folders with "a meaning" in the above listing is the top level directories below `config`. The `menus` sub folder is just added for better organization.
We use `TOML` in the example above, but Hugo also supports `JSON` and `YAML` as configuration formats. These can be mixed.
Fixes#5422
This means that the current `.Site` and ´.Hugo` is available as a globals, so you can do `site.IsServer`, `hugo.Version` etc.
Fixes#5470Fixes#5467Fixes#5503
When the page parser was rewritten in 0.51, this was interpreted literally, but commented out front matter is used in the wild to "hide it from GitHub", e.g:
```
<!--
+++
title = "hello"
+++
-->
```
Fixes#5478
The main item in this commit is showing of errors with a file context when running `hugo server`.
This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).
But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.
Fixes#5284Fixes#5290
See #5325
See #5324
This bug was introduced in Hugo 0.40. It is when you use the `<!--more-->` summary marker.
Note that this affects the word stats only. The related `PlainWords`, `Plain`, `Content` all return correct values.
Fixes#4675Fixes#4682
This is a follow-up to #4632. There were some assumptions in that implementation that did not hold water in all situations.
This commit simplifies the content lazy initalization making it more robust.
Fixes#4664
This resolves some surprising behaviour when reading other pages' content from shortcodes. Before this commit, that behaviour was undefined. Note that this has never been an issue from regular templates.
It will still not be possible to get **the current shortcode's page's rendered content**. That would have impressed Einstein.
The new and well defined rules are:
* `.Page.Content` from a shortcode will be empty. The related `.Page.Truncated` `.Page.Summary`, `.Page.WordCount`, `.Page.ReadingTime`, `.Page.Plain` and `.Page.PlainWords` will also have empty values.
* For _other pages_ (retrieved via `.Page.Site.GetPage`, `.Site.Pages` etc.) the `.Content` is there to use as you please as long as you don't have infinite content recursion in your shortcode/content setup. See below.
* `.Page.TableOfContents` is good to go (but does not support shortcodes in headlines; this is unchanged)
If you get into a situation of infinite recursion, the `.Content` will be empty. Run `hugo -v` for more information.
Fixes#4632Fixes#4653Fixes#4655
A sample config:
```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true
[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"
[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```
The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.
The content files will be assigned a language by
1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.
The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.
Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.
If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.
Fixes#4523Fixes#4552Fixes#4553
This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter.
This should make it easier to move content from Jekyll to Hugo.
To enable, put this in your `config.toml`:
```toml
[frontmatter]
date = [":filename", ":default"]
```
This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc.
So, if you want to use the `file modification time`, this can be a good configuration:
```toml
[frontmatter]
date = [ "date",":fileModTime", ":default"]
lastmod = ["lastmod" ,":fileModTime", ":default"]
```
The current `:default` values for the different dates are
```toml
[frontmatter]
date = ["date","publishDate", "lastmod"]
lastmod = ["lastmod", "date","publishDate"]
publishDate = ["publishDate", "date"]
expiryDate = ["expiryDate"]
```
The above will now be the same as:
```toml
[frontmatter]
date = [":default"]
lastmod = [":default"]
publishDate = [":default"]
expiryDate = [":default"]
```
Note:
* We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate.
* If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved.
* All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`).
* The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values.
* The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list.
Fixes#285Closes#3310Closes#3762Closes#4340
Also:
- Remove unnecessary space from `figure` tag if no class is specified.
- Update related tests.
- Add test cases for the changes made to the figure shortcode.
- Document the newly added target and rel parameters
- Add more detail to the documentation of all figure shortcode parameters.
* Page without front matter now treated same as a page with empty front matter.
* Test cases added to cover this and repro issue #4320.
* Type safety of front matter code improved.
Fixes#4320
This commit expands the Resource interface with 3 new methods:
* Name
* Title
* Params
All of these can be set in the Page front matter. `Name` will get its default value from the base filename, and is the value used in the ByPrefix and GetByPrefix lookup methods.
Fixes#4244
We still do lowering of the param strings in some internal use of this, but the exported `GetParam` method is changed to a more sensible default.
This was used for the `disqus_title` etc. in the internal Disqus template, which was obviously not right.
If you really want to lowercase your params, do it with `.GetParam "myparam" | lower` or similar.
Fixes#4187
This commit is not the smallest in Hugo's history.
Some hightlights include:
* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).
A site building benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:
```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 101785785 78067944 -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 185481057 149159919 -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 103149918 85679409 -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 203515478 169208775 -16.86%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 532464 391539 -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 1056549 772702 -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 555974 406630 -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 1086545 789922 -27.30%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 53243246 43598155 -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 105811617 86087116 -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 54558852 44545097 -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 106903858 86978413 -18.64%
```
Fixes#3651Closes#3158Fixes#1014Closes#2021Fixes#1240
Updates #3757
Recognize the Pandoc format under the file extension .pandoc or .pdc,
and shell out to pandoc as an external helper to format Pandoc content.
Refactor out repeated code with external helpers. Change the error
output formatting. I did not see any of the external helpers print the
string "<input>" to represent stdin as a file; just prepending the file
name to error output is more general and doesn't sacrifice that much in
terms of readability.
Closes#234
This commit refines the key used to map translations:
* Use `translationKey` set in front matter
* Fall back to path + base filename (i.e. the filename without extension and language code)
Note that the Page Kinde will be prepended to both cases above. It does not make sense to have a section as translation for the home page.
Fixes#2699
This is a pretty fundamental change in Hugo, but absolutely needed if we should have any hope of getting "multiple outputs" done.
This commit's goal is to say:
* Every file target path is created by `createTargetPath`, i.e. one function for all.
* That function takes every page and site parameter into account, to avoid fragile string parsing to uglify etc. later on.
* The path creation logic has full test coverage.
* All permalinks, paginator URLs etc. are then built on top of that same logic.
Fixes#1252Fixes#2110Closes#2374Fixes#1885Fixes#3102Fixes#3179Fixes#1641Fixes#1989
This fixes the behavior of .Truncated that was introduced with commit
bef496b97e which was later broken. The
desired behavior is that .Truncated would evaluate to false when there
was nothing after the user defined summary marker.
This also adds a simple unit test to ensure that this feature isn't
broken again. The check for content after the user defined summary
marker is done on the raw content instead of the working copy because
some of the markup renderers add elements after the marker, making it
difficult to determine if there is actually any content.
The behavior (evaluating to false when there is no content, just
summary) is also now documented.
The Param method currently assumes that its argument is a single,
distinct, top-level key to look up in the Params map. This enhances the
Param method; it will now also attempt to see if the key can be
interpreted as a nested chain of keys to look up in Params.
Fixes#2598
Note that this looks like overkill for just the logger, and that is correct,
but this will make sense once we start with the template handling etc.
Updates #2701
