This resolves some surprising behaviour when reading other pages' content from shortcodes. Before this commit, that behaviour was undefined. Note that this has never been an issue from regular templates.
It will still not be possible to get **the current shortcode's page's rendered content**. That would have impressed Einstein.
The new and well defined rules are:
* `.Page.Content` from a shortcode will be empty. The related `.Page.Truncated` `.Page.Summary`, `.Page.WordCount`, `.Page.ReadingTime`, `.Page.Plain` and `.Page.PlainWords` will also have empty values.
* For _other pages_ (retrieved via `.Page.Site.GetPage`, `.Site.Pages` etc.) the `.Content` is there to use as you please as long as you don't have infinite content recursion in your shortcode/content setup. See below.
* `.Page.TableOfContents` is good to go (but does not support shortcodes in headlines; this is unchanged)
If you get into a situation of infinite recursion, the `.Content` will be empty. Run `hugo -v` for more information.
Fixes#4632Fixes#4653Fixes#4655
As an example:
```html
{{ $pages := .Site.RegularPages | lang.Merge $frSite.RegularPages | lang.Merge $enSite.RegularPages }}
```
Will "fill in the gaps" in the current site with, from left to right, content from the French site, and lastly the English.
Fixes#4463
Also:
- Remove unnecessary space from `figure` tag if no class is specified.
- Update related tests.
- Add test cases for the changes made to the figure shortcode.
- Document the newly added target and rel parameters
- Add more detail to the documentation of all figure shortcode parameters.
This commit is not the smallest in Hugo's history.
Some hightlights include:
* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).
A site building benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:
```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 101785785 78067944 -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 185481057 149159919 -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 103149918 85679409 -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 203515478 169208775 -16.86%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 532464 391539 -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 1056549 772702 -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 555974 406630 -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 1086545 789922 -27.30%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 53243246 43598155 -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 105811617 86087116 -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 54558852 44545097 -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 106903858 86978413 -18.64%
```
Fixes#3651Closes#3158Fixes#1014Closes#2021Fixes#1240
Updates #3757
Recognize the Pandoc format under the file extension .pandoc or .pdc,
and shell out to pandoc as an external helper to format Pandoc content.
Refactor out repeated code with external helpers. Change the error
output formatting. I did not see any of the external helpers print the
string "<input>" to represent stdin as a file; just prepending the file
name to error output is more general and doesn't sacrifice that much in
terms of readability.
Closes#234
This applies to both regular templates and shortcodes. So, if the site language is French and the output format is AMP, this is the (start) of the lookup order for the home page:
1. index.fr.amp.html
2. index.amp.html
3. index.fr.html
4. index.html
5. ...
Fixes#3360
This commit allows shortcode per output format, a typical use case would be the special AMP media tags.
Note that this will only re-render the "overridden" shortcodes and only in pages where these are used, so performance in the normal case should not suffer.
Closes#3220
Before this commit, Hugo used `html/template` for all Go templates.
While this is a fine choice for HTML and maybe also RSS feeds, it is painful for plain text formats such as CSV, JSON etc.
This commit fixes that by using the `IsPlainText` attribute on the output format to decide what to use.
A couple of notes:
* The above requires a nonambiguous template name to type mapping. I.e. `/layouts/_default/list.json` will only work if there is only one JSON output format, `/layouts/_default/list.mytype.json` will always work.
* Ambiguous types will fall back to HTML.
* Partials inherits the text vs HTML identificator of the container template. This also means that plain text templates can only include plain text partials.
* Shortcode templates are, by definition, currently HTML templates only.
Fixes#3221
Before this commit, Hugo used `html/template` for all Go templates.
While this is a fine choice for HTML and maybe also RSS feeds, it is painful for plain text formats such as CSV, JSON etc.
This commit fixes that by using the `IsPlainText` attribute on the output format to decide what to use.
A couple of notes:
* The above requires a nonambiguous template name to type mapping. I.e. `/layouts/_default/list.json` will only work if there is only one JSON output format, `/layouts/_default/list.mytype.json` will always work.
* Ambiguous types will fall back to HTML.
* Partials inherits the text vs HTML identificator of the container template. This also means that plain text templates can only include plain text partials.
* Shortcode templates are, by definition, currently HTML templates only.
Fixes#3221
This commit fixes two different, but related issues:
1) Live-reload when a new shortcode was defined in the content file before the shortcode itself was created.
2) Live-reload when a newly defined shortcode changed its "inner content" status.
This commit also improves the shortcode related error messages to include the full path to the content file in question.
Fixes#3156
Note that this looks like overkill for just the logger, and that is correct,
but this will make sense once we start with the template handling etc.
Updates #2701
All config variables starts with low-case and uses camelCase.
If there is abbreviation at the beginning of the name, the whole
abbreviation will be written in low-case.
If there is abbreviation at the end of the name, the
whole abbreviation will be written in upper-case.
For example, rssURI.
This is the nth attempt to fix an issue by changing the placeholder token pattern, but
now we actually have tests for all the historic trouble cases.
Fixes#2223
The current "rendering language" is needed outside of Site. This commit moves the Language type to the helpers package, and then used to get correct correct language configuration in the markdownify template func.
This commit also adds two new template funcs: relLangURL and absLangURL.
See #2309
This is needed to make shortcode users happy with the new multilanguage support,
but it will also solve many other related posts about "stuff not available in the shortcode".
We will have to revisit this re the handler chain at some point, but that will be easier
now as the integration test story has improved so much.
As part of this commit, the site-building tests in page_test.go is refreshed, they now
tests for all the rendering engines (when available), and all of them now uses the
same code-path as used in production.
Fixes#1229Fixes#2323
Fixes ##1076
Work In Progress!
This commit makes a rework of the build and rebuild process to better suit a multi-site setup.
This also includes a complete overhaul of the site tests. Previous these were a messy mix that
were testing just small parts of the build chain, some of it testing code-paths not even used in
"real life". Now all tests that depends on a built site follows the same and real production code path.
See #2309Closes#2211Closes#477Closes#1744
And a Hugo global variable which contains the site under build.
This is really needed to get some level of control of the "multiple languages" in play.
There are still work related to this scattered around, but that will come.
With this commit, the multilingual feature is starting to work.
This issue was introduced as a fix to shortcode not working in RST.
One could argue that Blackfriday and friends should handle `#` in titles, but that will be a discussion
for another day.
The new placeholder pattern should be RST safe and work with titles.
And now with a test so this doesn't break again.
Fixes#2192Fixes#2209Closes#2210
This also includes a refactor of the hugofs package and its usage.
The motivation for that is:
The Afero filesystems are brilliant. Hugo's way of adding a dozen of global variables for the different filesystems was a mistake. In readFile (and also in some other places in Hugo today) we need a way to restrict the access inside the working dir. We could use ioutil.ReadFile and implement the path checking, checking the base path and the dots ("..") etc. But it is obviously better to use an Afero BasePathFs combined witha ReadOnlyFs. We could create a use-once-filesystem and handle the initialization ourselves, but since this is also useful to others and the initialization depends on some other global state (which would mean to create a new file system on every invocation), we might as well do it properly and encapsulate the predefined set of filesystems. This change also leads the way, if needed, to encapsulate the file systems in a struct, making it possible to have several file system sets in action at once (parallel multilanguage site building? With Moore's law and all...)
Fixes#1551
It would be helpful to know whether a shortcode was called with positional or
named parameters. This commit adds a boolean `IsNamedParams` property to the
`ShortcodeWithPage` struct.
Currently a `[]byte` copy is returned. In most cases this is the safe thing to do, but we should just modify/grow the slice as needed.
This is faster and consumes less memory:
```
benchmark old ns/op new ns/op delta
BenchmarkReplaceShortcodeTokens-4 7350 4419 -39.88%
benchmark old allocs new allocs delta
BenchmarkReplaceShortcodeTokens-4 5 1 -80.00%
benchmark old bytes new bytes delta
BenchmarkReplaceShortcodeTokens-4 4816 1152 -76.08%
```
This commit is aso a small spring cleaning of duplicated code in the different `PageConvert` methods.
Fixes#1516
This commit replaces the regexp driven `replaceShortcodeTokens` with a handwritten one.
It wasnt't possible to handle the p-tags case without breaking performance.
This fix actually improves in that area:
```
benchmark old ns/op new ns/op delta
BenchmarkParsePage 142738 142667 -0.05%
BenchmarkReplaceShortcodeTokens 665590 575645 -13.51%
BenchmarkShortcodeLexer 176038 181074 +2.86%
benchmark old allocs new allocs delta
BenchmarkParsePage 87 87 +0.00%
BenchmarkReplaceShortcodeTokens 9631 9424 -2.15%
BenchmarkShortcodeLexer 274 274 +0.00%
benchmark old bytes new bytes delta
BenchmarkParsePage 141830 141830 +0.00%
BenchmarkReplaceShortcodeTokens 52275 35219 -32.63%
BenchmarkShortcodeLexer 30177 30178 +0.00%
```
Fixes#1148
Many minor fixes to make test logs more consistent and correct a
mispelling.
Standardize on "[%i] got X but expected Y" for log messages. Using
a consistent layout makes it easier to read the test results. This
was mostly changing "Got" to "got". Swapped the order of values on
several calls to bring them in line with the convention.
A few log messages had a sequence number added to identify the
exact scenario that failed. Otherwise, there would be no way to
ascertain which failed When there are many scenarios.
Correct spelling of "expected."
Fixes#1028
Merged be2097e1ad
[close#1040]
- `.Ref` and `.RelRef` take a reference (the logical filename for a
page, including extension and/or a document fragment ID) and return
a permalink (or relative permalink) to the referenced document.
- If the reference is a page name (such as `about.md`), the page
will be discovered and the permalink will be returned: `/about/`
- If the reference is a page name with a fragment (such as
`about.md#who`), the page will be discovered and used to add the
`page.UniqueID()` to the resulting fragment and permalink:
`/about/#who:deadbeef`.
- If the reference is a fragment and `.*Ref` has been called from
a `Node` or `SiteInfo`, it will be returned as is: `#who`.
- If the reference is a fragment and `.*Ref` has been called from
a `Page`, it will be returned with the page’s unique ID:
`#who:deadbeef`.
- `.*Ref` can be called from either `Node`, `SiteInfo` (e.g.,
`Node.Site`), `Page` objects, or `ShortcodeWithPage` objects in
templates.
- `.*Ref` cannot be used in content, so two shortcodes have been
created to provide the functionality to content: `ref` and `relref`.
These are intended to be used within markup, like `[Who]({{% ref
about.md#who %}})` or `<a href="{{% ref about.md#who %}}">Who</a>`.
- There are also `ref` and `relref` template functions (used to create
the shortcodes) that expect a `Page` or `Node` object and the
reference string (e.g., `{{ relref . "about.md" }}` or `{{
"about.md" | ref . }}`). It actually looks for `.*Ref` as defined on
`Node` or `Page` objects.
- Shortcode handling had to use a *differently unique* wrapper in
`createShortcodePlaceholder` because of the way that the `ref` and
`relref` are intended to be used in content.
Enable blackfriday.EXTENSION_AUTO_HEADER_IDS to generate the name of the
header ID from the text in the header. Works for prefix and underline
headers.
- TOC extraction had to be modified to look for `<li><a href="#`>
instead of `#toc_` because of this change.
- Fixed a number of tests that depended on the presence of `toc_` with
as an `id` or as a `href` value.
- Renames the earlier parameter `footnoteref` to `documentId` as it more
accurately represents the nature of the parameter. The `documentId` is
appended to all generated headers through the new HTML renderer
parameter `HeaderIDSuffix`.
Some newly added shortcode tests compared maps in assertions.
This failed on Travis, as iteration order isn't guaranteed for maps since Go 1.
This commit fixes that by do a sort of the keys in the shortcode String() function.
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes#565Fixes#480Fixes#461
And probably some others.
