Commit graph

394 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
086ae81a98
hugolib: Fix possible .Content cut
There have been one report of a site with truncated `.Content` after the Hugo `0.40.1` release.

This commit fixes this so that race should not be possible anymore. It also adds a stress test with focus on content rendering and multiple output formats.

Fixes #4706
2018-05-08 16:52:51 +02:00
Bjørn Erik Pedersen
001a28c2f9 Fix .WordCount, .FuzzyWordCount, .ReadingTime when summary marker is set
This bug was introduced in Hugo 0.40. It is when you use the `<!--more-->` summary marker.

Note that this affects the word stats only. The related `PlainWords`, `Plain`, `Content` all return correct values.

Fixes #4675
Fixes #4682
2018-04-29 10:58:58 +02:00
Bjørn Erik Pedersen
288c396439
hugolib: Fix some shortcode vs .Content corner cases
This is a follow-up to #4632. There were some assumptions in that implementation that did not hold water in all situations.

This commit simplifies the content lazy initalization making it more robust.

Fixes #4664
2018-04-25 08:56:46 +02:00
Bjørn Erik Pedersen
402f6788ee
hugolib: Add .Page.BundleType
I eturn either:

1. leaf
2. branch
3. empty string

The above sits well with constructs like:

```
{{ with .BundleType }}
// Now we know it is a bundle
{{ end }}
```

Fixes #4662
2018-04-23 08:41:19 +02:00
Bjørn Erik Pedersen
4d26ab33dc
Make .Content (almost) always available in shortcodes
This resolves some surprising behaviour when reading other pages' content from shortcodes. Before this commit, that behaviour was undefined. Note that this has never been an issue from regular templates.

It will still not be possible to get **the current shortcode's  page's rendered content**. That would have impressed Einstein.

The new and well defined rules are:

* `.Page.Content` from a shortcode will be empty. The related `.Page.Truncated` `.Page.Summary`, `.Page.WordCount`, `.Page.ReadingTime`, `.Page.Plain` and `.Page.PlainWords` will also have empty values.
* For _other pages_ (retrieved via `.Page.Site.GetPage`, `.Site.Pages` etc.) the `.Content` is there to use as you please as long as you don't have infinite content recursion in your shortcode/content setup. See below.
* `.Page.TableOfContents` is good to go (but does not support shortcodes in headlines; this is unchanged)

If you get into a situation of infinite recursion, the `.Content` will be empty. Run `hugo -v` for more information.

Fixes #4632
Fixes #4653
Fixes #4655
2018-04-21 22:02:56 +02:00
Bjørn Erik Pedersen
d6a2024e6b
Revert "Improve .Content vs shortcodes"
This reverts commit e590cc26eb.
2018-04-19 17:40:57 +02:00
Bjørn Erik Pedersen
d6982ac0ac
Revert "hugolib: Prepare child page resources before the page itself"
This reverts commit 3238e14fdf.
2018-04-19 17:40:54 +02:00
Bjørn Erik Pedersen
3238e14fdf
hugolib: Prepare child page resources before the page itself
To make sure that the child resources' content is ready before any shortcode is processed in their owner.

See #4632
2018-04-19 16:35:21 +02:00
Bjørn Erik Pedersen
e590cc26eb
Improve .Content vs shortcodes
For the content from other pages in shortcodes there are some chicken and
egg dependencies that is hard to get around. But we can improve on this  by preparing the pages in a certain order:

 1. The headless bundles goes first. These are page typically page and image collections..
 2. Leaf bundles
 3. Regular single pages
 4. Branch bundles

Fixes #4632
2018-04-19 14:46:50 +02:00
Bjørn Erik Pedersen
d681ea55a0
Improve error message in metadata parse
Fixes #3696
2018-04-17 15:37:05 +02:00
Bjørn Erik Pedersen
417c5e2b67 Make Page.Content a method that returns interface{}
To prepare for a `Resource.Content` method.

See #4622
2018-04-15 18:08:06 +02:00
Bjørn Erik Pedersen
a4deaeff0c
Fix some livereload content regressions
Introduced in Hugo 0.38.

Fixes #4566
2018-04-05 14:28:45 +02:00
Bjørn Erik Pedersen
eb42774e58
Add support for a content dir set per language
A sample config:

```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true

[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"

[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```

The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.

The content files will be assigned a language by

1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.

The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.

Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.

If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.

Fixes #4523
Fixes #4552
Fixes #4553
2018-04-02 08:06:21 +02:00
Bjørn Erik Pedersen
ffaec4ca8c
Add a way to merge pages by language
As an example:

```html
{{ $pages := .Site.RegularPages | lang.Merge $frSite.RegularPages | lang.Merge $enSite.RegularPages }}
```

Will "fill in the gaps" in the current site with, from left to right, content from the French site, and lastly the English.

Fixes #4463
2018-03-16 10:10:01 +01:00
Bjørn Erik Pedersen
ce6e4310fe
Refactor the GitInfo into the date handlers
Fixes #4495
2018-03-11 21:32:05 +01:00
Bjørn Erik Pedersen
68bf1511f2
hugolib: Extract date and slug from filename
This commit makes it possible to extract the date from the content filename. Also, the filenames in these cases will make for very poor permalinks, so we will also use the remaining part as the page `slug` if that value is not set in front matter.

This should make it easier to move content from Jekyll to Hugo.

To enable, put this in your `config.toml`:

```toml
[frontmatter]
date  = [":filename", ":default"]
```

This commit is also a spring cleaning of how the different dates are configured in Hugo. Hugo will check for dates following the configuration from left to right, starting with `:filename` etc.

So, if you want to use the `file modification time`, this can be a good configuration:

 ```toml
[frontmatter]
date = [ "date",":fileModTime", ":default"]
lastmod = ["lastmod" ,":fileModTime", ":default"]
```

The current `:default` values for the different dates are

```toml
[frontmatter]
date = ["date","publishDate", "lastmod"]
lastmod = ["lastmod", "date","publishDate"]
publishDate = ["publishDate", "date"]
expiryDate = ["expiryDate"]
```

The above will now be the same as:

```toml
[frontmatter]
date = [":default"]
lastmod = [":default"]
publishDate = [":default"]
expiryDate = [":default"]
```

Note:

* We have some built-in aliases to the above: lastmod => modified, publishDate => pubdate, published and expiryDate => unpublishdate.
* If you want a new configuration for, say, `date`, you can provide only that line, and the rest will be preserved.
* All the keywords to the right that does not start with a ":" maps to front matter parameters, and can be any date param (e.g. `myCustomDateParam`).
* The keywords to the left are the **4 predefined dates in Hugo**, i.e. they are constant values.
* The current "special date handlers" are `:fileModTime` and `:filename`. We will soon add `:git` to that list.

Fixes #285
Closes #3310
Closes #3762
Closes #4340
2018-03-11 18:26:18 +01:00
Bjørn Erik Pedersen
4eb2fec67c Fix handling of top-level page bundles
Fixes #4332
2018-01-27 19:13:34 +01:00
Vas Sudanagunta
322c567220 Fix "date" page param
Add test coverage for all date type page params that shadow date type page variables.

Fixes #4323
2018-01-26 09:17:27 +01:00
Vas Sudanagunta
91bb774ae4 Support pages without front matter
* Page without front matter now treated same as a page with empty front matter.
* Test cases added to cover this and repro issue #4320.
* Type safety of front matter code improved.

Fixes #4320
2018-01-26 09:17:27 +01:00
Vas Sudanagunta
3f0379adb7 Add page metadata dates tests 2018-01-26 09:17:27 +01:00
Bjørn Erik Pedersen
ae5a45be6f
hugolib: Remove unused code 2018-01-25 17:14:03 +01:00
Bjørn Erik Pedersen
d418c2c2ea
Remove and update deprecation status 2018-01-25 10:22:11 +01:00
Bjørn Erik Pedersen
0432c64dd2 Add headless bundle support
This commit adds  support for `headless bundles` for the `index` bundle type.

So:

```toml
headless = true
```

In front matter means that

* It will have no `Permalink` and no rendered HTML in /public
* It will not be part of `.Site.RegularPages` etc.

But you can get it by:

* `.Site.GetPage ...`

The use cases are many:

* Shared media galleries
* Reusable page content "snippets"
* ...

Fixes #4311
2018-01-24 09:00:21 +01:00
Vas Sudanagunta
8125b4b03d Tighten page kind logic, introduce tests 2018-01-22 13:55:08 +01:00
Bjørn Erik Pedersen
5d03086981 hugolib: Fix handling of pages bundled in sub-folders in ByPrefix etc.
Fixes #4295
2018-01-21 14:42:54 +01:00
Bjørn Erik Pedersen
20c9b6ec81
resource: Add front matter metadata to Resource
This commit expands the Resource interface with 3 new methods:

* Name
* Title
* Params

All of these can be set in the Page front matter. `Name` will get its default value from the base filename, and is the value used in the ByPrefix and GetByPrefix lookup methods.

Fixes #4244
2018-01-17 16:22:33 +01:00
Bjørn Erik Pedersen
8a409894bd hugolib: Allow url in front matter for list type pages
This enables some potential foot-shooting, but is needed for some special URL requirements.

Fixes #4263
2018-01-12 18:10:40 +01:00
Bjørn Erik Pedersen
f0eecc6a4f Fix non-ASCII path handling for Page resources
Fixes #4241
2018-01-10 18:23:41 +01:00
Bjørn Erik Pedersen
f25d8a9e17 Fix sub-folder baseURL handling for Page resources
I.e. images etc.

Fixes #4228
2018-01-06 10:29:13 +01:00
Bjørn Erik Pedersen
1c114d539b
hugolib: Do not tolower result from Page.GetParam
We still do lowering of the param strings in some internal use of this, but the exported `GetParam` method is changed to a more sensible default.

This was used for the `disqus_title` etc. in the internal Disqus template, which was obviously not right.

If you really want to lowercase your params, do it with `.GetParam "myparam" | lower` or similar.

Fixes #4187
2017-12-29 08:58:38 +01:00
Bjørn Erik Pedersen
1b0780dbeb
source: Make sure .File.Dir() ends with a slash
Updates #4190
2017-12-28 11:32:02 +01:00
Bjørn Erik Pedersen
3cdf19e9b7
Implement Page bundling and image handling
This commit is not the smallest in Hugo's history.

Some hightlights include:

* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).

A site building  benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:

```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"

benchmark                                                                                                         old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      101785785     78067944      -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     185481057     149159919     -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      103149918     85679409      -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     203515478     169208775     -16.86%

benchmark                                                                                                         old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      532464         391539         -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1056549        772702         -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      555974         406630         -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1086545        789922         -27.30%

benchmark                                                                                                         old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      53243246      43598155      -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     105811617     86087116      -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      54558852      44545097      -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     106903858     86978413      -18.64%
```

Fixes #3651
Closes #3158
Fixes #1014
Closes #2021
Fixes #1240
Updates #3757
2017-12-27 18:44:47 +01:00
Bjørn Erik Pedersen
db4b7a5c67 Reuse the BlackFriday instance when possible
This is in heavy use in rendering, so this makes a difference:

```bash
benchmark                                                                                    old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4     124551144     107743429     -13.49%

benchmark                                                                                    old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4     528684         435118         -17.70%

benchmark                                                                                    old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4     53306848      45147832      -15.31%
```
2017-12-16 19:44:33 +01:00
Richard Metzler
9df3736fec Add .Title and .Page to MenuEntry
It uses `title` if configured on the menu entry. If not, it uses the `Page.Title` when possible.

Fixes #2784
2017-12-15 18:49:21 +01:00
Bjørn Erik Pedersen
b3daa1f4bf hugolib: Fix .IsTranslated with identical filenames
This commit refines the key used to map translations:

* Use `translationKey` set in front matter
* Fall back to path + base filename (i.e. the filename without extension and language code)

Note that the Page Kinde will be prepended to both cases above. It does not make sense to have a section as translation for the home page.

Fixes #2699
2017-11-17 19:46:32 +01:00
Bjørn Erik Pedersen
60dfb9a6e0 Add support for multiple staticDirs
This commit adds support for multiple statDirs both on the global and language level.

A simple `config.toml` example:

```bash
staticDir = ["static1", "static2"]
[languages]
[languages.no]
staticDir = ["staticDir_override", "static_no"]
baseURL = "https://example.no"
languageName = "Norsk"
weight = 1
title = "På norsk"

[languages.en]
staticDir2 = "static_en"
baseURL = "https://example.com"
languageName = "English"
weight = 2
title = "In English"
```

In the above, with no theme used:

the English site will get its static files as a union of "static1", "static2" and "static_en". On file duplicates, the right-most version will win.
the Norwegian site will get its static files as a union of "staticDir_override" and "static_no".

This commit also concludes the Multihost support in #4027.

Fixes #36
Closes #4027
2017-11-17 11:01:46 +01:00
Bjørn Erik Pedersen
2e0465764b Add multilingual multihost support
This commit adds multihost support when more than one language is configured and `baseURL` is set per language.

Updates #4027
2017-11-17 11:01:46 +01:00
Brian Chen
d9a78b61ad Handle Taxonomy permalinks
Return the correct virtual Section for Taxonomy and TaxonomyTerm.
Restrict permalink expansion to only Pages and Taxonomies, but then
actually use expanded permalinks even for non-Pages.

Fixes #1208.
2017-11-07 23:05:48 +01:00
Bjørn Erik Pedersen
6a30874f19 Make sure Date and PublishDate is always set to a value if one is available
Fixes #3854
2017-10-13 13:18:23 +02:00
Cameron Moore
47fdfd5196 Clean up lint in various packages
Changes fall into one of the following:

- gofmt -s
- receiver name is inconsistent
- omit unused 2nd value from range
- godoc comment formed incorrectly
- err assigned and not used
- if block ends with a return statement followed by else
2017-09-29 16:23:16 +02:00
Brendan Roy
8717a60cc0 Change SummaryLength to be configurable (#3924)
Move SummaryLength into the ContentSpec struct and refactor the
relevant summary functions to be methods of ContentSpec. The new
summaryLength struct member is configurable by the summaryLength config
value, and the default remains 70. Also updates hugolib/page to use the
refactored methods.

Resolves #3734
2017-09-29 09:04:55 +02:00
Bjørn Erik Pedersen
202510fdc9 hugolib: Fix "published" front matter handling
Also related:

* support "modified" as an optional way to signal "last modified"
* make sure all relevant page dates are also added to params

Fixes #3867
2017-09-08 12:28:15 +02:00
Bjørn Erik Pedersen
3b4f17bbc9 hugolib: Implement "related content"
This closes #98, even if this commit does not do full content text search.

We may revisit that problem in the future, but that deserves its own issue.

Fixes #98
2017-09-06 00:20:02 +02:00
Bjørn Erik Pedersen
08f48b91d6 compare, hugolib, tpl: Add Eqer interface
And use it in `eq` and `ne` so `Page` values can be compared directly in the templates without thinking about it being a `Page` or a `PageOutput` wrapper.

Fixes #3807
2017-08-18 07:36:32 +02:00
Bjørn Erik Pedersen
71ae9b4533 hugolib: Rewrite replaceDivider to reduce memory allocation
```bash
name              old time/op    new time/op    delta
ReplaceDivider-4   9.76µs ±105%    7.96µs ±24%     ~     (p=0.690 n=5+5)

name              old alloc/op   new alloc/op   delta
ReplaceDivider-4    3.46kB ± 0%    1.54kB ± 0%  -55.56%  (p=0.008 n=5+5)

name              old allocs/op  new allocs/op  delta
ReplaceDivider-4      6.00 ± 0%      1.00 ± 0%  -83.33%  (p=0.008 n=5+5)
```
2017-08-14 09:42:55 +02:00
Bjørn Erik Pedersen
199816fddd hugolib: Extract replaceDivider logic
To its own function and add a test and a benchmark for it.
2017-08-14 09:42:55 +02:00
Bjørn Erik Pedersen
9891c0fb0e Remove sourceRelativeLinks
Fixes #3766
2017-08-02 22:04:38 +02:00
Bjørn Erik Pedersen
deccc54004 hubolib: Add HasShortcode
Fixes #3707
2017-07-17 23:20:13 +02:00
Bjørn Erik Pedersen
aa6b1b9be7 output: Support templates per site/language
This applies to both regular templates and shortcodes. So, if the site language is French and the output format is AMP, this is the (start) of the lookup order for the home page:

1. index.fr.amp.html
2. index.amp.html
3. index.fr.html
4. index.html
5. ...

Fixes #3360
2017-07-04 09:12:44 +02:00
Bjørn Erik Pedersen
fb53987a4f hugolib: Fix error handling for JSON front matter
Fixes #3610
2017-06-19 14:41:03 +02:00