Commit graph

81 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
76b2afe642
Apply missing go fmt 2020-02-21 10:12:33 +01:00
Bjørn Erik Pedersen
f460530347
Fix panic on no output formats
A page needs its output formats even if it should not be rendered or its resources should not be published.

Fixes #6924
2020-02-21 09:41:39 +01:00
Bjørn Erik Pedersen
fa520a2d98 Add Page.GetTerms
Fixes #6905
2020-02-19 14:52:23 +01:00
Bjørn Erik Pedersen
eada236f87
Introduce a tree map for all content
This commit introduces a new data structure to store pages and their resources.

This data structure is backed by radix trees.

This simplies tree operations, makes all pages a bundle,  and paves the way for #6310.

It also solves a set of annoying issues (see list below).

Not a motivation behind this, but this commit also makes Hugo in general a little bit faster and more memory effective (see benchmarks). Especially for partial rebuilds on content edits, but also when taxonomies is in use.

```
name                                   old time/op    new time/op    delta
SiteNew/Bundle_with_image/Edit-16        1.32ms ± 8%    1.00ms ± 9%  -24.42%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16    1.28ms ± 0%    0.94ms ± 0%  -26.26%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16      33.9ms ± 2%    21.8ms ± 1%  -35.67%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16            40.6ms ± 1%    37.7ms ± 3%   -7.20%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16        56.7ms ± 0%    51.7ms ± 1%   -8.82%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16      19.9ms ± 2%    18.3ms ± 3%   -7.64%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16         37.9ms ± 4%    34.0ms ± 2%  -10.28%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16             10.7ms ± 0%    10.6ms ± 0%   -1.15%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16         10.8ms ± 0%    10.7ms ± 0%   -1.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16           43.2ms ± 1%    39.6ms ± 1%   -8.35%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                 47.6ms ± 1%    47.3ms ± 0%     ~     (p=0.057 n=4+4)
SiteNew/Deep_content_tree-16             73.0ms ± 1%    74.2ms ± 1%     ~     (p=0.114 n=4+4)
SiteNew/Many_HTML_templates-16           37.9ms ± 0%    38.1ms ± 1%     ~     (p=0.114 n=4+4)
SiteNew/Page_collections-16              53.6ms ± 1%    54.7ms ± 1%   +2.09%  (p=0.029 n=4+4)

name                                   old alloc/op   new alloc/op   delta
SiteNew/Bundle_with_image/Edit-16         486kB ± 0%     430kB ± 0%  -11.47%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16     265kB ± 0%     209kB ± 0%  -21.06%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16      13.6MB ± 0%     8.8MB ± 0%  -34.93%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16            66.5MB ± 0%    63.9MB ± 0%   -3.95%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16        28.8MB ± 0%    25.8MB ± 0%  -10.55%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16      6.16MB ± 0%    5.56MB ± 0%   -9.86%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16         16.9MB ± 0%    16.0MB ± 0%   -5.19%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16             2.28MB ± 0%    2.29MB ± 0%   +0.35%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16         2.07MB ± 0%    2.07MB ± 0%     ~     (p=0.114 n=4+4)
SiteNew/Tags_and_categories-16           14.3MB ± 0%    13.2MB ± 0%   -7.30%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                 69.1MB ± 0%    69.0MB ± 0%     ~     (p=0.343 n=4+4)
SiteNew/Deep_content_tree-16             31.3MB ± 0%    31.8MB ± 0%   +1.49%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16           10.8MB ± 0%    10.9MB ± 0%   +1.11%  (p=0.029 n=4+4)
SiteNew/Page_collections-16              21.4MB ± 0%    21.6MB ± 0%   +1.15%  (p=0.029 n=4+4)

name                                   old allocs/op  new allocs/op  delta
SiteNew/Bundle_with_image/Edit-16         4.74k ± 0%     3.86k ± 0%  -18.57%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16     4.73k ± 0%     3.85k ± 0%  -18.58%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16        301k ± 0%      198k ± 0%  -34.14%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16              389k ± 0%      373k ± 0%   -4.07%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16          338k ± 0%      262k ± 0%  -22.63%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16        102k ± 0%       88k ± 0%  -13.81%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16           176k ± 0%      152k ± 0%  -13.32%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16              26.8k ± 0%     26.8k ± 0%   +0.05%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16          26.8k ± 0%     26.8k ± 0%   +0.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16             273k ± 0%      245k ± 0%  -10.36%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                   396k ± 0%      398k ± 0%   +0.39%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16               317k ± 0%      325k ± 0%   +2.53%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16             146k ± 0%      147k ± 0%   +0.98%  (p=0.029 n=4+4)
SiteNew/Page_collections-16                210k ± 0%      215k ± 0%   +2.44%  (p=0.029 n=4+4)
```

Fixes #6312
Fixes #6087
Fixes #6738
Fixes #6412
Fixes #6743
Fixes #6875
Fixes #6034
Fixes #6902
Fixes #6173
Fixes #6590
2020-02-18 09:49:42 +01:00
Bjørn Erik Pedersen
e625088ef5
Add render template hooks for links and images
This commit also

* revises the change detection for templates used by content files in server mode.
* Adds a Page.RenderString method

Fixes #6545
Fixes #4663
Closes #6043
2019-12-18 11:44:40 +01:00
Bjørn Erik Pedersen
33d733300a
Deprecate mmark
Fixes #6486
2019-11-25 20:14:20 +01:00
Bjørn Erik Pedersen
a3fe5e5e35
Fix Params case handling in the index, sort and where func
This means that you can now do:

```
{{ range where .Site.Pages "Params.MYPARAM" "foo" }}
```
2019-11-22 18:41:50 +01:00
Luke Francl
70a1aa345b Support Go time format strings in permalinks
In the vein of an ancient TODO about supporting custom date formatting with
strftime, this allows `:`-prefixed Go time format strings in permalink segments.
This allows users to customize date-based permalinks any way they need to.

For example, with a date of 2019-11-09, the permalink `/:06/:1/:2` will render
as `/19/11/9`.

See:

07978e4a49 (diff-0688a3b65c7f5d01aa216f8d9b57fd00R111-R112)
https://discourse.gohugo.io/t/implementing-additional-date-formats-for-permalinks/17860
https://github.com/gohugoio/hugo/pull/6488
2019-11-10 11:00:02 +01:00
Bjørn Erik Pedersen
653e6856ea resources/page: Use binary search in Pages.Prev/Next if possible
This is obviously much faster for lager data sets:

```bash
name                         old time/op    new time/op    delta
SearchPage/ByWeight-100-4       267ns ± 4%     272ns ± 5%     ~     (p=0.457 n=4+4)
SearchPage/ByWeight-5000-4     10.8µs ± 3%     1.2µs ± 2%  -88.99%  (p=0.029 n=4+4)
SearchPage/ByWeight-10000-4    21.1µs ± 1%     1.4µs ±11%  -93.28%  (p=0.029 n=4+4)
```

See #4500
2019-10-13 12:36:17 +02:00
Bjørn Erik Pedersen
f4f566edf4 Make Pages.Prev/Next work like the other Prev/Next methods
Fixes #4500
2019-10-13 12:36:17 +02:00
Bjørn Erik Pedersen
9e57182705
tests: Convert from testify to quicktest 2019-08-12 13:26:32 +02:00
Christian Muehlhaeuser
6027ee1108 Avoid unnecessary conversions
No need to convert these types.
2019-08-10 20:12:38 +02:00
Bjørn Erik Pedersen
7ff0a8ee9f Simplify page tree logic
This is preparation for #6041.

For historic reasons, the code for bulding the section tree and the taxonomies were very much separate.

This works, but makes it hard to extend, maintain, and possibly not so fast as it could be.

This simplification also introduces 3 slightly breaking changes, which I suspect most people will be pleased about. See referenced issues:

This commit also switches the radix tree dependency to a mutable implementation: github.com/armon/go-radix.

Fixes #6154
Fixes #6153
Fixes #6152
2019-08-08 20:13:39 +02:00
Bjørn Erik Pedersen
de87624241 hugolib: Fix output format handling of mix cased page kinds
Fixes #4528
2019-08-05 11:22:03 +02:00
XhmikosR
f7f549e3a7 Fix assorted typos 2019-08-01 13:55:48 +02:00
Bjørn Erik Pedersen
53077b0da5
Merge pull request #6149 from bep/sort-caseinsensitive
Implement lexicographically string sorting
2019-08-01 10:19:19 +02:00
Bjørn Erik Pedersen
e5f2299741 Block symlink dir traversal for /static
This is in line with how it behaved before, but it was lifted a little for the project mount for Hugo Modules,
but that could create hard-to-detect loops.
2019-07-25 11:27:25 +02:00
Bjørn Erik Pedersen
9f5a92078a
Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092
2019-07-24 09:35:53 +02:00
Bjørn Erik Pedersen
e8a716b23a tpl/collections: Fix slice type handling in sort
The `sort` template func was producing a `[]page.Page` which did not work in `.Paginate`.

Fixes #6023
2019-06-10 08:32:44 +02:00
Bjørn Erik Pedersen
3e6cb2cb77 hugolib: Fix bundle path when slug is set
Fixes #4870
2019-06-09 16:55:08 +02:00
Bruno Amaral
d1cf53f5f4 Remove references to Google+
According to google, this is no longer supported and structured data should be used instead: https://support.google.com/webmasters/answer/6083347?hl=en
2019-05-26 19:24:45 +02:00
Bjørn Erik Pedersen
c7dd66bfe2 Fix links for non-HTML output formats
They were not correct for regular pages.

Fixes #5877
2019-04-20 15:01:20 +02:00
Bjørn Erik Pedersen
f2795d4d2c Fix WeightedPages in union etc.
We introduced a callback func() to get the owner Page in 0.55.0.

Sadly, funcs is  not comparable type in Go.

This commit replaces the func with a struct pointer that wraps the Page.

Fixes #5850
2019-04-13 18:52:38 +02:00
Bjørn Erik Pedersen
3b86b4a9f5
resources/page: Fix .RSSLinke deprecation message
Closes #4427
2019-04-09 16:47:29 +02:00
Bjørn Erik Pedersen
ebab291c0e
resources/page: Regenerate JSON wrapper 2019-04-05 16:58:52 +02:00
Bjørn Erik Pedersen
75467cd785
resources/page: Add missing GitInfo to Page 2019-04-05 16:52:37 +02:00
Bjørn Erik Pedersen
e91e222cd2
resources/page: Implement compare.ProbablyEqer for the core slices
Fixes #5808
2019-04-02 16:10:13 +02:00
Bjørn Erik Pedersen
5185fb065b hugolib: Allow relative URLs in front matter
Before this commit you would have to do this in multilingual setups:

```
---
title: "Custom!"
url: "/jp/custom/foo"
---
```

This commit allows for relative URLs, e.g:

```
---
title: "Custom!"
url: "custom/foo"
---
```

Which is obviously easier and more portable.

The meaning of relative may change to include more in the future (e.g. role based access).

Fixes #5704
2019-04-02 10:33:15 +02:00
Bjørn Erik Pedersen
4dae52af68 Avoid nilpointer on no File on Page
Fixes #5781
2019-03-26 10:20:40 +01:00
Bjørn Erik Pedersen
91ef9655aa
resources/page: Re-introduce .Page.Page
It was removed in 597e418cb0 -- it's not documented, but it's used in too many real sites on the web.

See #5784
2019-03-25 07:55:46 +01:00
Bjørn Erik Pedersen
597e418cb0
Make Page an interface
The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct.
This is all a preparation step for issue  #5074, "pages from other data sources".

But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes.

Most notable changes:

* The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday.
  This means that any markdown will partake in the global ToC and footnote context etc.
* The Custom Output formats are now "fully virtualized". This removes many of the current limitations.
* The taxonomy list type now has a reference to the `Page` object.
  This improves the taxonomy template `.Title` situation and make common template constructs much simpler.

See #5074
Fixes #5763
Fixes #5758
Fixes #5090
Fixes #5204
Fixes #4695
Fixes #5607
Fixes #5707
Fixes #5719
Fixes #3113
Fixes #5706
Fixes #5767
Fixes #5723
Fixes #5769
Fixes #5770
Fixes #5771
Fixes #5759
Fixes #5776
Fixes #5777
Fixes #5778
2019-03-23 18:51:22 +01:00