Commit graph

52 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
0bd6356c6d Fix panic when home page is drafted
Fixes #6927
2020-02-23 00:11:10 +01:00
Bjørn Erik Pedersen
7ef5a4c83e
hugolib: Fix 2 Paginator.Pages taxonomy regressions
Fixes #6921
Fixes #6918
2020-02-20 14:13:04 +01:00
Bjørn Erik Pedersen
fa520a2d98 Add Page.GetTerms
Fixes #6905
2020-02-19 14:52:23 +01:00
Bjørn Erik Pedersen
b2dcd53e3c
Use the tree for taxonomy.Pages() 2020-02-18 19:47:45 +01:00
Bjørn Erik Pedersen
aa3e183056
tpl: Fix RSS template for the terms listing
Fixes #6909
2020-02-18 16:55:30 +01:00
Bjørn Erik Pedersen
1b7acfe763
Fix taxonomy
Recently introduced in master.

See https://github.com/gohugoio/hugo/issues/6897#issuecomment-587499907
2020-02-18 16:43:36 +01:00
Bjørn Erik Pedersen
eada236f87
Introduce a tree map for all content
This commit introduces a new data structure to store pages and their resources.

This data structure is backed by radix trees.

This simplies tree operations, makes all pages a bundle,  and paves the way for #6310.

It also solves a set of annoying issues (see list below).

Not a motivation behind this, but this commit also makes Hugo in general a little bit faster and more memory effective (see benchmarks). Especially for partial rebuilds on content edits, but also when taxonomies is in use.

```
name                                   old time/op    new time/op    delta
SiteNew/Bundle_with_image/Edit-16        1.32ms ± 8%    1.00ms ± 9%  -24.42%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16    1.28ms ± 0%    0.94ms ± 0%  -26.26%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16      33.9ms ± 2%    21.8ms ± 1%  -35.67%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16            40.6ms ± 1%    37.7ms ± 3%   -7.20%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16        56.7ms ± 0%    51.7ms ± 1%   -8.82%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16      19.9ms ± 2%    18.3ms ± 3%   -7.64%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16         37.9ms ± 4%    34.0ms ± 2%  -10.28%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16             10.7ms ± 0%    10.6ms ± 0%   -1.15%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16         10.8ms ± 0%    10.7ms ± 0%   -1.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16           43.2ms ± 1%    39.6ms ± 1%   -8.35%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                 47.6ms ± 1%    47.3ms ± 0%     ~     (p=0.057 n=4+4)
SiteNew/Deep_content_tree-16             73.0ms ± 1%    74.2ms ± 1%     ~     (p=0.114 n=4+4)
SiteNew/Many_HTML_templates-16           37.9ms ± 0%    38.1ms ± 1%     ~     (p=0.114 n=4+4)
SiteNew/Page_collections-16              53.6ms ± 1%    54.7ms ± 1%   +2.09%  (p=0.029 n=4+4)

name                                   old alloc/op   new alloc/op   delta
SiteNew/Bundle_with_image/Edit-16         486kB ± 0%     430kB ± 0%  -11.47%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16     265kB ± 0%     209kB ± 0%  -21.06%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16      13.6MB ± 0%     8.8MB ± 0%  -34.93%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16            66.5MB ± 0%    63.9MB ± 0%   -3.95%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16        28.8MB ± 0%    25.8MB ± 0%  -10.55%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16      6.16MB ± 0%    5.56MB ± 0%   -9.86%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16         16.9MB ± 0%    16.0MB ± 0%   -5.19%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16             2.28MB ± 0%    2.29MB ± 0%   +0.35%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16         2.07MB ± 0%    2.07MB ± 0%     ~     (p=0.114 n=4+4)
SiteNew/Tags_and_categories-16           14.3MB ± 0%    13.2MB ± 0%   -7.30%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                 69.1MB ± 0%    69.0MB ± 0%     ~     (p=0.343 n=4+4)
SiteNew/Deep_content_tree-16             31.3MB ± 0%    31.8MB ± 0%   +1.49%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16           10.8MB ± 0%    10.9MB ± 0%   +1.11%  (p=0.029 n=4+4)
SiteNew/Page_collections-16              21.4MB ± 0%    21.6MB ± 0%   +1.15%  (p=0.029 n=4+4)

name                                   old allocs/op  new allocs/op  delta
SiteNew/Bundle_with_image/Edit-16         4.74k ± 0%     3.86k ± 0%  -18.57%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16     4.73k ± 0%     3.85k ± 0%  -18.58%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16        301k ± 0%      198k ± 0%  -34.14%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16              389k ± 0%      373k ± 0%   -4.07%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16          338k ± 0%      262k ± 0%  -22.63%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16        102k ± 0%       88k ± 0%  -13.81%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16           176k ± 0%      152k ± 0%  -13.32%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16              26.8k ± 0%     26.8k ± 0%   +0.05%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16          26.8k ± 0%     26.8k ± 0%   +0.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16             273k ± 0%      245k ± 0%  -10.36%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                   396k ± 0%      398k ± 0%   +0.39%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16               317k ± 0%      325k ± 0%   +2.53%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16             146k ± 0%      147k ± 0%   +0.98%  (p=0.029 n=4+4)
SiteNew/Page_collections-16                210k ± 0%      215k ± 0%   +2.44%  (p=0.029 n=4+4)
```

Fixes #6312
Fixes #6087
Fixes #6738
Fixes #6412
Fixes #6743
Fixes #6875
Fixes #6034
Fixes #6902
Fixes #6173
Fixes #6590
2020-02-18 09:49:42 +01:00
Bjørn Erik Pedersen
6ccf50ea7b hugolib: Fix draft etc. handling of _index.md pages
We will need to revisit this with a proper spec, but this commit makes sure that draft/expiryDate etc. set in front matter on _index.md content files that should disable the page will:

* Not crash
* Make the rendered page not render any `.Content`.

Fixes #6222
Fixes #6210
2019-08-15 20:41:48 +02:00
Bjørn Erik Pedersen
9475f61a37
hugolib: Fix taxonomies vs expired
In Hugo 0.57 we needed to delay the page metadata initialization until we had built the page graph.

This introduced a regression in that we now created taxonomy entries for expired pages.

This fixes that by moving the "should not build" filter before we assemble the taxonomies.

Fixes #6213
2019-08-15 09:52:08 +02:00
Bjørn Erik Pedersen
9e57182705
tests: Convert from testify to quicktest 2019-08-12 13:26:32 +02:00
Bjørn Erik Pedersen
7ff0a8ee9f Simplify page tree logic
This is preparation for #6041.

For historic reasons, the code for bulding the section tree and the taxonomies were very much separate.

This works, but makes it hard to extend, maintain, and possibly not so fast as it could be.

This simplification also introduces 3 slightly breaking changes, which I suspect most people will be pleased about. See referenced issues:

This commit also switches the radix tree dependency to a mutable implementation: github.com/armon/go-radix.

Fixes #6154
Fixes #6153
Fixes #6152
2019-08-08 20:13:39 +02:00
Bjørn Erik Pedersen
9f5a92078a
Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092
2019-07-24 09:35:53 +02:00
Bjørn Erik Pedersen
b799b12f4a hugolib: Fix panic for unused taxonomy content files
In Hugo 0.55 we connected the taxonomy nodes with their owning Page.

This failed if you had, say, a content file for a author that did not author anything in the site:

```
content/authors/silent-persin/_index.md
```

Fixes #5847
2019-04-15 13:36:05 +02:00
Bjørn Erik Pedersen
597e418cb0
Make Page an interface
The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct.
This is all a preparation step for issue  #5074, "pages from other data sources".

But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes.

Most notable changes:

* The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday.
  This means that any markdown will partake in the global ToC and footnote context etc.
* The Custom Output formats are now "fully virtualized". This removes many of the current limitations.
* The taxonomy list type now has a reference to the `Page` object.
  This improves the taxonomy template `.Title` situation and make common template constructs much simpler.

See #5074
Fixes #5763
Fixes #5758
Fixes #5090
Fixes #5204
Fixes #4695
Fixes #5607
Fixes #5707
Fixes #5719
Fixes #3113
Fixes #5706
Fixes #5767
Fixes #5723
Fixes #5769
Fixes #5770
Fixes #5771
Fixes #5759
Fixes #5776
Fixes #5777
Fixes #5778
2019-03-23 18:51:22 +01:00
Bjørn Erik Pedersen
40ffb0484b hugolib: Restore 0.48 slash handling in taxonomies
Fixes #5571
2018-12-30 18:50:09 +01:00
Bjørn Erik Pedersen
4bae8b04aa Revert " Fix handling of taxonomy terms containing slashes"
See #4090
See #5571

This reverts commit fff132537b.
2018-12-30 18:50:09 +01:00
Bjørn Erik Pedersen
65fa069271
Revert "hugolib: Restore taxonomy term path separation"
See #5571

This reverts commit 9ce0a1fb70.
2018-12-29 10:00:17 +01:00
Bjørn Erik Pedersen
9ce0a1fb70 hugolib: Restore taxonomy term path separation
Fixes #5513
2018-12-14 10:29:32 +01:00
Cameron Moore
fae48d7457 hugolib: Normalize permalink path segments
When constructing permalinks, ensure that most inputs used as path
segments are normalized with PathSpec.MakeSegment instead of
PathSpec.URLize.

The primary exception to that rule is with taxonomy titles in
pageToPermalinkTitle(). The approach taken here is to use URLize for
taxonomy pages. Everything else will use MakeSegment. The reason for
this exception is that people use taxonomies such as "s1/p1" to generate
URLs precisely they way they wish (see #5223). Tests have been added to
check for this case.

Fixes #4926
2018-10-03 11:02:15 +03:00
Thomas Jost
be3ae3ec92 hugolib: Do not FirstUpper taxonomy titles
Doing so was probably a mistake. This may be a breaking change for some people,
but it's easy to restore the previous behaviour in the layouts.

Fixes #5172
2018-09-06 20:29:21 +02:00
Cameron Moore
fff132537b Fix handling of taxonomy terms containing slashes
Fixes #4090
2018-08-22 07:18:37 +02:00
Bjørn Erik Pedersen
3eb313fef4 Simplify .Site.GetPage etc.
This commit is a follow up to a recent overhaul of the GetPage/ref/relref implemenation.

The most important change in this commit is the update to `.Site.GetPage`:

* To reduce the amount of breakage in the wild to its minimum, I have reworked .Site.GetPage with some rules:

* We cannot support more than 2 arguments, i.e. .Site.GetPage "page" "posts" "mypage.md" will now throw an error. I think this is the most uncommon syntax and should be OK. It is an easy fix to change the above to .Site.GetPage "/posts/mypage.md" or similar.
* .Site.GetPage "home", .Site.GetPage "home" "" and .Site.GetPage "home" "/" will give you the home page. This means that if you have page in root with the name home.md you need to do .Site.GetPage "/home.md" or similar

This commit also fixes some multilingual issues, most notable it is now possible to do cross-language ref/relref lookups by prepending the language code to the path, e.g. `/jp/posts/mypage.md`.

This commit also reverts the site building tests related to this to "Hugo 0.44 state", to get better control of the changes made.

Closes #4147
Closes #4727
Closes #4728
Closes #4728
Closes #4726
Closes #4652
2018-07-18 00:07:20 +02:00
Vas Sudanagunta
b93417aa1d Unify page lookups
This commit unifies the core internal page index for all page kinds.

This enables the `ref` and `relref` shortcodes to support all pages kinds, and adds a new page-relative  `.GetPage` method with simplified signature.

See #4147
See #4727
See #4728
See #4728
See #4726
See #4652
2018-07-18 00:07:20 +02:00
Bjørn Erik Pedersen
eb42774e58
Add support for a content dir set per language
A sample config:

```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true

[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"

[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```

The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.

The content files will be assigned a language by

1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.

The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.

Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.

If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.

Fixes #4523
Fixes #4552
Fixes #4553
2018-04-02 08:06:21 +02:00
Bjørn Erik Pedersen
20c9b6ec81
resource: Add front matter metadata to Resource
This commit expands the Resource interface with 3 new methods:

* Name
* Title
* Params

All of these can be set in the Page front matter. `Name` will get its default value from the base filename, and is the value used in the ByPrefix and GetByPrefix lookup methods.

Fixes #4244
2018-01-17 16:22:33 +01:00
Bjørn Erik Pedersen
2d3189b227 hugolib: Fix handling of mixed-case taxonomy folders with content file
* We match by path vs taxonomy to determine if we have a content page for that taxonomy
* The taxonomy name is (if `preserveTaxonomyNames` is not set) normalized to `maxmustermann` while you have the disk folder called `MaxMustermann`.
* This isn't a new issue, but I suspect most people will just name the folder `authors/maxmustermann` and it will just work.
* The inconsistent behaviour you see here is that you will end up with two pages with the same target filename, so it is a little random who will win.

This fixes that by also normalizing the taxonomy path when doing the comparison.

Fixes #4238
2018-01-08 11:47:26 +01:00
Bjørn Erik Pedersen
764f1304e1
hugolib: Add some more taxonomy permalinks test cases
See #1208
2017-11-07 23:42:47 +01:00
Brian Chen
d9a78b61ad Handle Taxonomy permalinks
Return the correct virtual Section for Taxonomy and TaxonomyTerm.
Restrict permalink expansion to only Pages and Taxonomies, but then
actually use expanded permalinks even for non-Pages.

Fixes #1208.
2017-11-07 23:05:48 +01:00
Bjørn Erik Pedersen
d8717cd4c7 all: Update import paths to gohugoio/hugo 2017-06-13 18:42:45 +02:00
Bjørn Erik Pedersen
1f55cb767d hugolib: Simplify some test loops 2017-06-06 08:43:33 +02:00
Bjørn Erik Pedersen
6bf010fed4 hugolib: Refactor/-work the permalink/target path logic
This is a pretty fundamental change in Hugo, but absolutely needed if we should have any hope of getting "multiple outputs" done.

This commit's goal is to say:

* Every file target path is created by `createTargetPath`, i.e. one function for all.
* That function takes every page and site parameter into account, to avoid fragile string parsing to uglify etc. later on.
* The path creation logic has full test coverage.
* All permalinks, paginator URLs etc. are then built on top of that same logic.

Fixes #1252
Fixes #2110
Closes #2374
Fixes #1885
Fixes #3102
Fixes #3179
Fixes #1641
Fixes #1989
2017-03-27 15:43:56 +02:00
Anton Staaf
c950699907 hugolib: Populate the taxonomy term Pages list
Previously this was left empty, but it is very handy to have a list
of term pages for a given taxonomy.  This list can now be paginated
like other page lists.  It makes it possible to render summary
content from each terms index page for instance.  It also makes it
possible to sort the term pages in the same way that other page
lists can be sorted.  An RSS feed is now generated for
KindTaxonomyTerm pages as well.

This also fixes a bug in HugoSites.createMissingPages.  Previously
it would only check for existing KindTaxonomyTerm pages if the
taxonomy had any terms defined.  So for a taxonomy with no terms
but a taxonomy terms page it would generate a second empty terms
page.
2017-03-05 21:24:14 +01:00
Bjørn Erik Pedersen
9671162a22 hugolib: Make sure that empty terms lists are still created
Fixes #2977
2017-03-01 17:07:38 +01:00
Bjørn Erik Pedersen
cc15864744 hugolib: Only return RSSLink when RSS is available
Fixes #1302
2017-03-01 12:30:41 +01:00
Bjørn Erik Pedersen
0eb76b0863 Revert "hugolib: Add terms' pages to .Data.Pages"
This reverts commit e117b1b2c7.

Will have to think a little about this.
2017-02-25 18:14:48 +01:00
Bjørn Erik Pedersen
c1425a166d hugolib: Fix preserveTaxonomyNames regression
Fixes #3070
2017-02-23 10:09:42 +01:00
Bjørn Erik Pedersen
e117b1b2c7 hugolib: Add terms' pages to .Data.Pages
Fixes #2826
2017-02-22 21:13:21 +01:00
Bjørn Erik Pedersen
a3af4fe46e hugolib: Finish menu vs section content pages
This commit also fixes the default menu sort when the weight is 0.

Closes #2974
2017-02-20 22:20:02 +01:00
Bjørn Erik Pedersen
4493e8ed9f hugolib: Fix issue with taxonomies when only some have content page
Fixes #2992
2017-02-18 08:37:11 +01:00
Bjørn Erik Pedersen
93ca7c9e95 all: Refactor to nonglobal Viper, i18n etc.
This is a final rewrite that removes all the global state in Hugo, which also enables
the use if `t.Parallel` in tests.

Updates #2701
Fixes #3016
2017-02-17 17:15:26 +01:00
Bjørn Erik Pedersen
c71e1b106e all: Refactor to nonglobal file systems
Updates #2701
Fixes #2951
2017-02-04 11:37:25 +07:00
Bjørn Erik Pedersen
45e3ed517a all: Refactor to non-global logger
Note that this looks like overkill for just the logger, and that is correct,
but this will make sense once we start with the template handling etc.

Updates #2701
2017-01-07 17:06:35 +01:00
Cameron Moore
d1b89f5c7c hugolib: Use reflect.DeepEqual in tests 2016-12-27 11:19:25 +01:00
Bjørn Erik Pedersen
ec2d502b4f node to page: Handle translations
Updates #2297
2016-11-22 09:57:03 +01:00
Bjørn Erik Pedersen
90de511017 Make taxonomies configurable per language
See #2312
2016-09-06 18:32:17 +03:00
Bjørn Erik Pedersen
708bc78770 Optimize the multilanguage build process
Work In Progress!

This commit makes a rework of the build and rebuild process to better suit a multi-site setup.

This also includes a complete overhaul of the site tests. Previous these were a messy mix that
were testing just small parts of the build chain, some of it testing code-paths not even used in
"real life". Now all tests that depends on a built site follows the same and real production code path.

See #2309
Closes #2211
Closes #477
Closes #1744
2016-09-06 18:32:16 +03:00
Alexandre Bourget
ec33732fbe Add multilingual support in Hugo
Implements:
* support to render:
  * content/post/whatever.en.md to /en/2015/12/22/whatever/index.html
  * content/post/whatever.fr.md to /fr/2015/12/22/whatever/index.html
* gets enabled when `Multilingual:` is specified in config.
* support having language switchers in templates, that know
  where the translated page is (with .Page.Translations)
  (when you're on /en/about/, you can have a "Francais" link pointing to
   /fr/a-propos/)
  * all translations are in the `.Page.Translations` map, including the current one.
* easily tweak themes to support Multilingual mode
* renders in a single swift, no need for two config files.

Adds a couple of variables useful for multilingual sites

Adds documentation (content/multilingual.md)

Added language prefixing for all URL generation/permalinking see in the
code base.

Implements i18n. Leverages the great github.com/nicksnyder/go-i18n lib.. thanks Nick.
* Adds "i18n" and "T" template functions..
2016-09-06 18:32:15 +03:00
Robert Basic
7d5c9fbf44 Make ByCount sort consistently
When two or more taxonomies have the same number of pages,
sort them by name to have consistent ByCount sorting of
taxonomies.

Fixes #1930
2016-04-02 22:59:10 +02:00
Bjørn Erik Pedersen
937592cb85 hugolib: Fix some more Golint warnings 2016-03-23 10:10:28 +01:00
Anthony Fok
8509727fe8 Add copyright header to that source files that don’t have one.
See #1646
2015-12-10 15:19:38 -07:00