Commit graph

1537 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
729593c842 hugolib: Fix deadlock when content building times out
Fixes #5375
2018-10-30 12:18:29 +01:00
Bjørn Erik Pedersen
b27ccf34bf resource: Allow .Data.Integrity to be accessed on its own
Fixes #5296
2018-10-27 11:34:15 +02:00
Bjørn Erik Pedersen
78578632f5 Fix archetype handling of directories in theme
Fixes #5318
2018-10-26 12:16:28 +02:00
Bjørn Erik Pedersen
acc14b4646 hugolib: Adjust error test to make it pass on Go tip 2018-10-24 15:19:49 +02:00
Bjørn Erik Pedersen
2d7709d155 tpl: Handle truncated identifiers in Go template errors
Long identifiers will give errors on the format:

```bash
 _default/single.html:5:14: executing "main" at <.ThisIsAVeryLongTitl...>: can't evaluate field ThisIsAVeryLongTitle
```

Hugo use this value to match the "base template or not", so we need to strip the "...".

Fixes #5346
2018-10-24 13:54:04 +02:00
Bjørn Erik Pedersen
889aca054a Run gofmt -s 2018-10-24 00:12:32 +02:00
Bjørn Erik Pedersen
e3ed4a83b8
hugolib: Rename some page_* files
To make it easier to see/work with the source files that is about the `Page` struct.
2018-10-23 22:21:21 +02:00
Bjørn Erik Pedersen
6636cf1bea
Resolve error handling/parser related TODOs
See #5324
2018-10-23 19:41:22 +02:00
Bjørn Erik Pedersen
f669ef6bec
herrors: Improve handling of JSON errors
`*json.UnmarshalTypeError` and `*json.SyntaxError` has a byte `Offset`, so use that.

This commit also reworks/simplifies the errror line matching logic. This also makes the file reading unbuffered, but that should be fine in this error case.

See #5324
2018-10-23 14:35:43 +02:00
Bjørn Erik Pedersen
ed7b3e2619
commands, hugolib: Get file context in "config parse failed" errors
Fixes #5325
2018-10-23 08:09:41 +02:00
Bjørn Erik Pedersen
2bf686ee21
hugolib: Improve errors in /i18n handlling
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen
9f74dc2a52
hugolib: Improve errors in /data handlling
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen
d1661b823a
hugolib: Continue the file context/line number errors work
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen
7930d2132a
hugolib: Remove the now superflous Source struct
See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen
eb038cfa0a
Convert the rest to new page parser code paths
And remove some now unused code.

See #5324
2018-10-22 20:46:14 +02:00
Bjørn Erik Pedersen
129c27ee6e
parser/metadecoders: Consolidate the metadata decoders
See #5324
2018-10-22 20:46:13 +02:00
Bjørn Erik Pedersen
44da60d869
hugolib: Redo the summary delimiter logic
Now that we have a proper page parse tree, this can be greatly simplified.

See #5324
2018-10-22 20:46:13 +02:00
Bjørn Erik Pedersen
1e3e34002d
hugolib: Integrate new page parser
See #5324
2018-10-22 20:46:13 +02:00
Bjørn Erik Pedersen
1b7ecfc2e1
hugolib: Use []byte in shortcode parsing
See #5324
2018-10-22 19:57:44 +02:00
Bjørn Erik Pedersen
d6c16afde0
Move the shortcode parser to the new pageparser package
See #5324
2018-10-22 19:57:43 +02:00
Bjørn Erik Pedersen
4a366fcfee Prevent stale content in Fast Render Mode
We do that by re-render visited pages that is not already in the stack. This may potentially do some double work, but that small penalty should be well worth it.

Fixes #5281
2018-10-17 10:15:22 +02:00
Bjørn Erik Pedersen
083311d033
hugolib: Fix test on Windows 2018-10-17 08:37:02 +02:00
Bjørn Erik Pedersen
0fe4ff1875
tpl: Improve the Execute panic error message
See #5327
2018-10-17 08:25:57 +02:00
Janus
1f42e47e47 Allow date and slug from filename for leaf bundles
Fixes #4558
2018-10-16 23:51:48 +02:00
Bjørn Erik Pedersen
35fbfb19a1
commands: Show server error info in browser
The main item in this commit is showing of errors with a file context when running `hugo server`.

This can be turned off: `hugo server --disableBrowserError` (can also be set in `config.toml`).

But to get there, the error handling in Hugo needed a revision. There are some items left TODO for commits soon to follow, most notable errors in content and config files.

Fixes #5284
Fixes #5290
See #5325
See #5324
2018-10-16 22:10:56 +02:00
Bjørn Erik Pedersen
e421696d02 helpers: Consolidate MakeSegment vs MakePathSanitized
In short:

* Avoid double tolower in MakeSegment
* Use MakePathSanitized for taxonomies in pageToPermalinkTitle; this matches what MakeSegment does.
* Move the "double hyphen and space" logic into UnicodeSanitize

The last bullet may be slightly breaking for some that now does not get the "--" in some URLs, but we need to reduce the amount of URL logic.

See #4926
2018-10-03 15:55:25 +02:00
Gregorio Martinez
4b4af2c52e hugolib: Use stdlib context package 2018-10-03 14:59:35 +02:00
Cameron Moore
fae48d7457 hugolib: Normalize permalink path segments
When constructing permalinks, ensure that most inputs used as path
segments are normalized with PathSpec.MakeSegment instead of
PathSpec.URLize.

The primary exception to that rule is with taxonomy titles in
pageToPermalinkTitle(). The approach taken here is to use URLize for
taxonomy pages. Everything else will use MakeSegment. The reason for
this exception is that people use taxonomies such as "s1/p1" to generate
URLs precisely they way they wish (see #5223). Tests have been added to
check for this case.

Fixes #4926
2018-10-03 11:02:15 +03:00
Gregorio Martinez
06d28a464d hugolib: Fix FuzzyWordCount test error message 2018-10-03 10:01:30 +03:00
Kaushal Modi
6818170308 Render Markdown in figure shortcode "caption" and "attr" params
Fixes https://github.com/gohugoio/hugo/issues/4406.
2018-10-03 09:55:53 +03:00
Kaushal Modi
c5279064df Re-organize the figure shortcode for better readability 2018-10-03 09:55:53 +03:00
Cameron Moore
d3b81ee58e hugolib: Improve error message for bad taxonomy weights
Error is now "Unable to convert taxonomy weight []string{"40"} to int
for variables/files.md"
2018-10-03 09:34:45 +03:00
Cameron Moore
1fd30d462e hugolib: Cast taxonomy weight parameters to int
Fixes #4628
2018-10-03 09:34:45 +03:00
Cameron Moore
498d629958 hugolib: Allow nil to be unwrapped as *Page
Previously, calls to *Page.Eq(nil) would always return false because the
unwrapPage func didn't support the nil case.  Add support for unwrapping
nil to a *Page.

Fixes #5043
2018-10-03 09:33:42 +03:00
Bjørn Erik Pedersen
10ac2ec446 tpl/collections: Fix handling of different interface types in Slice
In Hugo `0.49` we improved type support in `slice`. This has an unfortunate side effect in that `resources.Concat` now expects something that can resolve to `resource.Resources`.

This worked for most situations, but when you try to `slice` different `Resource` objects, you would be getting `[]interface {}` and not `resource.Resources`. And `concat` would fail:

```bash
error calling Concat: slice []interface {} not supported in concat.
```

This commit fixes that by simplifying the type checking logic in `Slice`:

* If the first item implements the `Slicer` interface, we try that
* If the above fails or the first item does not implement `Slicer`, we just return the `[]interface {}`

Fixes #5269
2018-10-02 23:54:16 +03:00
Bjørn Erik Pedersen
fb732d5322
hugolib: Be a litle more specific in NextPage TODO 2018-09-26 09:26:53 +02:00
Ricardo N Feliciano
ad705aac06 hugolib: Introduce Page.NextPage and Page.PrevPage
Introduce new page position variables in order to fix the ordering issue
of `.Next` and `.Prev` while also allowing an upgrade path via
deprecation.

`.NextInSection` becomes `.NextPageInSection`.
`.PrevInSection` becomes `.PrevPageInSection`.

`.Next` becomes a function returning `.PrevPage`.
`.Prev` becomes a function returning `.NextPage`.

Fixes #1061
2018-09-26 09:22:43 +02:00
Bjørn Erik Pedersen
2650fa772b Add directory based archetypes
Given this content:

```bash
archetypes
├── default.md
└── post-bundle
    ├── bio.md
    ├── images
    │   └── featured.jpg
    └── index.md
```

```bash
hugo new --kind post-bundle post/my-post
```

Will create a new folder in `/content/post/my-post` with the same set of files as in the `post-bundle` archetypes folder.

This commit also improves the archetype language detection, so, if you use template code in your content files, the `.Site` you get is for the correct language. This also means that it is now possible to translate strings defined in  the `i18n` bundles,  e.g. `{{ i18n "hello" }}`.

Fixes #4535
2018-09-23 19:27:23 +02:00
Bjørn Erik Pedersen
f916814697 Prevent symbolic links from themes 2018-09-23 17:18:35 +02:00
Bjørn Erik Pedersen
f1a00b2069
hugolib: Remove deprecated rssURI 2018-09-21 14:30:57 +02:00
Bjørn Erik Pedersen
2e2e34a935
hugolib: Deprecate Pages.Sort
In favour of ByWeight.
2018-09-21 14:23:00 +02:00
Bjørn Erik Pedersen
2eed35c826
hugolib: Minor cleaning in the sorting code 2018-09-21 14:13:45 +02:00
Bjørn Erik Pedersen
dd692c710a hugolib: Clear the page cache per render
This is a slight performance optimization with no functional effect.

See #5239
2018-09-21 11:21:15 +02:00
Bjørn Erik Pedersen
ed4f1edbd7 hugolib: Compare every element in pages cache
It is slightly slower, but correctnes is, of course, more important:

```bash
benchmark                             old ns/op     new ns/op     delta
BenchmarkSortByWeightAndReverse-4     367           645           +75.75%

benchmark                             old allocs     new allocs     delta
BenchmarkSortByWeightAndReverse-4     2              2              +0.00%

benchmark                             old bytes     new bytes     delta
BenchmarkSortByWeightAndReverse-4     64            64            +0.00%
```

Running the same benchmark without any cache (i.e. resorting the slice on every iteration) and then compare it to the current version shows that it still is plenty worth it:

```bash
▶ benchcmp 2.bench 1.bench
benchmark                             old ns/op     new ns/op     delta
BenchmarkSortByWeightAndReverse-4     1358757       645           -99.95%

benchmark                             old allocs     new allocs     delta
BenchmarkSortByWeightAndReverse-4     17159          2              -99.99%

benchmark                             old bytes     new bytes     delta
BenchmarkSortByWeightAndReverse-4     274573        64            -99.98%
```

Closes #5239
2018-09-21 11:21:15 +02:00
Bjørn Erik Pedersen
058cc6c2c3
Revert "hugolib: Normalize permalink path segments"
This reverts commit 06976ebb87.

Fixes #5223
2018-09-18 09:27:49 +02:00
Cameron Moore
06976ebb87 hugolib: Normalize permalink path segments
When constructing permalinks, ensure that path segments are normalized
with PathSpec.MakeSegment instead of PathSpec.URLize.

Fixes #4926
2018-09-14 11:35:26 +02:00
Bjørn Erik Pedersen
555a5612b2 hugolib: Do not set RSS as Kind in RSS output
Before Hugo this commit  we set the pseudo page kind RSS on the page when output to RSS. This had some unintended side effects, esp. when the only output format for that page  was RSS.

For the page kinds that can have multiple output formats, the Kind should be one of the standard home, page etc.

Fixes #5138
2018-09-14 11:33:02 +02:00
Bjørn Erik Pedersen
75e54345f9 hugolib: Make sure ambiguous lookups in GetPage gets an error
See #5138
2018-09-14 11:33:02 +02:00
Bjørn Erik Pedersen
e27fd4c1b8 tpl/collections: Add collections.Append
Before this commit you would typically use `.Scratch.Add` to manually create slices in a loop.

With variable overwrite in Go 1.11, we can do better. This commit adds the `append` template func.

A made-up example:

```bash
{{ $p1 := index .Site.RegularPages 0 }}{{ $p2 := index .Site.RegularPages 1 }}
{{ $pages := slice }}
{{ if true }}
  {{ $pages = $pages | append $p2 $p1 }}
{{ end }}
```

Note that with 2 slices as arguments, the two examples below will give the same result:

```bash
{{ $s1 := slice "a" "b" | append (slice "c" "d") }}
{{ $s2 := slice "a" "b" | append "c" "d" }}
```

Both of the above will give `[]string{a, b, c, d}`.

This commit also improves the type handling in the `slice` template function. Now `slice "a" "b"` will give a `[]string` slice. The old behaviour was to return a `[]interface{}`.

Fixes #5190
2018-09-14 10:12:08 +02:00
Bjørn Erik Pedersen
fe6676c775 tpl/collections: Improve type handling in collections.Slice
Fixes #5188
2018-09-10 09:19:01 +02:00
Bjørn Erik Pedersen
6667c6d743 tpl/collections: Add group template func
This extends the page grouping in Hugo with a template function that allows for ad-hoc grouping.

A made-up example:

```
{{ $cool := where .Site.RegularPages "Params.cool" true | group "cool" }}
{{ $blue := where .Site.RegularPages "Params.blue" true | group "blue" }}
{{ $paginator := .Paginate (slice $cool $blue) }}
```

Closes #4865
2018-09-08 20:20:26 +02:00
Vincent Danjean
cfda13b363 hugolib: Allow creating page groups from any page collection
This also adjusts the pagination logic to allow for these new collections.

Note that we will follow up with a template function named `group` that will be the end user API. The `.Group` method on `Page` should be considered as internal.

Updates #4865
2018-09-08 11:14:09 +02:00
Thomas Jost
be3ae3ec92 hugolib: Do not FirstUpper taxonomy titles
Doing so was probably a mistake. This may be a breaking change for some people,
but it's easy to restore the previous behaviour in the layouts.

Fixes #5172
2018-09-06 20:29:21 +02:00
Cameron Moore
e38e881248 hugolib: Simplify some code 2018-08-30 22:31:29 +02:00
Cameron Moore
0665a3951b hugolib: Add missing error checks 2018-08-30 22:31:29 +02:00
Cameron Moore
1ed8c36336 hugolib: Remove extraneous createStaticFs call
This looks like a copy+paste error.  We have the same line a few lines
below this call and we actually handle the err value there.
2018-08-30 22:31:29 +02:00
Cameron Moore
e03eb90a36 hugolib: Fix godoc comment 2018-08-30 22:31:29 +02:00
Cameron Moore
c915d0d325 hugolib: Fix typo in private func name 2018-08-30 22:31:29 +02:00
Bjørn Erik Pedersen
6022f219f1
hugolib: Only run variable variable overwrite test on Go 1.11 2018-08-28 18:00:19 +02:00
Bjørn Erik Pedersen
0c8a415483
hugolib: Add a test for template variable overwrite
New in Go 1.11.
2018-08-28 17:35:33 +02:00
Bjørn Erik Pedersen
ebb56e8bdb Improve minifier MIME type resolution
This commit also removes the deprecated `Suffix` from MediaType. Now use `Suffixes` and put the MIME type suffix in the type, e.g. `application/svg+xml`.

Fixes #5093
2018-08-28 17:00:53 +02:00
Bjørn Erik Pedersen
94d0e79d33
hugolib: Include language code in REF_NOT_FOUND errors
Fixes #5110
2018-08-24 09:23:01 +02:00
Cameron Moore
fff132537b Fix handling of taxonomy terms containing slashes
Fixes #4090
2018-08-22 07:18:37 +02:00
Bjørn Erik Pedersen
baa62d0abb
hugolib: Adjust tests for shortcode p-issue
See #1642
2018-08-17 09:13:39 +02:00
gllera
78c99463fd
hugolib: Fix shortcode output wrapped in p
Fixes #1642
2018-08-17 09:13:30 +02:00
Bjørn Erik Pedersen
9d973004f5 hugolib: Fix image cache-clearing for sub-languages
Fixes #5084
2018-08-16 12:54:59 +02:00
Bjørn Erik Pedersen
2247546017 hugolib: Force render of any changed page, even in Fast Render Mode
Fixes #5083
2018-08-16 12:54:59 +02:00
Bjørn Erik Pedersen
d139a037d9 hugoblib: Fix "adding a bundle" in server mode
Before this commit, the live reload logic in `hugo server` got confused when you dropped a new bundle into the project while the server was running. The workaround was to restart the server.

This commit fixes the "live reload bundle detection" in server mode, and also makes sure that the bundle headers are always processed first.

Fixes #5075
2018-08-16 12:54:59 +02:00
Bjørn Erik Pedersen
0a88741fe8 hugolib: Fix error when deleting a bundle in server mode
Fixes #5077
2018-08-16 12:54:59 +02:00
Bjørn Erik Pedersen
a6f199f7a6
hugolib: Use the interface value when doing Related search
Currently it makes no practical difference, but this is more a protection if we in the future creates index from the content related fields. That will not work from a shortcode.

See #5071
2018-08-14 18:19:45 +02:00
Bjørn Erik Pedersen
0dd06bdac0
hugolib: Fix Related when called from shortcode
Fixes #5071
2018-08-14 18:11:36 +02:00
Bjørn Erik Pedersen
56c61559b2
Only duplicate resource output in multihost mode
Yesterday's commit was a little too agressive.

This commit makes sure that the duplication of resources to public/en etc. is only performed in multihost mode.

See #5058
2018-08-14 12:04:17 +02:00
Bjørn Erik Pedersen
78f8475a05 Fix Resource output in multihost setups
In Hugo 0.46 we made the output of what you get from resources.Get and similar static, i.e. language agnostic. This makes total sense, as it is wasteful and time-consuming to do SASS/SCSS/PostCSS processing for lots of languages when the output is lots of duplicates with different filenames.

But since we now output the result once only, this had a negative side effect for multihost setups: We publish the resource once only to the root folder (i.e. not to the language "domain folder").

This commit removes the language code from the processed image keys. This creates less duplication in the file cache, but it means that you should do a `hugo --gc` to clean up stale files.

Fixes #5058
2018-08-13 19:00:51 +02:00
Bjørn Erik Pedersen
2182ecfd34 hugolib: Fix GitInfo when multiple content dirs
Fixes #5054
2018-08-11 19:51:19 +02:00
Bjørn Erik Pedersen
e85833d868 hugolib: Add multiple content dirs to GitInfo test site
See #5054
2018-08-11 19:51:19 +02:00
Bjørn Erik Pedersen
43a5aaa778
Fix broken tests 2018-08-09 20:58:31 +02:00
Christian Oliff
06bd013641 Update alias.go
less markup :-)
2018-08-07 09:42:49 +02:00
Bjørn Erik Pedersen
789ef8c639
Add support for minification of final output
Hugo Pipes added minification support for resources fetched via ´resources.Get` and similar.

This also adds support for minification of the final output for supported output formats: HTML, XML, SVG, CSS, JavaScript, JSON.

To enable, run Hugo with the `--minify` flag:

```bash
hugo --minify
```

This commit is also a major spring cleaning of the `transform` package to allow the new minification step fit into that processing chain.

Fixes #1251
2018-08-06 19:58:41 +02:00
Bjørn Erik Pedersen
88e447c449
tocss/scss: Improve _ prefix handling in SCSS imports
See #5008
2018-07-31 16:53:03 +02:00
Bjørn Erik Pedersen
f219ac09f6 tocss/scss: Improve SCSS project vs themes import resolution
Before this commit, only SASS/SCSS  components imported from main.scss at first level can be overwritten by homonymous files in projects or over-preceding theme components.

This commit fixes that by implementing a custom import resolver which will be tried first. This resolver will make sure that the project/theme hierarchy is always respected.

Fixes #5008
2018-07-31 10:54:10 +02:00
Bjørn Erik Pedersen
786f72302f
Fix image cache eviction for sites with subdir in baseURL
Fixes #5006
2018-07-29 14:26:45 +02:00
Bjørn Erik Pedersen
04d4c08dbc hugolib: Fix .Site.GetPage regression
In Hugo 0.44 we simplified the `.Site.GetPage` API and added code to handle the old-style syntax in most cases.

This logic did not handle the lookup of the home page via `.Site.GetPage "section" ""` and similar. This commit fixes that.

Fixes #4989
2018-07-24 11:37:59 +02:00
Bjørn Erik Pedersen
016dd4a69a
Add Page.FirstSection
It was added and then removed by accident some time ago. Let us add it again, as it is useful.
2018-07-23 20:20:19 +02:00
Bjørn Erik Pedersen
c6b599a06d hugolib: Fix "borrow content from another language" issue
If a content file contains shortcode(s), we have logic in place to re-render it per output format.

We also have logic in place that avoids making a copy of the content used for this process if we don't need it.

This was before this commit limited to server mode and if the page should be output to multiple formats.

But there is a third case: If a site (language) borrows and renders `.Content` from another language. This would, before this commit, behave oddly for content with shortcodes.

Fixes #4986
2018-07-23 20:12:41 +02:00
Bjørn Erik Pedersen
a451c49fde hugolib: Print a WARNING about relative non-relative ref/relref matches
Fixes #4973
2018-07-21 11:42:07 +02:00
Bjørn Erik Pedersen
ff16c42ed0
hugolib: Allow untyped nil to be merged in lang.Merge
Fixes #4977
2018-07-21 11:41:23 +02:00
Bjørn Erik Pedersen
166483fe12 resource/scss: Add IncludePaths config option
Takes paths relative to the current working dir.

Fixes #4921
2018-07-20 17:50:44 +02:00
Vas Sudanagunta
8278384b96 Increase refLinker test coverage 2018-07-20 07:02:41 +02:00
Bjørn Erik Pedersen
65e610e161
hugolib: Avoid nilpointer in absoluteSourceRef 2018-07-19 22:39:24 +02:00
Vas Sudanagunta
b56d9a1294 Fix typo-logic bug in GetPage 2018-07-19 21:36:10 +02:00
Vas Sudanagunta
d6fde8fa13 Enable test case fixed by commit 501543d4 2018-07-19 20:40:50 +02:00
Vas Sudanagunta
2bac371544
Add test coverage for recent ref overhaul
Closes #4969
2018-07-19 18:32:23 +02:00
Bjørn Erik Pedersen
9b4b97a722
hugolib: Create an adapter from old to new getPage
To make sure we confirm that the existing tests run the correct code path.

Updates #4969
2018-07-19 18:26:10 +02:00
Bjørn Erik Pedersen
501543d4b6
hugolib: Only do page-relative getPage for relative paths
Updates #4969
2018-07-19 17:47:05 +02:00
Bjørn Erik Pedersen
6ffa882349
hugolib: Improve error handling in refLink 2018-07-19 17:17:35 +02:00
Bjørn Erik Pedersen
e25aa655f4 Add configurable ref/relref error handling and notFoundURL
Two new settings:

* refLinksErrorLevel: ERROR (default) or WARNING. ERROR will fail the build.
* refLinksNotFoundURL: Used as a placeholder when page references cannot be found.

Fixes #4964
2018-07-19 14:32:43 +02:00
Bjørn Erik Pedersen
12679b4083 hugolib: Mark shortcode changes as content changes in server mode
This is unfortunate, but is needed to re-create the taxonomies collections etc. that may be referenced from them.

Fixes #4965
2018-07-19 13:58:18 +02:00
Bjørn Erik Pedersen
c39ac5738d
Revert "hugolib: Fix potential server panic with drafts/future enabled"
This had a side-effect. I have understand this better to apply the correct fix. Will come soon.

This reverts commit 1ab4658c0d.
2018-07-19 12:20:01 +02:00
Bjørn Erik Pedersen
1ab4658c0d
hugolib: Fix potential server panic with drafts/future enabled
Fixes #4965
2018-07-18 17:01:50 +02:00
Bjørn Erik Pedersen
75acff5f20
hugolib: Avoid server panic on TOML mistake in i18n
Fixes #4942
2018-07-18 11:53:04 +02:00
Bjørn Erik Pedersen
5c9d5413a4 hugolib/filesystems: Fix theme config for Work Fs
Which was the reason why `PostCSS´ from theme(s) was not picked up.

Fixes #4951
2018-07-18 11:39:04 +02:00
Bjørn Erik Pedersen
d741064beb Add optional lang as argument to rel/relref
Fixes #4956
2018-07-18 00:07:20 +02:00
Bjørn Erik Pedersen
3eb313fef4 Simplify .Site.GetPage etc.
This commit is a follow up to a recent overhaul of the GetPage/ref/relref implemenation.

The most important change in this commit is the update to `.Site.GetPage`:

* To reduce the amount of breakage in the wild to its minimum, I have reworked .Site.GetPage with some rules:

* We cannot support more than 2 arguments, i.e. .Site.GetPage "page" "posts" "mypage.md" will now throw an error. I think this is the most uncommon syntax and should be OK. It is an easy fix to change the above to .Site.GetPage "/posts/mypage.md" or similar.
* .Site.GetPage "home", .Site.GetPage "home" "" and .Site.GetPage "home" "/" will give you the home page. This means that if you have page in root with the name home.md you need to do .Site.GetPage "/home.md" or similar

This commit also fixes some multilingual issues, most notable it is now possible to do cross-language ref/relref lookups by prepending the language code to the path, e.g. `/jp/posts/mypage.md`.

This commit also reverts the site building tests related to this to "Hugo 0.44 state", to get better control of the changes made.

Closes #4147
Closes #4727
Closes #4728
Closes #4728
Closes #4726
Closes #4652
2018-07-18 00:07:20 +02:00
Vas Sudanagunta
b93417aa1d Unify page lookups
This commit unifies the core internal page index for all page kinds.

This enables the `ref` and `relref` shortcodes to support all pages kinds, and adds a new page-relative  `.GetPage` method with simplified signature.

See #4147
See #4727
See #4728
See #4728
See #4726
See #4652
2018-07-18 00:07:20 +02:00
Garry McNulty
38204c4ab6 hugolib: Only set 'allThemes' if there are themes in the config file
When the 'allThemes' configuration setting is read, it will panic if
there are no themes. This was a regression introduced in Hugo 0.42.

Also updated a unit test to check for this.

Fixes #4851
2018-07-15 16:13:48 +02:00
Bjørn Erik Pedersen
beec1fc98e resource: Fix resources.Concat for transformed resources
Fixes #4936
2018-07-12 19:36:12 +02:00
Bjørn Erik Pedersen
306573def0 Improve type support in resources.Concat
This allows the result of `.Resources.Match` and similar to be concatenated.

Fixes #4934
2018-07-12 13:43:27 +02:00
Bjørn Erik Pedersen
80c8f3b81a hugolib: Fix static filesystem for themed multihost sites
Multihost is where each language has its own `baseURL`. In this configuration, static files from the theme was not picked up.

This was a regression in Hugo `0.42`. This commit also adds proper tests for this, so that does not happen again.

Fixes #4929
2018-07-12 12:18:56 +02:00
Bjørn Erik Pedersen
6b6dcb44a0 Flush partialCached cache on rebuilds
Fixes #4931
2018-07-11 20:40:04 +02:00
Bjørn Erik Pedersen
b874a1ba7a media: Allow multiple file suffixes per media type
Before this commit, `Suffix` on `MediaType` was used both to set a custom file suffix and as a way to augment the mediatype definition (what you see after the "+", e.g. "image/svg+xml").

This had its limitations. For one, it was only possible with one file extension per MIME type.

Now you can specify multiple file suffixes using "suffixes", but you need to specify the full MIME type
identifier:

[mediaTypes]
[mediaTypes."image/svg+xml"]
suffixes = ["svg", "abc ]

In most cases, it will be enough to just change:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffix = "txt"

To:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffixes = ["txt"]

Hugo will still respect values set in "suffix" if no value for "suffixes" is provided, but this will be removed in a future release.

Note that you can still get the Media Type's suffix from a template: {{ $mediaType.Suffix }}. But this will now map to the MIME type filename.

Fixes #4920
2018-07-10 22:13:52 +02:00
Anthony Fok
4108705934 hugolib: Remove empty resources/ dir after TestNewSiteDefaultLang 2018-07-10 09:01:54 +02:00
Bjørn Erik Pedersen
179de5f5bc
Revert "Consider root and current section's content type if set in front matter"
This reverts commit c790029e1d.
2018-07-09 10:29:18 +02:00
Bjørn Erik Pedersen
5dc1f95b63
Revert "Restrict the new type layout resolution to page and section kinds"
This reverts commit 78e8a744b3.
2018-07-09 10:29:16 +02:00
Yang Li
3cea2932e1 Fix typos 2018-07-07 12:29:56 +02:00
Bjørn Erik Pedersen
2b8d907ab7 Add a newScratch template func
Fixes #4685
2018-07-06 17:51:38 +02:00
Bjørn Erik Pedersen
43338c3a99 hugolib: Do not create paginator pages for the other output formats
This is a recent regression in Hugo, where we have started to produce `/page/30/index.json` when the main output format (usually `HTML`) is set up with pagination.

For JSON this is potentially lot of superflous work and hurts performance.

This commit reinstates the earlier behaviour: We only create paginators if in use in the main output format.

And add a test for it to prevent this from happening again.

Fixes #4890
2018-07-06 14:01:27 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
78e8a744b3
Restrict the new type layout resolution to page and section kinds
Will have to test and evaluate this a little.

See  #4891
2018-06-30 16:50:02 +02:00
Bjørn Erik Pedersen
c790029e1d
Consider root and current section's content type if set in front matter
This should allow for less duplication of templates. Before this commit it was possible to override the content page of a given page/section, but only one page at a time.

Full "template sets" can now be inherited by setting `type: blog` etc. in the section content page's front matter, and that type will be considered when looking for layouts for all pages in that section.

For nested sections, it will use consider both `type` set in the current section first, then `type` set in the first section below home, e.g. `/docs`.

This commit also adds a new Page method: `FirstSection`. This navigates up to the first section below home (e.g. `/docs`). For the home page it will return itself.

Fixes #4891
2018-06-30 13:10:04 +02:00
Bjørn Erik Pedersen
de37455ec7
hugolib: Allow forward slash in shortcode names
Fixes #4886
2018-06-28 16:06:16 +02:00
Anthony Fok
42ed602580 hugolib: Replace deprecated {Get,}ByPrefix with {Get,}Match 2018-06-15 09:03:56 +02:00
Bjørn Erik Pedersen
128f14efad hugolib: Reset the global pages cache on server rebuilds
In Hugo 0.42, this could lead to errors of type `runtime error: invalid memory address or nil pointer dereference` in some rare situations.

Note that this was also an issue before 0.42, but the symptom was then potentially stale list content on rebuilds on content changes.

This commit also improves the above error message.

Fixes #4845
2018-06-13 12:00:30 +02:00
Bjørn Erik Pedersen
80230f26a3
Add support for theme composition and inheritance
This commit adds support for theme composition and inheritance in Hugo.

With this, it helps thinking about a theme as a set of ordered components:

```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```

The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.

So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.

Hugo uses two different algorithms to merge the filesystems, depending on the file type:

* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.

The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are  plans to improve on this and get a URL scheme so this can be resolved automatically.

Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:

* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`

The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.

A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.

Fixes #4460
Fixes #4450
2018-06-10 23:55:20 +02:00
Bjørn Erik Pedersen
6464981adb
hugolib: Make sure that .Site.Taxonomies is always set on rebuilds
Fixes #4838
2018-06-10 20:55:17 +02:00
Christian Oliff
2e6712e281 tpl: Always load GA script over HTTPS 2018-06-09 23:47:44 +02:00
Alexandros
65deb72dc4 tplimpl: Remove speakerdeck shortcode
Fixes #4830
2018-06-09 11:13:36 +02:00
Bjørn Erik Pedersen
dc4226a8b2 hugolib: Reset Page's main output on server rebuilds
Not doing so prevents the paginators to be rebuilt on changes.

Fixes #4819
2018-06-05 10:45:24 +03:00
Bjørn Erik Pedersen
bf5f10faa9 Reset the "distinct error logger" on rebuilds
Fixes #4818
2018-06-05 10:45:24 +03:00
Alex
ceaff7cafc tpl: Remove frameborder attr YT iframe + CSS fixes 2018-05-30 14:06:32 +02:00
Bjørn Erik Pedersen
2174525cec
hugolib: A little more on shortcode ordinal 2018-05-28 10:29:48 +02:00
Vas Sudanagunta
4f0665f476 Enhance Page and Resource String() 2018-05-26 09:43:47 +02:00
Bjørn Erik Pedersen
1f1d955b56
Add anonymizeIP to GA privacy config
See #4751
2018-05-25 10:53:39 +02:00
Bjørn Erik Pedersen
35ccf06dae
Fix some recently broken embedded templates
And add tests for them.

Fixes #4757
2018-05-23 10:03:11 +02:00
Bjørn Erik Pedersen
4ddcf52ccc Add an unified .Site.Config with a services section
Fixes #4751
2018-05-22 18:11:03 +02:00
Bjørn Erik Pedersen
353148c2bc Move the privacy config into a parent
See #4751
2018-05-22 18:11:03 +02:00
Bjørn Erik Pedersen
2f17f9378a Do not return error on .Get "class" and vice versa in shortcodes
The current error handling makes parameter checking in shortcodes too verbose for no good reason.

Fixes #4745
2018-05-21 18:52:13 +02:00
Bjørn Erik Pedersen
5f24a2c047 Add PrivacyEnhanced mode for YouTube to the GDPR Policy
See #4616
2018-05-21 16:49:00 +02:00
Bjørn Erik Pedersen
0bbdef986d config: Add the foundation for GDPR privacy configuration
See #4616
2018-05-21 00:41:42 +02:00
Anthony Fok
9bd4236e1b
Show site build warning in TestPageBundlerSiteRegular
So that a timeout warning does appear if it does happen
especially on a slow machine with soft floating-point CPU.

Special thanks to @bep for the solution.

See #4672
2018-05-14 00:09:39 -06:00
Anthony Fok
b2b500f563
hugolib: Use double quotes instead of back quotes
so that '\n' is interpreted in the timeout warning.
2018-05-14 00:03:03 -06:00
Vas Sudanagunta
2fb9af59c1 Improve markup determination logic
Sets Page.markup earlier (as early as possible, when the page is
loaded). Sets it once and only once, removing many redundant calls
to determineMarkupType().

This kills a sleeping bug that was avoided by the parts of the
code depending on this value making those redundant calls.
2018-05-12 20:16:26 +02:00
Bjørn Erik Pedersen
086ae81a98
hugolib: Fix possible .Content cut
There have been one report of a site with truncated `.Content` after the Hugo `0.40.1` release.

This commit fixes this so that race should not be possible anymore. It also adds a stress test with focus on content rendering and multiple output formats.

Fixes #4706
2018-05-08 16:52:51 +02:00
Bjørn Erik Pedersen
34ad9a4f17
tpl/tplimpl: Extract internal templates
Having them in separate files should make maintainance easier.

When adding new or making changes to the templates:

```bash
mage generate
```

This will get the Go code in sync.

Fixes #4457
2018-05-04 23:12:10 +02:00
Bjørn Erik Pedersen
655983a22d Reset .Page.Scratch on live-reload
Fixes #4683
2018-04-30 08:21:22 +02:00
Bjørn Erik Pedersen
001a28c2f9 Fix .WordCount, .FuzzyWordCount, .ReadingTime when summary marker is set
This bug was introduced in Hugo 0.40. It is when you use the `<!--more-->` summary marker.

Note that this affects the word stats only. The related `PlainWords`, `Plain`, `Content` all return correct values.

Fixes #4675
Fixes #4682
2018-04-29 10:58:58 +02:00
Bjørn Erik Pedersen
288c396439
hugolib: Fix some shortcode vs .Content corner cases
This is a follow-up to #4632. There were some assumptions in that implementation that did not hold water in all situations.

This commit simplifies the content lazy initalization making it more robust.

Fixes #4664
2018-04-25 08:56:46 +02:00
Bjørn Erik Pedersen
402f6788ee
hugolib: Add .Page.BundleType
I eturn either:

1. leaf
2. branch
3. empty string

The above sits well with constructs like:

```
{{ with .BundleType }}
// Now we know it is a bundle
{{ end }}
```

Fixes #4662
2018-04-23 08:41:19 +02:00
Bjørn Erik Pedersen
3decf4a327
hugolib: Add zero-based Ordinal to shortcode
The count starts at 0 relative to the shortcode's parent: Either the page or the surrounding shortcode.

Access it in a shortcode like this:

```bash
Ordinal is {{ .Ordinal }}
```

Note that this is a shared ordinal for all shortcodes in the relevant context, so, as an example, you have this in a content page:

```markdown
This is a shortcode:

{{< hello >}}

This is another shortcode:

{{< hugo >}}

The `.Ordinal` you get in the two shortcodes above is 0 and 1.
```

See #3359
2018-04-23 08:09:56 +02:00
Bjørn Erik Pedersen
24c662ce6b
hugolib: Defer the unlock in orderedMap.Add
Just in case someone tries to recover from the potential panic.
2018-04-22 21:32:05 +02:00
Bjørn Erik Pedersen
85535084de
hugolib: Process and render shortcodes in their order of appearance
Fixes #3359
2018-04-22 17:40:51 +02:00