Commit graph

118 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
1b7ecfc2e1
hugolib: Use []byte in shortcode parsing
See #5324
2018-10-22 19:57:44 +02:00
Bjørn Erik Pedersen
27f5a906a2
parser/pageparser: Use []byte in page lexer
See #5324
2018-10-22 19:57:44 +02:00
Bjørn Erik Pedersen
2fdc4a24d5
parser/pageparser: Add front matter etc. support
See #5324
2018-10-22 19:57:43 +02:00
Bjørn Erik Pedersen
f6863e1ef7
parser/pageparser: File renames and splitting
See #5324
2018-10-22 19:57:43 +02:00
Bjørn Erik Pedersen
d6c16afde0
Move the shortcode parser to the new pageparser package
See #5324
2018-10-22 19:57:43 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
159bed34c3
parser: Add some context to front matter parse error
Fixes #4638
2018-04-17 10:20:38 +02:00
Bjørn Erik Pedersen
0816a97a46
parser: Add WARNING for integer YAML keys
```bash
benchmark                                               old ns/op     new ns/op     delta
BenchmarkStringifyMapKeysStringsOnlyInterfaceMaps-4     3053          2015          -34.00%
BenchmarkStringifyMapKeysStringsOnlyStringMaps-4        5.23          5.18          -0.96%
BenchmarkStringifyMapKeysIntegers-4                     2320          5177          +123.15%

benchmark                                               old allocs     new allocs     delta
BenchmarkStringifyMapKeysStringsOnlyInterfaceMaps-4     6              6              +0.00%
BenchmarkStringifyMapKeysStringsOnlyStringMaps-4        0              0              +0.00%
BenchmarkStringifyMapKeysIntegers-4                     6              14             +133.33%

benchmark                                               old bytes     new bytes     delta
BenchmarkStringifyMapKeysStringsOnlyInterfaceMaps-4     1008          1008          +0.00%
BenchmarkStringifyMapKeysStringsOnlyStringMaps-4        0             0             +0.00%
BenchmarkStringifyMapKeysIntegers-4                     1008          1776          +76.19%
```
Closes #4393
2018-02-12 19:16:12 +01:00
Bjørn Erik Pedersen
10a917dfdc
parser: Tune stringifyMapKeys
```bash
benchmark                                               old ns/op     new ns/op     delta
BenchmarkStringifyMapKeysStringsOnlyInterfaceMaps-4     3269          3053          -6.61%
BenchmarkStringifyMapKeysStringsOnlyStringMaps-4        4.79          5.23          +9.19%
BenchmarkStringifyMapKeysIntegers-4                     2707          2320          -14.30%

benchmark                                               old allocs     new allocs     delta
BenchmarkStringifyMapKeysStringsOnlyInterfaceMaps-4     16             6              -62.50%
BenchmarkStringifyMapKeysStringsOnlyStringMaps-4        0              0              +0.00%
BenchmarkStringifyMapKeysIntegers-4                     16             6              -62.50%

benchmark                                               old bytes     new bytes     delta
BenchmarkStringifyMapKeysStringsOnlyInterfaceMaps-4     1080          1008          -6.67%
BenchmarkStringifyMapKeysStringsOnlyStringMaps-4        0             0             +0.00%
BenchmarkStringifyMapKeysIntegers-4                     1080          1008          -6.67%
```
2018-02-12 19:14:02 +01:00
Bjørn Erik Pedersen
d4beef0d2b
parser: Rename stringifyYAMLMapKeys to stringifyMapKeys 2018-02-12 18:17:25 +01:00
Bjørn Erik Pedersen
51213e0be1
parser: Add benchmarks for stringifyYAMLMapKeys
```bash
BenchmarkStringifyMapKeysStringsOnlyInterfaceMaps-4       500000              3269 ns/op            1080 B/op         16 allocs/op
BenchmarkStringifyMapKeysStringsOnlyStringMaps-4        300000000                4.79 ns/op            0 B/op          0 allocs/op
BenchmarkStringifyMapKeysIntegers-4                       500000              2707 ns/op            1080 B/op         16 allocs/op
```
2018-02-12 18:16:48 +01:00
Vas Sudanagunta
1fa2417777 Add support for YAML array data files
* Unmarshaled YAML arrays indistinguishable from JSON arrays.
* Fixes #3890
2018-02-12 17:14:40 +01:00
Dawid Gaweł
16a5c74519 parser: Fix YAML maps key type
Recurse through result of yaml package parsing and change all
maps from map[interface{}]interface{} to map[string]interface{}
making them jsonable and sortable.

Fixes #2441, #4083
2018-02-09 10:27:17 +01:00
Vas Sudanagunta
4402c07775 Fix JSON array-based data file handling regression
This bug was introduced in Hugo 0.35.

Fixes #4361
2018-02-02 09:14:37 +01:00
Vas Sudanagunta
91bb774ae4 Support pages without front matter
* Page without front matter now treated same as a page with empty front matter.
* Test cases added to cover this and repro issue #4320.
* Type safety of front matter code improved.

Fixes #4320
2018-01-26 09:17:27 +01:00
Bjørn Erik Pedersen
7f82b41a24 parser: Final (!) fix for issue with escaped JSON front matter
Fixes #3682
2017-07-08 18:43:36 +02:00
Bjørn Erik Pedersen
84db6c74a0 parser: Fix issue with escaped JSON front matter
Fixes #3682
2017-07-08 10:56:56 +02:00
Bjørn Erik Pedersen
e10e51a008 parser: Fix handling of JSON front matter with escaped quotes
Fixes #3661
2017-07-03 09:01:54 +02:00
Bjørn Erik Pedersen
3183b9a29d parser: Fix handling of quoted brackets in JSON front matter
Also remove a broken JSON test.

Fixes #3511
2017-06-19 16:45:52 +02:00
Bjørn Erik Pedersen
422057f607 create: Use archetype template as-is as a Go template
This commit removes the fragile front matter decoding, and takes the provided archetype file as-is and processes it as a template.

This also means that we no longer will attempt to fill in default values for `title` and `date`.

The upside is that it is now easy to create these values in a dynamic way:

```toml
+++
title = {{ .BaseFileName | title }}
date = {{ .Date }}
draft = true
+++
```

You can currently use all of Hugo's template funcs, but the data context is currently very shallow:

* `.Type` gives the archetype kind provided
* `.Name` gives the target file name without extension.
* `.Path` gives the target file name
* `.Date` gives the current time as RFC3339 formatted string

The above  will probably be extended in #1629.

Fixes #452
Updates #1629
2017-06-18 19:06:28 +02:00
Bjørn Erik Pedersen
2c2ce33a39 parser: Add horizontal YAML tags to benchmark 2017-06-03 12:35:28 +02:00
Bjørn Erik Pedersen
0907a5c1c2 all: Temporarily revert to BurntSushi for TOML front matter handling
We still have go-toml as a transitive dependency, and it is the way to go eventually, but we care about speed, so let us wait that one out.

Note that the issue this fixes is about taxonomies, but I guess this is a general issue for sites with many pages that uses TOML as front matter.

```
benchmark                              old ns/op     new ns/op     delta
BenchmarkFrontmatterTags/TOML:1-4      23206         8543          -63.19%
BenchmarkFrontmatterTags/TOML:11-4     80117         18495         -76.92%
BenchmarkFrontmatterTags/TOML:21-4     140676        28727         -79.58%

benchmark                              old allocs     new allocs     delta
BenchmarkFrontmatterTags/TOML:1-4      173            60             -65.32%
BenchmarkFrontmatterTags/TOML:11-4     625            138            -77.92%
BenchmarkFrontmatterTags/TOML:21-4     1106           210            -81.01%

benchmark                              old bytes     new bytes     delta
BenchmarkFrontmatterTags/TOML:1-4      9231          2912          -68.45%
BenchmarkFrontmatterTags/TOML:11-4     19808         5184          -73.83%
BenchmarkFrontmatterTags/TOML:21-4     31200         7536          -75.85%
```

See #3541
Updates #3464
2017-06-03 09:22:57 +02:00
Bjørn Erik Pedersen
3d9c4f513b parser: Add BenchmarkFrontmatterTags
The list handling is surprisingly expensive:

```
▶ go test -run="NONE" -bench="BenchmarkFrontmatterTags" -test.benchmem=true ./parser | prettybench
PASS
benchmark                               iter         time/iter   bytes alloc           allocs
---------                               ----         ---------   -----------           ------
BenchmarkFrontmatterTags/JSON:1-4    1000000     2039.00 ns/op      912 B/op     20 allocs/op
BenchmarkFrontmatterTags/JSON:11-4    300000     5202.00 ns/op     1640 B/op     44 allocs/op
BenchmarkFrontmatterTags/JSON:21-4    200000     7993.00 ns/op     2392 B/op     65 allocs/op
BenchmarkFrontmatterTags/YAML:1-4     200000     9359.00 ns/op     5928 B/op     66 allocs/op
BenchmarkFrontmatterTags/YAML:11-4    100000    21218.00 ns/op     8408 B/op    140 allocs/op
BenchmarkFrontmatterTags/YAML:21-4     50000    32852.00 ns/op    10920 B/op    211 allocs/op
BenchmarkFrontmatterTags/TOML:1-4     100000    21505.00 ns/op     9231 B/op    173 allocs/op
BenchmarkFrontmatterTags/TOML:11-4     20000    82919.00 ns/op    19808 B/op    625 allocs/op
BenchmarkFrontmatterTags/TOML:21-4     10000   141847.00 ns/op    31200 B/op   1106 allocs/op
ok      github.com/spf13/hugo/parser    17.890s
```

See #3464
2017-06-03 08:53:56 +02:00
Albert Nigmatzianov
d3cd68e656 parser: Improve TOML frontmatter parser performance
Difference between toml.Load(string(datum)) and
toml.LoadReader(bytes.NewReader(datum)):
benchmark           old ns/op     new ns/op     delta
BenchmarkLoad-4     82068         78489         -4.36%

benchmark           old allocs     new allocs     delta
BenchmarkLoad-4     494            493            -0.20%

benchmark           old bytes     new bytes     delta
BenchmarkLoad-4     17009         16913         -0.56%
2017-03-20 09:02:35 +01:00
Bjørn Erik Pedersen
d8923eb676 Update to new go-toml API
Closes #3142
2017-03-15 08:04:52 +01:00
crasm
ede452d34e parser: Accept JSON frontmatter without leading "{\n"
Accept JSON frontmatter without leading "{\n" so that one line
frontmatters such as `{"param":"paramvalue"}` no longer silently render
empty html.
2017-03-13 17:19:03 +01:00
Cameron Moore
f039e3be9e parser: Refactor frontmatter parser and add tests
Lots of cleanups here:

- Refactor InterfaceToConfig and InterfaceToFrontMatter to use io.Writer.
- Simplify InterfaceToFrontMatter by wrapping InterfaceToConfig.
- Export FrontmatterType since we return it in DetectFrontMatter.
- Refactor removeTOMLIdentifier to avoid blindly replacing "+++".
- Update HandleJSONMetaData to return an empty map on nil input.
- Updates vendored goorgeous package and test for org-mode frontmatter.
- Add tests and godoc comments.

Coverage for parser package increased from 45.2% to 85.2%.
2017-03-11 17:52:25 +01:00
Chase Adams
86e8dd62f0 all: Add org-mode support
Fixes #1483 
See #936
2017-02-21 08:46:03 +01:00
bogem
dec1706ae0 commands, hugolib, parser, tpl: Use errors.New instead of fmt.Errorf 2016-11-22 23:43:55 +01:00
bogem
4d1cb50003 parser: Use strings.Contains instead of strings.Index
Closes #2400
2016-09-11 12:23:15 +02:00
bogem
798bf60c56 parser: Simplify err conditions 2016-09-11 12:23:09 +02:00
Bjørn Erik Pedersen
20c4311df4 Switch to a more up to date TOML library
Fixes #2089
2016-08-20 20:33:06 +01:00
Henrique Dias
af34613f75 Update frontmatter.go 2016-07-13 23:53:06 +02:00
Henrique Dias
16b71bbbb4 Export "detectFrontMatter" to fix caddy-hugo
Closes #2108
2016-04-26 22:28:47 +02:00
Mattias Wadman
5d50c46482 Chomp Unicode BOM if present
Useful if using or sharing files with users that use editors that
append a unicode byte order marker header (like Windows notepad).

This will still assume files are UTF-8 encoded.

Closes #2075
2016-04-14 10:46:27 +02:00
Bjørn Erik Pedersen
b71f391cd7 parser: Remove unused vars 2016-03-24 21:34:34 +01:00
Bjørn Erik Pedersen
c20dee9d7f parser: Make the constant vars ... constants 2016-03-23 15:05:32 +01:00
Bjørn Erik Pedersen
bf76e74432 parser: Fix ALL_CAPS var names in test 2016-03-23 15:02:00 +01:00
Bjørn Erik Pedersen
1cb7ed6ac7 parser: Spring code cleaning 2016-03-23 14:51:16 +01:00
Bjørn Erik Pedersen
3a82ae7114 parser: Unexport some internals 2016-03-14 17:52:11 +01:00
Bjørn Erik Pedersen
df3f2af0a8 parser: Remove unused code 2016-03-14 15:42:59 +01:00
Derek Perkins
bac1ba4655 Fix spelling and go vet errors 2016-02-13 19:08:48 +08:00
Anthony Fok
8509727fe8 Add copyright header to that source files that don’t have one.
See #1646
2015-12-10 15:19:38 -07:00
Bjørn Erik Pedersen
e445c35d6a Fix copyright headers in source files
Still need to add some missing headers and an AUTHORS file.

See #1646
2015-12-07 19:57:01 +01:00
Steve Francia
f045d7a611 Change the license to Apache 2.0 2015-11-23 22:16:36 -05:00
Sven Dowideit
a1e32439fb Allow for any (short) line begining or ending with html comment
Fixes #1428
2015-09-25 18:32:32 +02:00
Icaro Seara
e791835e6d Fix typo, "delemiters" -> "delimiters" 2015-08-19 08:06:13 -06:00
Tatsushi Demachi
98659bf3b8 Fix searching YAML/TOML delimiters in frontmatter
When a YAML/TOML's delimiter character sequence is included in a
frontmatter string, parser mistakes it as a delimiter. This fixes it by
checking a character right before the delimiter sequence is '\n' or it
is the beginning of the frontmatter.

Fix #1320
2015-08-03 18:30:55 +02:00
Jeff Hodges
8d28686edc Trim trailing spaces from YAML and TOML delimiters
When someone hits space after typing "---" (or "+++") but before they
hit return, hugo silently failed to parse the file. This corrects that.
2015-08-02 14:46:46 +02:00
bep
a52e508d46 Update test logs for uniformity and consistency
Many minor fixes to make test logs more consistent and correct a
mispelling.

Standardize on "[%i] got X but expected Y" for log messages. Using
a consistent layout makes it easier to read the test results. This
was mostly changing "Got" to "got". Swapped the order of values on
several calls to bring them in line with the convention.

A few log messages had a sequence number added to identify the
exact scenario that failed. Otherwise, there would be no way to
ascertain which failed When there are many scenarios.

Correct spelling of "expected."

Fixes #1028
Merged be2097e1ad

[close #1040]
2015-05-08 22:27:00 -04:00
Joel Scoble
b4871787f0 add undraft command 2015-05-08 22:18:51 -04:00
Anthony Fok
67df33f500 Correct initialisms as suggested by golint
First step to use initialisms that golint suggests,
for example:

    Line 116: func GetHtmlRenderer should be GetHTMLRenderer

as see on http://goreportcard.com/report/spf13/hugo

Thanks to @bep for the idea!

Note that command-line flags (cobra and pflag)
as well as struct fields like .BaseUrl and .Url
that are used in Go HTML templates need more work
to maintain backward-compatibility, and thus
are NOT yet dealt with in this commit.

First step in fixing #959.
2015-03-11 21:55:00 +01:00
bep
f85d1a7da2 parser: add some frontmatter test cases 2015-03-10 23:17:39 +01:00
bep
63ffb916d6 parser: apply some Golint rules 2015-03-07 12:59:04 +01:00
Anthony Fok
a0c6dba305 Upgrade from gopkg.in/yaml.v1 to gopkg.in/yaml.v2 2015-01-24 04:30:00 -07:00
Tatsushi Demachi
78e9229c52 Fix "hugo new" EOF error with an archetype file without the final EOL
This rewrites `extractFrontMatterDelims` function to make it work with
an archetype file without the final EOL and adds more detailed error
messages and comments.

It also removes `matches` and `matches_quick` functions which aren't
called anywhere.
2015-01-10 19:48:35 +01:00
Andrew Gerrand
634d7b2638 switch to new location of goyaml 2014-08-18 11:31:55 -04:00
spf13
6b9d4a93da Adding new commands (new site [path], new theme [name]) 2014-05-08 18:30:58 -04:00
spf13
6d9a2d2497 adding a front matter format to lead rune method 2014-05-02 01:01:44 -04:00
spf13
cbd9506c29 moving front matter parsing (and creation) to parse package 2014-05-01 13:20:09 -04:00
Mike Keesey
2540d884d8 Fixing issues go vet reports. 2014-03-01 09:56:17 -05:00
Ask Bjørn Hansen
1dbed5ee06 Call the json delimiter 'json_lead' instead of 'java_lead' 2014-02-18 18:44:27 -05:00
spf13
8055838c70 Allowing empty files (ignored) so you can touch a new file while watching 2013-12-28 13:47:34 -05:00
Egon Elbre
1c60d5bf20 Don't shutdown hugo on an empty file. 2013-12-28 13:47:21 -05:00
spf13
18f2b82658 Switching to the rjson library which is more friendly to human generated json. 2013-10-01 22:45:24 -04:00
Noah Campbell
67b2abaf09 Add IsRenderable to Page
As pages are read from the target, they will be assessed if they should
be rendered or not.  The logic for IsRenderable is in the parser/page.go
and looks for anything exception '<'.
2013-09-18 10:17:43 -07:00
Noah Campbell
d8e1834910 Fix parsing edge case of frontmatter
When the frontmatter contains a - (or other delimiter) close to the
closing frontmatter delimiter, frontmatter detection would fail.
2013-09-18 09:15:46 -07:00
Noah Campbell
a82efe5bb1 Merge remote-tracking branch 'origin/parser' into mrg_praser
Also brought in parse for github.com/noahcampbell/akebia

Conflicts:
	hugolib/page.go
	hugolib/page_test.go
2013-09-17 15:52:40 -07:00