Commit graph

16 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
822dc627a1
tpl/transform: Add transform.Unmarshal func
Fixes #5428
2018-12-23 10:02:42 +01:00
Bjørn Erik Pedersen
78f8475a05 Fix Resource output in multihost setups
In Hugo 0.46 we made the output of what you get from resources.Get and similar static, i.e. language agnostic. This makes total sense, as it is wasteful and time-consuming to do SASS/SCSS/PostCSS processing for lots of languages when the output is lots of duplicates with different filenames.

But since we now output the result once only, this had a negative side effect for multihost setups: We publish the resource once only to the root folder (i.e. not to the language "domain folder").

This commit removes the language code from the processed image keys. This creates less duplication in the file cache, but it means that you should do a `hugo --gc` to clean up stale files.

Fixes #5058
2018-08-13 19:00:51 +02:00
Bjørn Erik Pedersen
b874a1ba7a media: Allow multiple file suffixes per media type
Before this commit, `Suffix` on `MediaType` was used both to set a custom file suffix and as a way to augment the mediatype definition (what you see after the "+", e.g. "image/svg+xml").

This had its limitations. For one, it was only possible with one file extension per MIME type.

Now you can specify multiple file suffixes using "suffixes", but you need to specify the full MIME type
identifier:

[mediaTypes]
[mediaTypes."image/svg+xml"]
suffixes = ["svg", "abc ]

In most cases, it will be enough to just change:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffix = "txt"

To:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffixes = ["txt"]

Hugo will still respect values set in "suffix" if no value for "suffixes" is provided, but this will be removed in a future release.

Note that you can still get the Media Type's suffix from a template: {{ $mediaType.Suffix }}. But this will now map to the MIME type filename.

Fixes #4920
2018-07-10 22:13:52 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
eb42774e58
Add support for a content dir set per language
A sample config:

```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true

[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"

[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```

The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.

The content files will be assigned a language by

1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.

The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.

Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.

If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.

Fixes #4523
Fixes #4552
Fixes #4553
2018-04-02 08:06:21 +02:00
Bjørn Erik Pedersen
3fbc75534d
resource: Fix path duplication/flattening in processed images
Fixes #4502
Closes #4501
2018-03-14 17:04:14 +01:00
Bjørn Erik Pedersen
772128485a Run gofmt -s with Go 1.10
See #4434
2018-02-21 09:59:33 +01:00
Bjørn Erik Pedersen
df20b05463
resource: Make resource counters for name and title independent
This is the most flexible with the current syntax, and probably what most people would expcect.

Updates #4335
2018-01-29 10:44:09 +01:00
Bjørn Erik Pedersen
7b472e4608
resource: Start Resources :counter first time they're used
This is less surprising and more flexible than the original implementation.

Given:

```toml
[[resources]]
  src = "documents/photo_specs.pdf"
  title = "Photo Specifications"
[[resources]]
  src = "**.pdf"
  name = "pdf-file-:counter"
```

Every `pdf` in the bundle will have an unique counter, but the `photo_specs.pdf` is still allowed to have its specific `title`.

If you change the above example to:

```toml
[[resources]]
  src = "documents/*specs.pdf"
  title = "Photo Specifications #:conter"
[[resources]]
  src = "**.pdf"
  name = "pdf-file-:counter"
```

We are talking about two different groups of documents, each with its own counters starting at 1.

Fixes #4335
2018-01-27 10:22:42 +01:00
Bjørn Erik Pedersen
5a0819b9b5 Merge matching resources params maps
This allows setting default params values in the more general resource matchers. I also allows override with more specific values if needed.

```toml
[[resources]]
src = "documents/photo_specs.pdf"
title = "Photo Specifications"
[resources.params]
ref = 90564687
icon = "photo"
[[resources]]
src = "documents/guide.pdf"
title = "Instruction Guide"
[resources.params]
ref = 90564568
[[resources]]
src = "documents/checklist.pdf"
title = "Document Checklist"
[resources.params]
ref = 90564572
[[resources]]
src = "documents/payment.docx"
title = "Proof of Payment"
[[resources]]
src = "documents/*.pdf"
title = "PDF file"
[resources.params]
icon = "pdf"
[[resources]]
src = "documents/*.docx"
title = "Word document"
[resources.params]
icon = "word"

```

In the above `TOML` example, `photo_specs.pdf` will get the `photo` icon, the other pdf files will get the default `pdf` icon.

Note that in the example above, the order matters: It will take the first value for a given params key, title or name that it finds.

Fixes #4315
2018-01-23 17:11:38 +01:00
Bjørn Erik Pedersen
9421380168
resource: Add Match and GetMatch
These methods takes a glob pattern as argument:

* by default matching from the bundle root
* matching is case insensitive and the separator is Unix style slashes: "/"
* the bundle root does (by default) not start with a leading slash
* if you renames the `Name` for the rsource in front matter (`src=...`), then that is the value used in `Match`.
* double asterisk matches beyond directory borders, so "**.jpg" will match any JPEG image in the bundle

See https://github.com/gobwas/glob

This commit also deprecates `ByPrefix` and `GetByPrefix`.

This should also be more effective, given a fair amount of reuse of the glob patterns:

```bash
BenchmarkResourcesByPrefix-4         300000          4284 ns/op        1130 B/op           7 allocs/op
BenchmarkResourcesMatch-4            300000          5220 ns/op         505 B/op           3 allocs/op
```

Fixes #4301
2018-01-22 10:28:12 +01:00
Bjørn Erik Pedersen
20c9b6ec81
resource: Add front matter metadata to Resource
This commit expands the Resource interface with 3 new methods:

* Name
* Title
* Params

All of these can be set in the Page front matter. `Name` will get its default value from the base filename, and is the value used in the ByPrefix and GetByPrefix lookup methods.

Fixes #4244
2018-01-17 16:22:33 +01:00
Bjørn Erik Pedersen
46db900dab
resource: Implement Resources.ByPrefix
Fixes #4266
2018-01-12 18:06:35 +01:00
Bjørn Erik Pedersen
db85e83403
resource: Make .Resources.GetByPrefix case insensitive
Fixes #4258
2018-01-11 18:58:53 +01:00
Bjørn Erik Pedersen
f25d8a9e17 Fix sub-folder baseURL handling for Page resources
I.e. images etc.

Fixes #4228
2018-01-06 10:29:13 +01:00
Bjørn Erik Pedersen
3cdf19e9b7
Implement Page bundling and image handling
This commit is not the smallest in Hugo's history.

Some hightlights include:

* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).

A site building  benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:

```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"

benchmark                                                                                                         old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      101785785     78067944      -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     185481057     149159919     -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      103149918     85679409      -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     203515478     169208775     -16.86%

benchmark                                                                                                         old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      532464         391539         -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1056549        772702         -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      555974         406630         -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1086545        789922         -27.30%

benchmark                                                                                                         old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      53243246      43598155      -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     105811617     86087116      -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      54558852      44545097      -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     106903858     86978413      -18.64%
```

Fixes #3651
Closes #3158
Fixes #1014
Closes #2021
Fixes #1240
Updates #3757
2017-12-27 18:44:47 +01:00