Commit graph

51 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
ebe4d39f17
resource/postcss: Try node_modules/postcss-cli/bin/postcss first
Fixes #4952
2018-07-18 10:00:50 +02:00
Bjørn Erik Pedersen
4c240800a4
resource/bundler: Improve error message 2018-07-16 10:42:07 +02:00
Bjørn Erik Pedersen
2f2bc7ff70
resource/integrity: Remove unused code 2018-07-14 11:17:48 +02:00
Bjørn Erik Pedersen
47d38628ec resource: Clean up the in-memory Resource reader usage
Turns out `strings.Reader` implements both `io.Reader` and `io.Seeker`, so we don't need anything special.

Updates #4936
2018-07-12 21:32:52 +02:00
Bjørn Erik Pedersen
0024dcfe3e resource: Move opening of the transformed resources after cache check
The old version should be functionally the same, but opening up these readers (potentially files) every time is wasteful if we don't read from them.
2018-07-12 19:36:12 +02:00
Bjørn Erik Pedersen
beec1fc98e resource: Fix resources.Concat for transformed resources
Fixes #4936
2018-07-12 19:36:12 +02:00
Bjørn Erik Pedersen
306573def0 Improve type support in resources.Concat
This allows the result of `.Resources.Match` and similar to be concatenated.

Fixes #4934
2018-07-12 13:43:27 +02:00
Bjørn Erik Pedersen
d96f2a460f
resource: Include the transformation step in the error message
Fixes #4924
2018-07-11 11:03:15 +02:00
Bjørn Erik Pedersen
b874a1ba7a media: Allow multiple file suffixes per media type
Before this commit, `Suffix` on `MediaType` was used both to set a custom file suffix and as a way to augment the mediatype definition (what you see after the "+", e.g. "image/svg+xml").

This had its limitations. For one, it was only possible with one file extension per MIME type.

Now you can specify multiple file suffixes using "suffixes", but you need to specify the full MIME type
identifier:

[mediaTypes]
[mediaTypes."image/svg+xml"]
suffixes = ["svg", "abc ]

In most cases, it will be enough to just change:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffix = "txt"

To:

[mediaTypes]
[mediaTypes."my/custom-mediatype"]
suffixes = ["txt"]

Hugo will still respect values set in "suffix" if no value for "suffixes" is provided, but this will be removed in a future release.

Note that you can still get the Media Type's suffix from a template: {{ $mediaType.Suffix }}. But this will now map to the MIME type filename.

Fixes #4920
2018-07-10 22:13:52 +02:00
Yang Li
3cea2932e1 Fix typos 2018-07-07 12:29:56 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
80230f26a3
Add support for theme composition and inheritance
This commit adds support for theme composition and inheritance in Hugo.

With this, it helps thinking about a theme as a set of ordered components:

```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```

The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.

So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.

Hugo uses two different algorithms to merge the filesystems, depending on the file type:

* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.

The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are  plans to improve on this and get a URL scheme so this can be resolved automatically.

Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:

* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`

The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.

A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.

Fixes #4460
Fixes #4450
2018-06-10 23:55:20 +02:00
Vas Sudanagunta
4f0665f476 Enhance Page and Resource String() 2018-05-26 09:43:47 +02:00
Bjørn Erik Pedersen
47c05c47e0 Add language merge support for Pages in resource.Resources
Fixes #4644
2018-04-19 16:23:00 +02:00
Bjørn Erik Pedersen
0e7716a424 resource: Implement Resource.Content
Fixes #4622
2018-04-15 18:08:06 +02:00
Bjørn Erik Pedersen
eb42774e58
Add support for a content dir set per language
A sample config:

```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true

[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"

[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```

The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.

The content files will be assigned a language by

1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.

The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.

Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.

If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.

Fixes #4523
Fixes #4552
Fixes #4553
2018-04-02 08:06:21 +02:00
Bjørn Erik Pedersen
b6798ee867
Bump some deprecations 2018-03-20 21:13:44 +01:00
Bjørn Erik Pedersen
3fbc75534d
resource: Fix path duplication/flattening in processed images
Fixes #4502
Closes #4501
2018-03-14 17:04:14 +01:00
Bjørn Erik Pedersen
ba94abbf5d
resource: Fix SVG and similar resource handling
The validation of if we could process the image (resize etc.) was moved up in Hugo 0.37, which meant SVG and other "non-processable" images would fail.

This commit fixes that by creating a regular resource for these image formats. They will not have `.Resize` or any of the other image methods.

Fixes #4455
2018-02-27 18:29:15 +01:00
Bjørn Erik Pedersen
13ea1e7c35 resource: Use Floyd-Steinberg dithering for PNGs
Fixes #4453
2018-02-26 19:44:36 +01:00
Bjørn Erik Pedersen
772128485a Run gofmt -s with Go 1.10
See #4434
2018-02-21 09:59:33 +01:00
Bjørn Erik Pedersen
799c654b0d resource: Preserve color palette for PNG images
This commit will force a reprocessing of PNG images with new names, so it is adviced to run a `hugo --gc` to remove stale files.

Fixes #4416
2018-02-19 20:15:58 +01:00
Bjørn Erik Pedersen
58382e9572 resource: Fix multi-threaded image processing issue
When doing something like this with the same image from a partial used in, say, both the home page and the single page:

```bash
{{ with $img }}
{{ $big := .Fill "1024x512 top" }}
{{ $small := $big.Resize "512x" }}
{{ end }}
```

There would be timing issues making Hugo in some cases try to process the same image with the same instructions in parallel.

You would experience errors of type:

```bash
png: invalid format: not enough pixel data
```

This commit works around that by adding a mutex per image. This should also improve the performance, sligthly, as it avoids duplicate work.

The current workaround before this fix is to always operate on the original:

```bash
{{ with $img }}
{{ $big := .Fill "1024x512 top" }}
{{ $small := .Fill "512x256 top" }}
{{ end }}
```

Fixes #4404
2018-02-14 20:59:02 +01:00
Bjørn Erik Pedersen
53dac9a506
resource: Improve error processing error message 2018-02-13 09:14:44 +01:00
Bjørn Erik Pedersen
722086b4ed
resource: Add smart cropping
This commit `smart` as a new and default anchor in `Fill`.

So:

```html
{{ $image.Fill "200x200" }}
```

Is, with default configuration, the same as:

```html
{{ $image.Fill "200x200" "smart" }}
```

You can change this default in your `config.toml`:

```toml
[imaging]
[imaging]
resampleFilter = "box"

quality = 68

anchor = "Smart"
```

Fixes #4375
2018-02-05 13:59:15 +01:00
Bjørn Erik Pedersen
df20b05463
resource: Make resource counters for name and title independent
This is the most flexible with the current syntax, and probably what most people would expcect.

Updates #4335
2018-01-29 10:44:09 +01:00
Bjørn Erik Pedersen
7b472e4608
resource: Start Resources :counter first time they're used
This is less surprising and more flexible than the original implementation.

Given:

```toml
[[resources]]
  src = "documents/photo_specs.pdf"
  title = "Photo Specifications"
[[resources]]
  src = "**.pdf"
  name = "pdf-file-:counter"
```

Every `pdf` in the bundle will have an unique counter, but the `photo_specs.pdf` is still allowed to have its specific `title`.

If you change the above example to:

```toml
[[resources]]
  src = "documents/*specs.pdf"
  title = "Photo Specifications #:conter"
[[resources]]
  src = "**.pdf"
  name = "pdf-file-:counter"
```

We are talking about two different groups of documents, each with its own counters starting at 1.

Fixes #4335
2018-01-27 10:22:42 +01:00
Alexey Grachov
912147ab89 resource: Fix typo in comment 2018-01-24 10:24:21 +01:00
Bjørn Erik Pedersen
5a0819b9b5 Merge matching resources params maps
This allows setting default params values in the more general resource matchers. I also allows override with more specific values if needed.

```toml
[[resources]]
src = "documents/photo_specs.pdf"
title = "Photo Specifications"
[resources.params]
ref = 90564687
icon = "photo"
[[resources]]
src = "documents/guide.pdf"
title = "Instruction Guide"
[resources.params]
ref = 90564568
[[resources]]
src = "documents/checklist.pdf"
title = "Document Checklist"
[resources.params]
ref = 90564572
[[resources]]
src = "documents/payment.docx"
title = "Proof of Payment"
[[resources]]
src = "documents/*.pdf"
title = "PDF file"
[resources.params]
icon = "pdf"
[[resources]]
src = "documents/*.docx"
title = "Word document"
[resources.params]
icon = "word"

```

In the above `TOML` example, `photo_specs.pdf` will get the `photo` icon, the other pdf files will get the default `pdf` icon.

Note that in the example above, the order matters: It will take the first value for a given params key, title or name that it finds.

Fixes #4315
2018-01-23 17:11:38 +01:00
Bjørn Erik Pedersen
34a216fe67
resource: Avoid some strings.ToLower in globbing
See #4301
2018-01-22 11:29:33 +01:00
Bjørn Erik Pedersen
9421380168
resource: Add Match and GetMatch
These methods takes a glob pattern as argument:

* by default matching from the bundle root
* matching is case insensitive and the separator is Unix style slashes: "/"
* the bundle root does (by default) not start with a leading slash
* if you renames the `Name` for the rsource in front matter (`src=...`), then that is the value used in `Match`.
* double asterisk matches beyond directory borders, so "**.jpg" will match any JPEG image in the bundle

See https://github.com/gobwas/glob

This commit also deprecates `ByPrefix` and `GetByPrefix`.

This should also be more effective, given a fair amount of reuse of the glob patterns:

```bash
BenchmarkResourcesByPrefix-4         300000          4284 ns/op        1130 B/op           7 allocs/op
BenchmarkResourcesMatch-4            300000          5220 ns/op         505 B/op           3 allocs/op
```

Fixes #4301
2018-01-22 10:28:12 +01:00
Bjørn Erik Pedersen
4eb1650bec
resource: Use path.Match instead of filepath.Match
They behave similar, but it is a path we're matching.

See #4244
2018-01-17 20:48:31 +01:00
Bjørn Erik Pedersen
20c9b6ec81
resource: Add front matter metadata to Resource
This commit expands the Resource interface with 3 new methods:

* Name
* Title
* Params

All of these can be set in the Page front matter. `Name` will get its default value from the base filename, and is the value used in the ByPrefix and GetByPrefix lookup methods.

Fixes #4244
2018-01-17 16:22:33 +01:00
Bjørn Erik Pedersen
ecaf14514e
resource: Fix handling of very long image file names
Fixes #4261
2018-01-15 20:22:14 +01:00
Bjørn Erik Pedersen
4dac1781e3
resource: Add some GoDoc 2018-01-13 00:08:21 +01:00
Bjørn Erik Pedersen
46db900dab
resource: Implement Resources.ByPrefix
Fixes #4266
2018-01-12 18:06:35 +01:00
Bjørn Erik Pedersen
60c9f3b1c3
resource: Make GetByPrefix work for Page resources
Fixes #4264
2018-01-12 17:46:50 +01:00
Bjørn Erik Pedersen
db85e83403
resource: Make .Resources.GetByPrefix case insensitive
Fixes #4258
2018-01-11 18:58:53 +01:00
Bjørn Erik Pedersen
50fb49c3d9
resource: Remove superflous comment 2018-01-11 09:17:48 +01:00
Bjørn Erik Pedersen
f0eecc6a4f Fix non-ASCII path handling for Page resources
Fixes #4241
2018-01-10 18:23:41 +01:00
Bjørn Erik Pedersen
4b04db0f08 resource: Avoid processing and storing same image for each language
Fixes #4231
2018-01-07 11:58:16 +01:00
Bjørn Erik Pedersen
97c1866e32
resource: Resources.ByType should return Resources
Currently it returns []Resource.

This way the invocations can be nested.

Fixes #4234
2018-01-07 10:53:45 +01:00
Bjørn Erik Pedersen
ab82a27d05 Fix URLs for bundle resources in multihost mode
Fixes #4217
2018-01-06 10:29:13 +01:00
Bjørn Erik Pedersen
f25d8a9e17 Fix sub-folder baseURL handling for Page resources
I.e. images etc.

Fixes #4228
2018-01-06 10:29:13 +01:00
Bjørn Erik Pedersen
196da49c9d
resource: Handle publish to /public on fresh build
Fixes #4213
2018-01-03 10:28:16 +01:00
Bjørn Erik Pedersen
a54fd7df21
resource: Use the correct Destination FS
Source and destination will be the same when this happens, but it should be correct.

See #4202
2018-01-01 14:28:19 +01:00
Bjørn Erik Pedersen
16e1d99c6d
resource: Create target dir if not exists
This is the case where image processing is triggered from shortcodes, i.e. before the target page's folder in /public is created.

Fixes #4202
2018-01-01 14:23:51 +01:00
Bjørn Erik Pedersen
e141294619
resource: Avoid potential case issue in image names 2017-12-28 22:56:48 +01:00
Bjørn Erik Pedersen
612dcc1944
helpers: Avoid writing the last MD5 buff part twice 2017-12-28 22:52:27 +01:00
Bjørn Erik Pedersen
e50a8c7a14 resource: Use MD5 to identify image files
But only a set of byte chunks spread around in the image file to calculate the fingerprint, which is much faster than reading the whole file:

```bash
BenchmarkMD5FromFileFast/full=false-4         	  300000	      4356 ns/op	     240 B/op	       5 allocs/op
BenchmarkMD5FromFileFast/full=true-4          	   30000	     42899 ns/op	   32944 B/op	       5 allocs/op
```

Fixes #4186
2017-12-28 17:41:51 +01:00