Commit graph

85 commits

Author SHA1 Message Date
Cameron Moore
600047ff1c source: Fix golint godoc issues 2018-09-07 08:25:51 +02:00
Bjørn Erik Pedersen
dea71670c0
Add Hugo Piper with SCSS support and much more
Before this commit, you would have to use page bundles to do image processing etc. in Hugo.

This commit adds

* A new `/assets` top-level project or theme dir (configurable via `assetDir`)
* A new template func, `resources.Get` which can be used to "get a resource" that can be further processed.

This means that you can now do this in your templates (or shortcodes):

```bash
{{ $sunset := (resources.Get "images/sunset.jpg").Fill "300x200" }}
```

This also adds a new `extended` build tag that enables powerful SCSS/SASS support with source maps. To compile this from source, you will also need a C compiler installed:

```
HUGO_BUILD_TAGS=extended mage install
```

Note that you can use output of the SCSS processing later in a non-SCSSS-enabled Hugo.

The `SCSS` processor is a _Resource transformation step_ and it can be chained with the many others in a pipeline:

```bash
{{ $css := resources.Get "styles.scss" | resources.ToCSS | resources.PostCSS | resources.Minify | resources.Fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

The transformation funcs above have aliases, so it can be shortened to:

```bash
{{ $css := resources.Get "styles.scss" | toCSS | postCSS | minify | fingerprint }}
<link rel="stylesheet" href="{{ $styles.RelPermalink }}" integrity="{{ $styles.Data.Digest }}" media="screen">
```

A quick tip would be to avoid the fingerprinting part, and possibly also the not-superfast `postCSS` when you're doing development, as it allows Hugo to be smarter about the rebuilding.

Documentation will follow, but have a look at the demo repo in https://github.com/bep/hugo-sass-test

New functions to create `Resource` objects:

* `resources.Get` (see above)
* `resources.FromString`: Create a Resource from a string.

New `Resource` transformation funcs:

* `resources.ToCSS`: Compile `SCSS` or `SASS` into `CSS`.
* `resources.PostCSS`: Process your CSS with PostCSS. Config file support (project or theme or passed as an option).
* `resources.Minify`: Currently supports `css`, `js`, `json`, `html`, `svg`, `xml`.
* `resources.Fingerprint`: Creates a fingerprinted version of the given Resource with Subresource Integrity..
* `resources.Concat`: Concatenates a list of Resource objects. Think of this as a poor man's bundler.
* `resources.ExecuteAsTemplate`: Parses and executes the given Resource and data context (e.g. .Site) as a Go template.

Fixes #4381
Fixes #4903
Fixes #4858
2018-07-06 11:46:12 +02:00
Bjørn Erik Pedersen
80230f26a3
Add support for theme composition and inheritance
This commit adds support for theme composition and inheritance in Hugo.

With this, it helps thinking about a theme as a set of ordered components:

```toml
theme = ["my-shortcodes", "base-theme", "hyde"]
```

The theme definition example above in `config.toml` creates a theme with the 3 components with presedence from left to right.

So, Hugo will, for any given file, data entry etc., look first in the project, and then in `my-shortcode`, `base-theme` and lastly `hyde`.

Hugo uses two different algorithms to merge the filesystems, depending on the file type:

* For `i18n` and `data` files, Hugo merges deeply using the translation id and data key inside the files.
* For `static`, `layouts` (templates) and `archetypes` files, these are merged on file level. So the left-most file will be chosen.

The name used in the `theme` definition above must match a folder in `/your-site/themes`, e.g. `/your-site/themes/my-shortcodes`. There are  plans to improve on this and get a URL scheme so this can be resolved automatically.

Also note that a component that is part of a theme can have its own configuration file, e.g. `config.toml`. There are currently some restrictions to what a theme component can configure:

* `params` (global and per language)
* `menu` (global and per language)
* `outputformats` and `mediatypes`

The same rules apply here: The left-most param/menu etc. with the same ID will win. There are some hidden and experimental namespace support in the above, which we will work to improve in the future, but theme authors are encouraged to create their own namespaces to avoid naming conflicts.

A final note: Themes/components can also have a `theme` definition in their `config.toml` and similar, which is the "inheritance" part of this commit's title. This is currently not supported by the Hugo theme site. We will have to wait for some "auto dependency" feature to be implemented for that to happen, but this can be a powerful feature if you want to create your own theme-variant based on others.

Fixes #4460
Fixes #4450
2018-06-10 23:55:20 +02:00
Bjørn Erik Pedersen
94c8b29c39
source: Remove deprecated File.Bytes 2018-04-15 14:34:34 +02:00
Bjørn Erik Pedersen
2817e84240 Fix handling of content files with "." in them
As in, more dots than just to separate the extension and any language indicator.

Fixes #4559
2018-04-07 10:57:29 +02:00
Bjørn Erik Pedersen
eb42774e58
Add support for a content dir set per language
A sample config:

```toml
defaultContentLanguage = "en"
defaultContentLanguageInSubdir = true

[Languages]
[Languages.en]
weight = 10
title = "In English"
languageName = "English"
contentDir = "content/english"

[Languages.nn]
weight = 20
title = "På Norsk"
languageName = "Norsk"
contentDir = "content/norwegian"
```

The value of `contentDir` can be any valid path, even absolute path references. The only restriction is that the content dirs cannot overlap.

The content files will be assigned a language by

1. The placement: `content/norwegian/post/my-post.md` will be read as Norwegian content.
2. The filename: `content/english/post/my-post.nn.md` will be read as Norwegian even if it lives in the English content folder.

The content directories will be merged into a big virtual filesystem with one simple rule: The most specific language file will win.
This means that if both `content/norwegian/post/my-post.md` and `content/english/post/my-post.nn.md` exists, they will be considered duplicates and the version inside `content/norwegian` will win.

Note that translations will be automatically assigned by Hugo by the content file's relative placement, so `content/norwegian/post/my-post.md` will be a translation of `content/english/post/my-post.md`.

If this does not work for you, you can connect the translations together by setting a `translationKey` in the content files' front matter.

Fixes #4523
Fixes #4552
Fixes #4553
2018-04-02 08:06:21 +02:00
Bjørn Erik Pedersen
b6798ee867
Bump some deprecations 2018-03-20 21:13:44 +01:00
Bjørn Erik Pedersen
4eb2fec67c Fix handling of top-level page bundles
Fixes #4332
2018-01-27 19:13:34 +01:00
Bjørn Erik Pedersen
6413559f75 Add a way to disable one or more languages
This commit adds a new config setting:

```toml
disableLanguages = ["fr"]
```

If this is a multilingual site:

* No site for the French language will be created
* French content pages will be ignored/not read
* The French language configuration (menus etc.) will also be ignored

This makes it possible to start translating new languages and turn it on when you're happy etc.

Fixes #4297
Fixed #4329
2018-01-26 14:04:14 +01:00
Bjørn Erik Pedersen
7e76a6fd3b
source: Fix test on Windows 2017-12-28 13:54:23 +01:00
Bjørn Erik Pedersen
1b0780dbeb
source: Make sure .File.Dir() ends with a slash
Updates #4190
2017-12-28 11:32:02 +01:00
Bjørn Erik Pedersen
3cdf19e9b7
Implement Page bundling and image handling
This commit is not the smallest in Hugo's history.

Some hightlights include:

* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).

A site building  benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:

```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"

benchmark                                                                                                         old ns/op     new ns/op     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      101785785     78067944      -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     185481057     149159919     -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      103149918     85679409      -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     203515478     169208775     -16.86%

benchmark                                                                                                         old allocs     new allocs     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      532464         391539         -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1056549        772702         -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      555974         406630         -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     1086545        789922         -27.30%

benchmark                                                                                                         old bytes     new bytes     delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      53243246      43598155      -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     105811617     86087116      -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4      54558852      44545097      -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4     106903858     86978413      -18.64%
```

Fixes #3651
Closes #3158
Fixes #1014
Closes #2021
Fixes #1240
Updates #3757
2017-12-27 18:44:47 +01:00
Bjørn Erik Pedersen
adbd5bc47f
source: Enable disabled test 2017-11-19 14:35:49 +01:00
Bjørn Erik Pedersen
60dfb9a6e0 Add support for multiple staticDirs
This commit adds support for multiple statDirs both on the global and language level.

A simple `config.toml` example:

```bash
staticDir = ["static1", "static2"]
[languages]
[languages.no]
staticDir = ["staticDir_override", "static_no"]
baseURL = "https://example.no"
languageName = "Norsk"
weight = 1
title = "På norsk"

[languages.en]
staticDir2 = "static_en"
baseURL = "https://example.com"
languageName = "English"
weight = 2
title = "In English"
```

In the above, with no theme used:

the English site will get its static files as a union of "static1", "static2" and "static_en". On file duplicates, the right-most version will win.
the Norwegian site will get its static files as a union of "staticDir_override" and "static_no".

This commit also concludes the Multihost support in #4027.

Fixes #36
Closes #4027
2017-11-17 11:01:46 +01:00
Cameron Moore
47fdfd5196 Clean up lint in various packages
Changes fall into one of the following:

- gofmt -s
- receiver name is inconsistent
- omit unused 2nd value from range
- godoc comment formed incorrectly
- err assigned and not used
- if block ends with a return statement followed by else
2017-09-29 16:23:16 +02:00
Matthieu Harlé
0abdeeef67 source: Normalize UniqueID between Windows & Linux 2017-08-09 19:35:40 +02:00
Jorin Vogel
81c13171a9 Add some missing doc comments
As pointed out by the linter, some exported functions and types are
missing doc comments.
The linter warnings have been reduced from 194 to 116.
Not all missing comments have been added in this commit though.
2017-08-03 15:57:51 +02:00
Bjørn Erik Pedersen
873a6f1885 Run gofmt to get imports in line vs gohugoio/hugo 2017-06-13 19:12:10 +02:00
Bjørn Erik Pedersen
d8717cd4c7 all: Update import paths to gohugoio/hugo 2017-06-13 18:42:45 +02:00
Bjørn Erik Pedersen
bef5048580 Revert "hugolib: Fix live-reload regression for add/removal of dirs"
Never mind. The "adding dir" scenario didn't work anyway, so that will need another fix.

This reverts commit 49a104309d.

Closes ##3325
2017-06-07 22:31:57 +02:00
Bjørn Erik Pedersen
49a104309d hugolib: Fix live-reload regression for add/removal of dirs
This reverts commit b5b6e81c02.

That change breaks watching of new directories (new dirs, deleted dirs).

Reopens #3325
Fixes #3569
2017-06-07 22:11:15 +02:00
Bjørn Erik Pedersen
4aff2b6e74 source: Cache language config
```
benchmark                                                                                                                     old ns/op     new ns/op     delta
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=5000|tags_per_page=0|shortcodes=false|render=false-4     552742744     501838152     -9.21%

benchmark                                                                                                                     old allocs     new allocs     delta
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=5000|tags_per_page=0|shortcodes=false|render=false-4     3047393        2957378        -2.95%

benchmark                                                                                                                     old bytes     new bytes     delta
BenchmarkSiteBuilding/frontmatter=YAML|num_root_sections=1|num_pages=5000|tags_per_page=0|shortcodes=false|render=false-4     356533864     354291924     -0.63%
```
2017-05-30 19:29:55 +03:00
xofyarg
b5b6e81c02 hugolib: Ignore non-source files on partial rebuild
Partial rebuild does not have the same logic as normal rebuild on
selecting which file to build. This change makes it possible to
share the file select logic between two kinds of build.

Fix #3325.
2017-04-22 22:38:54 +02:00
Bjørn Erik Pedersen
93ca7c9e95 all: Refactor to nonglobal Viper, i18n etc.
This is a final rewrite that removes all the global state in Hugo, which also enables
the use if `t.Parallel` in tests.

Updates #2701
Fixes #3016
2017-02-17 17:15:26 +01:00
Bjørn Erik Pedersen
c71e1b106e all: Refactor to nonglobal file systems
Updates #2701
Fixes #2951
2017-02-04 11:37:25 +07:00
digitalcraftsman
707d3cf137 docs, tpl: Fix documentation of UniqueID
See #2861
2016-12-31 17:55:03 +01:00
Bjørn Erik Pedersen
7d072fbbe6 source: Make UniqueID ... unique
Fixes #2861
2016-12-31 17:46:11 +01:00
bogem
75e55cd06f hugolib, source, tpl: Fix docs
See #2014
2016-11-23 09:13:00 +01:00
bogem
e81c06c3f0 helpers: Rename WalkRootTooShortError to ErrWalkRootTooShort
Fix golint warning: helpers/path.go:473:5: error var
WalkRootTooShortError should have name of the form ErrFoo

See #2014
2016-11-23 09:13:00 +01:00
bogem
1f130fd692 commands, hugolib, source, target, tpl: Get rid of some fmt statements 2016-11-22 23:43:55 +01:00
Albert Nigmatzianov
950034db5c source, tpl: Fix staticcheck complaints
tpl/template_funcs.go:1019:3: the surrounding loop is unconditionally terminated
source/lazy_file_reader.go:66:5: err != nil is always true for all possible
values ([nil:error] != [nil:error])
2016-11-15 21:22:43 +01:00
Prashant Karmakar
186db7cd7a Fix page names that contain dot
changes:
    - in hugolib/page.go, `func permalink` and `func TargetPath`
      Fixed the attempt to *replace* the extension of something
      that was *already* a basename.
    - in source/file.go, `func NewFile`
      added check for allowed languages before translating filename

Fixes #2555
2016-11-01 14:18:24 +01:00
Bjørn Erik Pedersen
342b6fe8a5 Make it clear that Path is relative to content root
Fixes #2290
2016-10-28 10:35:19 +02:00
Albert Nigmatzianov
f21e2f25c9 all: Unify case of config variable names
All config variables starts with low-case and uses camelCase.

If there is abbreviation at the beginning of the name, the whole
abbreviation will be written in low-case.
If there is abbreviation at the end of the name, the
whole abbreviation will be written in upper-case.
For example, rssURI.
2016-10-24 20:56:00 +02:00
Bjørn Erik Pedersen
c700cdc39c Replace some leftover os.Stat with hugofs.Source 2016-09-11 20:00:38 +02:00
Bjørn Erik Pedersen
e56ecab157 Multilingual TODO-fixes, take 1
See #2309
2016-09-06 18:32:18 +03:00
Bjørn Erik Pedersen
708bc78770 Optimize the multilanguage build process
Work In Progress!

This commit makes a rework of the build and rebuild process to better suit a multi-site setup.

This also includes a complete overhaul of the site tests. Previous these were a messy mix that
were testing just small parts of the build chain, some of it testing code-paths not even used in
"real life". Now all tests that depends on a built site follows the same and real production code path.

See #2309
Closes #2211
Closes #477
Closes #1744
2016-09-06 18:32:16 +03:00
Alexandre Bourget
ec33732fbe Add multilingual support in Hugo
Implements:
* support to render:
  * content/post/whatever.en.md to /en/2015/12/22/whatever/index.html
  * content/post/whatever.fr.md to /fr/2015/12/22/whatever/index.html
* gets enabled when `Multilingual:` is specified in config.
* support having language switchers in templates, that know
  where the translated page is (with .Page.Translations)
  (when you're on /en/about/, you can have a "Francais" link pointing to
   /fr/a-propos/)
  * all translations are in the `.Page.Translations` map, including the current one.
* easily tweak themes to support Multilingual mode
* renders in a single swift, no need for two config files.

Adds a couple of variables useful for multilingual sites

Adds documentation (content/multilingual.md)

Added language prefixing for all URL generation/permalinking see in the
code base.

Implements i18n. Leverages the great github.com/nicksnyder/go-i18n lib.. thanks Nick.
* Adds "i18n" and "T" template functions..
2016-09-06 18:32:15 +03:00
Kishin Yagami
39df7724ad source: Normalize file name to NFC
Fixes #2203
2016-08-08 20:25:00 +02:00
Bjørn Erik Pedersen
4ddd5361c1 Use the Afero source fs where relevant
Fixes #2319
2016-07-30 15:37:03 +02:00
Bjørn Erik Pedersen
a0859dc672 Make LazyFileReader use the Afero source fs
Fixes #2317
2016-07-30 15:21:02 +02:00
Alexandre Bourget
32d82a4496 Ignore emacs temp files 2016-07-11 02:01:45 +02:00
Cameron Moore
e2aea65170 helpers: Remove ToReader funcs
Remove StringToReader and BytesToReader in favor of using the stdlib directly.
2016-06-25 17:57:05 -05:00
Bjørn Erik Pedersen
85bcac530a source: Add missing GoDoc in File
See #2014
2016-04-07 15:44:53 +02:00
Bjørn Erik Pedersen
4f66f790b1 Add readFile template func
This also includes a refactor of the hugofs package and its usage.

The motivation for that is:

The Afero filesystems are brilliant. Hugo's way of adding a dozen of global variables for the different filesystems was a mistake. In readFile (and also in some other places in Hugo today) we need a way to restrict the access inside the working dir. We could use ioutil.ReadFile and implement the path checking, checking the base path and the dots ("..") etc. But it is obviously better to use an Afero BasePathFs combined witha ReadOnlyFs. We could create a use-once-filesystem and handle the initialization ourselves, but since this is also useful to others and the initialization depends on some other global state (which would mean to create a new file system on every invocation), we might as well do it properly and encapsulate the predefined set of filesystems. This change also leads the way, if needed, to encapsulate the file systems in a struct, making it possible to have several file system sets in action at once (parallel multilanguage site building? With Moore's law and all...)

Fixes #1551
2016-03-31 21:24:18 +02:00
srinivasreddy
b319c18969 source: Consolidate if conditions 2016-03-22 19:49:16 +01:00
Bjørn Erik Pedersen
e5aa477491 Add support for symbolic links for content, layout, static, theme
Note: This is for the content roots only, but that should satisfy most needs.

Fixes #1855
2016-03-21 19:13:29 -04:00
srinivasreddy
21054f730a source: Consolidate filepath.Split calls 2016-03-17 20:11:29 +01:00
Bjørn Erik Pedersen
4ada1cab4d source: Remove unused code 2016-03-14 15:43:51 +01:00
Steve Francia
9f3796a31d Read/reread individual source content files
next is incremental conversion
2016-01-26 14:26:23 -05:00