Commit graph

28 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
180195aa34
cache/filecache: Recover from file corruption
Fixes #6401
2019-10-21 09:51:51 +02:00
Bjørn Erik Pedersen
fcfa6f33bb
filecache: Ignore "does not exist" errors in prune
Fixes #6326
Fixes #5745
2019-09-12 17:37:41 +02:00
Bjørn Erik Pedersen
9e57182705
tests: Convert from testify to quicktest 2019-08-12 13:26:32 +02:00
Bjørn Erik Pedersen
e28bd4c0f8
Fix Jekyll import
Fixes #6131
2019-07-30 09:14:14 +02:00
Bjørn Erik Pedersen
508db1906b
Revert "cache/filecache: Skip resource cache init if the fs is missing"
I need to revisit this.

This reverts commit da4c4a7789.
2019-07-26 16:29:33 +02:00
Bjørn Erik Pedersen
da4c4a7789
cache/filecache: Skip resource cache init if the fs is missing
Fixes #6113
2019-07-26 10:45:07 +02:00
Bjørn Erik Pedersen
e5f2299741 Block symlink dir traversal for /static
This is in line with how it behaved before, but it was lifted a little for the project mount for Hugo Modules,
but that could create hard-to-detect loops.
2019-07-25 11:27:25 +02:00
Bjørn Erik Pedersen
9f5a92078a
Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092
2019-07-24 09:35:53 +02:00
Bjørn Erik Pedersen
b5f39d23b8 all: Apply staticcheck recommendations 2019-03-24 16:14:51 +01:00
Bjørn Erik Pedersen
d30e845485
Run gofmt -s 2019-03-23 20:15:06 +01:00
Bjørn Erik Pedersen
3f3187de0f
cache/namedmemcache: Fix data race 2018-12-27 12:05:42 +01:00
Bjørn Erik Pedersen
822dc627a1
tpl/transform: Add transform.Unmarshal func
Fixes #5428
2018-12-23 10:02:42 +01:00
Bjørn Erik Pedersen
514e18dc27 cache/filecache: Simplify test
See #5497
2018-12-04 19:19:20 +01:00
Bjørn Erik Pedersen
b804a70881 cache/filecache: Use OS fs for test
See #5497
2018-12-04 18:03:49 +01:00
Bjørn Erik Pedersen
7e75aeca80
Remove duplicate mapstructure depdendency 2018-11-28 00:17:43 +01:00
Bjørn Erik Pedersen
f9b4eb4f39 Handle themes in the new file cache (for images, assets)
In the newly consolidated file cache implementation, we forgot that we also look in the theme(s) for assets (SCSS transformations etc.), which is not good for Netlify and the demo sites.

Fixes #5460
2018-11-23 21:09:00 +01:00
Bjørn Erik Pedersen
94f0f7e597
cache/filecache: Add a :project placeholder
This allows for "cache per Hugo project", making `hugo --gc` work as expected, even if you have several Hugo projects running on the same PC.

See #5439
2018-11-14 23:14:51 +01:00
Bjørn Erik Pedersen
3c29c5af8e
cache/filecache: Add a cache prune func
Fixes #5439
2018-11-14 23:14:51 +01:00
Bjørn Erik Pedersen
33502667fb
cache/filecache: Add a filecache root dir
This is just a safe guard to make sure we don't evict/remove files that do not belong to the cache.
2018-11-14 23:14:51 +01:00
Bjørn Erik Pedersen
d3489eba5d
cache/filecache: Use time.Duration for maxAge
Fixes #5438
2018-11-14 23:14:50 +01:00
Bjørn Erik Pedersen
17d7ecde2b
cache/filecache: Split implementation and config into separate files 2018-11-14 23:14:50 +01:00
Bjørn Erik Pedersen
782dd15858
More spelling corrections 2018-11-13 18:28:40 +01:00
Bjørn Erik Pedersen
aff9c09166
cache/filecache: Spelling corrections 2018-11-13 18:16:53 +01:00
Bjørn Erik Pedersen
f7aeaa6129 Add a consolidated file cache
This commits reworks how file caching is performed in Hugo. Now there is only one way, and it can be configured.

This is the default configuration:

```toml
[caches]
[caches.getjson]
dir = ":cacheDir"
maxAge = -1
[caches.getcsv]
dir = ":cacheDir"
maxAge = -1
[caches.images]
dir = ":resourceDir/_gen"
maxAge = -1
[caches.assets]
dir = ":resourceDir/_gen"
maxAge = -1
```

You can override any of these cache setting in your own `config.toml`.

The placeholders explained:

`:cacheDir`: This is the value of the `cacheDir` config option if set (can also be set via OS env variable `HUGO_CACHEDIR`). It will fall back to `/opt/build/cache/hugo_cache/` on Netlify, or a `hugo_cache` directory below the OS temp dir for the others.
`:resourceDir`: This is the value of the `resourceDir` config option.

`maxAge` is the time in seconds before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off.

This means that if you run your builds on Netlify, all caches configured with `:cacheDir` will be saved and restored on the next build. For other CI vendors, please read their documentation. For an CircleCI example, see 6c3960a8f4/.circleci/config.yml

Fixes #5404
2018-11-13 14:19:42 +01:00
Bjørn Erik Pedersen
3eb313fef4 Simplify .Site.GetPage etc.
This commit is a follow up to a recent overhaul of the GetPage/ref/relref implemenation.

The most important change in this commit is the update to `.Site.GetPage`:

* To reduce the amount of breakage in the wild to its minimum, I have reworked .Site.GetPage with some rules:

* We cannot support more than 2 arguments, i.e. .Site.GetPage "page" "posts" "mypage.md" will now throw an error. I think this is the most uncommon syntax and should be OK. It is an easy fix to change the above to .Site.GetPage "/posts/mypage.md" or similar.
* .Site.GetPage "home", .Site.GetPage "home" "" and .Site.GetPage "home" "/" will give you the home page. This means that if you have page in root with the name home.md you need to do .Site.GetPage "/home.md" or similar

This commit also fixes some multilingual issues, most notable it is now possible to do cross-language ref/relref lookups by prepending the language code to the path, e.g. `/jp/posts/mypage.md`.

This commit also reverts the site building tests related to this to "Hugo 0.44 state", to get better control of the changes made.

Closes #4147
Closes #4727
Closes #4728
Closes #4728
Closes #4726
Closes #4652
2018-07-18 00:07:20 +02:00
Bjørn Erik Pedersen
3aa0e16d0c cache: Add even more concurrency to test 2017-06-28 22:47:28 +02:00
Bjørn Erik Pedersen
b3c8056de2 cache: Add concurrent cache test 2017-06-28 22:44:37 +02:00
Bjørn Erik Pedersen
87203139c3 cache: Add a simple partitioned lazy cache 2017-05-26 10:42:45 +03:00