Commit graph

890 commits

Author SHA1 Message Date
Joe Mooring
85e2ac1a44 commands: Create assets directory with new site
Closes #10460
2022-11-21 09:42:40 +01:00
Bjørn Erik Pedersen
20ef6dcf95
Skip flakey server tests on GitHub Action on Windows 2022-10-26 09:29:55 +02:00
Anthony Fok
0addb302ac server: Fix flaky TestServerPathEncodingIssues tests
Set getNumHomes: 1 to enable 567 ms or 2 s of wait for the server
to be ready in TestServerPathEncodingIssues/Unicode_paths and
TestServerPathEncodingIssues/Windows_multilingual_404.

Fixes #10332
2022-10-04 14:48:29 +02:00
Bjørn Erik Pedersen
4611b6920b Fix htimes /: operation not permitted error on config changes
Regression introduced in v0.104.1.
2022-09-29 12:27:25 +02:00
Bjørn Erik Pedersen
29ccb36069 Fix /static performance regression from Hugo 0.103.0
In `v0.103.0` we added support for `resources.PostProcess` for all file types, not just HTML. We had benchmarks that said we were fine in that department, but those did not consider the static file syncing.

This fixes that by:

* Making sure that the /static syncer always gets its own file system without any checks for the post process token.
* For dynamic files (e.g. rendered HTML files) we add an additional check to make sure that we skip binary files (e.g. images)

Fixes #10328
2022-09-26 19:02:25 +02:00
Bjørn Erik Pedersen
f3560aa0e1 server: Fix 404 redirects on Windows
Fixes #10314
2022-09-23 13:07:56 +02:00
Bjørn Erik Pedersen
08f0984f91
commands: Skip flaky test on CI 2022-09-22 11:24:42 +02:00
Bjørn Erik Pedersen
6be6752c8a server: Fix redirects when file path contains bytes > 0x80
Fixes #10287
2022-09-18 15:16:11 +02:00
Bjørn Erik Pedersen
74daca6b30 Support PostProcess for all file types
Not just HTML.

Fixes #10269
2022-09-14 19:09:20 +02:00
Bjørn Erik Pedersen
a5cda5ca4d server: Add 404 support 2022-09-14 14:25:33 +02:00
satotake
ab5ce59894 Fix usage description 2022-09-08 15:35:11 +02:00
satotake
7d40da876c Add --force to hugo new
Closes #9243
2022-09-08 15:35:11 +02:00
Bjørn Erik Pedersen
45e1084ff2
Add linux/arm64 extended to release setup
Fixes #8257
2022-08-28 18:23:01 +02:00
Bjørn Erik Pedersen
35fa192838 deps: Udpate to github.com/alecthomas/chroma/v2
Fixes #9932
Fixes #9931
2022-06-14 14:08:40 +02:00
Bjørn Erik Pedersen
cbc35c48d2 Respect NO_COLOR
Fixes #10004
2022-06-13 18:07:37 +02:00
Paul van Brouwershaven
c7d5f9f067
resources: Register MediaTypes before build
Fixes #9971
2022-06-03 21:37:50 +02:00
Bjørn Erik Pedersen
653ab2cc1f
commands: Fix case where languages cannot be configured
There are some commands that needs to complete without a complete configuration.
2022-05-27 13:34:20 +02:00
Bjørn Erik Pedersen
8ca7052528
server: Skip watching dirs in ignoreFiles
Fixes #9838
2022-05-25 18:31:00 +02:00
satotake
6f7fbe03b1 basefs: add noBuildLock flag
If the flag is enabled, `.hugo_build.lock` will not be created.
This ensures safe running on read-only filesystem etc.
Close #9780
2022-05-24 13:57:23 +02:00
satotake
2fc2e9c871 import: Fix importing jekyll site
Just use `afero.OsFs` without building site.
Close #9817
2022-05-21 23:44:40 +02:00
Bjørn Erik Pedersen
2f9eac480f server: Fix multihost crash
As introduced in v0.99.0.

Fixes #9901
2022-05-18 11:10:04 +02:00
Bjørn Erik Pedersen
657d1a2d95
server: Refresh the error template
We cannot cache it forever, as that will not allow the end user to edit and preview it.
2022-05-16 09:24:30 +02:00
Bjørn Erik Pedersen
87a22eb6d6 server: Fix SIGINT handling after loading bad configuration
Also fix the config error messages.

Fixes #9664
2022-05-15 22:58:05 +02:00
Bjørn Erik Pedersen
fc9f315d86 Improve SASS errors
Fixes #9897
2022-05-15 20:25:25 +02:00
Bjørn Erik Pedersen
5c96bda70a
errors: Misc improvements
* Redo the server error template
* Always add the content file context if relevant
* Remove some now superflous error string matching
* Move the server error template to _server/error.html
* Add file context (with position) to codeblock render blocks
* Improve JS build errors

Fixes #9892
Fixes #9891
Fixes #9893
2022-05-14 13:40:56 +02:00
Bjørn Erik Pedersen
4a96df96d9
server: Always rebuild the files involved in an error
Fixes #9884
2022-05-14 13:40:56 +02:00
Bjørn Erik Pedersen
51f08b0b6a Revise the use of htime.Since/htime.Now
We cannot (also, it doesn't add any value)  use that when the `clock` is set,

* To measure time (before that global is set)
* To compare file timestamps re cache eviction

Fixes #9868
2022-05-09 14:18:40 +02:00
Bjørn Erik Pedersen
35c88a7f90 Use configured timeZone for the clock
And some other related adjustments.

Updates #8787
2022-05-08 16:56:26 +02:00
satotake
e77ca3c105 Add clock cli flag
Close #8787
2022-05-08 16:56:26 +02:00
Bjørn Erik Pedersen
f2946da9e8 Improve error messages, esp. when the server is running
* Add file context to minifier errors when publishing
* Misc fixes (see issues)
* Allow custom server error template in layouts/server/error.html

To get to this, this commit also cleans up and simplifies the code surrounding errors and files. This also removes the usage of `github.com/pkg/errors`, mostly because of https://github.com/pkg/errors/issues/223 -- but also because most of this is now built-in to Go.

Fixes #9852
Fixes #9857
Fixes #9863
2022-05-06 19:43:22 +02:00
Bjørn Erik Pedersen
9b352f04a3 Fix syncing of /static regression
As introduced in Hugo `v0.76.1`.

And add a proper test for it.

Fixes  #9794
Closes #9788
2022-04-18 19:17:27 +02:00
Bjørn Erik Pedersen
e66e2e9ce5 Revert "Revert "Fix PostProcess regression for hugo server""
This reverts commit 6c35a1a9ea.

Updates #9794
2022-04-18 19:17:27 +02:00
Bjørn Erik Pedersen
6c35a1a9ea
Revert "Fix PostProcess regression for hugo server"
This reverts commit 4deb5c6066.
2022-04-17 10:35:01 +02:00
Bjørn Erik Pedersen
4deb5c6066 Fix PostProcess regression for hugo server
Fixes #9788
2022-04-16 18:43:13 +02:00
Bjørn Erik Pedersen
d070bdf10f
Rework the Destination filesystem to make --renderStaticToDisk work
See #9626
2022-04-08 13:26:17 +02:00
Bjørn Erik Pedersen
b08193971a
Revert "Revert "Some minor adjustments to the new static filesystem logic""
This reverts commit 5ef8a9f32c.
2022-04-08 13:26:17 +02:00
Bjørn Erik Pedersen
0a56f2af4e
Revert "Revert "Allow rendering static files to disk and dynamic to memory in server mode""
This reverts commit 64b7b7a897.
2022-04-08 13:26:16 +02:00
Joe Mooring
a6fa290f67 commands: Change link protocol to https 2022-03-25 15:29:50 +01:00
Bjørn Erik Pedersen
9539069f5e commands: Improve server startup to make tests less flaky
Do this by announcing/listen on the local address before we start the server.
2022-03-21 09:32:35 +01:00
Bjørn Erik Pedersen
b80853de90
all: gofmt -w -r 'interface{} -> any' .
Updates #9687
2022-03-17 22:03:27 +01:00
Bjørn Erik Pedersen
31fbc081c9 Improve server startup/shutdown
Closes #9671
2022-03-14 19:38:17 +01:00
Bjørn Erik Pedersen
cebd886ac1 commands: Improve server tests
Updates #9647
2022-03-14 09:16:33 +01:00
Bjørn Erik Pedersen
64b7b7a897 Revert "Allow rendering static files to disk and dynamic to memory in server mode"
This reverts commit 7d8011ed63.

Updates #9647
2022-03-11 13:10:47 +01:00
Bjørn Erik Pedersen
5ef8a9f32c Revert "Some minor adjustments to the new static filesystem logic"
This reverts commit 63bb2a5b18.
2022-03-11 13:10:47 +01:00
Bjørn Erik Pedersen
63bb2a5b18 Some minor adjustments to the new static filesystem logic
Closes #9625
2022-03-08 19:27:54 +01:00
SatowTakeshi
7d8011ed63 Allow rendering static files to disk and dynamic to memory in server mode
Updates #9625
2022-03-08 19:27:54 +01:00
Marshall Cottrell
06bac57ab0 Add support for CODEOWNERS
Fixes #9474
2022-02-23 22:43:19 +01:00
Bjørn Erik Pedersen
aebde49b88 commands: Fix server panic regression
And now with a proper server test.

Fixes #9518
Fixes #9530
Fixes #9539
2022-02-21 21:43:26 +01:00
Bjørn Erik Pedersen
f2e7b49acf Add --printUnusedTemplates
Fixes #9502
2022-02-15 20:01:57 +01:00
Bjørn Erik Pedersen
837fdfdf45
commands: Rename --i18n-warnings to printI18nWarnings
To get it in line with the others.
2022-02-15 13:03:36 +01:00
Bjørn Erik Pedersen
6819feab6c
commands: Rename --path-warnings, --print-men to --printPathWarnings, --printMemoryUsage
To get it in line with the others.
2022-02-15 13:00:06 +01:00
Bjørn Erik Pedersen
a2a660ed12 commands: Fix server deadlock on config error
Fixes #9486
2022-02-10 19:43:47 +01:00
Joe Mooring
54f8d8a70a Remove the "check" command
Closes #9454
2022-02-10 16:25:23 +01:00
Joe Mooring
3336762939 Remove hugo gen autocomplete
Closes #8862
2022-02-01 23:23:30 +01:00
Bjørn Erik Pedersen
e334a40661
commands: Fix CLI help text for hugo new
Updates #9363
2022-01-07 12:36:12 +01:00
Bjørn Erik Pedersen
c8b5ab75b7 Add --panicOnWarning flag
Fixes #9357
Fixes #9359
2022-01-06 12:27:04 +01:00
Bjørn Erik Pedersen
2b6063c3e3 Misc depreation updates
* Deprecate .Page.Path when backed by a file
* site.Permalinks
* --ignoreVendor (use --ignoreVendorPaths)

Closes #9348
Closes #9349
2022-01-04 17:10:39 +01:00
Bjørn Erik Pedersen
fdad91fd96
commands: Make sure pollInterval is always set
Fixes #9165
2021-11-13 21:45:51 +01:00
Bjørn Erik Pedersen
ab5c6990a5
commands: Fix missing file locking in server partial render
Fixes #9162
2021-11-12 09:29:04 +01:00
Bjørn Erik Pedersen
096f5e1921 Fix the "page picker" logic in --navigateToChanged
Fixes #9051
2021-10-18 12:13:13 +02:00
Bjørn Erik Pedersen
ba35e69856 Add a cross process build lock and use it in the archetype content builder
Fixes #9048
2021-10-18 12:13:13 +02:00
Bjørn Erik Pedersen
9185e11eff Reimplement archetypes
The old implementation had some issues, mostly related to the context (e.g. name, file paths) passed to the template.

This new implementation is using the exact same code path for evaluating the pages as in a regular build.

This also makes it more robust and easier to reason about in a multilingual setup.

Now, if you are explicit about the target path, Hugo will now always pick the correct mount and language:

```bash
hugo new content/en/posts/my-first-post.md
```

Fixes #9032
Fixes #7589
Fixes #9043
Fixes #9046
Fixes #9047
2021-10-16 15:22:03 +02:00
Bjørn Erik Pedersen
13ad8408fc
commands: Make the error handling for the mod commands more lenient
So it at least is possible to do a `hugo mod clean --all` if the cache is in a funky state.
2021-09-17 17:16:34 +02:00
Bjørn Erik Pedersen
7d1f806ecb commands: Don't fail on template errors on go mod graph etc.
Fixes #8942
2021-08-31 22:16:12 +02:00
Bjørn Erik Pedersen
a0489c2dfd
Avoid failing with "module not found" for hugo mod init and similar
Fixes #8940
2021-08-31 12:08:11 +02:00
Bjørn Erik Pedersen
ffa2fe6117
Revert "commands: Avoid too many watch file handles causing the server to fail to start"
This reverts commit 3f38c785b7.
2021-08-21 16:32:20 +02:00
Helder Pereira
d966f5d08d highlight: Remove some pygments references 2021-08-21 15:50:49 +02:00
Shiming Zhang
3f38c785b7
commands: Avoid too many watch file handles causing the server to fail to start
Fixes #8904
2021-08-19 11:04:09 +02:00
Bjørn Erik Pedersen
58b6742cfe
Import time/tzdata on Windows
Updates #8892
2021-08-14 12:09:41 +02:00
Bjørn Erik Pedersen
a3701e0931 Switch to go-toml v2
We have been using `go-toml` for language files only. This commit makes it the only TOML library.

It's spec compliant and very fast.

A benchark building a site with 200 pages with TOML front matter:

```bash
name                                  old time/op    new time/op    delta
SiteNew/Regular_TOML_front_matter-16    48.5ms ± 1%    47.1ms ± 1%  -2.85%  (p=0.029 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_TOML_front_matter-16    16.9MB ± 0%    16.7MB ± 0%  -1.56%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_TOML_front_matter-16      302k ± 0%      296k ± 0%  -2.20%  (p=0.029 n=4+4)
```

Note that the front matter unmarshaling is only a small part of building a site, so the above is very good.

Fixes #8801
2021-07-28 11:51:13 +02:00
Bjørn Erik Pedersen
351ed0f569
commands: Fix panic on invalid config in "hugo mod get" and similar
Fixes #8773
2021-07-20 18:23:09 +02:00
Bjørn Erik Pedersen
022c479551
hugofs: Make FileMeta a struct
This commit started out investigating a `concurrent map read write` issue, ending by replacing the map with a struct.

This is easier to reason about, and it's more effective:

```
name                                  old time/op    new time/op    delta
SiteNew/Regular_Deep_content_tree-16    71.5ms ± 3%    69.4ms ± 5%    ~     (p=0.200 n=4+4)

name                                  old alloc/op   new alloc/op   delta
SiteNew/Regular_Deep_content_tree-16    29.7MB ± 0%    27.9MB ± 0%  -5.82%  (p=0.029 n=4+4)

name                                  old allocs/op  new allocs/op  delta
SiteNew/Regular_Deep_content_tree-16      313k ± 0%      303k ± 0%  -3.35%  (p=0.029 n=4+4)
```

See #8749
2021-07-15 17:14:26 +02:00
John Hollowell
04dc469fbd
commands: Move time notification to after any build errors
This allows error parsers (VSCode problemMatchers) to use the time notification as bounds for detecting errors.

Closes #8403
2021-07-05 10:38:54 +02:00
Bjørn Erik Pedersen
e31b1d1946
commands: Make the --poll flag a duration
So you can do:

```
hugo server --poll 700ms
```

See #8720
2021-07-05 10:23:29 +02:00
Bjørn Erik Pedersen
24ce98b6d1
Add polling as a fallback to native filesystem events in server watch
Fixes #8720
Fixes #6849
Fixes #7930
2021-07-04 16:12:28 +02:00
Bjørn Erik Pedersen
3a6dc6d3f4
modules: Use value type for module.Time
Which is in line with how we do it elsewhere.
2021-06-27 18:06:52 +02:00
Bjørn Erik Pedersen
6cd2110ab2
commands: Add version time to "hugo config mounts" 2021-06-27 18:00:20 +02:00
Bjørn Erik Pedersen
6a365c2712
commands: Add some more info to "hugo config mounts"
* Add owner path and version.
* Also add thme meta info and Hugo version when run with -v flag
2021-06-27 16:08:28 +02:00
Bjørn Erik Pedersen
93aad3c543
Split out the puthe path/filepath functions into common/paths
So they can be used from the config package without cyclic troubles.

Updates #8654
2021-06-18 10:55:00 +02:00
Bjørn Erik Pedersen
31fb29fb3f
Do not read config from os.Environ when running tests
Fixes #8655
2021-06-16 18:58:40 +02:00
Bjørn Erik Pedersen
d392893cd7
Misc config loading fixes
The main motivation behind this is simplicity and correctnes, but the new small config library is also faster:

```
BenchmarkDefaultConfigProvider/Viper-16         	  252418	      4546 ns/op	    2720 B/op	      30 allocs/op
BenchmarkDefaultConfigProvider/Custom-16        	  450756	      2651 ns/op	    1008 B/op	       6 allocs/op
```

Fixes #8633
Fixes #8618
Fixes #8630
Updates #8591
Closes #6680
Closes #5192
2021-06-14 17:00:32 +02:00
Bjørn Erik Pedersen
f55d2f4376
tpl/fmt: Add erroridf template func
Fixes #8613
2021-06-07 19:11:03 +02:00
Joe Mooring
76c95f55a5
Display version when building site (#8533)
Closes #8531
2021-05-14 16:45:13 +02:00
Bjørn Erik Pedersen
4227cc1bd3
commands: Remove all dates from gendoc
It creates lots of diffs even if there are no changes.
2021-05-01 12:03:56 +02:00
Bjørn Erik Pedersen
9b83f45b6d
Add complete dependency list in "hugo env -v"
Fixes #8400
2021-04-09 08:07:22 +02:00
Avinash Sonawane
24c716cac3 Fix new theme command description
`hugo new theme foo` creates theme `foo` in `./themes` and not in current directory.
2021-03-14 18:47:23 +01:00
Bjørn Erik Pedersen
c8f45d1d86
commands: Fix autocomplete docs 2021-02-20 14:08:51 +01:00
Anthony Fok
5f621df257
commands: Add PowerShell completion support
Revert "Refactor: Remove powershell support" with fixes

Thanks to Ben Mezger (@benmezger) for the original code.
See #8122

This reverts commit a7c515e1b5.
2021-02-09 13:37:36 -07:00
Ben Mezger
d36fd5b3ee Refactor: Write to stdout by default 2021-02-04 15:32:51 -07:00
Ben Mezger
a7c515e1b5 Refactor: Remove powershell support 2021-02-04 15:32:51 -07:00
Ben Mezger
216b00f358 Feat: Add zsh, fish and powershell completion support
See issue #4296
2021-02-04 15:32:51 -07:00
Bjørn Erik Pedersen
cea1574023
Add Dart Sass support
But note that the Dart Sass Embedded Protocol is still in beta (beta 5), a main release scheduled for Q1 2021.

Fixes #7380
Fixes #8102
2020-12-30 17:32:25 +01:00
Anthony Fok
c84ad8db82
deps: Bump github.com/spf13/cobra from 0.15.0 to 0.20.0
The substring match in TestExecute deploy test is adjusted accordingly.
2020-12-04 20:43:03 -07:00
Bjørn Erik Pedersen
d90e37e0c6 all: Format code with gofumpt
See https://github.com/mvdan/gofumpt
2020-12-03 13:12:58 +01:00
sth
0ad378b09c
Use --baseURL path for live-reload URL
Fixes #6595
2020-12-02 12:52:26 +01:00
Bjørn Erik Pedersen
78f227b664
js: Let ESBuild handle all imports from node_modules
This commit fixes some issues where modules in /assets share the same name as in node_modules.

This was not intended, and with this commit the node_modules-components should be isolated. If you want to redefine something inside node_modules, use the `defines` option.

Fixes #7948
2020-11-13 08:54:29 +01:00
Bjørn Erik Pedersen
85e4dd7370 Make js.Build fully support modules
Fixes #7816
Fixes #7777
Fixes #7916
2020-11-03 13:04:37 +01:00
Eric Hagman
f465c5c307
build: Allow optional "nodeploy" tag to exclude deploy command from bin
Fixes #7826
2020-10-23 09:03:41 +02:00
Bjørn Erik Pedersen
fdfa4a5fe6 Allow getJSON errors to be ignored
This change is mostly motivated to get a more stable CI build (we're building the Hugo site there, with Instagram and Twitter shortcodes sometimes failing).

Fixes #7866
2020-10-22 09:09:29 +02:00
Bjørn Erik Pedersen
5e2a547cb5 Add force flag to server redirects config
Fixes #7778
2020-10-05 22:20:00 +02:00
Daniel Holbach
292b0e26ec typo: already -> already 2020-09-14 10:15:59 +02:00
Bjørn Erik Pedersen
85ba9bfffb Add "hugo mod npm pack"
This commit also introduces a convention where these common JS config files, including `package.hugo.json`, gets mounted into:

```
assets/_jsconfig
´``

These files mapped to their real filename will be added to the environment when running PostCSS, Babel etc., so you can do `process.env.HUGO_FILE_TAILWIND_CONFIG_JS` to resolve the real filename.

But do note that `assets` is a composite/union filesystem, so if your config file is not meant to be overridden, name them something specific.

This commit also adds adds `workDir/node_modules` to `NODE_PATH` and `HUGO_WORKDIR` to the env when running the JS tools above.

Fixes #7644
Fixes #7656
Fixes #7675
2020-09-13 20:55:29 +02:00
Bjørn Erik Pedersen
9a1e6d15a3 modules: Make ignoreVendor a glob pattern
Fixes #7642
2020-09-10 08:47:05 +02:00
Jeff Warner
d39636a5fc
commands: Remove logic that hides 'Building Sites' message after build completes
Append newline to the message instead.

Fixes #7579
2020-08-22 09:56:43 +02:00
Robert van Gent
a1c3e3c1f3 deploy: Ensure that non-trivial default flag values are passed through. 2020-07-04 21:19:25 +02:00
Bjørn Erik Pedersen
48dbb593f7
commands: Add an option to print memory usage at intervals
Use it with `hugo --print-mem
2020-06-25 12:19:21 +02:00
Bjørn Erik Pedersen
fc0f13b684 commands: Fix URL rewrites vs fast render server mode
Fixes #7357
2020-06-06 12:37:42 +02:00
Bjørn Erik Pedersen
6a3e89743c Add redirect support to the server
Fixes #7323
2020-05-28 16:25:34 +02:00
Bjørn Erik Pedersen
518d149646
commands: Use WARN log level also for the early initialization
Fixes #7285
2020-05-18 16:52:45 +02:00
Steve
102ec2da7a
commands: Modify gen chromastyles to output all CSS classes
Chroma was updated to include a new function to output all CSS
classes without skipping any considered redundant with the current
style.

This will modify the `hugo gen chromastyles` command to use this new
function by default, which avoids potential problems if the style is
later modified.

Also includes requires the updated Chroma (v0.7.2).

Resolves #7167
2020-04-13 18:31:33 +02:00
Bjørn Erik Pedersen
ee67dbeff5
Fix query parameter handling in server fast render mode
There may be a simpler way to get to the raw path, but this will have to do for now.

Fixes #7163
2020-04-12 19:18:54 +02:00
Jay Williams
efc61d6f3b
commands: Use semver for min_version per recommendations
See https://github.com/gohugoio/hugoThemes#themetoml
2020-04-07 19:42:45 +02:00
Bjørn Erik Pedersen
7204b354a9 Some minify configuration adjustments 2020-03-20 20:35:57 +01:00
Bjørn Erik Pedersen
df298558a5
Improve Tailwind/PostCSS error messages
Fixes #7041
Fixes #7042
2020-03-11 14:13:03 +01:00
Bjørn Erik Pedersen
108314444b Add HTTP header support for the dev server
Fixes #7031
2020-03-08 19:57:30 +01:00
Bjørn Erik Pedersen
760a87a45a commands: Add --all flag to hugo mod clean 2020-03-03 13:29:58 +01:00
Bjørn Erik Pedersen
dce210ab56
modules: Improve "hugo mod clean"
* Only clean project modules
* Optional glob pattern of module paths to clean

Closes #6907
2020-02-19 17:14:35 +01:00
Bjørn Erik Pedersen
0b96aba022
commands: Add "hugo mod verify"
See #6907
2020-02-19 17:14:35 +01:00
Bjørn Erik Pedersen
775c7c2474
commands: Support "hugo mod get -u ./..."
Fixes #6828
2020-02-18 12:17:16 +01:00
Bjørn Erik Pedersen
eada236f87
Introduce a tree map for all content
This commit introduces a new data structure to store pages and their resources.

This data structure is backed by radix trees.

This simplies tree operations, makes all pages a bundle,  and paves the way for #6310.

It also solves a set of annoying issues (see list below).

Not a motivation behind this, but this commit also makes Hugo in general a little bit faster and more memory effective (see benchmarks). Especially for partial rebuilds on content edits, but also when taxonomies is in use.

```
name                                   old time/op    new time/op    delta
SiteNew/Bundle_with_image/Edit-16        1.32ms ± 8%    1.00ms ± 9%  -24.42%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16    1.28ms ± 0%    0.94ms ± 0%  -26.26%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16      33.9ms ± 2%    21.8ms ± 1%  -35.67%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16            40.6ms ± 1%    37.7ms ± 3%   -7.20%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16        56.7ms ± 0%    51.7ms ± 1%   -8.82%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16      19.9ms ± 2%    18.3ms ± 3%   -7.64%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16         37.9ms ± 4%    34.0ms ± 2%  -10.28%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16             10.7ms ± 0%    10.6ms ± 0%   -1.15%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16         10.8ms ± 0%    10.7ms ± 0%   -1.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16           43.2ms ± 1%    39.6ms ± 1%   -8.35%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                 47.6ms ± 1%    47.3ms ± 0%     ~     (p=0.057 n=4+4)
SiteNew/Deep_content_tree-16             73.0ms ± 1%    74.2ms ± 1%     ~     (p=0.114 n=4+4)
SiteNew/Many_HTML_templates-16           37.9ms ± 0%    38.1ms ± 1%     ~     (p=0.114 n=4+4)
SiteNew/Page_collections-16              53.6ms ± 1%    54.7ms ± 1%   +2.09%  (p=0.029 n=4+4)

name                                   old alloc/op   new alloc/op   delta
SiteNew/Bundle_with_image/Edit-16         486kB ± 0%     430kB ± 0%  -11.47%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16     265kB ± 0%     209kB ± 0%  -21.06%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16      13.6MB ± 0%     8.8MB ± 0%  -34.93%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16            66.5MB ± 0%    63.9MB ± 0%   -3.95%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16        28.8MB ± 0%    25.8MB ± 0%  -10.55%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16      6.16MB ± 0%    5.56MB ± 0%   -9.86%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16         16.9MB ± 0%    16.0MB ± 0%   -5.19%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16             2.28MB ± 0%    2.29MB ± 0%   +0.35%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16         2.07MB ± 0%    2.07MB ± 0%     ~     (p=0.114 n=4+4)
SiteNew/Tags_and_categories-16           14.3MB ± 0%    13.2MB ± 0%   -7.30%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                 69.1MB ± 0%    69.0MB ± 0%     ~     (p=0.343 n=4+4)
SiteNew/Deep_content_tree-16             31.3MB ± 0%    31.8MB ± 0%   +1.49%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16           10.8MB ± 0%    10.9MB ± 0%   +1.11%  (p=0.029 n=4+4)
SiteNew/Page_collections-16              21.4MB ± 0%    21.6MB ± 0%   +1.15%  (p=0.029 n=4+4)

name                                   old allocs/op  new allocs/op  delta
SiteNew/Bundle_with_image/Edit-16         4.74k ± 0%     3.86k ± 0%  -18.57%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file/Edit-16     4.73k ± 0%     3.85k ± 0%  -18.58%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories/Edit-16        301k ± 0%      198k ± 0%  -34.14%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs/Edit-16              389k ± 0%      373k ± 0%   -4.07%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree/Edit-16          338k ± 0%      262k ± 0%  -22.63%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates/Edit-16        102k ± 0%       88k ± 0%  -13.81%  (p=0.029 n=4+4)
SiteNew/Page_collections/Edit-16           176k ± 0%      152k ± 0%  -13.32%  (p=0.029 n=4+4)
SiteNew/Bundle_with_image-16              26.8k ± 0%     26.8k ± 0%   +0.05%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16          26.8k ± 0%     26.8k ± 0%   +0.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16             273k ± 0%      245k ± 0%  -10.36%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16                   396k ± 0%      398k ± 0%   +0.39%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16               317k ± 0%      325k ± 0%   +2.53%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16             146k ± 0%      147k ± 0%   +0.98%  (p=0.029 n=4+4)
SiteNew/Page_collections-16                210k ± 0%      215k ± 0%   +2.44%  (p=0.029 n=4+4)
```

Fixes #6312
Fixes #6087
Fixes #6738
Fixes #6412
Fixes #6743
Fixes #6875
Fixes #6034
Fixes #6902
Fixes #6173
Fixes #6590
2020-02-18 09:49:42 +01:00
Mark Rosemaker
8a5124d6b3
commands: Rename doWithCommandeer to cfgInit/cfgSetAndInit
This will make it clearer what it does and make the code more consistent.
2020-02-13 00:37:49 +01:00
Bjørn Erik Pedersen
2bbc865f7b commands: Fix config environment handling
Fixes #6503
Fixes #6824
2020-01-31 16:36:36 +01:00
Bjørn Erik Pedersen
49ef647203 modules: Fix "hugo mod get -u" with no arguments
Fixes #6826
Closes #6825
2020-01-30 17:15:35 +01:00
Bjørn Erik Pedersen
c6d650c8c8
tpl/tplimpl: Rework template management to get rid of concurrency issues
This more or less completes the simplification of the template handling code in Hugo started in v0.62.

The main motivation was to fix a long lasting issue about a crash in HTML content files  without front matter.

But this commit also comes with a big functional improvement.

As we now have moved the base template evaluation to the build stage we now use the same lookup rules for `baseof` as for `list` etc. type of templates.

This means that in this simple example you can have a `baseof` template for the `blog` section without having to duplicate the others:

```
layouts
├── _default
│   ├── baseof.html
│   ├── list.html
│   └── single.html
└── blog
    └── baseof.html
```

Also, when simplifying code, you often get rid of some double work, as shown in the "site building" benchmarks below.

These benchmarks looks suspiciously good, but I have repeated the below with ca. the same result. Compared to master:

```
name                              old time/op    new time/op    delta
SiteNew/Bundle_with_image-16        13.1ms ± 1%    10.5ms ± 1%  -19.34%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16    13.0ms ± 0%    10.7ms ± 1%  -18.05%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16      46.4ms ± 2%    43.1ms ± 1%   -7.15%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16            52.2ms ± 2%    47.8ms ± 1%   -8.30%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16        77.9ms ± 1%    70.9ms ± 1%   -9.01%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16      43.0ms ± 0%    37.2ms ± 1%  -13.54%  (p=0.029 n=4+4)
SiteNew/Page_collections-16         58.2ms ± 1%    52.4ms ± 1%   -9.95%  (p=0.029 n=4+4)

name                              old alloc/op   new alloc/op   delta
SiteNew/Bundle_with_image-16        3.81MB ± 0%    2.22MB ± 0%  -41.70%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16    3.60MB ± 0%    2.01MB ± 0%  -44.20%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16      19.3MB ± 1%    14.1MB ± 0%  -26.91%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16            70.7MB ± 0%    69.0MB ± 0%   -2.40%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16        37.1MB ± 0%    31.2MB ± 0%  -15.94%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16      17.6MB ± 0%    10.6MB ± 0%  -39.92%  (p=0.029 n=4+4)
SiteNew/Page_collections-16         25.9MB ± 0%    21.2MB ± 0%  -17.99%  (p=0.029 n=4+4)

name                              old allocs/op  new allocs/op  delta
SiteNew/Bundle_with_image-16         52.3k ± 0%     26.1k ± 0%  -50.18%  (p=0.029 n=4+4)
SiteNew/Bundle_with_JSON_file-16     52.3k ± 0%     26.1k ± 0%  -50.16%  (p=0.029 n=4+4)
SiteNew/Tags_and_categories-16        336k ± 1%      269k ± 0%  -19.90%  (p=0.029 n=4+4)
SiteNew/Canonify_URLs-16              422k ± 0%      395k ± 0%   -6.43%  (p=0.029 n=4+4)
SiteNew/Deep_content_tree-16          401k ± 0%      313k ± 0%  -21.79%  (p=0.029 n=4+4)
SiteNew/Many_HTML_templates-16        247k ± 0%      143k ± 0%  -42.17%  (p=0.029 n=4+4)
SiteNew/Page_collections-16           282k ± 0%      207k ± 0%  -26.55%  (p=0.029 n=4+4)
```

Fixes #6716
Fixes #6760
Fixes #6768
Fixes #6778
2020-01-22 09:39:49 +01:00
Bjørn Erik Pedersen
ff6253bc7c Support files in content mounts
This commit is a general improvement of handling if single file mounts.

Fixes #6684
Fixes #6696
2020-01-01 18:19:49 +01:00
Bjørn Erik Pedersen
8a58ebb311 hugolib: Improve error and reload handling of hook templates in server mode
Fixes #6635
2019-12-20 11:38:44 +01:00
Bjørn Erik Pedersen
a03c631c42
Rework template handling for function and map lookups
This is a big commit, but it deletes lots of code and simplifies a lot.

* Resolving the template funcs at execution time means we don't have to create template clones per site
* Having a custom map resolver means that we can remove the AST lower case transformation for the special lower case Params map

Not only is the above easier to reason about, it's also faster, especially if you have more than one language, as in the benchmark below:

```
name                          old time/op    new time/op    delta
SiteNew/Deep_content_tree-16    53.7ms ± 0%    48.1ms ± 2%  -10.38%  (p=0.029 n=4+4)

name                          old alloc/op   new alloc/op   delta
SiteNew/Deep_content_tree-16    41.0MB ± 0%    36.8MB ± 0%  -10.26%  (p=0.029 n=4+4)

name                          old allocs/op  new allocs/op  delta
SiteNew/Deep_content_tree-16      481k ± 0%      410k ± 0%  -14.66%  (p=0.029 n=4+4)
```

This should be even better if you also have lots of templates.

Closes #6594
2019-12-12 10:04:35 +01:00
Bjørn Erik Pedersen
5c5231e09e
commands: Use HUGO_ENV if set
Fixes #6456
2019-11-25 19:12:14 +01:00
Chris Trimble
8a89b8582f commands: Fix jekyll metadata import on individual posts
Prior refactor had eliminated writing FrontMatter for Jekyll imports. This
fixes that bug as well as adds a regression test.

Also removed unused site var, replaced some raw strings, and added regression
test for \r\n removal.

Fixes #5576
2019-11-25 09:46:44 +01:00
Bjørn Erik Pedersen
bfb9613a14
Add Goldmark as the new default markdown handler
This commit adds the fast and CommonMark compliant Goldmark as the new default markdown handler in Hugo.

If you want to continue using BlackFriday as the default for md/markdown extensions, you can use this configuration:

```toml
[markup]
defaultMarkdownHandler="blackfriday"
```

Fixes #5963
Fixes #1778
Fixes #6355
2019-11-23 14:12:24 +01:00
Bjørn Erik Pedersen
a3fe5e5e35
Fix Params case handling in the index, sort and where func
This means that you can now do:

```
{{ range where .Site.Pages "Params.MYPARAM" "foo" }}
```
2019-11-22 18:41:50 +01:00
Bjørn Erik Pedersen
2dcc1318d1 Add some more output if loading modules takes time
Also include the time to collect modules etc. in the "Total in ..." time reported for the `hugo` command.

Fixes #6519
2019-11-21 18:11:07 +01:00
reuben honigwachs
1a36ce9b09 commands: Add hint when dir not empty
Many users seem to stumble over creating new sites in existing non-empty directories. And that `--force` is the necessary option. So I added *See --force* as a hint to the error.

Fixes #4825.
2019-11-11 21:38:10 +01:00
Nick Krichevsky
e6aa6edb4c Do not attempt to build if there is no config file
Fixes #5896
2019-10-22 00:10:36 +02:00
Garret Kelly
674e81ae87 deploy: Add ability to invalidate Google Cloud CDN 2019-10-03 16:53:45 +02:00
HyeonGyu Lee
ea9261e856 commands: Make sure the hugo field is always initialized before it's used
Wrap the field to make it accessible after initialization.

Fixes #6193
2019-08-15 09:33:47 +02:00
Bjørn Erik Pedersen
17ca8f0c4c
commands: Fix faulty -h logic in hugo mod get
Fixes #6197
2019-08-13 11:17:19 +02:00
Bjørn Erik Pedersen
9e57182705
tests: Convert from testify to quicktest 2019-08-12 13:26:32 +02:00
Bjørn Erik Pedersen
166a394a2f Fix static sync issue with virtual mounts
Fixes #6165
2019-08-09 10:18:28 +02:00
Bjørn Erik Pedersen
d7c233afee commands: Add "hugo config mounts" command
This prints the effective file mounts in a project.

Fixes #6144
2019-07-31 12:10:05 +02:00
Bjørn Erik Pedersen
45ee8a7a52 commands: Cleanup the hugo config command
Most importantly filter out some keys not relevant for the end user.

See #6144
2019-07-31 12:10:05 +02:00
Bjørn Erik Pedersen
6eca0a3dee
commands: Fix config reloading in Vim and similar
The config path was simplified in Hugo 0.56.0 to support more config dirs/files (go.mod etc.),
and the new code path assumed that every file change would trigger a `Write` event.

This is not true for Vim etc. which triggers a `Chmod` and then a `Rename`.

Lesson learned: Be really careful changing Os/editor specific code without proper tests.

Fixes #6139
2019-07-30 09:35:01 +02:00
Bjørn Erik Pedersen
9f5a92078a
Add Hugo Modules
This commit implements Hugo Modules.

This is a broad subject, but some keywords include:

* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`,  `hugo mod get`,  `hugo mod graph`,  `hugo mod tidy`, and  `hugo mod vendor`.

All of the above is backed by Go Modules.

Fixes #5973
Fixes #5996
Fixes #6010
Fixes #5911
Fixes #5940
Fixes #6074
Fixes #6082
Fixes #6092
2019-07-24 09:35:53 +02:00
Joshua Arnott
de7b9475c0 commands: Remove comma after URL in new site output
Some terminals include the comma in the URL when hyperlinking, resulting in a 404
2019-06-19 23:03:30 +02:00
Robert van Gent
9df57154ee deploy: Default --target to the first deployment target 2019-06-08 00:41:59 +02:00
Robert van Gent
8914fe7ed7 Update CLI doc for "long" form 2019-05-31 22:08:12 +02:00
Robert van Gent
b0f536fb27 Drop dashes in http header matcher attributes; other changes from code review 2019-05-31 22:08:12 +02:00
Ruslan Nasonov
5b4b8bb3c1 commands: Create new 'hugo list all' command
New:
- command `hugo list all`, return all posts meta in csv format

Refactoring:
- move common parts in commands/list.go to function `buildSites`
- change way to detect path to content

See #5904
2019-05-25 00:09:51 +02:00
Robert van Gent
f4956d9aae deploy: Support invalidating a CloudFront CDN cache 2019-05-06 21:09:33 +02:00
Robert van Gent
c7165589b3 Add a "deploy" command 2019-05-03 17:58:40 +02:00