This commmit prepares for the addition of Goldmark as the new Markdown renderer in Hugo.
This introduces a new `markup` package with some common interfaces and each implementation in its own package.
See #5963
This commit implements Hugo Modules.
This is a broad subject, but some keywords include:
* A new `module` configuration section where you can import almost anything. You can configure both your own file mounts nd the file mounts of the modules you import. This is the new recommended way of configuring what you earlier put in `configDir`, `staticDir` etc. And it also allows you to mount folders in non-Hugo-projects, e.g. the `SCSS` folder in the Bootstrap GitHub project.
* A module consists of a set of mounts to the standard 7 component types in Hugo: `static`, `content`, `layouts`, `data`, `assets`, `i18n`, and `archetypes`. Yes, Theme Components can now include content, which should be very useful, especially in bigger multilingual projects.
* Modules not in your local file cache will be downloaded automatically and even "hot replaced" while the server is running.
* Hugo Modules supports and encourages semver versioned modules, and uses the minimal version selection algorithm to resolve versions.
* A new set of CLI commands are provided to manage all of this: `hugo mod init`, `hugo mod get`, `hugo mod graph`, `hugo mod tidy`, and `hugo mod vendor`.
All of the above is backed by Go Modules.
Fixes#5973Fixes#5996Fixes#6010Fixes#5911Fixes#5940Fixes#6074Fixes#6082Fixes#6092
Add the ability to have a `summary` page variable that overrides
the auto-generated summary. Logic for obtaining summary becomes:
* if summary divider is present in content, use the text above it
* if summary variables is present in page metadata, use that
* auto-generate summary from first _x_ words of the content
Fixes#5800
The main motivation of this commit is to add a `page.Page` interface to replace the very file-oriented `hugolib.Page` struct.
This is all a preparation step for issue #5074, "pages from other data sources".
But this also fixes a set of annoying limitations, especially related to custom output formats, and shortcodes.
Most notable changes:
* The inner content of shortcodes using the `{{%` as the outer-most delimiter will now be sent to the content renderer, e.g. Blackfriday.
This means that any markdown will partake in the global ToC and footnote context etc.
* The Custom Output formats are now "fully virtualized". This removes many of the current limitations.
* The taxonomy list type now has a reference to the `Page` object.
This improves the taxonomy template `.Title` situation and make common template constructs much simpler.
See #5074Fixes#5763Fixes#5758Fixes#5090Fixes#5204Fixes#4695Fixes#5607Fixes#5707Fixes#5719Fixes#3113Fixes#5706Fixes#5767Fixes#5723Fixes#5769Fixes#5770Fixes#5771Fixes#5759Fixes#5776Fixes#5777Fixes#5778
Add a configuration option "noreferrerLinks". When set to "true" the "HTML_NOREFERRER_LINKS" flag is being passed to Blackfriday. Thereby all *absolute* links will get a "noreferrer" value for their "rel" attribute.
See #4722
Add a configuration option "nofollowLinks". When set to "true" the "HTML_NOFOLLOW_LINKS" flag is being passed to Blackfriday. Thereby all *absolute* links will get a "nofollow" value for the "rel" attribute.
Fixes#4722
This commit is not the smallest in Hugo's history.
Some hightlights include:
* Page bundles (for complete articles, keeping images and content together etc.).
* Bundled images can be processed in as many versions/sizes as you need with the three methods `Resize`, `Fill` and `Fit`.
* Processed images are cached inside `resources/_gen/images` (default) in your project.
* Symbolic links (both files and dirs) are now allowed anywhere inside /content
* A new table based build summary
* The "Total in nn ms" now reports the total including the handling of the files inside /static. So if it now reports more than you're used to, it is just **more real** and probably faster than before (see below).
A site building benchmark run compared to `v0.31.1` shows that this should be slightly faster and use less memory:
```bash
▶ ./benchSite.sh "TOML,num_langs=.*,num_root_sections=5,num_pages=(500|1000),tags_per_page=5,shortcodes,render"
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 101785785 78067944 -23.30%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 185481057 149159919 -19.58%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 103149918 85679409 -16.94%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 203515478 169208775 -16.86%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 532464 391539 -26.47%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 1056549 772702 -26.87%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 555974 406630 -26.86%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 1086545 789922 -27.30%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 53243246 43598155 -18.12%
BenchmarkSiteBuilding/TOML,num_langs=1,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 105811617 86087116 -18.64%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=500,tags_per_page=5,shortcodes,render-4 54558852 44545097 -18.35%
BenchmarkSiteBuilding/TOML,num_langs=3,num_root_sections=5,num_pages=1000,tags_per_page=5,shortcodes,render-4 106903858 86978413 -18.64%
```
Fixes#3651Closes#3158Fixes#1014Closes#2021Fixes#1240
Updates #3757
This is in heavy use in rendering, so this makes a difference:
```bash
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 124551144 107743429 -13.49%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 528684 435118 -17.70%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=1,num_pages=500,tags_per_page=5,shortcodes,render-4 53306848 45147832 -15.31%
```
Move SummaryLength into the ContentSpec struct and refactor the
relevant summary functions to be methods of ContentSpec. The new
summaryLength struct member is configurable by the summaryLength config
value, and the default remains 70. Also updates hugolib/page to use the
refactored methods.
Resolves#3734
This avoids having to execute these expensive operations for sites not using these values.
This commit sums up a set of wordcounting and autosummary related performance improvements.
The effect of these kind of depends on what features your site use, but a benchmark from 4 Hugo sites in the wild shows promise:
```
benchmark old ns/op new ns/op delta
BenchmarkHugo-4 21293005843 20032857342 -5.92%
benchmark old allocs new allocs delta
BenchmarkHugo-4 65290922 65186032 -0.16%
benchmark old bytes new bytes delta
BenchmarkHugo-4 9771213416 9681866464 -0.91%
```
Closes#2378
It is obviously more efficient when we do not care about the actual words.
```
BenchmarkTotalWords-4 100000 18795 ns/op 0 B/op 0 allocs/op
BenchmarkTotalWordsOld-4 30000 46751 ns/op 6400 B/op 1 allocs/op
```
For people using autogenerated summaries, this is one of the hot spots in the memory department.
We don't need to split al the content into words to do proper summary truncation.
This is obviously more effective:
```
BenchmarkTestTruncateWordsToWholeSentence-4 300000 4720 ns/op 0 B/op 0 allocs/op
BenchmarkTestTruncateWordsToWholeSentenceOld-4 100000 17699 ns/op 3072 B/op 3 allocs/op
```
Started to increase coverage in helpers package, now at 74.9% of statements.
In the process, also a few minor changes have been applied to content.go.
* Content.go has undergone a formatting refactor regarding comments
* Unused function TruncateWords has been removed
* RenderingContext's "mmark" has been changed to use MmarkRender
* Content_test.go added to cover content.go's functionality
* add global `hasCJKLanguage` flag, if true, turn on auto-detecting CJKLanguage
* add `isCJKLanguage` frontmatter to force specify whether is CJKLanguage or not
* For .Summary: If isCJKLanguage is true, use the runes as basis for truncation, else keep as today.
* For WordCount: If isCJKLanguage is true, use the runes as basis for calculation, else keep as today.
* Unexport RuneCount
Fixes#1377
First step to use initialisms that golint suggests,
for example:
Line 116: func GetHtmlRenderer should be GetHTMLRenderer
as see on http://goreportcard.com/report/spf13/hugo
Thanks to @bep for the idea!
Note that command-line flags (cobra and pflag)
as well as struct fields like .BaseUrl and .Url
that are used in Go HTML templates need more work
to maintain backward-compatibility, and thus
are NOT yet dealt with in this commit.
First step in fixing #959.
go test -test.run=NONE -bench=".*" -test.benchmem=true ./helpers
Old vs new impl (string.Replace vs string.Replacer):
benchmark old ns/op new ns/op delta
BenchmarkStripHTML 10210 6572 -35.63%
benchmark old allocs new allocs delta
BenchmarkStripHTML 6 5 -16.67%
benchmark old bytes new bytes delta
BenchmarkStripHTML 1456 848 -41.76%
