Meny people, including me, have a custom robots.txt in static.
Also remove that option from the command line; it doesn't feel
important enough.
Fixes ##2049
This also includes a refactor of the hugofs package and its usage.
The motivation for that is:
The Afero filesystems are brilliant. Hugo's way of adding a dozen of global variables for the different filesystems was a mistake. In readFile (and also in some other places in Hugo today) we need a way to restrict the access inside the working dir. We could use ioutil.ReadFile and implement the path checking, checking the base path and the dots ("..") etc. But it is obviously better to use an Afero BasePathFs combined witha ReadOnlyFs. We could create a use-once-filesystem and handle the initialization ourselves, but since this is also useful to others and the initialization depends on some other global state (which would mean to create a new file system on every invocation), we might as well do it properly and encapsulate the predefined set of filesystems. This change also leads the way, if needed, to encapsulate the file systems in a struct, making it possible to have several file system sets in action at once (parallel multilanguage site building? With Moore's law and all...)
Fixes#1551
It is a fairly costly operation:
```
benchmark old ns/op new ns/op delta
BenchmarkScratchGet-4 109 31.6 -71.01%
benchmark old allocs new allocs delta
BenchmarkScratchGet-4 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkScratchGet-4 0 0 +0.00%
´´´
This uses the Emoji map from https://github.com/kyokomi/emoji -- but with a custom replacement implementation.
The built-in are fine for most use cases, but in Hugo we do care about pure speed.
The benchmarks below are skewed in Hugo's direction as the source and result is a byte slice,
Kyokomi's implementation works best with strings.
Curious: The easy-to-use `strings.Replacer` is also plenty fast.
```
BenchmarkEmojiKyokomiFprint-4 20000 86038 ns/op 33960 B/op 117 allocs/op
BenchmarkEmojiKyokomiSprint-4 20000 83252 ns/op 38232 B/op 122 allocs/op
BenchmarkEmojiStringsReplacer-4 100000 21092 ns/op 17248 B/op 25 allocs/op
BenchmarkHugoEmoji-4 500000 5728 ns/op 624 B/op 13 allocs/op
```
Fixes#1891
We can of course skip reading the entire byte slice again and again.
This was a slip in the original implementation; functionally the same,
but is slightly faster, esp. for larger data sets with many shortcodes:
```
benchmark old ns/op new ns/op delta
BenchmarkReplaceShortcodeTokens-4 15505 14753 -4.85%
benchmark old allocs new allocs delta
BenchmarkReplaceShortcodeTokens-4 1 1 +0.00%
benchmark old bytes new bytes delta
BenchmarkReplaceShortcodeTokens-4 3072 3072 +0.00%
```
Site.ReadPagesFromSource returns nil chan error value when a site
content directory is empty but its receiver expects to be passed
something error values via the channel.
This fixes it by returning a channel which will be immediately closed.
Fix#1797
Cobra, the CLI commander in use in Hugo, has some long awaited improvements in the error handling department.
This enables a more centralized error handling approach.
This commit introduces that by changing all the command funcs to `RunE`:
* The core part of the error logging, usage logging and `os.Exit(-1)` is now performed in one place and that one place only.
* The usage text is now only shown on invalid arguments etc. (user errors)
Fixes#1502
It would be helpful to know whether a shortcode was called with positional or
named parameters. This commit adds a boolean `IsNamedParams` property to the
`ShortcodeWithPage` struct.
Currently a `[]byte` copy is returned. In most cases this is the safe thing to do, but we should just modify/grow the slice as needed.
This is faster and consumes less memory:
```
benchmark old ns/op new ns/op delta
BenchmarkReplaceShortcodeTokens-4 7350 4419 -39.88%
benchmark old allocs new allocs delta
BenchmarkReplaceShortcodeTokens-4 5 1 -80.00%
benchmark old bytes new bytes delta
BenchmarkReplaceShortcodeTokens-4 4816 1152 -76.08%
```
This commit is aso a small spring cleaning of duplicated code in the different `PageConvert` methods.
Fixes#1516
Page.GroupByParam function internally uses Page.GetParam to get a
parameter value for a key of a page group but now Page.GetParam returns
a lowercase character string every time. It has no need to using
lowercase character string as a group key value and it confuse a
function user.
This fixes it to keep and return an original parameter string as a group
key value.
Fix#1564
* add global `hasCJKLanguage` flag, if true, turn on auto-detecting CJKLanguage
* add `isCJKLanguage` frontmatter to force specify whether is CJKLanguage or not
* For .Summary: If isCJKLanguage is true, use the runes as basis for truncation, else keep as today.
* For WordCount: If isCJKLanguage is true, use the runes as basis for calculation, else keep as today.
* Unexport RuneCount
Fixes#1377
Add validation before creating aliases:
* Prevent creating aliases outside webroot (public/ dir)
* Skip empty "" alias
* Skip "/" → "/index.html", which gets overwritten anyway
* Refuse to create Windows-invalid filenames on Windows;
warn on other platforms
* In case of invalid aliases, after skipping them,
return `err = nil` to prevent the error passing up
all the way to `hugolib.Render()` and causing Hugo to abort.
* Update alias tests.
Fixes#701: Add support for alias with whitespace
Fixes#1418: Add validation for alias
A new "published" setting that is the opposite of "draft" is added and
left intentionally undocumented.
This setting comes from jekyll and eases the transition to hugo
greatly. We leave it undocumented so that folks don't rely on it, but
also don't shoot themselves in the foot during a jekyll migration.
The foot-shooting occurs if they have only a few documents that were
drafts ("published: false") in the jekyll version of their site and
don't notice that they were published in the migration to hugo.
The custom sort functions used from the templates had some subtle data race- and related issues,
especially when used in the single page template.
This commit fixes this by making copies and protect the read and writes with a RWMutex.
The results are cached (it will typically be invoked *number of pages* times with exactly the same data).
This is, not surprisingly, also faster:
```
benchmark old ns/op new ns/op delta
BenchmarkSortByWeightAndReverse 14228 491 -96.55%
benchmark old allocs new allocs delta
BenchmarkSortByWeightAndReverse 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkSortByWeightAndReverse 32 0 -100.00%
```
Fixes#1293
I could be wrong here, but it looks to me like .Site.Social.facebook is used in tpl/template_embedded.go, but the variable is never set. I've added a line to initializeSiteInfo to map the info from config into this variable.
In particular, RawContent() excludes the metadata header.
This is necessary in the use case of embedding remarkjs.com slides, as it needs
the unprocessed Markdown content to generate the slides.
Even as a copy at the end is needed, this consumes way less memory on Go 1.4.2:
```benchmark old ns/op new ns/op delta
BenchmarkParsePage 145979 139964 -4.12%
BenchmarkReplaceShortcodeTokens 633574 631946 -0.26%
BenchmarkShortcodeLexer 195842 187938 -4.04%
benchmark old allocs new allocs delta
BenchmarkParsePage 87 87 +0.00%
BenchmarkReplaceShortcodeTokens 9424 9415 -0.10%
BenchmarkShortcodeLexer 274 274 +0.00%
benchmark old bytes new bytes delta
BenchmarkParsePage 141830 141830 +0.00%
BenchmarkReplaceShortcodeTokens 35219 25385 -27.92%
BenchmarkShortcodeLexer 30178 30177 -0.00%
```
See #1148
This commit replaces the regexp driven `replaceShortcodeTokens` with a handwritten one.
It wasnt't possible to handle the p-tags case without breaking performance.
This fix actually improves in that area:
```
benchmark old ns/op new ns/op delta
BenchmarkParsePage 142738 142667 -0.05%
BenchmarkReplaceShortcodeTokens 665590 575645 -13.51%
BenchmarkShortcodeLexer 176038 181074 +2.86%
benchmark old allocs new allocs delta
BenchmarkParsePage 87 87 +0.00%
BenchmarkReplaceShortcodeTokens 9631 9424 -2.15%
BenchmarkShortcodeLexer 274 274 +0.00%
benchmark old bytes new bytes delta
BenchmarkParsePage 141830 141830 +0.00%
BenchmarkReplaceShortcodeTokens 52275 35219 -32.63%
BenchmarkShortcodeLexer 30177 30178 +0.00%
```
Fixes#1148
Before this commit, taxonomy names were hyphenated, lower-cased and normalized -- then fixed and titleized on the archive page.
So what you entered in the front matter isn't necessarily what you got in the final site.
To preserve backwards compability, `PreserveTaxonomyNames` is default `false`.
Setting it to `true` will preserve what you type (the first characters is made toupper for titles), but normalized in URLs.
This also means that, if you manually construct URLs to the archive pages, you will have to pass the Taxonomy names through the `urlize` func.
Fixes#1180
Section names are also used as the title of the list pages, but naming section folders as `Fish and Chips` and similar didn't work very well.
This commit fixes that.
This commit also changes the title casing of the section titles. Some may argue that this is a breaking change, but the old behaviour was also pretty broken,
even for languages that use title capitalizations, as it didn't follow any particular style guide, `fish and chips` became `Fish And Chips` etc.
Now it just turns the first letter into upper case, so `Fish and Chips` will be left as `Fish and Chips`.
People wanting the good old behaviour can use the `title` template func.
Fixes#1176
To determine if a page is the "Home Page" has inspired lots of creativity in the template department.
This commit makes it simpler: IsHome will tell the truth.
Setting `RelativeURLs` to `true` will make all relative URLs in the site *really* relative.
And will do so with speed.
So:
In `/post/myblogpost.html`:
`/mycss.css` becomes `../mycss.css`
The same in `/index.html` will become:
`./mycss.css` etc.
Note that absolute URLs will not be touched (either external resources, or URLs constructed with `BaseURL`).
The speediness is about the same as before:
```
benchmark old ns/op new ns/op delta
BenchmarkAbsURL 17462 18164 +4.02%
BenchmarkAbsURLSrcset 18842 19632 +4.19%
BenchmarkXMLAbsURLSrcset 18643 19313 +3.59%
BenchmarkXMLAbsURL 9283 9656 +4.02%
benchmark old allocs new allocs delta
BenchmarkAbsURL 24 28 +16.67%
BenchmarkAbsURLSrcset 29 32 +10.34%
BenchmarkXMLAbsURLSrcset 27 30 +11.11%
BenchmarkXMLAbsURL 12 14 +16.67%
benchmark old bytes new bytes delta
BenchmarkAbsURL 3154 3404 +7.93%
BenchmarkAbsURLSrcset 2376 2573 +8.29%
BenchmarkXMLAbsURLSrcset 2569 2763 +7.55%
BenchmarkXMLAbsURL 1888 1998 +5.83%
```
Fixes#1104Fixes#622Fixes#937Fixes#157
Create new field in Node
Update Page to look for lastmod field in the front matter. If not present, then assign Date to Lastmod
Update Site, to assign a value to Lastmod (based on the same logic used for Date)
Fixes#733
Pretty sure it has worked at some point, but that PR probably has been rebased to pieces.
This refactors the fix by @dannys42 into a method, as this URL fix is applied several places.
Fixes#1114
Many minor fixes to make test logs more consistent and correct a
mispelling.
Standardize on "[%i] got X but expected Y" for log messages. Using
a consistent layout makes it easier to read the test results. This
was mostly changing "Got" to "got". Swapped the order of values on
several calls to bring them in line with the convention.
A few log messages had a sequence number added to identify the
exact scenario that failed. Otherwise, there would be no way to
ascertain which failed When there are many scenarios.
Correct spelling of "expected."
Fixes#1028
Merged be2097e1ad
[close#1040]
The current menu system works great, but is too much work if all you want is a simple menu with the sections as menu items, and having these menu items connected to the pages in a way that enables setting the correct menu item as active for both the section lists and the pages itself.
This commit adds a new option `SectionPagesMenu' which, if set, will create a new menu with that name with all the sections as menu items. The pages in the sections will behave as "shadow members" of these section items as `blogpage.HasMenuCurrent "sectionmenu" $sectionmenuitem` will return true.
If a menu item with the same `identifier` is defined in site config, *that* item will take precedence.
`Paginate`now returns error when
1) `.Paginate` is called after `.Paginator`
2) `.Paginate` is repeatedly called with different arguments
This should help remove some confusion.
This commit also introduces DistinctErrorLogger, to prevent spamming the log for duplicate rendering errors from the pagers.
Fixes#993
Commit 358dcce supposedly added ".adoc" extension recognition
for AsciiDoc, but one place was missed.
Thanks to @sjfloat for reporting the bug!
See discussions at #470.
Put version handling into the helpers package so it can be used by many,
and split version and suffix to make it possible to calculate the next Hugo version.
Thanks to @bep's new, brilliant helpers.Deprecated() function,
the following functions or variables are transitioned to their
new names, preserving backward compatibility for v0.14
and warning the user of upcoming obsolescence in v0.15:
* .Url → .URL (for node, menu and paginator)
* .Site.BaseUrl → .Site.BaseURL
* .Site.Indexes → .Site.Taxonomies
* .Site.Recent → .Site.Pages
* getJson → getJSON
* getCsv → getCSV
* safeHtml → safeHTML
* safeCss → safeCSS
* safeUrl → safeURL
Also fix related initialisms in strings and comments.
Continued effort in fixing #959.
See #470
* Based on existing support for reStructuredText files
* Handles content files with extensions `.asciidoc` and `.ad`
* Pipes content through `asciidoctor --safe -`.
If `asciidoctor` is not installed, then `asciidoc --safe -`.
* To make sure `asciidoctor` or `asciidoc` is found, after adding
a piece of AsciiDoc content, run `hugo` with the `-v` flag
and look for this message:
INFO: 2015/01/23 Rendering with /usr/bin/asciidoctor ...
Caveats:
* The final "Last updated" timestamp is currently not stripped.
* When `hugo` is run with `-v`, you may see a lot of these messages
INFO: 2015/01/23 Rendering with /usr/bin/asciidoctor ...
if you have lots of `*.ad`, `*.adoc` or `*.asciidoc` files.
* Some versions of `asciidoc` may have trouble with its safe mode.
To test if you are affected, try this:
$ echo "Hello" | asciidoc --safe -
asciidoc: ERROR: unsafe: ifeval invalid
asciidoc: FAILED: ifeval invalid safe document
If so, I recommend that you install `asciidoctor` instead.
Feedback and patches welcome!
Ideally, we should be using https://github.com/VonC/asciidocgo,
@VonC's wonderful Go implementation of Asciidoctor. However,
there is still a bit of work needed for asciidocgo to expose
its API so that Hugo can actually use it.
Until then, hope this "experimental AsciiDoc support through external
helpers" can serve as a stopgap solution for our community. :-)
2015-01-30: Updated for the replaceShortcodeTokens() syntax change
2015-02-21: Add `.adoc` extension as suggested by @Fale
Conflicts:
helpers/content.go
First step to use initialisms that golint suggests,
for example:
Line 116: func GetHtmlRenderer should be GetHTMLRenderer
as see on http://goreportcard.com/report/spf13/hugo
Thanks to @bep for the idea!
Note that command-line flags (cobra and pflag)
as well as struct fields like .BaseUrl and .Url
that are used in Go HTML templates need more work
to maintain backward-compatibility, and thus
are NOT yet dealt with in this commit.
First step in fixing #959.
...
Prevent 404.html from prettifying into 404/index.html
Restore @realchaseadams's commit 348e123
"Force `UglyUrls` option to force `404.html` file name"
which got lost after some refactoring (commit 8db3c0b).
Remove the equivalent "force `UglyUrls`" code for `sitemap.xml`
because the refactored code now calls `renderAndWriteXML()`
which uses `WriteDestFile()` which does not prettify a filename.
Fixes#939 (reverted from commit c4c19ad303)
Restore @realchaseadams's commit 348e123
"Force `UglyUrls` option to force `404.html` file name"
which got lost after some refactoring (commit 8db3c0b).
Remove the equivalent "force `UglyUrls`" code for `sitemap.xml`
because the refactored code now calls `renderAndWriteXML()`
which uses `WriteDestFile()` which does not prettify a filename.
Fixes#939
Added a new Template.PrintErrors() function call,
used in hugolib/site.go#Process() so it does not clutter
up `go test -v ./...` results.
Special thanks to @tatsushid for mapping out the call trace
which makes it a lot easier to find the appropriate places
to place the Template.PrintErrors() call.
Fixes#316
The variable scope in the Go templates makes it hard, if possible at all, to write templates with counter variables or similar state.
This commit fixes that by adding a writable context to Node, backed by a map: Scratch.
This context has three methods, Get, Set and Add. The Add is tailored for counter variables, but can be used for any built-in numeric values or strings.
* Add meta author, description and generator tags
* Add Hugo version beside the logo and in the footer
* Suggest the user to run `go get -u -v` to update dependencies
* Requires Go 1.3+ rather than Go 1.1+
* Improve rendering/formatting in some places
* Add trailing slash to URLs where appropriate
* GitHub redirects all http requests to https, update accordingly
Two new configuration properties, `Paginate` (default `0`) and `PaginatePath` (default `page`) are added.
Setting `paginate` to a positive value will split the list pages for the home page, sections and taxonomies into chunks of size of the `paginate` property.
A `.Paginator` is provided to help building a pager menu.
There are two ways to configure a `.Paginator`:
1. The simplest way is just to call `.Paginator.Pages` from a template. It will contain the pages for "that page" (`.Data.Pages` will (like today) contain all the pages).
2. Select a sub-set of the pages with the available template functions and pass the slice to `.Paginate` : `{{ range (.Paginate (where .Data.Pages "Type" "post")).Pages }}`
**NOTE:** For a given Node, it's one of the options above. It's perfectly legitimate to iterate over the same pager more than once, but it's static and cannot change.
The `.Paginator` contains enough information to build a full-blown paginator interface.
The pages are built on the form (note: BLANK means no value, i.e. home page):
```
[SECTION/TAXONOMY/BLANK]/index.html
[SECTION/TAXONOMY/BLANK]/page/1/index.html => redirect to [SECTION/TAXONOMY/BLANK]/index.html
[SECTION/TAXONOMY/BLANK]/page/2/index.html
....
```
Fixes#96
When we have an absolute menu url specified in the config file
(e.g., `menu: { main: { name: "News", url: "/news" } }`),
its menu entry is generated by prefixing it with the BaseUrl.
The result is then run through prepUrl(), which uses helpers.Urlize to
convert urls such as 'My First Link' to 'my-first-link'.
The behaviour is backwards: we do not want to run helpers.Urlize on the
BaseUrl, only on the absolute component. Currently, a BaseUrl such as
'http://my.edu/ENG101' will be converted to 'http://my.edu/eng101',
resulting in broken links in all of my menus.
This commit switches the URL prep and BaseUrl prepending actions around. I
would argue that these URLs shouldn't be run through prepUrl anyway
because the site developer has specified them explicitly in a config file
and might be surprised for, e.g., URLs to change case, but that's another
commit for another time.
Setting per-page Blackfriday angledQuotes did not work
with TOML or JSON front matter, but it does work with YAML.
It turns out that page.Params("blackfriday") returns
type map[interface{}]interface{} for YAML, but
type map[string]interface{} for JSON and TOML.
This patch updates page.GetParam() to catch the latter,
with an error message if page.GetParam() does not recognize
a type. A test is also added.
canonifyUrls=true, RelPermalink and baseUrl with sub-path did not work.
This fixes that by adding a check for canonifyUrl=trues=true in RelPermalink().
So given
- baseUrl "http://somehost.com/sub/"
- the path "some-path/file.html"
For canonifyUrls=false RelPermalink() returns "/sub/some-path/file.html"
For canonifyUrls=true RelPermalink() returns "/some-path/file.html"
In the last case, the Url will be made absolute and clickable in a later step.
This commit also makes the menu urls defined in site config releative. To make them work with canonifying of urls, the context root is prepended if canonifying is turned off.
Fixes#519Fixes#711
This map can potentially be used many times for a given page, and altough the cost of re-creating the map should be minimal, caching it is simple -- and could save some GC and CPU cycles.
(Experimental) reStructuredText support was working in v0.12,
but was no longer handled after some refactoring in v0.13-DEV.
That experimental support is now restored.
Furthermore, check for both rst2html and rst2html.py in the PATH,
and execute whichever is found.
See #472 for more information.
Instead of `strings.TrimSpace()`, use `strings.Join(strings.Fields(s), " ")`
to collapse all whitespaces into single spaces, in order to match the
behaviour of helpers.TruncateWordsToWholeSentence(),
in order to detect non-truncated content correctly.
