Work In Progress!
This commit makes a rework of the build and rebuild process to better suit a multi-site setup.
This also includes a complete overhaul of the site tests. Previous these were a messy mix that
were testing just small parts of the build chain, some of it testing code-paths not even used in
"real life". Now all tests that depends on a built site follows the same and real production code path.
See #2309Closes#2211Closes#477Closes#1744
This commit also consolidates URLs on Node vs Page, so now .Permalink should be interoperable.
Note that this implementations should be fairly short-livded, waiting for #2297, but the API should be stable.
Implements:
* support to render:
* content/post/whatever.en.md to /en/2015/12/22/whatever/index.html
* content/post/whatever.fr.md to /fr/2015/12/22/whatever/index.html
* gets enabled when `Multilingual:` is specified in config.
* support having language switchers in templates, that know
where the translated page is (with .Page.Translations)
(when you're on /en/about/, you can have a "Francais" link pointing to
/fr/a-propos/)
* all translations are in the `.Page.Translations` map, including the current one.
* easily tweak themes to support Multilingual mode
* renders in a single swift, no need for two config files.
Adds a couple of variables useful for multilingual sites
Adds documentation (content/multilingual.md)
Added language prefixing for all URL generation/permalinking see in the
code base.
Implements i18n. Leverages the great github.com/nicksnyder/go-i18n lib.. thanks Nick.
* Adds "i18n" and "T" template functions..
If Page.Markup was not set by the user, it will now be set after
guessing from the file extension. This means, Page.Markup will be set in
any case. It can be used by a theme to differentiate between markup
types.
Fixes#1950
This also includes a refactor of the hugofs package and its usage.
The motivation for that is:
The Afero filesystems are brilliant. Hugo's way of adding a dozen of global variables for the different filesystems was a mistake. In readFile (and also in some other places in Hugo today) we need a way to restrict the access inside the working dir. We could use ioutil.ReadFile and implement the path checking, checking the base path and the dots ("..") etc. But it is obviously better to use an Afero BasePathFs combined witha ReadOnlyFs. We could create a use-once-filesystem and handle the initialization ourselves, but since this is also useful to others and the initialization depends on some other global state (which would mean to create a new file system on every invocation), we might as well do it properly and encapsulate the predefined set of filesystems. This change also leads the way, if needed, to encapsulate the file systems in a struct, making it possible to have several file system sets in action at once (parallel multilanguage site building? With Moore's law and all...)
Fixes#1551
Currently a `[]byte` copy is returned. In most cases this is the safe thing to do, but we should just modify/grow the slice as needed.
This is faster and consumes less memory:
```
benchmark old ns/op new ns/op delta
BenchmarkReplaceShortcodeTokens-4 7350 4419 -39.88%
benchmark old allocs new allocs delta
BenchmarkReplaceShortcodeTokens-4 5 1 -80.00%
benchmark old bytes new bytes delta
BenchmarkReplaceShortcodeTokens-4 4816 1152 -76.08%
```
This commit is aso a small spring cleaning of duplicated code in the different `PageConvert` methods.
Fixes#1516
* add global `hasCJKLanguage` flag, if true, turn on auto-detecting CJKLanguage
* add `isCJKLanguage` frontmatter to force specify whether is CJKLanguage or not
* For .Summary: If isCJKLanguage is true, use the runes as basis for truncation, else keep as today.
* For WordCount: If isCJKLanguage is true, use the runes as basis for calculation, else keep as today.
* Unexport RuneCount
Fixes#1377
A new "published" setting that is the opposite of "draft" is added and
left intentionally undocumented.
This setting comes from jekyll and eases the transition to hugo
greatly. We leave it undocumented so that folks don't rely on it, but
also don't shoot themselves in the foot during a jekyll migration.
The foot-shooting occurs if they have only a few documents that were
drafts ("published: false") in the jekyll version of their site and
don't notice that they were published in the migration to hugo.
In particular, RawContent() excludes the metadata header.
This is necessary in the use case of embedding remarkjs.com slides, as it needs
the unprocessed Markdown content to generate the slides.
This commit replaces the regexp driven `replaceShortcodeTokens` with a handwritten one.
It wasnt't possible to handle the p-tags case without breaking performance.
This fix actually improves in that area:
```
benchmark old ns/op new ns/op delta
BenchmarkParsePage 142738 142667 -0.05%
BenchmarkReplaceShortcodeTokens 665590 575645 -13.51%
BenchmarkShortcodeLexer 176038 181074 +2.86%
benchmark old allocs new allocs delta
BenchmarkParsePage 87 87 +0.00%
BenchmarkReplaceShortcodeTokens 9631 9424 -2.15%
BenchmarkShortcodeLexer 274 274 +0.00%
benchmark old bytes new bytes delta
BenchmarkParsePage 141830 141830 +0.00%
BenchmarkReplaceShortcodeTokens 52275 35219 -32.63%
BenchmarkShortcodeLexer 30177 30178 +0.00%
```
Fixes#1148
Before this commit, taxonomy names were hyphenated, lower-cased and normalized -- then fixed and titleized on the archive page.
So what you entered in the front matter isn't necessarily what you got in the final site.
To preserve backwards compability, `PreserveTaxonomyNames` is default `false`.
Setting it to `true` will preserve what you type (the first characters is made toupper for titles), but normalized in URLs.
This also means that, if you manually construct URLs to the archive pages, you will have to pass the Taxonomy names through the `urlize` func.
Fixes#1180
Section names are also used as the title of the list pages, but naming section folders as `Fish and Chips` and similar didn't work very well.
This commit fixes that.
This commit also changes the title casing of the section titles. Some may argue that this is a breaking change, but the old behaviour was also pretty broken,
even for languages that use title capitalizations, as it didn't follow any particular style guide, `fish and chips` became `Fish And Chips` etc.
Now it just turns the first letter into upper case, so `Fish and Chips` will be left as `Fish and Chips`.
People wanting the good old behaviour can use the `title` template func.
Fixes#1176
Create new field in Node
Update Page to look for lastmod field in the front matter. If not present, then assign Date to Lastmod
Update Site, to assign a value to Lastmod (based on the same logic used for Date)
Fixes#733
The current menu system works great, but is too much work if all you want is a simple menu with the sections as menu items, and having these menu items connected to the pages in a way that enables setting the correct menu item as active for both the section lists and the pages itself.
This commit adds a new option `SectionPagesMenu' which, if set, will create a new menu with that name with all the sections as menu items. The pages in the sections will behave as "shadow members" of these section items as `blogpage.HasMenuCurrent "sectionmenu" $sectionmenuitem` will return true.
If a menu item with the same `identifier` is defined in site config, *that* item will take precedence.
Thanks to @bep's new, brilliant helpers.Deprecated() function,
the following functions or variables are transitioned to their
new names, preserving backward compatibility for v0.14
and warning the user of upcoming obsolescence in v0.15:
* .Url → .URL (for node, menu and paginator)
* .Site.BaseUrl → .Site.BaseURL
* .Site.Indexes → .Site.Taxonomies
* .Site.Recent → .Site.Pages
* getJson → getJSON
* getCsv → getCSV
* safeHtml → safeHTML
* safeCss → safeCSS
* safeUrl → safeURL
Also fix related initialisms in strings and comments.
Continued effort in fixing #959.
First step to use initialisms that golint suggests,
for example:
Line 116: func GetHtmlRenderer should be GetHTMLRenderer
as see on http://goreportcard.com/report/spf13/hugo
Thanks to @bep for the idea!
Note that command-line flags (cobra and pflag)
as well as struct fields like .BaseUrl and .Url
that are used in Go HTML templates need more work
to maintain backward-compatibility, and thus
are NOT yet dealt with in this commit.
First step in fixing #959.
Setting per-page Blackfriday angledQuotes did not work
with TOML or JSON front matter, but it does work with YAML.
It turns out that page.Params("blackfriday") returns
type map[interface{}]interface{} for YAML, but
type map[string]interface{} for JSON and TOML.
This patch updates page.GetParam() to catch the latter,
with an error message if page.GetParam() does not recognize
a type. A test is also added.
canonifyUrls=true, RelPermalink and baseUrl with sub-path did not work.
This fixes that by adding a check for canonifyUrl=trues=true in RelPermalink().
So given
- baseUrl "http://somehost.com/sub/"
- the path "some-path/file.html"
For canonifyUrls=false RelPermalink() returns "/sub/some-path/file.html"
For canonifyUrls=true RelPermalink() returns "/some-path/file.html"
In the last case, the Url will be made absolute and clickable in a later step.
This commit also makes the menu urls defined in site config releative. To make them work with canonifying of urls, the context root is prepended if canonifying is turned off.
Fixes#519Fixes#711
This map can potentially be used many times for a given page, and altough the cost of re-creating the map should be minimal, caching it is simple -- and could save some GC and CPU cycles.
Instead of `strings.TrimSpace()`, use `strings.Join(strings.Fields(s), " ")`
to collapse all whitespaces into single spaces, in order to match the
behaviour of helpers.TruncateWordsToWholeSentence(),
in order to detect non-truncated content correctly.
Having correct dates is important in Hugo. Previously date parsing errors were swallowed, leading to confusing results.
This commit adds ERROR logging when date or publishdate in front matter cannot be parsed into a time.Time.
The flag `HTML_SMARTYPANTS_ANGLED_QUOTES` was added to Blackfriday on Black Friday. This configures rendering of double quotes as angled left and right quotes («
»).
Typical use cases would be either or, or combined, but never in the same
document. As an example would be a person from Norway; he has a blog in both
English and Norwegian (his native tongue); he would then configure Blackfriday
to use angled quotes for the Norwegian section, but keep them as reqular
double quotes for the English.
This commit adds configuration support for this new flag, configuration that can be set in the site configuration, but overridden in page front matter.
Fixes#605
- `.Ref` and `.RelRef` take a reference (the logical filename for a
page, including extension and/or a document fragment ID) and return
a permalink (or relative permalink) to the referenced document.
- If the reference is a page name (such as `about.md`), the page
will be discovered and the permalink will be returned: `/about/`
- If the reference is a page name with a fragment (such as
`about.md#who`), the page will be discovered and used to add the
`page.UniqueID()` to the resulting fragment and permalink:
`/about/#who:deadbeef`.
- If the reference is a fragment and `.*Ref` has been called from
a `Node` or `SiteInfo`, it will be returned as is: `#who`.
- If the reference is a fragment and `.*Ref` has been called from
a `Page`, it will be returned with the page’s unique ID:
`#who:deadbeef`.
- `.*Ref` can be called from either `Node`, `SiteInfo` (e.g.,
`Node.Site`), `Page` objects, or `ShortcodeWithPage` objects in
templates.
- `.*Ref` cannot be used in content, so two shortcodes have been
created to provide the functionality to content: `ref` and `relref`.
These are intended to be used within markup, like `[Who]({{% ref
about.md#who %}})` or `<a href="{{% ref about.md#who %}}">Who</a>`.
- There are also `ref` and `relref` template functions (used to create
the shortcodes) that expect a `Page` or `Node` object and the
reference string (e.g., `{{ relref . "about.md" }}` or `{{
"about.md" | ref . }}`). It actually looks for `.*Ref` as defined on
`Node` or `Page` objects.
- Shortcode handling had to use a *differently unique* wrapper in
`createShortcodePlaceholder` because of the way that the `ref` and
`relref` are intended to be used in content.
File handling was broken on Windows. This commit contains a revision of the path handling with separation of file paths and urls where needed.
There may be remaining issues and there may be better ways to do this, but it is easier to start that refactoring job with a set of passing tests.
Fixes#687Fixes#660
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes#565Fixes#480Fixes#461
And probably some others.
If content pages are fully rendered in a list page, footnotes that use
the same reference (`[^fn]`) will have duplicated anchors. This change
builds on #526 to put the page filename (`Page.File.Name`) as part of
the anchor for a footnote.
This would fix discussion [116](http://discuss.gohugo.io/t/footnote-references-are-duplicated-on-list-pages/116).
- The config file can provide FootnoteAnchorPrefix, which will be used
by blackfriday when rendering to HTML. A value of `q:` has the effect
of making the anchor for a footnote `[^footie]` be `fn:q:footie`. The
default is `""`.
- The config file can provide FootnoteReturnLinkContents, which will be
used by blackfriday when rendering to HTML. A value of `^` has the
effect of making the return link be `^` instead of `[return]`.
Modified markdownRender and markdownRenderWithTOC in hugolib/page.go to
use the same flags and extensions as were previously used when we simply
called blackfriday.MarkdownCommon to convert Markdown to HTML. These
flags/extensions were dropped during the refactor that added the
`.TableOfContents` page variable, and caused features like Markdown
tables to no longer work.
Modified the expected output for TestTableOfContents in page_test.go,
apparently changing the flags/extensions caused an `—` to become
`–`.
Added TableOfContents field to hugolib.Page struct. New function
getTableOfContents is used in convertMarkdown to set the TableOfContents
field.
Added new test file hugolib/page_toc_test.go with a simple test of the
new functionality.
Conflicts:
hugolib/page.go
A sample config.yaml for a site might contain:
```yaml
permalinks:
post: /:year/:month/:title/
```
Then, any article in the `post` section, will have the canonical URL
formed via the permalink specification given.
Signed-off-by: Noah Campbell <noahcampbell@gmail.com>
* Add `.Truncated` bool to each page; will be set true if the
`.Summary` is truncated and it's worth showing a "more" link of some
kind.
* Add `Params` to the site config, defining `.Site.Params` accessible
to each page; this lets the site maintainer associate arbitrary data
with names, on a site-wide basis.
* Provide a `First` function to templates:
* Use-case: `{{range First 5 .Site.Recent}}` or anything else which
is a simple iterable provided by hugolib
* Tests by me for `.Truncated` and `First`
Also @noahcampbell contributed towards this:
* Add UnitTest for `.Site.Params`:
> Digging into this test case a bit more, I'm realizing that we need
> to create a param test case to ensure that for each type we render
> (page, index, homepage, rss, etc.) that the proper fields are
> represented. This will help us refactor without fear in the
> future.
Sample config.yaml:
```yaml
title: "Test site"
params:
Subtitle: "More tests always good"
AuthorName: "John Doe"
SidebarRecentLimit: 5
```
Signed-off-by: Noah Campbell <noahcampbell@gmail.com>
The render code path would use a fallback if there was an exception.
This change instead relies on explicit declaration of the layout to use
and includes a check to see if the layout indeed exists before
attempting to render it.
If a file named index.html exists in a directory, or root, it will be
rendered as if ugly urls are turned on. This allows for top level
content to not need a supporting layout file and content in content.
This change should not affect anyone who is using the perscribed way.
I also cleaned up a bunch of one off functions in site.go.
Allow content that is not markdown and does not need to be rendered to
exists in the content directory. Currently any valid html or xml
document can exist. Templates are applied to these documents as well.
If you need to have content that doesn't have templates or AbsUrlify
like operations, then continue to put this content in static and it will
be copied over.
As pages are read from the target, they will be assessed if they should
be rendered or not. The logic for IsRenderable is in the parser/page.go
and looks for anything exception '<'.
filepath was used inconsistently throughout the hugolib. With the
introduction of source and target modules, all path are normalized to
"/". This simplifies the processing of paths. It does mean that
contributors need to be aware of using path/filepath in any module other
than source or target is not recommended. The current exception is
hugolib/config.go
It started with wanting to move templates in template bundles and the
rest followed. I did my best to start grouping related functions
together, but there are some that I missed. There is also the method
Urlize that seems to be a special function used in both worlds. I'll
need to revisit this method.
Allow full control of summaries which can be rendered as html rather
than text. Using a `<!--more-->` html comment in your markdown / rst
you can indiciate where the summary should end and have the summary
converted to html.
Signed-off-by: Noah Campbell <noahcampbell@gmail.com>
Conflicts:
hugolib/page_test.go
because the url lacks a trailing /, many webservers will issue a
redirect to the canonical url with trailing slash for directory index
w/index.htm(l).
Append a slash to avoid this.
The filename path was being split using a unix specific path seperator. This fix uses the os.PathSeperator to ensure proper evaluation regardless of platform.
