Restore @realchaseadams's commit 348e123
"Force `UglyUrls` option to force `404.html` file name"
which got lost after some refactoring (commit 8db3c0b).
Remove the equivalent "force `UglyUrls`" code for `sitemap.xml`
because the refactored code now calls `renderAndWriteXML()`
which uses `WriteDestFile()` which does not prettify a filename.
Fixes#939
Added a new Template.PrintErrors() function call,
used in hugolib/site.go#Process() so it does not clutter
up `go test -v ./...` results.
Special thanks to @tatsushid for mapping out the call trace
which makes it a lot easier to find the appropriate places
to place the Template.PrintErrors() call.
Fixes#316
The variable scope in the Go templates makes it hard, if possible at all, to write templates with counter variables or similar state.
This commit fixes that by adding a writable context to Node, backed by a map: Scratch.
This context has three methods, Get, Set and Add. The Add is tailored for counter variables, but can be used for any built-in numeric values or strings.
* Add meta author, description and generator tags
* Add Hugo version beside the logo and in the footer
* Suggest the user to run `go get -u -v` to update dependencies
* Requires Go 1.3+ rather than Go 1.1+
* Improve rendering/formatting in some places
* Add trailing slash to URLs where appropriate
* GitHub redirects all http requests to https, update accordingly
Two new configuration properties, `Paginate` (default `0`) and `PaginatePath` (default `page`) are added.
Setting `paginate` to a positive value will split the list pages for the home page, sections and taxonomies into chunks of size of the `paginate` property.
A `.Paginator` is provided to help building a pager menu.
There are two ways to configure a `.Paginator`:
1. The simplest way is just to call `.Paginator.Pages` from a template. It will contain the pages for "that page" (`.Data.Pages` will (like today) contain all the pages).
2. Select a sub-set of the pages with the available template functions and pass the slice to `.Paginate` : `{{ range (.Paginate (where .Data.Pages "Type" "post")).Pages }}`
**NOTE:** For a given Node, it's one of the options above. It's perfectly legitimate to iterate over the same pager more than once, but it's static and cannot change.
The `.Paginator` contains enough information to build a full-blown paginator interface.
The pages are built on the form (note: BLANK means no value, i.e. home page):
```
[SECTION/TAXONOMY/BLANK]/index.html
[SECTION/TAXONOMY/BLANK]/page/1/index.html => redirect to [SECTION/TAXONOMY/BLANK]/index.html
[SECTION/TAXONOMY/BLANK]/page/2/index.html
....
```
Fixes#96
When we have an absolute menu url specified in the config file
(e.g., `menu: { main: { name: "News", url: "/news" } }`),
its menu entry is generated by prefixing it with the BaseUrl.
The result is then run through prepUrl(), which uses helpers.Urlize to
convert urls such as 'My First Link' to 'my-first-link'.
The behaviour is backwards: we do not want to run helpers.Urlize on the
BaseUrl, only on the absolute component. Currently, a BaseUrl such as
'http://my.edu/ENG101' will be converted to 'http://my.edu/eng101',
resulting in broken links in all of my menus.
This commit switches the URL prep and BaseUrl prepending actions around. I
would argue that these URLs shouldn't be run through prepUrl anyway
because the site developer has specified them explicitly in a config file
and might be surprised for, e.g., URLs to change case, but that's another
commit for another time.
Setting per-page Blackfriday angledQuotes did not work
with TOML or JSON front matter, but it does work with YAML.
It turns out that page.Params("blackfriday") returns
type map[interface{}]interface{} for YAML, but
type map[string]interface{} for JSON and TOML.
This patch updates page.GetParam() to catch the latter,
with an error message if page.GetParam() does not recognize
a type. A test is also added.
canonifyUrls=true, RelPermalink and baseUrl with sub-path did not work.
This fixes that by adding a check for canonifyUrl=trues=true in RelPermalink().
So given
- baseUrl "http://somehost.com/sub/"
- the path "some-path/file.html"
For canonifyUrls=false RelPermalink() returns "/sub/some-path/file.html"
For canonifyUrls=true RelPermalink() returns "/some-path/file.html"
In the last case, the Url will be made absolute and clickable in a later step.
This commit also makes the menu urls defined in site config releative. To make them work with canonifying of urls, the context root is prepended if canonifying is turned off.
Fixes#519Fixes#711
This map can potentially be used many times for a given page, and altough the cost of re-creating the map should be minimal, caching it is simple -- and could save some GC and CPU cycles.
(Experimental) reStructuredText support was working in v0.12,
but was no longer handled after some refactoring in v0.13-DEV.
That experimental support is now restored.
Furthermore, check for both rst2html and rst2html.py in the PATH,
and execute whichever is found.
See #472 for more information.
Instead of `strings.TrimSpace()`, use `strings.Join(strings.Fields(s), " ")`
to collapse all whitespaces into single spaces, in order to match the
behaviour of helpers.TruncateWordsToWholeSentence(),
in order to detect non-truncated content correctly.
Added Version, CommitHash and BuildDate to hugolib/hugo.go and used it in build
Removed commitHash and buildDate from commands/version.go and used hugolib vars
Removed getDateFormat function from commands/version.go
Conflicts:
README.md
docs/content/templates/variables.md
This rewrites `extractFrontMatterDelims` function to make it work with
an archetype file without the final EOL and adds more detailed error
messages and comments.
It also removes `matches` and `matches_quick` functions which aren't
called anywhere.
Having correct dates is important in Hugo. Previously date parsing errors were swallowed, leading to confusing results.
This commit adds ERROR logging when date or publishdate in front matter cannot be parsed into a time.Time.
The flag `HTML_SMARTYPANTS_ANGLED_QUOTES` was added to Blackfriday on Black Friday. This configures rendering of double quotes as angled left and right quotes («
»).
Typical use cases would be either or, or combined, but never in the same
document. As an example would be a person from Norway; he has a blog in both
English and Norwegian (his native tongue); he would then configure Blackfriday
to use angled quotes for the Norwegian section, but keep them as reqular
double quotes for the English.
This commit adds configuration support for this new flag, configuration that can be set in the site configuration, but overridden in page front matter.
Fixes#605
Menu urls like /categories/новости-проекта would turn into /categories/d0bdd0bed0b2d0bed181d182d0b8-d0bfd180d0bed0b5d0bad182d0b0, which is illegal, while the directory under the categories/ is created with the original name. It results in 404 not found error.
This commit fixes that by make sure that SanitizeUrl() is called last.
Fixes#719
- `.Ref` and `.RelRef` take a reference (the logical filename for a
page, including extension and/or a document fragment ID) and return
a permalink (or relative permalink) to the referenced document.
- If the reference is a page name (such as `about.md`), the page
will be discovered and the permalink will be returned: `/about/`
- If the reference is a page name with a fragment (such as
`about.md#who`), the page will be discovered and used to add the
`page.UniqueID()` to the resulting fragment and permalink:
`/about/#who:deadbeef`.
- If the reference is a fragment and `.*Ref` has been called from
a `Node` or `SiteInfo`, it will be returned as is: `#who`.
- If the reference is a fragment and `.*Ref` has been called from
a `Page`, it will be returned with the page’s unique ID:
`#who:deadbeef`.
- `.*Ref` can be called from either `Node`, `SiteInfo` (e.g.,
`Node.Site`), `Page` objects, or `ShortcodeWithPage` objects in
templates.
- `.*Ref` cannot be used in content, so two shortcodes have been
created to provide the functionality to content: `ref` and `relref`.
These are intended to be used within markup, like `[Who]({{% ref
about.md#who %}})` or `<a href="{{% ref about.md#who %}}">Who</a>`.
- There are also `ref` and `relref` template functions (used to create
the shortcodes) that expect a `Page` or `Node` object and the
reference string (e.g., `{{ relref . "about.md" }}` or `{{
"about.md" | ref . }}`). It actually looks for `.*Ref` as defined on
`Node` or `Page` objects.
- Shortcode handling had to use a *differently unique* wrapper in
`createShortcodePlaceholder` because of the way that the `ref` and
`relref` are intended to be used in content.
File handling was broken on Windows. This commit contains a revision of the path handling with separation of file paths and urls where needed.
There may be remaining issues and there may be better ways to do this, but it is easier to start that refactoring job with a set of passing tests.
Fixes#687Fixes#660
Enable blackfriday.EXTENSION_AUTO_HEADER_IDS to generate the name of the
header ID from the text in the header. Works for prefix and underline
headers.
- TOC extraction had to be modified to look for `<li><a href="#`>
instead of `#toc_` because of this change.
- Fixed a number of tests that depended on the presence of `toc_` with
as an `id` or as a `href` value.
- Renames the earlier parameter `footnoteref` to `documentId` as it more
accurately represents the nature of the parameter. The `documentId` is
appended to all generated headers through the new HTML renderer
parameter `HeaderIDSuffix`.
Some newly added shortcode tests compared maps in assertions.
This failed on Travis, as iteration order isn't guaranteed for maps since Go 1.
This commit fixes that by do a sort of the keys in the shortcode String() function.
- Prevent `.xml` generation for root section
- Remove redundant check for DisableRSS
- Fix permalinks for rel="alternate"
- Rename generated xml file to <type>/index.xml
- Add required description element in default template
- Make default RSS template validate on w3c (timezone format is still an issue)
Conflicts:
hugolib/site.go
- Change order of HasPrefix to match correct order
- Remove theme concatenation to _internal in last loop of
appendthemetemplates so it looks in the right place for internal
templates
Conflicts:
hugolib/site.go
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes#565Fixes#480Fixes#461
And probably some others.
Following issues are fixed
1. Can't access fields and methods specified in GroupBy call
2. PagesGroup doesn't contain Pages. It's always empty.
3. When GroupBy is called with Section key, it doesn't work as expected
- In `layouts/_default/taxonomy.html`, the `.Data` result does not
provide the same information that `layouts/_default/terms.html` does
for being able to identify the plural value of the term.
- This change adds `.Data.Singular` and `.Data.Plural` to provide
similar capabilities.
- This *may* be incompatible with templates that check for `{{ if ne
$taxonomy "Pages" }}` if the `page.Params` has either the singular or
plural values as keys.
…`map[string]string` to `map[string]interface{}`.
This allows values other than `string` values to be saved to Author,
such as:
```toml
# config.toml
…
[Author]
name = "Austin Ziegler"
social-site = [ "Facebook", "Twitter", "GitHub" ]
```
My specific use-case is that I’m trying to make something work similar
whether it’s specified in `.Params.Author` or in `.Site.Author` without
introducing `.Site.Params.Author`.
`GroupBy` is modified to allow it to receive a method name argument for
example `Type` as its first argument. It is only allowed to call with
a method which takes no arguments and returns a result or a pair of
a result and an error.
The functions discussed at #443 are also added
- `ByPublishDate`: Order contents by `PublishDate` front matter variable
- `GroupByPublishDate(format, order)`: Group contents by `PublishDate`
front matter variable formatted in string like `GroupByDate`
- `GroupByParam(key, order)`: Group contents by `Param` front matter
variable specified by `key` argument
- `GroupByParamDate(key, format, order)`: Group contents by `Param`
front matter variable specified by `key` argument and formatted in
string like `GroupByDate`. It's effective against `time.Time` type
front matter variable
Prior to this commit, `HasMenuCurrent` and `IsMenuCurrent` on `Node` always returned false.
This made it hard (if possible at all) to mark the currently selected menu item/group for non-Page content (home page, category pages etc.), i.e. for menus defined in the site configuration.
This commit provides an implementation of these two methods.
Notable design choices:
* These menu items have a loose coupling to the the resources they navigate to; the `Url` is the best common identificator. To facilitate a consistent matching, and to get it in line with the menu items connected to `Page`, relative Urls (Urls starting with '/') for menu items in the site configuration are converted to permaLinks using the same rules used for others’.
* `IsMenuCurrent` only looks at the children of the current node; this is in line with the implementation on `Page`.
* Due to this loose coupling, `IsMenuCurrent` have to search downards in the tree to make sure that the node is inside the current menu. This could have been made simpler if it could answer `yes` to any match of any menu item matching the current resource.
This commit also adds a set of unit tests for the menu system.
Fixes#367
TOML and YAML handles integers differently, creating issues when using integer values from configuration or front matter in the First template function.
This currently works in YAML (parses into int), but not in TOML (parses into int64).
This commit modifies First so it accepts any int.
Fixes#551
The Intersect template-method would fail if one or both of the lists were nil (post vs page; post has tags, page has not).
This commit adds a nil-check and returns an empty result if any of the inputs are nil.
See #537
If content pages are fully rendered in a list page, footnotes that use
the same reference (`[^fn]`) will have duplicated anchors. This change
builds on #526 to put the page filename (`Page.File.Name`) as part of
the anchor for a footnote.
This would fix discussion [116](http://discuss.gohugo.io/t/footnote-references-are-duplicated-on-list-pages/116).
- The config file can provide FootnoteAnchorPrefix, which will be used
by blackfriday when rendering to HTML. A value of `q:` has the effect
of making the anchor for a footnote `[^footie]` be `fn:q:footie`. The
default is `""`.
- The config file can provide FootnoteReturnLinkContents, which will be
used by blackfriday when rendering to HTML. A value of `^` has the
effect of making the return link be `^` instead of `[return]`.
Node.Site.Recent is not really just recent pages, but all pages, so I figured it was better to add a new parameter with a more informative name.
I also changed the code slightly so that all pages are added to the list of pages before we start rendering shortcodes... this way you can use a shortcode to refer to another page. Previosuly, this had been broken, because the list ofg pages would not be fully populated while the shortcodes were being processed. The code that does this is not reading from disk or doing any rendering, so it shouldn't take any more time to do.
This fixes#450. There are two problems:
1.) We're creating a new goroutine for every page.
2.) We're calling s.Pages = append(s.Pages, page) inside each goroutine.
1 is a problem if in that if you have a ton of pages, that's a ton of goroutines. It's not really useful to have more than a few goroutines at a time, and lots can actually make your code much slower, and, evidently, crash.
2 is a problem in that append is not thread safe. Sometimes it returns a new slice with a larger capacity, when the original slice isn't large enough. This can cause problems if two goroutines do this at the same time.
The solution for 1 is to use a limited number of workers (I chose 2*GOMAXPROCS as a nice guess).
The solution for 2 is to serialize access to s.Pages, which I did by doing it in a single goroutine.
Mainly this was a change to helpers.MakePermalink, but to get the local server to run correctly,
we needed to redirect the path of the request from /foo to /. In addition, I added tests for the
server's code for fixing up the base url with different config file & CLI options.
git bisect identified 62dd1d4 as the breaking commit; when
github.com/spf13/viper was introduced, the Params field was always
empty.
Given a map in YAML in Viper, the return type is
`map[interface{}]interface{}`, _not_ `map[string]interface{}`, even if
`.SetDefault()` has been called with an item of
`map[string]interface{}{}` so the cast assertion on the `.Get("Params")`
always failed.
Viper stores Permalinks as a map[string]interface{}, so the type assertion
to PermalinkOverrides (map[string]PathPattern) will always fail.
We can, however, get Permalinks as a map[string]string, and convert each
value to a PathPattern.
If you don't have access to the root domain of your site (eg a GitHub project
page) and you try to generate custom permalinks, they must begin with a slash.
Go's URL resolution library sees the leading slash and thinks "this URL starts
at the root", just like a filesystem - so it discards your subdomain and maps
all custom permalinks from the root of your site. Fine if you control the root
domain, not so useful if you don't.
Removing the check for a leading slash fixes this problem. You can now specify
custom permalinks that do not start with a slash, and they will map safely
regardless of what subdomain you upload the generated site under.
Tests have been updated for this commit so that they continue to function.
Modified markdownRender and markdownRenderWithTOC in hugolib/page.go to
use the same flags and extensions as were previously used when we simply
called blackfriday.MarkdownCommon to convert Markdown to HTML. These
flags/extensions were dropped during the refactor that added the
`.TableOfContents` page variable, and caused features like Markdown
tables to no longer work.
Modified the expected output for TestTableOfContents in page_test.go,
apparently changing the flags/extensions caused an `—` to become
`–`.
Added TableOfContents field to hugolib.Page struct. New function
getTableOfContents is used in convertMarkdown to set the TableOfContents
field.
Added new test file hugolib/page_toc_test.go with a simple test of the
new functionality.
Conflicts:
hugolib/page.go
Be able to inhibit AbsURL canonicalization of content, on a site
configuration basis. Advantages of being able to inhibit this include
making it easier to rendering on other hostnames, and being able to
include resources on http or https depending on how this page was
retrieved, avoiding mixed-mode client complaints without adding latency
for plain http.
Fixed windows uses different filepath separator. The filepath.ToSlash
shouldn't be used, because it can cause errors in filepath suffix and prefix
testing since "c:\a" isn't a prefix of "c:/a/b/c".
A sample config.yaml for a site might contain:
```yaml
permalinks:
post: /:year/:month/:title/
```
Then, any article in the `post` section, will have the canonical URL
formed via the permalink specification given.
Signed-off-by: Noah Campbell <noahcampbell@gmail.com>
* Add `.Truncated` bool to each page; will be set true if the
`.Summary` is truncated and it's worth showing a "more" link of some
kind.
* Add `Params` to the site config, defining `.Site.Params` accessible
to each page; this lets the site maintainer associate arbitrary data
with names, on a site-wide basis.
* Provide a `First` function to templates:
* Use-case: `{{range First 5 .Site.Recent}}` or anything else which
is a simple iterable provided by hugolib
* Tests by me for `.Truncated` and `First`
Also @noahcampbell contributed towards this:
* Add UnitTest for `.Site.Params`:
> Digging into this test case a bit more, I'm realizing that we need
> to create a param test case to ensure that for each type we render
> (page, index, homepage, rss, etc.) that the proper fields are
> represented. This will help us refactor without fear in the
> future.
Sample config.yaml:
```yaml
title: "Test site"
params:
Subtitle: "More tests always good"
AuthorName: "John Doe"
SidebarRecentLimit: 5
```
Signed-off-by: Noah Campbell <noahcampbell@gmail.com>
Remove the hugo-nav since it relied on a slow library. The current
build reimplements the absurl functionality based on string replace.
Discovered that my prior implementation missed the requirement for
making absolute paths (/path) absolute with the host, whereas a relative
path is left untouched. Updated the test cases to support this if this
is reimplemented.
Checks to make sure the xml document starts with <?xml. Previously, the
html translate package would write additional details into the document
that caused it to fail.
50% speedup. Fix#91
to run the benchmark:
go test -test.run=NONE -bench=".*" -test.benchmem=true ./transform/ > new.txt
to compare the results:
/usr/local/go/misc/benchcmp baseline.txt new.txt
Speedup and memory improvements
benchmark old ns/op new ns/op delta
BenchmarkChain 101219 50453 -50.15%
BenchmarkTransform 51625 45531 -11.80%
benchmark old allocs new allocs delta
BenchmarkChain 222 103 -53.60%
BenchmarkTransform 135 106 -21.48%
benchmark old bytes new bytes delta
BenchmarkChain 23919 10998 -54.02%
BenchmarkTransform 11858 10665 -10.06%
Removed these checks so a single file in content can generate a site.
For example, given a site with a content directory and an index.html,
running hugo -s dir will generate a project without any more input.
The render code path would use a fallback if there was an exception.
This change instead relies on explicit declaration of the layout to use
and includes a check to see if the layout indeed exists before
attempting to render it.
If a file named index.html exists in a directory, or root, it will be
rendered as if ugly urls are turned on. This allows for top level
content to not need a supporting layout file and content in content.
This change should not affect anyone who is using the perscribed way.
I also cleaned up a bunch of one off functions in site.go.
Allow content that is not markdown and does not need to be rendered to
exists in the content directory. Currently any valid html or xml
document can exist. Templates are applied to these documents as well.
If you need to have content that doesn't have templates or AbsUrlify
like operations, then continue to put this content in static and it will
be copied over.
As pages are read from the target, they will be assessed if they should
be rendered or not. The logic for IsRenderable is in the parser/page.go
and looks for anything exception '<'.
I want to move all logic to writing aliases to target so I can pave the
way for writing aliases specific to other runtimes (like .htaccess for
apache or a script for updating AWS or symlinking on a filesystem).
filepath was used inconsistently throughout the hugolib. With the
introduction of source and target modules, all path are normalized to
"/". This simplifies the processing of paths. It does mean that
contributors need to be aware of using path/filepath in any module other
than source or target is not recommended. The current exception is
hugolib/config.go
It started with wanting to move templates in template bundles and the
rest followed. I did my best to start grouping related functions
together, but there are some that I missed. There is also the method
Urlize that seems to be a special function used in both worlds. I'll
need to revisit this method.
Tests to ensure rendering dates in templates is working correctly.
Actually, I was running into invalid templates not giving warnings when
I was trying to render a date.
Introducing the target module in hugo. This provides the simple
interface for writing content given a label (filename) and a io.Reader
containing the content to be written.
If site.Target is not set, it defaults back to the original behavior of
writing to file system.
In hugolib/site_url_test.go I have an InMemoryTarget for testing
purposes and use it to see if the final output of a render matches.
Allow full control of summaries which can be rendered as html rather
than text. Using a `<!--more-->` html comment in your markdown / rst
you can indiciate where the summary should end and have the summary
converted to html.
Signed-off-by: Noah Campbell <noahcampbell@gmail.com>
Conflicts:
hugolib/page_test.go
Provide unit test support RenderThing.
One observation is that creating the site.Tmpl variable is a one time
event. site.Tmpl doesn't like additional templates with the same name.
This means that updating a template while in --watch mode requires
throwing away the entire Site object and creating a new one. Not that
this is a bad idea, but it is something I discovered while working on
these unit tests.
An oversight on my behalf. The FromSlash method is used when writing
out the public file name. There is one place where the slashes are
required which is setting the output file. I replaced those instances
with filepath.Join which should do the right thing depending on the OS.
because the url lacks a trailing /, many webservers will issue a
redirect to the canonical url with trailing slash for directory index
w/index.htm(l).
Append a slash to avoid this.
When redirecting an alias from a .xhtml path, served with default content type,
a redirect only works if the html element has a xmlns attribute. This adds the
attribute when the alias path ends in .xhtml
sanitizeRegex was stripping dots in permalinks when generating
RenderIndexes (noted during feed/rss generation).
permalink was being set to `.../indexxml` instead of `.../index.xml`.
Adding "dot" to the regex whitelist fixed the issue.
when not using ugly urls, the feed permalink does not end up in the
expected location, and instead always behaves as if using ugly urls.
this fixes that behavior and inserts the feed xml file into the
directory as index.xml.
fixes#32
Interacting with timezones will result in checks against the filesystem.
This access, by definition, is an integration test. Creating a
*integration_test.go file will signify this change.
When interacting with Travis-ci.org, the ubuntu boxes plus go 1.1 do not
seem to support shortcode timezones, think PST. In this case, the tests
are skipped. This is not ideal, but the IRC #go-nuts channel has
indicated timezone support is still lacking. We should advise users of
hugo that timezone support may be an issue and report any odd behavior.
The workaround is to use numeric timezones (-08:00 for PST, etc.)
The filename path was being split using a unix specific path seperator. This fix uses the os.PathSeperator to ensure proper evaluation regardless of platform.
