Commit graph

77 commits

Author SHA1 Message Date
Bjørn Erik Pedersen
dd45e6d7e5 Lazy calculate WordCount, ReadingTime and FuzzyWordCount
This avoids having to execute these expensive operations for sites not using these values.

This commit sums up a set of wordcounting and autosummary related performance improvements.

The effect of these kind of depends on what features your site use, but a benchmark from 4 Hugo sites in the wild shows promise:

```
benchmark           old ns/op       new ns/op       delta
BenchmarkHugo-4     21293005843     20032857342     -5.92%

benchmark           old allocs     new allocs     delta
BenchmarkHugo-4     65290922       65186032       -0.16%

benchmark           old bytes      new bytes      delta
BenchmarkHugo-4     9771213416     9681866464     -0.91%
```

Closes #2378
2016-09-14 10:57:39 +02:00
Bjørn Erik Pedersen
0a7d1d0ddc Fix some corner cases in revised summary handling
And clean up the test.

See #2309
2016-09-06 18:32:18 +03:00
Bjørn Erik Pedersen
2079a23dd8 Make it possible to configure Blackfroday per language
See #2309
2016-09-06 18:32:17 +03:00
Bjørn Erik Pedersen
ed0985404d Render the shortcodes as late as possible
This is needed to make shortcode users happy with the new multilanguage support,
but it will also solve many other related posts about "stuff not available in the shortcode".

We will have to revisit this re the handler chain at some point, but that will be easier
now as the integration test story has improved so much.

As part of this commit, the site-building tests in page_test.go is refreshed, they now
tests for all the rendering engines (when available), and all of them now uses the
same code-path as used in production.

Fixes #1229
Fixes #2323
Fixes ##1076
2016-09-06 18:32:16 +03:00
Bjørn Erik Pedersen
708bc78770 Optimize the multilanguage build process
Work In Progress!

This commit makes a rework of the build and rebuild process to better suit a multi-site setup.

This also includes a complete overhaul of the site tests. Previous these were a messy mix that
were testing just small parts of the build chain, some of it testing code-paths not even used in
"real life". Now all tests that depends on a built site follows the same and real production code path.

See #2309
Closes #2211
Closes #477
Closes #1744
2016-09-06 18:32:16 +03:00
Bjørn Erik Pedersen
8588103316 Unexport ShouldBuild and AssertShouldBuild 2016-06-14 15:53:49 +02:00
Hanchen Wang
4724a5794e hugolib: Refactor page.ShouldBuild and table driven test 2016-06-14 15:45:26 +02:00
Mattias Wadman
5d50c46482 Chomp Unicode BOM if present
Useful if using or sharing files with users that use editors that
append a unicode byte order marker header (like Windows notepad).

This will still assume files are UTF-8 encoded.

Closes #2075
2016-04-14 10:46:27 +02:00
Bjørn Erik Pedersen
206440eef2 Fix broken tests 2016-04-12 19:18:02 +02:00
Bjørn Erik Pedersen
39c9ae3108 Revert "Use Node.ID for anchor ID"
This reverts commit cd558958a0.
2016-04-12 18:11:24 +02:00
Bjørn Erik Pedersen
cd558958a0 Use Node.ID for anchor ID
Fixes #2057
2016-04-11 13:17:25 +02:00
Bjørn Erik Pedersen
7c84e6b3e3 hugolib: Even more Golint fixes 2016-03-24 14:42:03 +01:00
Bjørn Erik Pedersen
dea185aa9b hugolib: Remove unused code 2016-03-14 15:41:03 +01:00
Bjørn Erik Pedersen
c462440dcd Add Page tests for simple methods 2016-02-07 13:32:08 +01:00
Anthony Fok
8509727fe8 Add copyright header to that source files that don’t have one.
See #1646
2015-12-10 15:19:38 -07:00
Bjørn Erik Pedersen
5bda0398e7 Export Page.Layout
And at the same time rename and unexport the badly named `Layout()` func. That method is not very usable outside the package.

Fixes #1539
2015-11-02 17:24:50 +01:00
coderzh
823334875d WordCount and Summary support CJK Language
* add global `hasCJKLanguage` flag, if true, turn on auto-detecting CJKLanguage
 * add `isCJKLanguage` frontmatter to force specify whether is CJKLanguage or not
 * For .Summary: If isCJKLanguage is true, use the runes as basis for truncation, else keep as today.
 * For WordCount: If isCJKLanguage is true, use the runes as basis for calculation, else keep as today.
 * Unexport RuneCount

Fixes #1377
2015-10-07 15:14:57 +02:00
Sven Dowideit
a1e32439fb Allow for any (short) line begining or ending with html comment
Fixes #1428
2015-09-25 18:32:32 +02:00
Jeff Hodges
49fe04c0bd Correct check of published boolean 2015-08-31 05:26:46 +02:00
Jeff Hodges
311593bff0 Add undocumented published setting to front matter
A new "published" setting that is the opposite of "draft" is added and
left intentionally undocumented.

This setting comes from jekyll and eases the transition to hugo
greatly. We leave it undocumented so that folks don't rely on it, but
also don't shoot themselves in the foot during a jekyll migration.

The foot-shooting occurs if they have only a few documents that were
drafts ("published: false") in the jekyll version of their site and
don't notice that they were published in the migration to hugo.
2015-08-30 20:40:56 +02:00
Bjørn Erik Pedersen
3037d200cb Fix inconsistent front matter handling
Fixes #768
2015-07-26 15:28:56 +02:00
Bjørn Erik Pedersen
a2fa3895ee Fix log message in test 2015-07-12 19:27:29 +02:00
Bjørn Erik Pedersen
77c60a3440 Add RuneCount to Page
Fixes #1266
2015-07-12 11:05:37 +02:00
bep
be0903c71a Fix FullFilePath
Fixes #732
2015-05-31 18:54:20 +02:00
bep
be535832f7 Add IsHome
To determine if a page is the "Home Page" has inspired lots of creativity in the template department.

This commit makes it simpler: IsHome will tell the truth.
2015-05-28 03:20:08 +02:00
spf13
a584ff207b Stop Viper from leaking across many of the tests (now tests pass regardless of order tested) 2015-05-20 02:21:21 -04:00
Takuya Wakisaka
8a96234b1f Add Page tests with UTF8 paths
See #988
2015-05-17 14:54:27 +02:00
bep
be01d04870 hugolib: make Page implement the ReaderFrom interface 2015-04-03 21:41:24 +02:00
Naoya Inada
407e80a9ab Add site-wide/per-page [blackfriday] extensions option 2015-01-26 09:55:37 +01:00
Anthony Fok
803865f870 Fix for page.GetParam() for JSON and TOML maps
Setting per-page Blackfriday angledQuotes did not work
with TOML or JSON front matter, but it does work with YAML.

It turns out that page.Params("blackfriday") returns
type map[interface{}]interface{} for YAML, but
type map[string]interface{} for JSON and TOML.

This patch updates page.GetParam() to catch the latter,
with an error message if page.GetParam() does not recognize
a type.  A test is also added.
2015-01-24 15:25:44 +01:00
Tatsushi Demachi
78e9229c52 Fix "hugo new" EOF error with an archetype file without the final EOL
This rewrites `extractFrontMatterDelims` function to make it work with
an archetype file without the final EOL and adds more detailed error
messages and comments.

It also removes `matches` and `matches_quick` functions which aren't
called anywhere.
2015-01-10 19:48:35 +01:00
bep
9f77f93071 Fix various Windows-issues
File handling was broken on Windows. This commit contains a revision of the path handling with separation of file paths and urls where needed.

There may be remaining issues and there may be better ways to do this, but it is easier to start that refactoring job with a set of passing tests.

Fixes #687
Fixes #660
2014-12-09 09:43:15 -05:00
Austin Ziegler
8f9cea7f58 Enable descriptive header IDs.
Enable blackfriday.EXTENSION_AUTO_HEADER_IDS to generate the name of the
header ID from the text in the header. Works for prefix and underline
headers.

- TOC extraction had to be modified to look for `<li><a href="#`>
  instead of `#toc_` because of this change.
- Fixed a number of tests that depended on the presence of `toc_` with
  as an `id` or as a `href` value.
- Renames the earlier parameter `footnoteref` to `documentId` as it more
  accurately represents the nature of the parameter. The `documentId` is
  appended to all generated headers through the new HTML renderer
  parameter `HeaderIDSuffix`.
2014-11-24 18:01:57 -05:00
spf13
4a2f16f91e refactor handlers to use types instead of structs. 2014-11-20 12:43:34 -05:00
bep
55fcd2f30f Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.

Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.

The new flow is:

1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes

The handling of summaries is also made simpler by this.

This commit also introduces some other chenges:

1. distinction between shortcodes that need further processing and those who do not:

* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)

The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.

2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.

The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:

* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`

Fixes #565
Fixes #480
Fixes #461

And probably some others.
2014-11-17 18:32:06 -05:00
Joel Scoble
2c51bba0c3 converted path 2 filepath 2014-11-13 22:38:21 -05:00
spf13
5dfc1dedb8 Big refactor of how source files are used. Also added default destination extension option. 2014-10-16 20:20:09 -04:00
Joel Scoble
4b9e4c90d9 #462 fix, remove leading and trailing dashes from urlized slug. includes test changes 2014-10-07 17:05:11 -04:00
Joel Scoble
4e9b04086a fix tags not being in lowercase, #491 2014-09-11 17:03:31 -04:00
spf13
640ab3c72e fixed #303. Adding theme support to views 2014-06-03 17:53:00 -04:00
Jake Mitchell
e0ea6b9955 Support embedding <script> tags in markdown pages 2014-06-03 15:14:38 -04:00
spf13
fb7d45e613 permitting empty files again. Fixing some tests to now pass. 2014-05-01 14:11:56 -04:00
Mike Keesey
2540d884d8 Fixing issues go vet reports. 2014-03-01 09:56:17 -05:00
Niels Widger
c1b9380dfd Add back blackfriday extensions during Markdown conversion
Modified markdownRender and markdownRenderWithTOC in hugolib/page.go to
use the same flags and extensions as were previously used when we simply
called blackfriday.MarkdownCommon to convert Markdown to HTML.  These
flags/extensions were dropped during the refactor that added the
`.TableOfContents` page variable, and caused features like Markdown
tables to no longer work.

Modified the expected output for TestTableOfContents in page_test.go,
apparently changing the flags/extensions caused an `&mdash;` to become
`&ndash;`.
2014-01-30 23:04:37 -05:00
Tibor Vass
6dd2e9a49a gofmt all go code 2014-01-29 18:03:35 -05:00
spf13
1882ffabc6 Adding support for boolean params 2014-01-28 23:24:59 -05:00
spf13
f45c6bc38a Moved TOC test into page_test. Some general page_test cleanup. 2014-01-28 23:09:24 -05:00
spf13
d0825a211a Big refactor of pages code. Changed TOC code to only parse when actually used 2014-01-27 17:20:20 -05:00
spf13
3851117c25 renaming MinRead to ReadingTime and adding to documentation. 2014-01-21 20:08:24 -05:00
spf13
8055838c70 Allowing empty files (ignored) so you can touch a new file while watching 2013-12-28 13:47:34 -05:00