2018-10-17 06:57:09 -04:00
// Copyright 2018 The Hugo Authors. All rights reserved.
2015-12-10 17:19:38 -05:00
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
2018-10-17 06:57:09 -04:00
package pageparser
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
2020-01-14 00:36:33 -05:00
import (
"testing"
)
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
var (
2019-09-29 08:51:51 -04:00
tstEOF = nti ( tEOF , "" )
tstLeftNoMD = nti ( tLeftDelimScNoMarkup , "{{<" )
tstRightNoMD = nti ( tRightDelimScNoMarkup , ">}}" )
tstLeftMD = nti ( tLeftDelimScWithMarkup , "{{%" )
tstRightMD = nti ( tRightDelimScWithMarkup , "%}}" )
tstSCClose = nti ( tScClose , "/" )
tstSC1 = nti ( tScName , "sc1" )
tstSC1Inline = nti ( tScNameInline , "sc1.inline" )
tstSC2Inline = nti ( tScNameInline , "sc2.inline" )
tstSC2 = nti ( tScName , "sc2" )
tstSC3 = nti ( tScName , "sc3" )
tstSCSlash = nti ( tScName , "sc/sub" )
tstParam1 = nti ( tScParam , "param1" )
tstParam2 = nti ( tScParam , "param2" )
tstParamBoolTrue = nti ( tScParam , "true" )
tstParamBoolFalse = nti ( tScParam , "false" )
tstParamInt = nti ( tScParam , "32" )
tstParamFloat = nti ( tScParam , "3.14" )
tstVal = nti ( tScParamVal , "Hello World" )
tstText = nti ( tText , "Hello World" )
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
)
2018-10-17 07:48:55 -04:00
var shortCodeLexerTests = [ ] lexerTest {
2018-10-17 06:57:09 -04:00
{ "empty" , "" , [ ] Item { tstEOF } } ,
2018-10-18 03:04:48 -04:00
{ "spaces" , " \t\n" , [ ] Item { nti ( tText , " \t\n" ) , tstEOF } } ,
{ "text" , ` to be or not ` , [ ] Item { nti ( tText , "to be or not" ) , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "no markup" , ` {{ < sc1 > }} ` , [ ] Item { tstLeftNoMD , tstSC1 , tstRightNoMD , tstEOF } } ,
{ "with EOL" , "{{< sc1 \n >}}" , [ ] Item { tstLeftNoMD , tstSC1 , tstRightNoMD , tstEOF } } ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
2018-10-17 06:57:09 -04:00
{ "forward slash inside name" , ` {{ < sc / sub > }} ` , [ ] Item { tstLeftNoMD , tstSCSlash , tstRightNoMD , tstEOF } } ,
2018-06-28 10:06:16 -04:00
2018-10-17 06:57:09 -04:00
{ "simple with markup" , ` {{ % sc1 % }} ` , [ ] Item { tstLeftMD , tstSC1 , tstRightMD , tstEOF } } ,
{ "with spaces" , ` {{ < sc1 > }} ` , [ ] Item { tstLeftNoMD , tstSC1 , tstRightNoMD , tstEOF } } ,
{ "mismatched rightDelim" , ` {{ < sc1 % }} ` , [ ] Item { tstLeftNoMD , tstSC1 ,
2018-10-18 03:04:48 -04:00
nti ( tError , "unrecognized character in shortcode action: U+0025 '%'. Note: Parameters with non-alphanumeric args must be quoted" ) } } ,
2018-10-17 06:57:09 -04:00
{ "inner, markup" , ` {{ % sc1 % }} inner {{ % / sc1 % }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftMD ,
tstSC1 ,
tstRightMD ,
2018-10-18 03:04:48 -04:00
nti ( tText , " inner " ) ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftMD ,
tstSCClose ,
tstSC1 ,
tstRightMD ,
tstEOF ,
} } ,
2018-10-17 06:57:09 -04:00
{ "close, but no open" , ` {{ < / sc1 > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
tstLeftNoMD , nti ( tError , "got closing shortcode, but none is open" ) } } ,
2018-10-17 06:57:09 -04:00
{ "close wrong" , ` {{ < sc1 > }} {{ < / another > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstRightNoMD , tstLeftNoMD , tstSCClose ,
2018-10-18 03:04:48 -04:00
nti ( tError , "closing tag for shortcode 'another' does not match start tag" ) } } ,
2018-10-17 06:57:09 -04:00
{ "close, but no open, more" , ` {{ < sc1 > }} {{ < / sc1 > }} {{ < / another > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstRightNoMD , tstLeftNoMD , tstSCClose , tstSC1 , tstRightNoMD , tstLeftNoMD , tstSCClose ,
2018-10-18 03:04:48 -04:00
nti ( tError , "closing tag for shortcode 'another' does not match start tag" ) } } ,
2018-10-17 06:57:09 -04:00
{ "close with extra keyword" , ` {{ < sc1 > }} {{ < / sc1 keyword > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstRightNoMD , tstLeftNoMD , tstSCClose , tstSC1 ,
2018-10-18 03:04:48 -04:00
nti ( tError , "unclosed shortcode" ) } } ,
2019-09-29 08:51:51 -04:00
{ "float param, positional" , ` {{ < sc1 3 .14 > }} ` , [ ] Item {
tstLeftNoMD , tstSC1 , nti ( tScParam , "3.14" ) , tstRightNoMD , tstEOF } } ,
{ "float param, named" , ` {{ < sc1 param1 = 3 .14 > }} ` , [ ] Item {
tstLeftNoMD , tstSC1 , tstParam1 , nti ( tScParamVal , "3.14" ) , tstRightNoMD , tstEOF } } ,
2020-01-14 00:36:33 -05:00
{ "named param, raw string" , ` {{ < sc1 param1 = ` + "`" + "Hello World" + "`" + " >}}" , [ ] Item {
tstLeftNoMD , tstSC1 , tstParam1 , nti ( tScParamVal , "Hello World" ) , tstRightNoMD , tstEOF } } ,
2019-09-29 08:51:51 -04:00
{ "float param, named, space before" , ` {{ < sc1 param1 = 3 .14 > }} ` , [ ] Item {
tstLeftNoMD , tstSC1 , tstParam1 , nti ( tScParamVal , "3.14" ) , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "Youtube id" , ` {{ < sc1 - ziL - Q_456igdO -4 > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
tstLeftNoMD , tstSC1 , nti ( tScParam , "-ziL-Q_456igdO-4" ) , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "non-alphanumerics param quoted" , ` {{ < sc1 "-ziL-.%QigdO-4" > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
tstLeftNoMD , tstSC1 , nti ( tScParam , "-ziL-.%QigdO-4" ) , tstRightNoMD , tstEOF } } ,
2020-01-14 00:36:33 -05:00
{ "raw string" , ` {{ < sc1 ` + "`" + "Hello World" + "`" + ` >}} ` , [ ] Item {
tstLeftNoMD , tstSC1 , nti ( tScParam , "Hello World" ) , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "two params" , ` {{ < sc1 param1 param2 > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstParam2 , tstRightNoMD , tstEOF } } ,
2015-03-02 15:23:16 -05:00
// issue #934
2018-10-17 06:57:09 -04:00
{ "self-closing" , ` {{ < sc1 / > }} ` , [ ] Item {
2015-03-02 15:23:16 -05:00
tstLeftNoMD , tstSC1 , tstSCClose , tstRightNoMD , tstEOF } } ,
2016-09-27 15:11:34 -04:00
// Issue 2498
2018-10-17 06:57:09 -04:00
{ "multiple self-closing" , ` {{ < sc1 / > }} {{ < sc1 / > }} ` , [ ] Item {
2016-09-27 15:11:34 -04:00
tstLeftNoMD , tstSC1 , tstSCClose , tstRightNoMD ,
tstLeftNoMD , tstSC1 , tstSCClose , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "self-closing with param" , ` {{ < sc1 param1 / > }} ` , [ ] Item {
2016-09-27 15:11:34 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstSCClose , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "multiple self-closing with param" , ` {{ < sc1 param1 / > }} {{ < sc1 param1 / > }} ` , [ ] Item {
2016-09-27 15:11:34 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstSCClose , tstRightNoMD ,
tstLeftNoMD , tstSC1 , tstParam1 , tstSCClose , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "multiple different self-closing with param" , ` {{ < sc1 param1 / > }} {{ < sc2 param1 / > }} ` , [ ] Item {
2016-09-27 15:11:34 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstSCClose , tstRightNoMD ,
tstLeftNoMD , tstSC2 , tstParam1 , tstSCClose , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "nested simple" , ` {{ < sc1 > }} {{ < sc2 > }} {{ < / sc1 > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstRightNoMD ,
tstLeftNoMD , tstSC2 , tstRightNoMD ,
tstLeftNoMD , tstSCClose , tstSC1 , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "nested complex" , ` {{ < sc1 > }} ab {{ % sc2 param1 % }} cd {{ < sc3 > }} ef {{ < / sc3 > }} gh {{ % / sc2 % }} ij {{ < / sc1 > }} kl ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstRightNoMD ,
2018-10-18 03:04:48 -04:00
nti ( tText , "ab" ) ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftMD , tstSC2 , tstParam1 , tstRightMD ,
2018-10-18 03:04:48 -04:00
nti ( tText , "cd" ) ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC3 , tstRightNoMD ,
2018-10-18 03:04:48 -04:00
nti ( tText , "ef" ) ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSCClose , tstSC3 , tstRightNoMD ,
2018-10-18 03:04:48 -04:00
nti ( tText , "gh" ) ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftMD , tstSCClose , tstSC2 , tstRightMD ,
2018-10-18 03:04:48 -04:00
nti ( tText , "ij" ) ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSCClose , tstSC1 , tstRightNoMD ,
2018-10-18 03:04:48 -04:00
nti ( tText , "kl" ) , tstEOF ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
} } ,
2018-10-17 06:57:09 -04:00
{ "two quoted params" , ` {{ < sc1 "param nr. 1" "param nr. 2" > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
tstLeftNoMD , tstSC1 , nti ( tScParam , "param nr. 1" ) , nti ( tScParam , "param nr. 2" ) , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "two named params" , ` {{ < sc1 param1 = "Hello World" param2 = "p2Val" > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstVal , tstParam2 , nti ( tScParamVal , "p2Val" ) , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "escaped quotes" , ` {{ < sc1 param1 = \ "Hello World\" > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstVal , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "escaped quotes, positional param" , ` {{ < sc1 \ "param1\" > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "escaped quotes inside escaped quotes" , ` {{ < sc1 param1 = \ "Hello \"escaped\" World\" > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstParam1 ,
2018-10-18 03:04:48 -04:00
nti ( tScParamVal , ` Hello ` ) , nti ( tError , ` got positional parameter 'escaped'. Cannot mix named and positional parameters ` ) } } ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
{ "escaped quotes inside nonescaped quotes" ,
2018-10-17 06:57:09 -04:00
` {{ < sc1 param1 = "Hello \"escaped\" World" > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , nti ( tScParamVal , ` Hello "escaped" World ` ) , tstRightNoMD , tstEOF } } ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
{ "escaped quotes inside nonescaped quotes in positional param" ,
2018-10-17 06:57:09 -04:00
` {{ < sc1 "Hello \"escaped\" World" > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
tstLeftNoMD , tstSC1 , nti ( tScParam , ` Hello "escaped" World ` ) , tstRightNoMD , tstEOF } } ,
2020-01-14 00:36:33 -05:00
{ "escaped raw string, named param" , ` {{ < sc1 param1 = ` + ` \ ` + "`" + "Hello World" + ` \ ` + "`" + ` >}} ` , [ ] Item {
tstLeftNoMD , tstSC1 , tstParam1 , nti ( tError , "unrecognized escape character" ) } } ,
{ "escaped raw string, positional param" , ` {{ < sc1 param1 ` + ` \ ` + "`" + "Hello World" + ` \ ` + "`" + ` >}} ` , [ ] Item {
tstLeftNoMD , tstSC1 , tstParam1 , nti ( tError , "unrecognized escape character" ) } } ,
{ "two raw string params" , ` {{ < sc1 ` + "`" + "Hello World" + "`" + "`" + "Second Param" + "`" + ` >}} ` , [ ] Item {
tstLeftNoMD , tstSC1 , nti ( tScParam , "Hello World" ) , nti ( tScParam , "Second Param" ) , tstRightNoMD , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "unterminated quote" , ` {{ < sc1 param2 = " Hello World > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
tstLeftNoMD , tstSC1 , tstParam2 , nti ( tError , "unterminated quoted string in shortcode parameter-argument: 'Hello World>}}'" ) } } ,
2020-01-14 00:36:33 -05:00
{ "unterminated raw string" , ` {{ < sc1 ` + "`" + "Hello World" + ` >}} ` , [ ] Item {
tstLeftNoMD , tstSC1 , nti ( tError , "unterminated raw string in shortcode parameter-argument: 'Hello World >}}'" ) } } ,
{ "unterminated raw string in second argument" , ` {{ < sc1 ` + "`" + "Hello World" + "`" + "`" + "Second Param" + ` >}} ` , [ ] Item {
tstLeftNoMD , tstSC1 , nti ( tScParam , "Hello World" ) , nti ( tError , "unterminated raw string in shortcode parameter-argument: 'Second Param >}}'" ) } } ,
2018-10-17 06:57:09 -04:00
{ "one named param, one not" , ` {{ < sc1 param1 = "Hello World" p2 > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstVal ,
2018-10-18 03:04:48 -04:00
nti ( tError , "got positional parameter 'p2'. Cannot mix named and positional parameters" ) } } ,
2020-01-14 00:36:33 -05:00
{ "one named param, one quoted positional param, both raw strings" , ` {{ < sc1 param1 = ` + "`" + "Hello World" + "`" + "`" + "Second Param" + "`" + ` >}} ` , [ ] Item {
tstLeftNoMD , tstSC1 , tstParam1 , tstVal ,
nti ( tError , "got quoted positional parameter. Cannot mix named and positional parameters" ) } } ,
2018-10-17 06:57:09 -04:00
{ "one named param, one quoted positional param" , ` {{ < sc1 param1 = "Hello World" "And Universe" > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstParam1 , tstVal ,
2018-10-18 03:04:48 -04:00
nti ( tError , "got quoted positional parameter. Cannot mix named and positional parameters" ) } } ,
2018-10-17 06:57:09 -04:00
{ "one quoted positional param, one named param" , ` {{ < sc1 "param1" param2 = "And Universe" > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstParam1 ,
2018-10-18 03:04:48 -04:00
nti ( tError , "got named parameter 'param2'. Cannot mix named and positional parameters" ) } } ,
2018-10-17 06:57:09 -04:00
{ "ono positional param, one not" , ` {{ < sc1 param1 param2 = "Hello World" > }} ` , [ ] Item {
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
tstLeftNoMD , tstSC1 , tstParam1 ,
2018-10-18 03:04:48 -04:00
nti ( tError , "got named parameter 'param2'. Cannot mix named and positional parameters" ) } } ,
2018-10-17 06:57:09 -04:00
{ "commented out" , ` {{ < / * sc1 * / > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
nti ( tText , "{{<" ) , nti ( tText , " sc1 " ) , nti ( tText , ">}}" ) , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "commented out, with asterisk inside" , ` {{ < / * sc1 "**/*.pdf" * / > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
nti ( tText , "{{<" ) , nti ( tText , " sc1 \"**/*.pdf\" " ) , nti ( tText , ">}}" ) , tstEOF } } ,
2018-10-17 06:57:09 -04:00
{ "commented out, missing close" , ` {{ < / * sc1 > }} ` , [ ] Item {
2018-10-18 03:04:48 -04:00
nti ( tError , "comment must be closed" ) } } ,
2018-10-17 06:57:09 -04:00
{ "commented out, misplaced close" , ` {{ < / * sc1 > }} */ ` , [ ] Item {
2018-10-18 03:04:48 -04:00
nti ( tError , "comment must be closed" ) } } ,
2018-11-26 05:01:27 -05:00
// Inline shortcodes
{ "basic inline" , ` {{ < sc1 .inline > }} Hello World {{ < / sc1 .inline > }} ` , [ ] Item { tstLeftNoMD , tstSC1Inline , tstRightNoMD , tstText , tstLeftNoMD , tstSCClose , tstSC1Inline , tstRightNoMD , tstEOF } } ,
{ "basic inline with space" , ` {{ < sc1 .inline > }} Hello World {{ < / sc1 .inline > }} ` , [ ] Item { tstLeftNoMD , tstSC1Inline , tstRightNoMD , tstText , tstLeftNoMD , tstSCClose , tstSC1Inline , tstRightNoMD , tstEOF } } ,
{ "inline self closing" , ` {{ < sc1 .inline > }} Hello World {{ < / sc1 .inline > }} Hello World {{ < sc1 .inline / > }} ` , [ ] Item { tstLeftNoMD , tstSC1Inline , tstRightNoMD , tstText , tstLeftNoMD , tstSCClose , tstSC1Inline , tstRightNoMD , tstText , tstLeftNoMD , tstSC1Inline , tstSCClose , tstRightNoMD , tstEOF } } ,
2019-01-31 05:53:21 -05:00
{ "inline self closing, then a new inline" , ` {{ < sc1 .inline > }} Hello World {{ < / sc1 .inline > }} Hello World {{ < sc1 .inline / > }} {{ < sc2 .inline > }} Hello World {{ < / sc2 .inline > }} ` , [ ] Item {
tstLeftNoMD , tstSC1Inline , tstRightNoMD , tstText , tstLeftNoMD , tstSCClose , tstSC1Inline , tstRightNoMD , tstText , tstLeftNoMD , tstSC1Inline , tstSCClose , tstRightNoMD ,
tstLeftNoMD , tstSC2Inline , tstRightNoMD , tstText , tstLeftNoMD , tstSCClose , tstSC2Inline , tstRightNoMD , tstEOF } } ,
Move the emoji parsing to pageparser
This avoids double parsing the page content when `enableEmoji=true`.
This commit also adds some general improvements to the parser, making it in general much faster:
```bash
benchmark old ns/op new ns/op delta
BenchmarkShortcodeLexer-4 90258 101730 +12.71%
BenchmarkParse-4 148940 15037 -89.90%
benchmark old allocs new allocs delta
BenchmarkShortcodeLexer-4 456 700 +53.51%
BenchmarkParse-4 28 33 +17.86%
benchmark old bytes new bytes delta
BenchmarkShortcodeLexer-4 69875 81014 +15.94%
BenchmarkParse-4 8128 8304 +2.17%
```
Running some site benchmarks with Emoji support turned on:
```bash
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 924556797 818115620 -11.51%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 4112613 4133787 +0.51%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 426982864 424363832 -0.61%
```
Fixes #5534
2018-12-17 15:03:23 -05:00
{ "inline with template syntax" , ` {{ < sc1 .inline > }} {{ .Get 0 }} {{ .Get 1 }} {{ < / sc1 .inline > }} ` , [ ] Item { tstLeftNoMD , tstSC1Inline , tstRightNoMD , nti ( tText , "{{ .Get 0 }}" ) , nti ( tText , "{{ .Get 1 }}" ) , tstLeftNoMD , tstSCClose , tstSC1Inline , tstRightNoMD , tstEOF } } ,
{ "inline with nested shortcode (not supported)" , ` {{ < sc1 .inline > }} Hello World {{ < sc1 > }} {{ < / sc1 .inline > }} ` , [ ] Item { tstLeftNoMD , tstSC1Inline , tstRightNoMD , tstText , nti ( tError , "inline shortcodes do not support nesting" ) } } ,
2018-11-26 05:01:27 -05:00
{ "inline case mismatch" , ` {{ < sc1 .Inline > }} Hello World {{ < / sc1 .Inline > }} ` , [ ] Item { tstLeftNoMD , nti ( tError , "period in shortcode name only allowed for inline identifiers" ) } } ,
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
}
2015-03-05 15:10:53 -05:00
func TestShortcodeLexer ( t * testing . T ) {
2017-02-04 22:20:06 -05:00
t . Parallel ( )
2016-09-27 15:11:34 -04:00
for i , test := range shortCodeLexerTests {
2019-01-31 05:53:21 -05:00
t . Run ( test . name , func ( t * testing . T ) {
items := collect ( [ ] byte ( test . input ) , true , lexMainSection )
if ! equal ( items , test . items ) {
t . Errorf ( "[%d] %s: got\n\t%v\nexpected\n\t%v" , i , test . name , items , test . items )
}
} )
Shortcode rewrite, take 2
This commit contains a restructuring and partial rewrite of the shortcode handling.
Prior to this commit rendering of the page content was mingled with handling of the shortcodes. This led to several oddities.
The new flow is:
1. Shortcodes are extracted from page and replaced with placeholders.
2. Shortcodes are processed and rendered
3. Page is processed
4. The placeholders are replaced with the rendered shortcodes
The handling of summaries is also made simpler by this.
This commit also introduces some other chenges:
1. distinction between shortcodes that need further processing and those who do not:
* `{{< >}}`: Typically raw HTML. Will not be processed.
* `{{% %}}`: Will be processed by the page's markup engine (Markdown or (infuture) Asciidoctor)
The above also involves a new shortcode-parser, with lexical scanning inspired by Rob Pike's talk called "Lexical Scanning in Go",
which should be easier to understand, give better error messages and perform better.
2. If you want to exclude a shortcode from being processed (for documentation etc.), the inner part of the shorcode must be commented out, i.e. `{{%/* movie 47238zzb */%}}`. See the updated shortcode section in the documentation for further examples.
The new parser supports nested shortcodes. This isn't new, but has two related design choices worth mentioning:
* The shortcodes will be rendered individually, so If both `{{< >}}` and `{{% %}}` are used in the nested hierarchy, one will be passed through the page's markdown processor, the other not.
* To avoid potential costly overhead of always looking far ahead for a possible closing tag, this implementation looks at the template itself, and is branded as a container with inner content if it contains a reference to `.Inner`
Fixes #565
Fixes #480
Fixes #461
And probably some others.
2014-10-27 16:48:30 -04:00
}
}
2015-03-05 15:10:53 -05:00
func BenchmarkShortcodeLexer ( b * testing . B ) {
2018-10-18 03:04:48 -04:00
testInputs := make ( [ ] [ ] byte , len ( shortCodeLexerTests ) )
for i , input := range shortCodeLexerTests {
testInputs [ i ] = [ ] byte ( input . input )
}
Move the emoji parsing to pageparser
This avoids double parsing the page content when `enableEmoji=true`.
This commit also adds some general improvements to the parser, making it in general much faster:
```bash
benchmark old ns/op new ns/op delta
BenchmarkShortcodeLexer-4 90258 101730 +12.71%
BenchmarkParse-4 148940 15037 -89.90%
benchmark old allocs new allocs delta
BenchmarkShortcodeLexer-4 456 700 +53.51%
BenchmarkParse-4 28 33 +17.86%
benchmark old bytes new bytes delta
BenchmarkShortcodeLexer-4 69875 81014 +15.94%
BenchmarkParse-4 8128 8304 +2.17%
```
Running some site benchmarks with Emoji support turned on:
```bash
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 924556797 818115620 -11.51%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 4112613 4133787 +0.51%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 426982864 424363832 -0.61%
```
Fixes #5534
2018-12-17 15:03:23 -05:00
var cfg Config
2015-03-05 15:10:53 -05:00
b . ResetTimer ( )
for i := 0 ; i < b . N ; i ++ {
2018-10-18 03:04:48 -04:00
for _ , input := range testInputs {
Move the emoji parsing to pageparser
This avoids double parsing the page content when `enableEmoji=true`.
This commit also adds some general improvements to the parser, making it in general much faster:
```bash
benchmark old ns/op new ns/op delta
BenchmarkShortcodeLexer-4 90258 101730 +12.71%
BenchmarkParse-4 148940 15037 -89.90%
benchmark old allocs new allocs delta
BenchmarkShortcodeLexer-4 456 700 +53.51%
BenchmarkParse-4 28 33 +17.86%
benchmark old bytes new bytes delta
BenchmarkShortcodeLexer-4 69875 81014 +15.94%
BenchmarkParse-4 8128 8304 +2.17%
```
Running some site benchmarks with Emoji support turned on:
```bash
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 924556797 818115620 -11.51%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 4112613 4133787 +0.51%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/TOML,num_langs=3,num_pages=5000,tags_per_page=5,shortcodes,render-4 426982864 424363832 -0.61%
```
Fixes #5534
2018-12-17 15:03:23 -05:00
items := collectWithConfig ( input , true , lexMainSection , cfg )
2018-10-18 03:04:48 -04:00
if len ( items ) == 0 {
2015-03-05 15:10:53 -05:00
}
2018-10-18 03:04:48 -04:00
2015-03-05 15:10:53 -05:00
}
}
}
