github/hugo
1
0
Fork 0
mirror of https://github.com/gohugoio/hugo.git synced 2024-11-07 20:30:36 -05:00
Code Issues Projects Releases Packages Wiki Activity
hugo/publisher/htmlElementsCollector_test.go

271 lines
9.2 KiB
Go
Raw Normal View History

Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
// Copyright 2020 The Hugo Authors. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package publisher
import (
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
"bytes"
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
"fmt"
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
"io"
"math/rand"
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
"strings"
"testing"
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
"time"
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
Create a struct with all of Hugo's config options Primary motivation is documentation, but it will also hopefully simplify the code. Also, * Lower case the default output format names; this is in line with the custom ones (map keys) and how it's treated all the places. This avoids doing `stringds.EqualFold` everywhere. Closes #10896 Closes #10620
2023-01-04 12:24:36 -05:00
"github.com/gohugoio/hugo/config/testconfig"
publisher: Also test minified HTML in the element collector Updates #7567
2021-04-07 02:33:31 -04:00
"github.com/gohugoio/hugo/media"
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
"github.com/gohugoio/hugo/minifiers"
publisher: Also test minified HTML in the element collector Updates #7567
2021-04-07 02:33:31 -04:00
"github.com/gohugoio/hugo/output"
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
qt "github.com/frankban/quicktest"
)
func TestClassCollector(t *testing.T) {
c := qt.New((t))
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
rnd := rand.New(rand.NewSource(time.Now().Unix()))
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
f := func(tags, classes, ids string) HTMLElements {
var tagss, classess, idss []string
if tags != "" {
tagss = strings.Split(tags, " ")
}
if classes != "" {
classess = strings.Split(classes, " ")
}
if ids != "" {
idss = strings.Split(ids, " ")
}
return HTMLElements{
Tags: tagss,
Classes: classess,
IDs: idss,
}
}
publisher: Also test minified HTML in the element collector Updates #7567
2021-04-07 02:33:31 -04:00
skipMinifyTest := map[string]bool{
"Script tags content should be skipped": true, // https://github.com/tdewolff/minify/issues/396
}
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
for _, test := range []struct {
name string
html string
expect HTMLElements
}{
{"basic", `<body class="b a"></body>`, f("body", "a b", "")},
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
{"duplicates", `<div class="b a b"></div><div class="b a b"></div>x'`, f("div", "a b", "")},
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
{"single quote", `<body class='b a'></body>`, f("body", "a b", "")},
{"no quote", `<body class=b id=myelement></body>`, f("body", "b", "myelement")},
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
{"short", `<i>`, f("i", "", "")},
{"invalid", `< body class="b a"></body><div></div>`, f("div", "", "")},
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
// https://github.com/gohugoio/hugo/issues/7318
{"thead", `<table class="cl1">
publisher: Fix tag collector for nested table elements Fixes #7318
2020-05-25 15:05:59 -04:00
<thead class="cl2"><tr class="cl3"><td class="cl4"></td></tr></thead>
<tbody class="cl5"><tr class="cl6"><td class="cl7"></td></tr></tbody>
</table>`, f("table tbody td thead tr", "cl1 cl2 cl3 cl4 cl5 cl6 cl7", "")},
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
{"thead uppercase", `<TABLE class="CL1">
<THEAD class="CL2"><TR class="CL3"><TD class="CL4"></TD></TR></THEAD>
<TBODY class="CL5"><TR class="CL6"><TD class="CL7"></TD></TR></TBODY>
</TABLE>`, f("table tbody td thead tr", "CL1 CL2 CL3 CL4 CL5 CL6 CL7", "")},
Fix class collector when running with --minify Also add a related stresstest. Fixes #7161
2020-04-21 06:57:45 -04:00
// https://github.com/gohugoio/hugo/issues/7161
{"minified a href", `<a class="b a" href=/></a>`, f("a", "a b", "")},
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
{"AlpineJS bind 1", `<body>
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
<div x-bind:class="{
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
'class1': data.open,
'class2 class3': data.foo == 'bar'
}">
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
</div>
</body>`, f("body div", "class1 class2 class3", "")},
{"AlpineJS bind 2", `<div x-bind:class="{ 'bg-black': filter.checked }" class="inline-block mr-1 mb-2 rounded bg-gray-300 px-2 py-2">FOO</div>`,
all: Format code with gofumpt See https://github.com/mvdan/gofumpt
2020-12-02 07:23:25 -05:00
f("div", "bg-black bg-gray-300 inline-block mb-2 mr-1 px-2 py-2 rounded", ""),
},
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
{"AlpineJS bind 3", `<div x-bind:class="{ 'text-gray-800': !checked, 'text-white': checked }"></div>`, f("div", "text-gray-800 text-white", "")},
{"AlpineJS bind 4", `<div x-bind:class="{ 'text-gray-800': !checked,
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
'text-white': checked }"></div>`, f("div", "text-gray-800 text-white", "")},
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
{"AlpineJS bind 5", `<a x-bind:class="{
Fix some missing JS class collector cases Fixes #7216
2020-04-27 11:49:51 -04:00
'text-a': a && b,
'text-b': !a && b || c,
'pl-3': a === 1,
pl-2: b == 3,
'text-gray-600': (a > 1)
}" class="block w-36 cursor-pointer pr-3 no-underline capitalize"></a>`, f("a", "block capitalize cursor-pointer no-underline pl-2 pl-3 pr-3 text-a text-b text-gray-600 w-36", "")},
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
{"AlpineJS transition 1", `<div x-transition:enter-start="opacity-0 transform mobile:-translate-x-8 sm:-translate-y-8">`, f("div", "mobile:-translate-x-8 opacity-0 sm:-translate-y-8 transform", "")},
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
{"Vue bind", `<div v-bind:class="{ active: isActive }"></div>`, f("div", "active", "")},
publisher: Skip script, pre and textarea content when looking for HTML elements Updates #7567
2021-04-06 12:19:25 -04:00
// Issue #7746
publisher: Fix writeStats with quote inside quotes Fixes #7746
2020-09-28 16:17:36 -04:00
{"Apostrophe inside attribute value", `<a class="missingclass" title="Plus d'information">my text</a><div></div>`, f("a div", "missingclass", "")},
publisher: Skip script, pre and textarea content when looking for HTML elements Updates #7567
2021-04-06 12:19:25 -04:00
// Issue #7567
{"Script tags content should be skipped", `<script><span>foo</span><span>bar</span></script><div class="foo"></div>`, f("div script", "foo", "")},
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
{"Style tags content should be skipped", `<style>p{color: red;font-size: 20px;}</style><div class="foo"></div>`, f("div style", "foo", "")},
publisher: Skip script, pre and textarea content when looking for HTML elements Updates #7567
2021-04-06 12:19:25 -04:00
{"Pre tags content should be skipped", `<pre class="preclass"><span>foo</span><span>bar</span></pre><div class="foo"></div>`, f("div pre", "foo preclass", "")},
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
{"Textarea tags content should be skipped", `<textarea class="textareaclass"><span>foo</span><span>bar</span></textarea><div class="foo"></div>`, f("div textarea", "foo textareaclass", "")},
{"DOCTYPE should beskipped", `<!DOCTYPE html>`, f("", "", "")},
{"Comments should be skipped", `<!-- example comment -->`, f("", "", "")},
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
{"Comments with elements before and after", `<div></div><!-- example comment --><span><span>`, f("div span", "", "")},
Make the HTML collector parsing more robust Most notably better handling self-closing elements Closes #10698
2023-02-06 11:29:12 -05:00
{"Self closing tag", `<div><hr/></div>`, f("div hr", "", "")},
// svg with self closing style tag.
{"SVG with self closing style tag", `<svg><style/><g><path class="foo"/></g></svg>`, f("g path style svg", "foo", "")},
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
// Issue #8530
{"Comment with single quote", `<!-- Hero Area Image d'accueil --><i class="foo">`, f("i", "foo", "")},
{"Uppercase tags", `<DIV></DIV>`, f("div", "", "")},
{"Predefined tags with distinct casing", `<script>if (a < b) { nothing(); }</SCRIPT><div></div>`, f("div script", "", "")},
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
// Issue #8417
{"Tabs inline", `<hr id="a" class="foo"><div class="bar">d</div>`, f("div hr", "bar foo", "a")},
{"Tabs on multiple rows", `<form
id="a"
action="www.example.com"
method="post"
></form>
<div id="b" class="foo">d</div>`, f("div form", "foo", "a b")},
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
{"Big input, multibyte runes", strings.Repeat(`神真美好 `, rnd.Intn(500)+1) + "<div id=\"神真美好\" class=\"foo\">" + strings.Repeat(`神真美好 `, rnd.Intn(100)+1) + " <span>神真美好</span>", f("div span", "foo", "神真美好")},
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
} {
publisher: Also test minified HTML in the element collector Updates #7567
2021-04-07 02:33:31 -04:00
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
for _, variant := range []struct {
minify bool
}{
{minify: false},
{minify: true},
} {
c.Run(fmt.Sprintf("%s--minify-%t", test.name, variant.minify), func(c *qt.C) {
publisher: Also test minified HTML in the element collector Updates #7567
2021-04-07 02:33:31 -04:00
w := newHTMLElementsCollectorWriter(newHTMLElementsCollector())
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
if variant.minify {
publisher: Also test minified HTML in the element collector Updates #7567
2021-04-07 02:33:31 -04:00
if skipMinifyTest[test.name] {
c.Skip("skip minify test")
}
Create a struct with all of Hugo's config options Primary motivation is documentation, but it will also hopefully simplify the code. Also, * Lower case the default output format names; this is in line with the custom ones (map keys) and how it's treated all the places. This avoids doing `stringds.EqualFold` everywhere. Closes #10896 Closes #10620
2023-01-04 12:24:36 -05:00
m, _ := minifiers.New(media.DefaultTypes, output.DefaultFormats, testconfig.GetTestConfig(nil, nil))
m.Minify(media.Builtin.HTMLType, w, strings.NewReader(test.html))
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
publisher: Also test minified HTML in the element collector Updates #7567
2021-04-07 02:33:31 -04:00
} else {
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
var buff bytes.Buffer
buff.WriteString(test.html)
io.Copy(w, &buff)
publisher: Also test minified HTML in the element collector Updates #7567
2021-04-07 02:33:31 -04:00
}
got := w.collector.getHTMLElements()
c.Assert(got, qt.DeepEquals, test.expect)
})
}
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
}
publisher: Make the HTML element collector more robust Fixes #8530
2021-05-13 07:10:32 -04:00
Collect HTML elements during the build to use in PurgeCSS etc. The main use case for this is to use with resources.PostProcess and resources.PostCSS with purgecss. You would normally set it up to extract keywords from your templates, doing it from the full /public takes forever for bigger sites. Doing the template thing misses dynamically created class names etc., and it's hard/impossible to set up in when using themes. You can enable this in your site config: ```toml [build] writeStats = true ``` It will then write a `hugo_stats.json` file to the project root as part of the build. If you're only using this for the production build, you should consider putting it below `config/production`. You can then set it up with PostCSS like this: ```js const purgecss = require('@fullhuman/postcss-purgecss')({ content: [ './hugo_stats.json' ], defaultExtractor: (content) => { let els = JSON.parse(content).htmlElements; return els.tags.concat(els.classes, els.ids); } }); module.exports = { plugins: [ require('tailwindcss'), require('autoprefixer'), ...(process.env.HUGO_ENVIRONMENT === 'production' ? [ purgecss ] : []) ] }; ``` Fixes #6999
2020-03-03 06:25:03 -05:00
}
publisher: Fix memory usage in writeStats ``` name old time/op new time/op delta ClassCollectorWriter-16 72.1µs ± 0% 32.3µs ± 0% -55.17% (p=0.029 n=4+4) name old alloc/op new alloc/op delta ClassCollectorWriter-16 85.9kB ± 0% 35.1kB ± 0% -59.16% (p=0.029 n=4+4) name old allocs/op new allocs/op delta ClassCollectorWriter-16 329 ± 0% 149 ± 0% -54.71% (p=0.029 n=4+4) ``` Fixes #7945
2020-11-27 02:46:58 -05:00
Fix slow HTML elements collector for the pre case ``` name old time/op new time/op delta ElementsCollectorWriterPre-10 25.2µs ± 1% 3.4µs ± 0% -86.54% (p=0.029 n=4+4) name old alloc/op new alloc/op delta ElementsCollectorWriterPre-10 624B ± 0% 142B ± 0% -77.18% (p=0.029 n=4+4) name old allocs/op new allocs/op delta ElementsCollectorWriterPre-10 16.0 ± 0% 6.0 ± 0% -62.50% (p=0.029 n=4+4) ``` Fixes #10698
2023-02-05 09:14:30 -05:00
func TestEndsWithTag(t *testing.T) {
c := qt.New((t))
for _, test := range []struct {
name string
s string
tagName string
expect bool
}{
{"empty", "", "div", false},
{"no match", "foo", "div", false},
{"no close", "foo<div>", "div", false},
{"no close 2", "foo/div>", "div", false},
{"no close 2", "foo//div>", "div", false},
{"no tag", "foo</>", "div", false},
{"match", "foo</div>", "div", true},
{"match space", "foo< / div>", "div", true},
{"match space 2", "foo< / div \n>", "div", true},
{"match case", "foo</DIV>", "div", true},
Make the HTML collector parsing more robust Most notably better handling self-closing elements Closes #10698
2023-02-06 11:29:12 -05:00
{"self closing", `</defs><g><g><path fill="#010101" d=asdf"/>`, "div", false},
Fix slow HTML elements collector for the pre case ``` name old time/op new time/op delta ElementsCollectorWriterPre-10 25.2µs ± 1% 3.4µs ± 0% -86.54% (p=0.029 n=4+4) name old alloc/op new alloc/op delta ElementsCollectorWriterPre-10 624B ± 0% 142B ± 0% -77.18% (p=0.029 n=4+4) name old allocs/op new allocs/op delta ElementsCollectorWriterPre-10 16.0 ± 0% 6.0 ± 0% -62.50% (p=0.029 n=4+4) ``` Fixes #10698
2023-02-05 09:14:30 -05:00
} {
c.Run(test.name, func(c *qt.C) {
got := isClosedByTag([]byte(test.s), []byte(test.tagName))
c.Assert(got, qt.Equals, test.expect)
})
}
}
publisher: Some performance tweaks for the HTML elements collector
2021-04-20 10:50:03 -04:00
func BenchmarkElementsCollectorWriter(b *testing.B) {
publisher: Fix memory usage in writeStats ``` name old time/op new time/op delta ClassCollectorWriter-16 72.1µs ± 0% 32.3µs ± 0% -55.17% (p=0.029 n=4+4) name old alloc/op new alloc/op delta ClassCollectorWriter-16 85.9kB ± 0% 35.1kB ± 0% -59.16% (p=0.029 n=4+4) name old allocs/op new allocs/op delta ClassCollectorWriter-16 329 ± 0% 149 ± 0% -54.71% (p=0.029 n=4+4) ``` Fixes #7945
2020-11-27 02:46:58 -05:00
const benchHTML = `
publisher: Exclude comment and doctype elements from writeStats - Reorder code blocks - Rename cssClassCollectorWriter to htmlElementCollectorWriter, as it just collect html element information - Expand benchmark to test for minified and unminified content Fixes #8396 Fixes #8417
2021-04-12 17:42:51 -04:00
<!DOCTYPE html>
<html>
<head>
<title>title</title>
<style>
a {color: red;}
.c {color: blue;}
</style>
</head>
<body id="i1" class="a b c d">
<a class="c d e"></a>
<hr>
<a class="c d e"></a>
<a class="c d e"></a>
<hr>
<a id="i2" class="c d e f"></a>
<a id="i3" class="c d e"></a>
<a class="c d e"></a>
<p>To force<br> line breaks<br> in a text,<br> use the br<br> element.</p>
<hr>
<a class="c d e"></a>
<a class="c d e"></a>
<a class="c d e"></a>
<a class="c d e"></a>
<table>
<thead class="ch">
<tr>
<th>Month</th>
<th>Savings</th>
</tr>
</thead>
<tbody class="cb">
<tr>
<td>January</td>
<td>$100</td>
</tr>
<tr>
<td>February</td>
<td>$200</td>
</tr>
</tbody>
<tfoot class="cf">
<tr>
<td></td>
<td>$300</td>
</tr>
</tfoot>
</table>
</body>
</html>
`
for i := 0; i < b.N; i++ {
w := newHTMLElementsCollectorWriter(newHTMLElementsCollector())
fmt.Fprint(w, benchHTML)
publisher: Fix memory usage in writeStats ``` name old time/op new time/op delta ClassCollectorWriter-16 72.1µs ± 0% 32.3µs ± 0% -55.17% (p=0.029 n=4+4) name old alloc/op new alloc/op delta ClassCollectorWriter-16 85.9kB ± 0% 35.1kB ± 0% -59.16% (p=0.029 n=4+4) name old allocs/op new allocs/op delta ClassCollectorWriter-16 329 ± 0% 149 ± 0% -54.71% (p=0.029 n=4+4) ``` Fixes #7945
2020-11-27 02:46:58 -05:00
}
}
publisher: Add benchmark
2023-02-05 10:39:31 -05:00
func BenchmarkElementsCollectorWriterPre(b *testing.B) {
const benchHTML = `
<pre class="preclass">
<span>foo</span><span>bar</span>
<!-- many more span elements -->
<span class="foo">foo</span>
<span class="bar">bar</span>
<span class="baz">baz</span>
<span class="qux">qux</span>
<span class="quux">quux</span>
<span class="quuz">quuz</span>
<span class="corge">corge</span>
</pre>
<div class="foo"></div>
`
w := newHTMLElementsCollectorWriter(newHTMLElementsCollector())
for i := 0; i < b.N; i++ {
fmt.Fprint(w, benchHTML)
}
}
Reference in a new issue Copy permalink