mirror of
https://github.com/gohugoio/hugo.git
synced 2024-09-20 01:15:13 -04:00
1,003 B
1,003 B
| lastmod | date | menu | next | prev | title | weight | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| 2016-01-06 | 2015-12-08 |
|
/community/mailing-list | /extras/urls | Custom robots.txt | 120 |
Hugo can generated a customized robots.txt in the [same way as any other templates]({{< ref "templates/go-templates.md" >}}).
By default, it generates a robots.txt, which allows everything, with the following content:
User-agent: *
To disable it, just set disableRobotsTXT option to false in the [command line]({{< ref "commands/hugo.md" >}}) or [configuration file]({{< ref "overview/configuration.md" >}}).
Hugo will use the template robots.txt according to the following list in descending precedence:
- /layouts/robots.txt
- /themes/
THEME/layout/robots.txt
An example of a robots.txt layout is:
User-agent: *
{{range .Data.Pages}}
Disallow: {{.RelPermalink}}{{end}}
This template disallows and all the pages of the site creating one Disallow entry for each one.