ba45da9d03
32356e4e Fix typo in header of shortcode-templates.md c8f1a2d2 Correct code example for index template function bfa6a55d Escape code fencing ff8b2f99 Fix typos in deployment with wercker tutorial 557c36e8 theme: Merge commit '7fbb4bed25001182bfeb91f79db0f0c1936582ee' 7fbb4bed Squashed 'themes/gohugoioTheme/' changes from 7dd8a302..ca53082d ce31cee0 Add "See Also" config 158cee1b Make the tags into keywords 61600be6 Add a note to the related section 49edb5a2 Relase 0.27.1 c9bbc001 releaser: Add release notes to /docs for release of 0.27.1 213c6c3b Add bugs poster 8b4590cd Add KeyCDN integration tutorial 2b277859 Add tutorial videos to several docs pages 950fef1f Update roadmap to link to the correct milestones page 496f5bf6 Rename relnotes d6f9378d Bump Netlify versions to 0.27 087fde7f Update 0.27 release notes 603f94ae docs: Document Related Content 3790f6a3 releaser: Bump versions for release of 0.27 0948868c releaser: Add release notes to /docs for release of 0.27 git-subtree-dir: docs git-subtree-split: 32356e4eabe357ae914f4d1d59e8ae31ce936723
1.6 KiB
| title | linktitle | description | date | publishdate | lastmod | categories | keywords | menu | weight | sections_weight | draft | aliases | toc | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Robots.txt File | Robots.txt | Hugo can generate a customized robots.txt in the same way as any other template. | 2017-02-01 | 2017-02-01 | 2017-02-01 |
|
|
|
165 | 165 | false |
|
false |
To create your robots.txt as a template, first set the enableRobotsTXT value to true in your configuration file. By default, this option generates a robots.txt with the following content, which tells search engines that they are allowed to crawl everything:
User-agent: *
Robots.txt Template Lookup Order
The lookup order for the robots.txt template is as follows:
/layouts/robots.txt/themes/<THEME>/layout/robots.txt
{{% note %}}
If you do not want Hugo to create a default robots.txt or leverage the robots.txt template, you can hand code your own and place the file in static. Remember that everything in the static directory is copied over as-is when Hugo builds your site.
{{% /note %}}
Robots.txt Template Example
The following is an example robots.txt layout:
{{< code file="layouts/robots.txt" download="robots.txt" >}} User-agent: *
{{range .Data.Pages}} Disallow: {{.RelPermalink}} {{end}} {{< /code >}}
This template disallows all the pages of the site by creating one Disallow entry for each page.