2c0d1ccdcd
73f355ce Update theme 83ff50c2 Use example.com in examples 71292134 Add alias news > release-notes 2e15f642 Update theme 8eef09d2 Add Pygments configuration 572b9e75 Clean up the code shortcode use a1b2fd3b Remove the code fence language codes 1473b1d9 Remove redundant text b92c2042 Update theme 8f439c28 Edit contributing section in README 8bcf8a19 Add contributing section to README 4c44ee1c Fix broken content file 2bdc7710 Clarify .Data.Pages sorting in lists.md 092271c2 Use infinitive mood for main titles b9b8abef Update theme to reflect change to home page content b897b71b Change copy to use sentence case fd675ee5 Enable RSS feed for sections 060a5e27 Correct movie title in taxonomies.md 6a5ca96a Update displayed site name for Hub 22f4b7a4 Add example of starting up the local server d9612cb3 Update theme a8c3988a Update theme 4198189d Update theme 12d6b016 Update theme 2b1c4197 Update theme b6d90a1e Fix News release titles cfe751db Add some build info to README git-subtree-dir: docs git-subtree-split: 73f355ce0dd88d032062ea70067431ab980cdd8d
1.6 KiB
| title | linktitle | description | date | publishdate | lastmod | categories | menu | weight | sections_weight | draft | aliases | toc | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Robots.txt File | Robots.txt | Hugo can generate a customized robots.txt in the same way as any other template. | 2017-02-01 | 2017-02-01 | 2017-02-01 |
|
|
165 | 165 | false |
|
false |
To create your robots.txt as a template, first set the enableRobotsTXT value to true in your configuration file. By default, this option generates a robots.txt with the following content, which tells search engines that they are allowed to crawl everything:
User-agent: *
Robots.txt Template Lookup Order
The lookup order for the robots.txt template is as follows:
/layouts/robots.txt/themes/<THEME>/layout/robots.txt
{{% note %}}
If you do not want Hugo to create a default robots.txt or leverage the robots.txt template, you can hand code your own and place the file in static. Remember that everything in the static directory is copied over as-is when Hugo builds your site.
{{% /note %}}
Robots.txt Template Example
The following is an example robots.txt layout:
{{< code file="layouts/robots.txt" download="robots.txt" >}} User-agent: *
{{range .Data.Pages}} Disallow: {{.RelPermalink}} {{end}} {{< /code >}}
This template disallows all the pages of the site by creating one Disallow entry for each page.