github/diceware
1
0
Fork 0
mirror of https://github.com/dmuth/diceware.git synced 2024-11-21 00:16:29 -05:00
Code Issues Projects Releases Packages Wiki Activity

Added robots.txt

Browse source
This commit is contained in:
Douglas Muth 2018-05-12 12:43:04 -04:00
parent 451536a9bb
commit cb8d301c7e
1 changed files with 62 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

62
robots.txt Normal file
View file

@ -0,0 +1,62 @@
#
# robots.txt
#
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
#
# This file will be ignored unless it is at the root of your host:
# Used: http://example.com/robots.txt
# Ignored: http://example.com/site/robots.txt
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/wc/robots.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html
User-agent: *
Crawl-delay: 10
#
# From http://www.gotfuturama.com/Multimedia/AsciiArt/dispatch.cgi?target=Bender1
#
# QR
# #t(@
# @R
# @S
# RSQ
# @K@
# #KQ7
# RSQ@
# @%GC(7Q/
# #@K%C6QKCCC6@
# (@%CC3%C3CC%CCCC6Q
# GCCC%C3CC66CCCsCCC@
# sQ3C6CCCC%CCC3CCCCC%C7
# @6%CCCCCOC6CsCCC66O%s@
# GsCCC36CC6CC6%%CC%CC%@
# GC36C3s%CCCCsCC%C%C%CsG
# sCCC666CCCCC6%CC%CCsRG%#7
# C6%C63CRQGC3C77%7(t((t((7Q^
# sC3CC@C((t((7QG#K#@@@@@@@@G7Q
# S6CCO#(t((@%/%O@@@@@@O(//R@@sC
# QsCC6st(%@ CG @@s@
# @OCCC%((%R/C@ RC @@7@
# @OO3s@(((@/ (R/ @#R(
# @s3CCCQC(t%%GQRQQQ##QGCGC((7R
# GGCCCsCCCG36SGQKSSSGsss
# Gs6C6CCCCCCCCCCs6CC6CC3
# /@CCC3Cs3GSQQQ#QRGQS@@Q
# @3%C@@(@///%///((((
# @CCOR/(@(O%O%%((((((
# GCCs@%(@///%////(((C
# CCCCG@(@%%CQ%%%%O(/#C
# sC3C%CCGGQQ#QQGGG6CC#@
# @((%@#3CC3%C3CC%CCCCsQC((Q7
# #Qt((t((7%Q#R@QK@Q#QG37((t((7@
# %3((t(((((((t((t((t((t(tt((t(t((t@
# %7t((t((t((((tt((((((((t(tt(t((t(((#C
#