User-agent: * Disallow: /~teddy/home/ Disallow: /doc/ Disallow: /~per/crux/cvs.cgi Disallow: /~devil/ # From Wikipedia’s robots.txt: # # Friendly, low-speed bots are welcome viewing article pages, but not # dynamically-generated pages please. # # Inktomi's "Slurp" can read a minimum delay between hits; if your # bot supports such a thing using the 'Crawl-delay' or another # instruction, please let us know. # User-agent: * Disallow: /w/ Disallow: /trap/ Disallow: /wiki/Special:Random Disallow: /wiki/Special%3ARandom Disallow: /wiki/Special:Search Disallow: /wiki/Special%3ASearch