# All robots will spider the domain User-agent: * Disallow: /cgi-bin/ Disallow: /_notes/ Disallow: /fish/ Disallow: /graphics/ Disallow: /images/ Disallow: /logs/ Disallow: /reference/ Disallow: /textfiles/ Disallow: /pics/ Disallow: /data/ Disallow: /logs/ Disallow: /menus/ Disallow: /charts/ Disallow: /reference/ Disallow: /serverchart/ Disallow: /tables/ User-agent: googlebot Disallow: *.zip