# Disallow all crawlers access to certain pages. User-agent: * Disallow: /code/ # these will soon disappear Disallow: /http500.asp Disallow: /online/add/ Disallow: /online/item/ Disallow: /online/checkout/ Disallow: /online/vinyl/ User-Agent: MediaPartners-Google Allow: /online/ Allow: /txtpage-ac.asp Allow: /txtpage-cd.asp Allow: /txtpage-ge.asp