# robots.txt -- Prevent robot access # The following directives apply to all robots User-agent: * Disallow: /cgi-bin/ Disallow: /test/ # These are restrictions for webflyer.com Disallow: /ap/ Disallow: /dealwatchdownload/ Disallow: /fg/ Disallow: /ft/ Disallow: /pf/ Disallow: /wf/ # The following directives apply to specified robots User-agent: Webclipping.com Disallow: