User-agent: * Disallow: /cgi-bin/ Disallow: /js/ Disallow: /inc/ Disallow: /tlog/ Disallow: /translation/ Disallow: /menu/ Disallow: /inc/ Disallow: /phrases/ Disallow: /validate/ User-agent: psbot Disallow: / User-agent: Microsoft URL Control Disallow: all User-agent: Offline Explorer Disallow: all User-agent: linkwalker Disallow: / User-agent: spidersoft Disallow: / User-agent: linksmanager Disallow: / User-agent: pavuk Disallow: / User-Agent: sitecheck.internetseer.com Disallow: /