# ------------------------------------------ # Robots Exclusion File (for help see: http://www.robotstxt.org/wc/exclusion.html) # # A sub-domain is considered as an entirely different site and should be treated as such by placing the robots.txt file in the root of the subdomain. # # An asterisk * can be used to identify collections of entries (i.e. /projects/client*.htm). # Multiple *'s may be included in any line. # # A dollar sign $ can be used to control access to a specific directory. # For example, Disallow: /project/text$ will disable access to the file /projects/text but will still allow access to /projects/text.doc and /projects/text/home.htm # The entry must exactly match for this to be effective. # ------------------------------------------ User-agent: * Disallow: