Disallowing files for robots

by Susan Vollmer <susanvollmer(at)yahoo.com>

 Date:  Sun, 26 Nov 2000 14:34:06 -0800 (PST)
 To:  hwg-techniques(at)hwg.org
  todo: View Thread, Original
Does someone have a sample text file that you could
share or post a link in regard to disallowing robots
to index specific html pages?

I'm working on a future site that doesn't have hosting
yet, but I need to temporarily post it on my own site
for testing, critiques and feedback.  When you create
a robot.txt file can you disallow for an entire
directory or must each individual html file be listed?

Thank you in advance for any input you can provide.

Regards,
Susan V.
http://www.fallenmartyrs.com


__________________________________________________
Do You Yahoo!?
Yahoo! Shopping - Thousands of Stores. Millions of Products.
http://shopping.yahoo.com/

HWG hwg-techniques mailing list archives, maintained by Webmasters @ IWA