robots.txt
by "Durwood Edwards" <durwoode(at)mindspring.com>
|
| Date: |
Fri, 15 Jun 2001 09:45:08 -0500 |
| To: |
<hwg-techniques(at)hwg.org> |
| |
todo: View
Thread,
Original
|
|
In a robots.txt file, which of the following examples would be correct to
disallow all robots from indexing the indicated directories:
User-agent: *
Disallow: /applets/
User-agent: *
Disallow: /issamples
User-agent: *
Disallow: /srchadm
or
User-agent: *
Disallow: /applets/
Disallow: /issamples
Disallow: /srchadm
Thanks,
Durwood Edwards
HWG hwg-techniques mailing list archives,
maintained by Webmasters @ IWA