Re:

by "Leland V. Lammert" <lvl(at)omnitec.net>

Date:	Tue, 28 Mar 2000 11:05:08 -0600
To:	Molly <Molly_Brewer(at)brown.edu>, hwg-techniques(at)hwg.org
In-Reply-To:
	todo: View Thread, Original

At 09:04 AM 3/28/00 , Molly wrote:
>Hi there,  I'm a newbie, first posting!
>
>Anyone out there willing to copy/paste the meta tags needed to tell a search engine not to index a specific page?
>
>Thanks!

Hi Molly,

You don't restrict search engines with meta tags,.. you do it with a robots.txt file. here is a sample (should be in your site root directory):

#Robots.txt 
User-agent:     * #Any robot
Disallow: /applets/         # JAVA
Disallow: /buttons/         # Site buttons
Disallow: /images/          # Images
Disallow: /Library/         # Library components
Disallow: /images/          # Images
#       Disallow Alexa agent
User-agent: ia_archiver

It's up to the search engine to 'obey' the robots file, . . but most do. Notice the exclusion that we included for Alexa, based on a conversation here last year.

        Lee
============================================
    Leland V. Lammert                                lvl(at)omnitec.net
       Chief Scientist                         Omnitec Corporation
   Network/Internet Consultants              www.omnitec.net
============================================

HWG hwg-techniques mailing list archives, maintained by Webmasters @ IWA