Re: Question on Robots.txt

Hello Elizabeth,

My understanding is the same as what you described
that the robots.txt file is used more to disallow
directories or to give full open access to search
engines and spiders depending on the content of the
text file.

For my own web site, I place different key words on 
each web page in the site.  Some of the words are the
same but my story pages will vary based on the
country, who the person was and what the person was
involved in.  This allows certain directories or
search engines (especially Yahoo and Google) to find
the specific page that will help the user.  Most
people only want to see the page with the information
they were looking for.  They don't necessarily want to
start at the beginning of the site and try to find it.

Regards,
Susan V.
http://www.fallenmartyrs.com



--- "Davies, Elizabeth H." <EHDavies(at)West.com> wrote:
> Has anyone ever seen or used the ROBOTS.TXT file as
> a place to put keywords
> or other META tags?
> 
> Reason: my company is having a ecommerce site
> builder being built for them
> by another company. The current product did not
> provide any area for META
> tags for search engines (this was a requirement). I
> pointed this out to my
> boss and they replied to him that they would have a
> robots.txt file where
> all keywords entered will be placed. I don't think
> that this will replace
> having regular META description and keyword tags.
> 
> I've only seen robots.txt used for exclusion. I have
> NEVER seen them replace
> the META tags. Also, non-robot search engines don't
> even touch that file. I
> know that the ability to be listed by Yahoo is
> pretty darned important to
> our clients. 
> 
> I've only been with this company for 3 months and
> previous to this was
> designing brochure-ware sites for non-profits and
> personal pages for 4
> years. I consider myself an intermediate web
> designer, not an expert by any
> stretch of the imagination. Even though I haven't
> found ANYWHERE on the web
> that puts this as a use for robots.txt.... maybe one
> of you has.
> 
> Is this a viable method? or are they simply blowing
> smoke. 
> 
> Elizabeth Davies
> West Interactive
> Systems Development
> Web Designer
> (402)573-3386 (West6: X206-7562) 


__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail - only $35 
a year!  http://personal.mail.yahoo.com/

HWG hwg-techniques mailing list archives, maintained by Webmasters @ IWA