>I may have missed it in the archives, but I didn't see any topics on which
>sections of a document a robot or spider or other Web traveller might examine
>to determine content information.
>
>I suppose that this sort of information could be contained in the <META> tag,
>like
>
><META HTTP-EQUIV="Keywords" CONTENT="Audio/Video Equipment">
><META HTTP-EQUIV="Keywords" CONTENT="Laserdiscs sold here!">
>
>but is this indeed what robots look for besides (I guess) the first few lines
>of the document itself?
>
>If there isn't a specific indexing method that is standard for all search
>engines/robots/crawlers, shouldn't there be one?
There is a mailing list at [email protected] (put subscribe in the
subject to subscribe) examining some of these issues as they relate to bots
which will go "shopping" on the web. There probably is a practical limit to the
amount of info which can be transferred along with the page before it slows the
page down. There is also an issue of human interface (which a web page is
designed to be) and a bot interface (which is designed to transfer info and
doesn't care about humans). Maybe a standard tag to a file or page with the bot
information would work.
-------------------------------------------------------------------------
! Frank Atkinson, DP Manager ! 614-228-3113 Voice !
! Hannah News Service ( Rotunda Inc. ) ! 614-228-5897 Fax !
! 16 E. Broad St. ! [email protected] !
! Columbus, Ohio 43215 ! [email protected] !
! http://han1.hannah.com/frank/frank.html !
! The next generation !
! in shopping - AGENTS http://han1.hannah.com/frank/autoshopper.htm !
-------------------------------------------------------------------------