[ic] Inktomi/Yahoo Search Engine Results include Session ID's

[Gary Norton]

I was wondering if anyone else was experiencing any problems with this as
well.

To illustrate, if you go to yahoo and search for "Toyota lift kits"
(http://search.yahoo.com/search?p=toyota+lift+kits&ei=UTF-8&fr=fp-tab-web-t&
n=20&fl=0&x=wrt)

And look at the current #2 listing (suspensionconnection.com) you will
notice that the session id has been indexed.

If you go even further and click "More pages from this site"
(http://search.yahoo.com/search?p=toyota+lift+kits&ei=UTF-8&n=20&fl=0&fr=fp-
tab-web-t&vst=0&vs=www.suspensionconnection.com)

It will display the "TOP 20 WEB RESULTS out of about 15,700". All together
this site should have less than 3000 pages.  If you look at many of the
links you can find the same page listed several times with a different
session ID.

This is bad as it potentially sets the site up to be flagged as "spamming"
content.

I am using the default RobotUA, RobotHost, RobotIP that is used with IC5.0
but recently added the following IP (66.196.65.37) after searching the log:

66.196.65.37 - - [20/Feb/2004:10:51:36 -0700] "GET /robots.txt HTTP/1.0" 200
7910 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp;
http://help.yahoo.com/help/us/ysearch/slurp)"

I am wondering if anyone has any idea why this is happening (I have not
noticed it in the past). I don't know if this is related to the
Yahoo/Inktomi changes or not.

Anybody have any ideas?

TIA,
Gary Norton
broadGap Technologies
http://www.broadgap.com