[ic] New Robot UAs

[Stefan Hornburg (Racke)]

On 01/15/2011 08:42 PM, Grant wrote:
>>>> Everyone,
>>>>
>>>> I've been going through Apache data lately and have discovered at
>>>> least two new robots
>>>> that ought to be added to robots.cfg:
>>>>
>>>> 1) 'bingbot'
>>>>
>>>> http://www.bing.com/toolbox/blogs/webmaster/archive/2010/06/28/bing-crawler-bingbot-on-the-horizon.aspx
>>>>
>>>> Looks like Microsoft has changed the UA for MSNBot, but Interchange is
>>>> no longer recognizing it.
>>>>
>>>
>>> It should still recognize it, as the simple term "bot" is in the RobotUA
>>> list.
>>> Neverless I added it.
>>>
>>> Thanks
>>>         Racke
>>
>> SurveyBot is also not triggering the spider detection.  Maybe the
>> "bot" entry only works if it appears after a delimiter in the UA?
>>
>> - Grant
>
> Also AdsBot of "AdsBot-Google-Mobile
> (+http://www.google.com/mobile/adsbot.html)".

That works for me (no session, $Session->{spider} == 1) with current
Interchange from Git.

Regards
	Racke


-- 
LinuXia Systems => http://www.linuxia.de/
Expert Interchange Consulting and System Administration
ICDEVGROUP => http://www.icdevgroup.org/
Interchange Development Team