[ic] New Robot UAs

[Grant]

>>> Everyone,
>>>
>>> I've been going through Apache data lately and have discovered at
>>> least two new robots
>>> that ought to be added to robots.cfg:
>>>
>>> 1) 'bingbot'
>>>
>>> http://www.bing.com/toolbox/blogs/webmaster/archive/2010/06/28/bing-crawler-bingbot-on-the-horizon.aspx
>>>
>>> Looks like Microsoft has changed the UA for MSNBot, but Interchange is
>>> no longer recognizing it.
>>>
>>
>> It should still recognize it, as the simple term "bot" is in the RobotUA
>> list.
>> Neverless I added it.
>>
>> Thanks
>>        Racke
>
> SurveyBot is also not triggering the spider detection.  Maybe the
> "bot" entry only works if it appears after a delimiter in the UA?
>
> - Grant

Also AdsBot of "AdsBot-Google-Mobile
(+http://www.google.com/mobile/adsbot.html)".

- Grant