[ic] Where is robots.cfg included?
Grant
emailgrant at gmail.com
Fri Mar 5 17:57:07 UTC 2010
>>>> I'd like to switch from my manually-built RobotsUA definition to the
>>>> IC robots.cfg file. I thought it would be included from catalog.cfg,
>>>> but I don't see a reference to it in the latest version. Where is
>>>> robots.cfg called from?
>>>>
>>> It's global, see:
>>>
>>> http://github.com/interchange/interchange/blob/master/dist/interchange.cfg.d
>>> ist
>>>
>>> # Include individual usertags.
>>> include usertag/*.tag
>>>
>>> # Include the robot recognition lists
>>> include robots.cfg
>>>
>>> # Include the list of country-specific standardised subdomains
>>> include subdomains.cfg
>>
>> Thank you Gert. Has anyone run into problems relying on robots.cfg?
>
> I've never had problems. Others have occasionally had problems with
> things such as recently the ask toolbar in a browser being confused for
> the ask.com spider bot (note this has been fixed in robots.cfg now).
>
>
> Peter
Yeah it seems like everybody wants a piece of the UA.
- Grant
More information about the interchange-users
mailing list