[interchange-cvs] interchange - kwalsh modified dist/robots.cfg
interchange-cvs at icdevgroup.org
interchange-cvs at icdevgroup.org
Thu Aug 9 07:11:48 EDT 2007
User: kwalsh
Date: 2007-08-09 11:11:47 GMT
Modified: dist robots.cfg
Log:
* Twiceler experimental web crawler.
* Various Wikis, such as ShopWiki.
Revision Changes Path
2.6 +2 -0 interchange/dist/robots.cfg
rev 2.6, prev_rev 2.5
Index: robots.cfg
===================================================================
RCS file: /var/cvs/interchange/dist/robots.cfg,v
retrieving revision 2.5
retrieving revision 2.6
diff -u -r2.5 -r2.6
--- robots.cfg 11 Jul 2007 19:34:41 -0000 2.5
+++ robots.cfg 9 Aug 2007 11:11:47 -0000 2.6
@@ -123,6 +123,7 @@
Toutatis,
TurnitinBot,
Tv*Merc,
+ Twiceler,
urllib,
VB Project,
Valkyrie,
@@ -132,6 +133,7 @@
wget,
WhizBang,
whowhere,
+ Wiki,
WinInet,
winona,
Wire,
More information about the interchange-cvs
mailing list