[ic] Rolling big tables (mysql)

[Kevin Walsh]

Grant <emailgrant at gmail.com> wrote:
> I do keep a separate table of robot UAs and match traffic rows to them
> with op=eq to populate another table with robot IPs and non-robot IPs
> for the day to speed up the report.  Don't you think it would be
> slower to match/no-match each IC request to a known robot UA and write
> to the traffic table based on that, instead of unconditionally writing
> all requests to the traffic table?  If not, excluding the robot
> requests from the traffic table would mean a lot less processing for
> the report and a lot fewer records for the traffic table.
> 
Perhaps you should create a column called "spider" in the traffic table
and save a true or false value depending upon the [data session spider]
value.  You can then generate reports "WHERE spider = 0", for ordinary
users, or "WHERE spider = 1" for robots etc.  An index on the spider column
would be nice, of course.

Then again, I wouldn't save traffic data to a table anyway.  I'd use
usertrack and/or the apache access_log for that.  There are lots of tools
that will allow you to analyse Apache log files.  You can even save some
Interchange usertrack info into a custom Apache access_log file.

See also:

    http://www.interchange.rtfm.info/icdocs/tags/data.html
    http://www.interchange.rtfm.info/icdocs/config/TrackFile.html
    http://www.interchange.rtfm.info/icdocs/config/TrackPageParam.html

-- 
   _/   _/  _/_/_/_/  _/    _/  _/_/_/  _/    _/
  _/_/_/   _/_/      _/    _/    _/    _/_/  _/   K e v i n   W a l s h
 _/ _/    _/          _/ _/     _/    _/  _/_/    kevin at cursor.biz
_/   _/  _/_/_/_/      _/    _/_/_/  _/    _/