[ic] Setting high RobotLimit ... where to set it?

[John Young]

I'm writing to correct something I posted a few days ago.

> Bryan D Gmyrek asked:
>> The documentation is a bit confusing to me.  It says:
>> "The RobotLimit directive defines the number of consecutive pages
>> a user session may access without a 30 second pause. If the limit
>> is exceeded, the command defined in the Global directive
>> LockoutCommand will be executed and catalog URLs will be
>> rewritten with host 127.0.0.1, sending the robot back to itself.
>> The default is 0, disabling the check."
>> So does it mean that you could access 200 pages as long as there
>> is a 30 second pause between each access or does it mean you can
>> access 200 pages as long as there is a total of a 30 second pause
>> when you sum all pauses up.

To which I somewhat erroneously replied:
> The number of pages accessed within a 30 second time period.
> In a normal user session, IC maintains a count of page accesses.
> You load your store's index.html, and accesses++.  Load the
> results page, and accesses++.  Load a flypage, accesses++.
> If accesses within the last 30 seconds > RobotLimit, write
> a warning to the error log, perform the lockout command if
> configured to do so, etc.

The part about accesses incrementing is correct.  The rest isn't.
The documentation and the log message IC outputs when the RobotLimit
is tripped are correct.

If 30 seconds have elapsed since the last visit in a particular
session, the access counter is reset to 0.  Otherwise, it increments
by 1.  If the access counter is then greater than your RobotLimit,
an error message is generated and a lockout command is executed per
your configuration.  I imagine one reason it is done this way is
to avoid the overhead of a more elaborate history/time-averaging scheme.

So, yes, you could access 200 pages, as long as 30-second pauses
were taken at intervals every RobotLimit-number-of-pages (or more
frequently, obviously).


-John Young