[ic] Search engine traffic

[Andrew Rich]

Hi All

IC 4.8.9

We are having a lot of search engine traffic.  Good for us.  We have not
set a robots.txt file to stop them from going in any particular page.
We did setup a robots.txt file once before (badly) but lost Google.
Google and others are back but are now at times putting a lot of
pressure on our system.

e.g. Yesterday > 7000 visits, first 12hrs today 4000 visits (info from
IC reports page).

Will the upgrade from 4.8.9 to 5.x assist with this load issue?

Some search engines add many items to the cart.  Do you think we should
prevent this or live with it? 

This traffic is also eating up disk space with sessions also.  I think
we are deleting after 2 days currently.

Any suggestions welcome.

Regards
Andrew Rich
Maps Downunder