[ic] caching search results, academic discussion

[interchange-users@icdevgroup.org]

On Thu, Jan 30, 2003 at 10:46:47PM -0500, Mike Heins wrote:
> Quoting Jeff Dafoe (jeff@badtz-maru.com):
> > Hello,
> > 
> >     I have noticed that, on my system, most of the load is generated by
> > product scans.  It seems to be the IO issue created when dozens of people
> > hit the first page of the search results at once and the system has to scan
> > the products and write the resultant match list out to the session
> > directory.  I suspect it is the writing of the results that creates the
> > highest load, as write operations are typically much slower than read
> > operations.  When pondering this, I realized that all of my users are doing
> > the same searches (via the category_vertical links) but each search is
> > resulting in redundant scans and writes.  I think there may be some
> > opportunity for caching here.  I am thinking it might be possible to create
> > one copy of the results that were returned for a given search instead of
> > each user having their own copy.  This global result list could be updated
> > at certain intervals, much like the timed-build tag.
> >     This is assuming I haven't missed some way to cache results page that
> > currently exists, which is not impossible.
> 
> [timed-build ...] with an embedded search-region would be the way to
> go.
> 
> 	[timed-build file="timed/cat.[cgi name=cat filter=filesafe]" minutes=60]
> 
> 	[search-region arg="
> 				sf=category
> 				se=[cgi cat]
> 			"]
> 
> 	...
> 
> 	[/search-region]
> 	[/timed-build]
> 

Bear in mind that one does not have to do this inside IC but can use an
external daemon/process of whatever sort to rebuild these caches on a periodic 
(or otherwise) basis.  It doesn't even have to be on the same machine.

There are COUNTLESS opportunities to cache or pre-cook data in ic.





-- 

Christopher F. Miller, Publisher                               cfm@maine.com
MaineStreet Communications, Inc           208 Portland Road, Gray, ME  04039
1.207.657.5078                                         http://www.maine.com/
Content/site management, online commerce, internet integration, Debian linux