[ic] caching search results, academic discussion
interchange-users@icdevgroup.org
interchange-users@icdevgroup.org
Thu Jan 30 23:19:01 2003
On Thu, Jan 30, 2003 at 10:46:47PM -0500, Mike Heins wrote:
> Quoting Jeff Dafoe (jeff@badtz-maru.com):
> > Hello,
> >
> > I have noticed that, on my system, most of the load is generated by
> > product scans. It seems to be the IO issue created when dozens of people
> > hit the first page of the search results at once and the system has to scan
> > the products and write the resultant match list out to the session
> > directory. I suspect it is the writing of the results that creates the
> > highest load, as write operations are typically much slower than read
> > operations. When pondering this, I realized that all of my users are doing
> > the same searches (via the category_vertical links) but each search is
> > resulting in redundant scans and writes. I think there may be some
> > opportunity for caching here. I am thinking it might be possible to create
> > one copy of the results that were returned for a given search instead of
> > each user having their own copy. This global result list could be updated
> > at certain intervals, much like the timed-build tag.
> > This is assuming I haven't missed some way to cache results page that
> > currently exists, which is not impossible.
>
> [timed-build ...] with an embedded search-region would be the way to
> go.
>
> [timed-build file="timed/cat.[cgi name=cat filter=filesafe]" minutes=60]
>
> [search-region arg="
> sf=category
> se=[cgi cat]
> "]
>
> ...
>
> [/search-region]
> [/timed-build]
>
Bear in mind that one does not have to do this inside IC but can use an
external daemon/process of whatever sort to rebuild these caches on a periodic
(or otherwise) basis. It doesn't even have to be on the same machine.
There are COUNTLESS opportunities to cache or pre-cook data in ic.
--
Christopher F. Miller, Publisher cfm@maine.com
MaineStreet Communications, Inc 208 Portland Road, Gray, ME 04039
1.207.657.5078 http://www.maine.com/
Content/site management, online commerce, internet integration, Debian linux