[Date Prev][Date Next][Thread Prev][Thread Next][Minivend by date
][Minivend by thread
]
Re: [mv] problems w/ MV 3.14-3
****** message to minivend-users from "kyle@invisio.com" <kyle@invisio.com> ******
Brian,
As far as keeping the session db under control, set the session
expire lower (I use 2 hours) and run expire via a crontab every
hour (it runs quickly if your session db is relatively small) and
keeps it small.
Also set up a crontab to compact the db once a night.
Here are my crontab entries used on a site that serves 2,000 unique
visitors per day (MV 3.11):
1 4 * * * /path/mvend/bin/compact /usr/home/USER/catalogs/SIMPLE
5 * * * * /path/mvend/bin/expire -c SIMPLE
I set compact for 4:01 am (but really it is quick and can be run
any time)
expire runs at :05 after each hour.
I have been running this setup for several months now, and
I never encounter problems associated with expire or compact
(they run quickly) and the session db stays under a quarter M
(250,000 bytes) most of the time!
Kyle (KC)
At 11:02 AM 3/2/00 -0500, you wrote:
>****** message to minivend-users from "Brian D. Davison"
><brian@davison.net> ******
>
>I've been running minivend for years, but haven't done much new minivend
>coding so I've been off this list for quite a while. But I've searched
>the archives and docs and can't seem to find some answers I need.
>
>Most of these issues have bothered me for a long time, and perhaps
>others will learn from some of the workarounds I've tried.
>
>Session growing uncontrollably:
>
> We're currently serving some 18k requests per day through minivend.
> I expire the catalog (using expire -r -c catalog) every day at a
> low-usage time period (which of course means people outside of the US
> have problems), because most days I get error messages in
> my http logs of failed requests that say that access to a minivend
> url failed for reason: Premature end of script headers. So this
> appears to be a problem with the expire process locking out regular
> requests. I give the expire 40 minutes to run, and then I
> automatically restart MV because sometimes it never recovers on its own.
> While all this is very annoying, we've lived with it because I haven't
> had time to dig deep enough to figure it out. But what really becomes
> a problem is that the expire doesn't seem to be helping -- over time
> (say 6-12 months) the session db grows to be huge (say >200MB) and
> at that size, minivend requests take much longer to complete and the
> machine gets much slower under the increased load. My not so good
> solution is to just delete the session and retire_id bd files
> when they get too big, but I really don't like that approach since
> it loses all current data. [I use a SaveExpire of 7 days, and leave
> SessionExpire unset (default of 1 day).]
>
>Cached pages:
>
> I'm sure this is a simple one. I use ClearCache Yes, PageCache Yes,
> ScratchDir cache, and SearchCache Yes. And when I restart MV, I
> get messages in my error log saying that .../cache/SearchCache
> and .../cache/PageCache have been cleared, except that those
> directories are always empty. The cache files are actually in
> .../cache and it never gets cleared unless I do it manually.
> The subdirectories are owned by the minivend user (although I see that
> the permissions are 700, which is unusual but shouldn't be a problem).
>
>Hung/stuck processes:
>
> Periodically, I'll find MV hung, with a single process using more than
> a few seconds of CPU time (say a few minutes). During this time, no
> other MV process is functioning, so all activity is halted. Most of
> the time the MV housekeeping will find this job and kill it, but
> sometimes it fails in a way that the job is no longer there but the
> other MV processes are not accepting new requests, and so I have to
> manually kill them all off and restart MV.
>
> What I'd really like is to keep MV from getting hung in the first
> place. When I see stuck jobs like that, sometimes I'll kill them
> manually (i.e. without waiting for the MV housekeeping), and I'll
> notice in the error log that the request was made by a robot
> (sometimes a search engine robot, sometimes a less-friendly one).
> And that these requests (made by the robot) are usually requests
> that include the session id in the URL. This is wild speculation,
> but I'm guessing that mv sometimes has a problem with 'random'
> session ids. Using robots.txt and apache deny directives are
> insufficient, as there are some robots that I want (see next item)
> and there are always new robots that I haven't filtered yet.
> Besides, they just hide the problem (that I assume is w/in MV).
>
>Can't eliminate session ids from URLs:
>
> My site has a large enough catalog (20k+) that I'm unwilling to
> create static pages for every item. So, we serve every page
> dynamically (except for those served by the cache). However,
> a large part of our traffic comes from search engines because
> we've managed to get pages from the catalog indexed. I'm familiar
> with the [set mv_no_session_id] and [set mv_no_count] and I have
> those on _every_ page, and it helps reduce the set of URL names
> for my pages that people might use. However, what I really want
> [I think] is to eliminate session ids from URLs in all cases.
> I realize this makes my catalog unusable by those who don't
> allow cookies, but we have alternate ordering methods (email, toll-free
> phone, etc.) that people can use if they want. The reason I want
> this is because search engine robots (or any robot that I know of)
> use cookies, so when they visit, all of the outgoing links get
> the session id embedded in them. Which means that periodically, I get
> search engine robots asking for a page multiple times with different
> session ids (they can't tell that it is really the same page).
> This caused me real problems a couple of days ago when Inktomi (a robot
> that I normally like because it gets us in lots of engines) did this,
> causing me loads of problems because of MV hanging (described above).
> So, if I can avoid generating URLs with session ids, I can eliminate
> multiple retrievals, and some of the stuck jobs from robots.
>
>OK, have I ranted enough? I'm sure I've got more questions, but these are
>the ones that have been a real pain. I'd really like pointers, or patches
>that would help these issues. Does MV4 fix these things?
>
>While I'm using MV3.14-3, these problems are _not_ specific to it because
>I've lived with them through multiple versions. I'm currently running
>under a Redhat 5.0 server, with perl 5.004_04 and GDBM db files, but I'm
>planning downtime this weekend to upgrade to RH 6.1 (for other reasons).
>
>-- Brian
>
>Brian D. Davison - brian@davison.net (732) 249-6492
>=============================================================================
>Brian's Books - online computer books catalog - http://www.briansbooks.com/
>============= WEB CACHING RESOURCES http://www.web-caching.com/ =============
>
>
>
>
>-
>To unsubscribe from the list, DO NOT REPLY to this message. Instead, send
>email with 'UNSUBSCRIBE minivend-users' in the body to Majordomo@minivend.com.
>Archive of past messages: http://www.minivend.com/minivend/minivend-list
-
To unsubscribe from the list, DO NOT REPLY to this message. Instead, send
email with 'UNSUBSCRIBE minivend-users' in the body to Majordomo@minivend.com.
Archive of past messages: http://www.minivend.com/minivend/minivend-list
