[ic] IC process memory and CPU usage growing

Kevin Old kold at kold.homelinux.com
Thu Oct 9 14:52:28 EDT 2003


On Thu, 2003-10-09 at 08:17, Kevin Walsh wrote:
> Kevin Old [kold at kold.homelinux.com] wrote:
> > I'm running IC 4.9.7, MySQL 3.23, 512MB RAM, 1Ghz processor.  IC has
> > been running just fine, but today I noticed that the IC process is
> > roughly using 36%-99.9% of the CPU and 28M-58M of RAM.
> > 
> > I'm only running one catalog and this is not a very high traffic site,
> > although I have it configured as High in interchange.cfg.
> > 
> > I've cleaned out the tmp directory under the catalog directory and
> > expired all the sessions under that directory in the catalog directory.
> > 
> > Any ideas what could be causing this?
> > 
> > No errors in either error.log.
> > 
> > Even with all this the site seems to respond pretty fast.  I'm just
> > concerned that it'll disable my server.
> > 
> What has changed between the time the system was running ok and the
> time you noticed the high CPU usage?  For instance, have you
> upgraded Interchange, Perl or any Perl modules?  Have you modified any
> of your pages in a way that you can determine what as changed there?

Absolutely no code, IC upgrades or Perl versions or Modules have changed
in over 2 months.

> 
> What operating system, and O/S version, are you using?  Also, what
> Perl version is in use and how was it built?

RedHat 7.3, RPM version of Perl 5.6.1:

 perl -V
Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration:
  Platform:
    osname=linux, osvers=2.4.17-0.13smp, archname=i386-linux
    uname='linux daffy.perf.redhat.com 2.4.17-0.13smp #1 smp fri feb 1
10:30:48 est 2002 i686 unknown '
    config_args='-des -Doptimize=-O2 -march=i386 -mcpu=i686 -Dcc=gcc
-Dcf_by=Red Hat, Inc. -Dcccdlflags=-fPIC -Dinstallprefix=/usr
-Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr
-Dsiteprefix=/usr -Uusethreads -Uuseithreads -Uuselargefiles -Dd_dosuid
-Dd_semctl_semun -Di_db -Di_ndbm -Di_gdbm -Di_shadow -Di_syslog
-Dman3ext=3pm'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
    useperlio=undef d_sfio=undef uselargefiles=undef usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
  Compiler:
    cc='gcc', ccflags ='-fno-strict-aliasing -I/usr/local/include',
    optimize='-O2 -march=i386 -mcpu=i686',
    cppflags='-fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.2
2.96-109)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=4
    alignbytes=4, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -ldl -lm -lc -lcrypt -lutil
 perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
    libc=/lib/libc-2.2.5.so, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl): 
  Compile-time options:
  Built under linux
  Compiled at Apr  1 2002 12:23:22
  @INC:
    /usr/lib/perl5/5.6.1/i386-linux
    /usr/lib/perl5/5.6.1
    /usr/lib/perl5/site_perl/5.6.1/i386-linux
    /usr/lib/perl5/site_perl/5.6.1
    /usr/lib/perl5/site_perl/5.6.0
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.6.1/i386-linux
    /usr/lib/perl5/vendor_perl/5.6.1
    /usr/lib/perl5/vendor_perl

> 
> Does the memory usage grow as you watch the high CPU process or does
> it remain static?  You said that the memory usage was 26-58MB.  What
> is normal for your setup?
> 

As of right now the process is running perfectly normal.  Using hardly
any memory and little or no CPU.  Last night the memory and CPU usage
would fluctuate every time top refreshed.

> Have you tried running "strace -p 1234", where 1234 is the process ID,
> to determine whether the process is spinning or simply doing some
> heavy-duty work?  If so, can you post a brief section from the end of
> the trace if it indicates that the process is stuck in a tight loop.

I did run it, but not last night.  As of now, here's what a strace on
the IC process returns:


[root at ec root]# strace -p 11932
select(16, [7 8], NULL, NULL, {1, 238266}) = 0 (Timeout)
time(NULL)                              = 1065706745
open("/usr/lib/interchange/etc",
O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 2
fstat64(2, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl64(2, F_SETFD, FD_CLOEXEC)         = 0
getdents64(0x2, 0x98541e0, 0x1000, 0)   = 568
getdents64(0x2, 0x98541e0, 0x1000, 0)   = 0
close(2)                                = 0
select(16, [7 8], NULL, NULL, {2, 0})   = 0 (Timeout)
time(NULL)                              = 1065706747
open("/usr/lib/interchange/etc",
O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 2
fstat64(2, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl64(2, F_SETFD, FD_CLOEXEC)         = 0
getdents64(0x2, 0x98541e0, 0x1000, 0)   = 568
getdents64(0x2, 0x98541e0, 0x1000, 0)   = 0
close(2)                                = 0
select(16, [7 8], NULL, NULL, {2, 0})   = 0 (Timeout)
time(NULL)                              = 1065706749
open("/usr/lib/interchange/etc",
O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 2
fstat64(2, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl64(2, F_SETFD, FD_CLOEXEC)         = 0
getdents64(0x2, 0x98541e0, 0x1000, 0)   = 568
getdents64(0x2, 0x98541e0, 0x1000, 0)   = 0
close(2)                                = 0
select(16, [7 8], NULL, NULL, {2, 0})   = 1 (in [8], left {0, 134757})
select(16, [7 8], NULL, NULL, {2, 0})   = 1 (in [8], left {2, 0})
accept(8, {sin_family=AF_UNIX, path="”Œ,!"}, [2]) = 2
fcntl64(2, F_GETFL)                     = 0x2 (flags O_RDWR)
fstat64(2, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x40591000
_llseek(2, 0, 0xbfffdbc0, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
fcntl64(2, F_GETFL)                     = 0x2 (flags O_RDWR)
fstat64(2, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x40592000
_llseek(2, 0, 0xbfffdbc0, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
fcntl64(2, F_SETFD, 0)                  = 0
fork()                                  = 9086
close(2)                                = 0
munmap(0x40592000, 4096)                = 0
close(2)                                = -1 EBADF (Bad file descriptor)
munmap(0x40591000, 4096)                = 0
wait4(-1, 0xbfffdd8c, 0, NULL)          = ? ERESTARTSYS (To be
restarted)
--- SIGUSR1 (User defined signal 1) ---


> 
> Does the process settle down after a while, or does it remain using
> a lot of CPU time until it is killed?

Well if you consider "after a while" to be over night, it does.  I
didn't check it overnight, but as of this morning it's running just
fine.

> 
> Lastly, is the process killable, or does it require a "kill -9" to
> get rid of it?  If you attempt to stop Interchange and then do a "ps",
> can you see that the high-CPU process has died, or does it linger
> until it is specifically killed?

Last night I had it running as RPC mode and the 6 processes that were
spawned didn't kill nicely....I had to kill -9 them.  After the
processes died, I then switched it to High mode and the 1 process that
was started had the same high memory & CPU usage as before.

It seems as it has taken care of itself, but I'll keep an eye on it and
keep you updated.

Thanks,
Kevin
-- 
Kevin Old <kold at kold.homelinux.com>



More information about the interchange-users mailing list