[ic] Unexplained Interchange crashes

[Phil Smith]

> On 23/08/22 22:35, Phil Smith wrote:
> > Hello All
> >
> > I am hoping for some advice on a problem I have.
> >
> > This Interchange server has been up working 24/7 for several years,
> > and interchange has been rock solid and never crashes.
> >
> >[ ....]
> 
> If it's crashing just since you moved the server to a new building and you
> literally changed nothing else then the issue has to be environmental, or
> hardware-related.  Something as simple as a power glitch could cause
issues
> that you did not anticipate.
> 
> That said, what you can do for now, and it's probably a good idea to do
> anyways, is set up a "watchdog" task that runs from cron.  Have it attempt
to
> fetch an IC page and if it cannot then it should restart the service (and
send
> you a notice that it did).
> 
> If you're running a Linux distribution with systemd then consider using a
> systemd .service file to start IC instead of an init.d script.  Here's one
I use,
> with an additional line I recently added to tell systemd to automatically
> restart IC on failure.  Note you can get this file by unpacking the
interchange
> package from GhettoForge.  Note also that I've changed the locations of
> many of the interchange files in this to be more appropriate to a packaged
> interchange, so you may have to change some of this stuff (or omit some of
> the *Directory directives alltogether).
> 
> See systemd documentation to see how all of this works:
> 
> # /usr/lib/systemd/system/interchange.service
> [Unit]
> Description=Interchange web application platform After=syslog.target
> network.target
> 
> [Service]
> Type=forking
> PIDFile=/var/run/interchange/interchange.pid
> EnvironmentFile=-/etc/sysconfig/network
> User=interch
> Group=interch
> ExecStart=/usr/sbin/interchange -q
> ExecStop=/usr/sbin/interchange -q --stop RuntimeDirectory=interchange
> CacheDirectory=interchange LogsDirectory=interchange
> StateDirectory=interchange ConfigurationDirectory=interchange
> 
> 
> [Install]
> WantedBy=multi-user.target
> 
> # /etc/systemd/system/interchange.service.d/override.conf
> [Service]
> Restart=on-failure
> 
> 
> 
> Peter

Thank you Peter, the problem with Environmental issues is  - this new Comms
room is 1000% better than old building - UPS backed and air conditioned. No
other service seems to be affected - just IC.

I have already set up a 'watchdog' process, it pulls the PID from
/etc/interchange/etc/interchange.pid  and periodically checks that is still
alive from `ps -p $pid`. If its not running it currently emails me to go
restart it.

I will have a look through your interchange.service above and see if I can
tweak it to fit my system (thanks).

I just wish something was logging why it has started crashing.

Thanks

Phil.