[ic] Authnet charges without orders

[Jon Jensen]

On Tue, 21 Jun 2011, Paul Jordan wrote:

> I realized the sites perl was updated 10 minutes before the problems 
> started to occur. The update was a minor one and remained in the 5.8.8 
> family I think - I didn't personally do it, and from what I understand, 
> it was not a yum style update.

If it wasn't a packaged update with careful attention to compatibility, 
it's possible some compile options changed, or different core module 
versions were included, so that could've been the source of the problem.

> I believe IC is running in LOW, because I have not specific TRAFFIC. I 
> have been reading fiddling with TRAFFIC might help as well - for 
> example, RPC mode appears to have helped others. I notice no 
> similarities with the failed transactions.

What you're experiencing sounds like the old Perl signals problem which 
comes up on the list every once in a while. It affects calls to 
subprocesses (sendmail, wget) or anything else relying on signals (LWP).

You can reproduce it on your own by creating an ITL page with a loop that 
sends an email a few hundred times, then write a shell script to hit that 
page repeatedly and run it 4 or so times concurrently. You'll see errors 
that you don't see when there's less load on the Interchange daemon.

There are two changes you need to make, which can be seen in this commit I 
made back in 2006:

http://git.icdevgroup.org/?p=interchange.git;a=commitdiff;h=e1c6f77db1974777645d4120924ad01c6279b605

You must have PERL_SIGNALS=unsafe in the environment, and MaxServers 0. 
Make sure to change it in the right server stanza for your traffic mode! 
Or just change it in all of them. (There's little difference between 
so-called low and high traffic modes, so no need to worry about that in 
this case.) Then restart Interchange.

Whenever I've seen this problem, once I make the changes, I cannot make a 
single email fail to send no matter how many I send, how fast.

In reality the MaxServers setting does not work very well to stop a 
runaway server anyway, since both Apache and CGI processes stack up 
whenever there's serious load. So I don't think you're giving up much 
and in exchange your email, LWP, and wget problems stop.

HTH,
Jon

-- 
Jon Jensen
End Point Corporation
http://www.endpoint.com/