A site I look after uses St. George's Webpay/IPG API to process credit cards. It was working perfectly happily until the Ruby version was upgraded to the newest 1.8.7. I didn't fancy down-grading it because running multiple copies of Ruby had made me run into library clobbering issues and mass failure (which had prompted the upgrade) so I set about investigating it over the course of a long, tedious evening.
The Webpay interface was basically a SWIG binding around a simple C wrapper that calls a binary library using dlopen. I suspected a Ruby version mismatch was at fault so I recompiled the binding, then upgraded SWIG to the latest version and tried again, but this didn't work. It looked like something in Rails was breaking the types that were being sent to SWIG and this caused it to fail to negotiate an SSL session with the bank (possibly because the certificate name wasn't being passed to the library, or something like that). Tracing with ltrace only caused segmentation faults.
The most interesting thing to note was that when I reduced the card processing code to a test program and ran it, it worked fine. It was only when I called it through a controller action that it failed. Further tests revealed that loading bits of the Rails framework from an IRB session also made it fail. I tried to load bits of the framework selectively, but without much luck as it seemed difficult to not load things like ActiveSupport, which is what I suspected was causing the problem.
Upgrading the Rails framework from the elderly one in use to 1.2.6 (still elderly, but probably a lot more recent and in the same 1.x track) did not help.
The solution was elegant and horrible: move the processing functions to a dedicated server application and call it via DRb.
This turned out to be very easy! The card functions had been ported from PHP by the original developer and were contained in dedicated functions that didn't touch ActiveRecord so they were able to be moved holus bolus to a new file. Using DRb is as simple as adding
require 'drb' DRb.start_service webpay = DRbObject.new nil, "drbunix://sock/location/webpay.sock"
The call to BankClass.process_card was then replaced with
result = webpay.process_card(params)
The server side is similarly easy, but I added some daemonizing and PID file management code from http://snippets.dzone.com/posts/show/2265 so that I could control and monitor the server with monit.
Not quite as satisfying as fixing the root of the problem, but it only took me five hours to do this and made me appreciate DRb's elegance.
Beware the combination of SWIG and Rails, as the latter monkeypatches everything you love and cherish. While the bank's API is nice enough, I would much prefer to use one that defines a nice HTTP interface - all self respecting HTTPS client implementations should be able to handle client certificates, and I can't see any security advantage in doing anything else.