Tuesday, March 9, 2010

SWIG, Python, and stopping C++ objects from being destroyed when they fall out of scope.

I recently had an issue with SWIG and Python. I had wrapped two C++ classes, lets call them class A and class B. Class B takes as an argument to it's constructor a pointer to an instance of class A and saves it in a member variable for later use.

EG C++ code:

class A {

class B {
    B(A* a) { this->a = a; }
    A* a;

Now in Python I do this:

def make_B():
    a = A()
    b = B(a)
    return b

b = make_B()
print b.a # garbage, or segfault

The problem is that Python doesn't know that b is holding a reference to a, and it destroys a when it falls out of scope.

I spent hours digging through the SWIG documentation trying to find an elegant way to inform SWIG that it should increment the reference count on a when I call the b constructor, to no avail. Finally I came up with a hack that works pretty well. I used the %feature("pythonappend") mechanism to extend the B proxy class constructor so that it saves a reference to the a it's passed. The code in the .i file looks like this

%feature("pythonappend") B(A*) %{
    self._ref_to_a = args[0] # a refcount +=1

which modifies the proxy class constructor thus:

  def __init__(self, *args): 
      this = _mymodule.new_B(*args)
      try: self.this.append(this)
      except: self.this = this
+     ref_to_a = args[0] # a refcount +=1

Now when a B is created it's proxy object keeps a reference to the A it's passed, causing python to increment the A's reference count, preventing it from being destroyed when it falls out of scope. When the B proxy object is destroyed, the refcount on A will be decremented, making it available for garbage collection as well.

You can make this a bit more convenient by defining a SWIG macro

%define %SAVE_ARGS_REF(function, argnum)
%feature("pythonappend") function %{
        self.__saved_ref_ ## argnum = args[ argnum ] 


The only potential issue I can see with this approach is that it clutters the namespace; it's up to the programmer not to clobber anything.

It's also not particularly elegant; it would be nice if SWIG offered a simple typemap or decorator or something to indicate that a particular object saves a reference to another object for the duration of it's life.

Note: I've confirmed this technique works, but I haven't tested the specific code in this post, cut and pasters beware.

No comments: