Sunday, March 28, 2010

How to compile the Apache QPID C++ broker from SVN without checking out the Java broker.

I was trying to check out and build the Apache Qpid C++ broker from source. The instructions say to check out  http://svn.apache.org/repos/asf/qpid/branches/0.6-release/qpid/.  But that Java broker dir is HUGE and I don't event WANT the Java broker!

So at first I just checked out the cpp dir. Why not? Well the build breaks in a strange way, something about cannot execute rubygen.mk.

make[1]: execvp: ./rubygen.mk: Permission denied 

It turns out that there is a dependency on the specs dir. But autoconf doesn't tell you about it. So you must check out BOTH dirs to a common dir, then build the cpp broker.

eg:

mkdir qpid
cd qpid
svn co http://svn.apache.org/repos/asf/qpid/branches/0.6-release/qpid/cpp cpp
svn co http://svn.apache.org/repos/asf/qpid/branches/0.6-release/qpid/specs specs
cd cpp
./bootstrap
./configure
make
make check
make install


That should do it.

Tuesday, March 9, 2010

SWIG, Python, and stopping C++ objects from being destroyed when they fall out of scope.

I recently had an issue with SWIG and Python. I had wrapped two C++ classes, lets call them class A and class B. Class B takes as an argument to it's constructor a pointer to an instance of class A and saves it in a member variable for later use.

EG C++ code:

class A {
    void;
};

class B {
  public:
    B(A* a) { this->a = a; }
    A* a;
};


Now in Python I do this:


def make_B():
    a = A()
    b = B(a)
    return b

b = make_B()
print b.a # garbage, or segfault


The problem is that Python doesn't know that b is holding a reference to a, and it destroys a when it falls out of scope.

I spent hours digging through the SWIG documentation trying to find an elegant way to inform SWIG that it should increment the reference count on a when I call the b constructor, to no avail. Finally I came up with a hack that works pretty well. I used the %feature("pythonappend") mechanism to extend the B proxy class constructor so that it saves a reference to the a it's passed. The code in the .i file looks like this


%feature("pythonappend") B(A*) %{
    self._ref_to_a = args[0] # a refcount +=1
%}


which modifies the proxy class constructor thus:


  def __init__(self, *args): 
      this = _mymodule.new_B(*args)
      try: self.this.append(this)
      except: self.this = this
+     ref_to_a = args[0] # a refcount +=1


Now when a B is created it's proxy object keeps a reference to the A it's passed, causing python to increment the A's reference count, preventing it from being destroyed when it falls out of scope. When the B proxy object is destroyed, the refcount on A will be decremented, making it available for garbage collection as well.

You can make this a bit more convenient by defining a SWIG macro


%define %SAVE_ARGS_REF(function, argnum)
%feature("pythonappend") function %{
        self.__saved_ref_ ## argnum = args[ argnum ] 
%}
%enddef

%SAVE_ARGS_REF( B(A*), 0)

The only potential issue I can see with this approach is that it clutters the namespace; it's up to the programmer not to clobber anything.

It's also not particularly elegant; it would be nice if SWIG offered a simple typemap or decorator or something to indicate that a particular object saves a reference to another object for the duration of it's life.

Note: I've confirmed this technique works, but I haven't tested the specific code in this post, cut and pasters beware.