C/C++ modules in your package¶
Bob massively relies on a mixture between the user-friendly and easy-to-develop Python interface, and a fast implementation of identified bottlenecks using C++.
Creating C++/Python bindings should be rather straightforward. Only few adaptations need to be performed to get the C/C++ code being compiled and added as an extension. For simplicity, we created an example package that includes a simple example of a C++ extension. You can check it out by:
$ git clone https://gitlab.idiap.ch/bob/bob.extension.git $ cp -R bob.extension/bob/extension/data/bob.example.extension ./ $ rm -rf bob.extension # optionally remove the cloned source of bob.extension $ cd bob.example.extension
Setting up your package¶
Typically, Python extensions written in C/C++ for Bob should use a set of standard APIs allowing C++ Blitz++ Arrays to be transparently converted to Python NumPy Arrays.
The build of your package will therefore depend on, at least, two packages: (1) bob.extension (this package): will provide build instructions and resources for defining and building your extension (2) bob.blitz: will provide a bridge between pure C++ code, depending on Blitz++ Arrays and NumPy arrays.
To be able to import
bob.blitz in the setup.py, we need to include some code:
setup_packages = ['bob.extension', 'bob.blitz'] # C++ modules needed at runtime of your package bob_packages =  from setuptools import setup, find_packages, dist dist.Distribution(dict(setup_requires = setup_packages + bob_packages))
We keep the
bob_packages in separate variables since we will need them later.
bob_packages contain a list of bob packages that this extension directly depends on.
In our example, we only depend on
bob.blitz, and we can leave the list empty.
bob.blitz is required in all C++/Python packages since it contains all the mechanisms
to deal with arrays amongst other things.
As the second step, we need to add some lines in the header of the file to tell the
setuptools system to compile our library with our
# import the Extension class and the build_ext function from bob.blitz from bob.blitz.extension import Extension, build_ext # load the requirements.txt for additional requirements from bob.extension.utils import load_requirements build_requires = setup_packages + bob_packages + load_requirements()
In fact, we don’t use the extension from
bob.extension.Extension, but the one from
bob.blitz.extension, which is a derivation of this package.
The difference is that in
bob.blitz.extension.Extension all header files and libraries for the
Blitz++ library are added.
Third, we have to add an extension using the
Extension class, by listing all C/C++ files that should be compiled into the extension:
# read version from version.txt file version = open("version.txt").read().rstrip() setup( ... setup_requires = build_requires, install_requires = build_requires, ... ext_modules = [ Extension("bob.example.extension._module", [ # the pure C++ code "bob/example/extension/Function.cpp", # the Python bindings "bob/example/extension/main.cpp", ], version = version, bob_packages = bob_packages ), ... #add more extensions if you wish ], ... )
These modifications will allow you to compile extensions that are linked against our core Python-C++ bridge
bob.blitz (by default).
You can specify any other
pkg-config module and that will be linked in (for example,
opencv) using the
boost packages, you might need to define, which boost modules are required.
By default, when using boost you should at least add the
system module, i.e., by:
setup( ... ext_modules = [ Extension( ... packages = ['boost'], boost_modules = ['system'], ), ... ], ... )
Other modules and options can be set manually using the standard options for Python extensions.
When your module compiles and links against the pure C++ code, you can simply use the
bob_packages to specify dependencies in your C++ code.
This will automatically add the desired include and library directories, as well as the libraries and the required preprocessor options.
In our example, we have defined a small C++ function, which also shows the basic bridge between
numpy.ndarray and our C++ pendant
Basically, there are two C++ files for our extension.
bob/example/extension/Function.cpp contains the pure C++ implementation of the function.
bob/example/extension/main.cpp, we define the Python bindings to that function.
Finally, the function
reverse from the module
_library is imported into our module in the
In the bindings of the
reverse function in
bob/example/extension/main.cpp, we make use of some C++ defines that makes the life easier.
see Helper utilities
Building your package¶
To compile your C++ Python bindings and the corresponding C++ implementation, just do:
$ buildout ...
By default, we compile the source code (of this and all dependent packages, both the ones installed as
eggs, and the ones developed using
mr.developer) in debug mode.
If you want to change that, switch the according flag in the
debug = False, and the compilation will be done with optimization flags and C++ exception handling enabled.
Now, we can use the script
./bin/bob_example_extension_reverse.py (that we have registered in the
setup.py) to reverse a list of floats, using the C++ implementation of the
$ ./bin/bob_example_extension_reverse.py 1 2 3 4 5 [1.0, 2.0, 3.0, 4.0, 5.0] reversed is [ 5. 4. 3. 2. 1.]
We can also see that the function documentation has made it into the module, too:
$ ./bin/python >>> import bob.example.extension >>> help(bob.example.extension)
and that we can list version and the dependencies of our package:
>>> print (bob.example.extension.version) 0.0.1a0 >>> print (bob.example.extension.get_config()) ...
In the header file
<bob.extension/defines.h> we have added some functions that help you to keep your code short and clean.
Particularly, we provide three preprocessor directives:
Starts a try-catch block to protect your bound function against exceptions of any kinds (which would lead to a Python interpreter crash otherwise).
BOB_CATCH_FUNCTION(char* message, void* ret)¶
Catches C++ exceptions of any kind, adds the
messagein case an unknown exception is caught, and returns with the given error return (which is usually 0 for normal functions or -1 for constructors and setter functions). This macro should be used when binding a stand-alone function, for binding class member functions, please use
BOB_CATCH_MEMBER(char* message, void* ret)¶
Catches C++ exceptions of any kind, adds the
messagein case an unknown exception is caught, and returns with the given error return (which is usually 0 for normal functions or -1 for constructors and setter functions). This macro should be used when binding a member function of a class, for binding stand-alone functions, please use
These preprocessor directives will catch any C++ exception that is raised inside the C/C++ code that you bind to python and translate them into proper Python exceptions.
These directives will only be active in release mode, when compiling
debug = true, they will not do anything. This is in order to
support C++ debuggers like
gdb-python to be able to handle
Additionally, we added some preprocessor directives that help in the bindings:
Checks if the given object
ois a number, i.e., an int, a long, a float or a complex.
After including the above mentioned header, we also re-define the functions
PyString_AS_STRING() (which doesn’t exist in the bindings for Python3)
so that they can be used in bindings for both Python2 and Python3.