Boost Python and GCC fvisibility option

I'm working on a robot project related to collision detection recently. The work is based on the code of Trajopt and we use python to do some mix programming.

The main framework is written in Python and we need to call some C++ function in the middle, which in turn needs to call some Python functions as subroutine.

The process is so difficult as described on stackoverflow.

In C++, declare a Boost Python Object and a register function

#include <boost/python.hpp>
#include <iostream>
#include <Bar.h>
namespace bp = boost::python;
using namespace std;
bp::object foo;

void register_foo(bp::object f)
{
  foo = f;
}

BOOST_PYTHON_MODULE(testmod)
{
  bp::def("register_foo", &register_foo, "Register the callback from Python");
}

In Python, register the callback function.

import testmod
def callback(val):
    print "I'm in Python!", "Argument = ", val
    return val
testmod.register_foo(callback)

Then in C++, you can write a call back function as follows.

void test_foo()
{
  double arg = 3;
  double rt = bp::extract<double>(foo(arg));
}

The problem occurred when I want to call the foo function in other C++ cpp file. If you have any experience in Boost.Python, it won't surprise you that there is one cpp file that is compiled and linked to generate the dynamic library to be imported in Python. Suppose the file is named A.cpp and my callback function foo is registered in A.cpp as well.

It works like a charm when I call foo inside A.cpp only. But once I call it in B.cpp, the Python interpreter crashes complaining that foo is Nonetype thus not callable.

I really hope it's a compilation error which is easier to ask Google but it's a runtime one with little helpful information. After checking all the materials I can find on web, I surrendered and preparing to ask on fabulous stackoverflow. To ask the problem, I need to prepare a minimal example that won't work. However, when I actually prepared one, it worked! So I guess there maybe something wrong with the compilation process and checking the CMakeLists.txt in trajopt line by line. Finally I found one line that is suspicious:

if (NOT APPLE)
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=hidden") 
endif()

By commenting out this line, the problem is solved.

I'm rather curious about the reason behind since it wastes me so much time.

The reason is that dynamic library on Linux (*.so files) is different with those (*.dll) on Windows where *.so files exports all the symbols by default. This greatly increases the import time when importing a large module generated by Boost.Python. So GCC provide the -fvisibility option to hide all symbols by default so my foo variable can't be accessed in B.cpp file.

Reader may argue that I need to consider the symbol exporting reason since I'm using a dynamic library instead of static. Oh, I never export my own code as dynamic libraries but always link them as static :)