Skip to content
This repository has been archived by the owner on Jan 7, 2022. It is now read-only.

Find boost-python reliably cross linux distributions #35

Closed
skindstrom opened this issue Oct 4, 2018 · 3 comments
Closed

Find boost-python reliably cross linux distributions #35

skindstrom opened this issue Oct 4, 2018 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@skindstrom
Copy link
Contributor

This issue breaks out the second point from #31 into a separate issue.

The below text is copied from the original issue
The _boost_py_component is not correctly set when building on Fedora, causing boost-python to not be found. I was able to make it work by setting _boost_py_component to python3, but that's obviously not an acceptable solution for all distros. Not sure how to solve this one, I remember hearing something about the struggles with this versioning last summer. There seems to be a solution for boost python libs merged into CMake 3.11, but I didn't look into it too deeply.

Error message when not finding the correct boost python version.

CMake Error at /usr/share/cmake/Modules/FindBoost.cmake:1879 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.60.0

  Boost include path: /usr/include

  Could not find the following Boost libraries:

          boost_python-py35

  Some (but not all) of the required Boost libraries were found.  You may
  need to install these additional Boost libraries.  Alternatively, set
  BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
  to the location of Boost.
Call Stack (most recent call first):
  CMake/logdevice-deps.cmake:16 (find_package)
  CMakeLists.txt:34 (include)
@skindstrom
Copy link
Contributor Author

While playing around with the Fedora docker containers I noticed that setting _boost_py_component to python3 works in the Ubuntu container as well! In both Fedora and Ubuntu (and Arch), libboost_python3 is symlinked to the file using the distro custom versioning scheme. Because of this, it can be detected when _boost_py_component is python3.

However, this may cause issues when a specific python version is needed, and I believe logdevice specifically requires python 3.6 (because of nubia facebookarchive/python-nubia#2). If the main goal is to simply get the containers working, I would suggest setting the _boost_py_component to python3 and be done with it, as they are using python 3.6.

If 3.7 support is achieved in nubia, I see no drawbacks to setting _boost_py_component to python3, as a check for a minimum python version is made here (which should be updated to 3.6) https://github.com/facebookincubator/LogDevice/blob/2c0d9d5d9a38dfc5e60886acabc0a036eb43031e/logdevice/CMake/logdevice-deps.cmake#L8

Let me know your thoughts and I'll send a PR your way.

@calebmarchent
Copy link

calebmarchent commented Nov 5, 2018

Sorry that its taken a while to get to this one, the python3 is good, except for on Ubuntu Xenial, a platform for which I still need LogDevice to be able to build. I had a good look around the internet to see if any reliable solutions existed and I finally conclude that they simply don't; I understand there is an aspiration to "re-vamp" the FindBoost logic; but of course that would only be deployed into newer distribution versions.

Re-writing (including) a customised version of the FindBoost.cmake file, might be an option, but there is a lot of logic there that would end up with maintaining divergent code.

This leaves us with trying the different expected combinations, until one works; using a CMake foreach() loop or the like?

Please let me know your thoughts?

@calebmarchent
Copy link

Fixed by: #55
where we first try python3, if that fails, retry with python-py35, which catches the Ubuntu 16.04 case.

facebook-github-bot pushed a commit that referenced this issue Oct 23, 2019
Summary:
I feel bad for such a hacky fix, but (a) I don't have better simple ideas, (b) it's only used in tests.

Stack trace:
  (gdb) bt
  #0  raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:50
  #1  0x000000000096052a in facebook::logdevice::handle_fatal_signal (sig=<optimized out>) at logdevice/server/fatalsignal.cpp:58
  #2  <signal handler called>
  #3  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
  #4  0x00007f0b92e3b935 in __GI_abort () at abort.c:90
  #5  0x00007f0b940fc395 in __gnu_cxx::__verbose_terminate_handler () at ../../.././libstdc++-v3/libsupc++/vterminate.cc:95
  #6  0x00007f0b940fa016 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../.././libstdc++-v3/libsupc++/eh_terminate.cc:47
  #7  0x00007f0b940fa061 in std::terminate () at ../../.././libstdc++-v3/libsupc++/eh_terminate.cc:57
  #8  0x00007f0b940fa33b in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x26e5800 <typeinfo for std::bad_function_call>, dest=0x415cf0 <std::bad_function_call::~bad_function_call()plt>) at ../../.././libstdc++-v3/libsupc++/eh_throw.cc:93
  #9  0x00000000004b5f92 in folly::throw_exception<std::bad_function_call> (ex=...) at folly/lang/Exception.h:36
  #10 0x00000000004b5f57 in folly::detail::throw_exception_<std::bad_function_call> () at folly/lang/Exception.h:64
  #11 0x0000000000d70019 in folly::throw_exception<std::bad_function_call> () at folly/lang/Exception.h:82
  #12 folly::detail::function::FunctionTraits<void (facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::uninitCall(folly::detail::function::Data&, facebook::logdevice::E&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&) () at folly/Function.h:372
  #13 0x0000000000f023f8 in folly::detail::function::FunctionTraits<void (facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::operator()(facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (this=<optimized out>, args=..., args=...) at folly/Function.h:377
  #14 facebook::logdevice::FileBasedVersionedConfigStore::getConfig(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, folly::Function<void (facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>, folly::Optional<facebook::logdevice::vcs_config_version_t>) const (
      this=<optimized out>, key=..., cb=..., base_version=...) at logdevice/common/FileBasedVersionedConfigStore.cpp:258
  #15 0x0000000000d76019 in facebook::logdevice::configuration::nodes::VersionedNodesConfigurationStore<facebook::logdevice::FileBasedVersionedConfigStore>::getConfig(folly::Function<void (facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>, folly::Optional<facebook::logdevice::vcs_config_version_t>) const (
      this=<optimized out>, cb=..., base_version=...) at logdevice/common/configuration/nodes/NodesConfigurationStore.h:88
  #16 0x0000000000d6de2f in facebook::logdevice::configuration::nodes::ncm::Dependencies::readFromStore (this=0x7f0b91967fa0, should_do_consistent_config_fetch=false) at logdevice/common/configuration/nodes/NodesConfigurationManagerDependencies.cpp:342
  #17 0x0000000000d5e64e in facebook::logdevice::configuration::nodes::NodesConfigurationManager::onHeartBeat (this=0x7f0b90d0df10) at logdevice/common/configuration/nodes/NodesConfigurationManager.cpp:538
  #18 0x0000000000d73073 in facebook::logdevice::configuration::nodes::ncm::Dependencies::scheduleHeartBeat()::$_4::operator()() const (this=<optimized out>) at logdevice/common/configuration/nodes/NodesConfigurationManagerDependencies.cpp:365
  #19 std::_Function_handler<void (), facebook::logdevice::configuration::nodes::ncm::Dependencies::scheduleHeartBeat()::$_4>::_M_invoke(std::_Any_data const&) (__functor=...) at third-party-buck/platform007/build/libgcc/include/c++/trunk/bits/std_function.h:316
  #20 0x0000000000e250a6 in std::function<void ()>::operator()() const (this=<optimized out>) at third-party-buck/platform007/build/libgcc/include/c++/trunk/bits/std_function.h:706
  #21 facebook::logdevice::(anonymous namespace)::WheelTimerDispatchImpl::makeWheelTimerInternalExecutor(facebook::logdevice::Worker*)::$_0::operator()()::{lambda()#1}::operator()() const (this=0x7f0b8bf34070) at logdevice/common/Timer.cpp:123
  #22 folly::detail::function::FunctionTraits<void ()>::callSmall<facebook::logdevice::(anonymous namespace)::WheelTimerDispatchImpl::makeWheelTimerInternalExecutor(facebook::logdevice::Worker*)::$_0::operator()()::{lambda()#1}>(folly::detail::function::Data&) (p=...) at folly/Function.h:361
  #23 0x0000000000e63602 in folly::detail::function::FunctionTraits<void ()>::operator()() (this=<optimized out>) at folly/Function.h:377
  #24 facebook::logdevice::Worker::addWithPriority(folly::Function<void ()>, signed char)::$_9::operator()() (this=0x7f0b8bf34060) at logdevice/common/Worker.cpp:1373
  #25 folly::detail::function::FunctionTraits<void ()>::callBig<facebook::logdevice::Worker::addWithPriority(folly::Function<void ()>, signed char)::$_9>(folly::detail::function::Data&) (p=...) at folly/Function.h:368
  #26 0x00000000010d7e75 in folly::detail::function::FunctionTraits<void ()>::operator()() (this=0x7f0b8bf34060) at folly/Function.h:377
  #27 facebook::logdevice::EventLoopTaskQueue::executeTasks (this=0x7f0b74023000, tokens=<optimized out>) at logdevice/common/EventLoopTaskQueue.cpp:154
  #28 0x00000000010d7699 in facebook::logdevice::EventLoopTaskQueue::haveTasksEventHandler()::$_1::operator()(unsigned int) const (this=<optimized out>, n=1) at logdevice/common/EventLoopTaskQueue.cpp:101
  #29 facebook::logdevice::LifoEventSemImpl<std::atomic>::AsyncWaiter::processBatch<facebook::logdevice::EventLoopTaskQueue::haveTasksEventHandler()::$_1&>(facebook::logdevice::EventLoopTaskQueue::haveTasksEventHandler()::$_1&, unsigned int) (this=<optimized out>, func=..., maxBatchSize=<optimized out>) at logdevice/common/LifoEventSem.h:351
  #30 facebook::logdevice::EventLoopTaskQueue::haveTasksEventHandler (this=0x7f0b74023000) at logdevice/common/EventLoopTaskQueue.cpp:106
  #31 0x0000000001a907f2 in event_process_active_single_queue (base=<optimized out>, activeq=0x7f0b74020010, max_to_process=2147483647, endtime=0x0) at logdevice/external/libevent-2.1.3-alpha/event.c:1449
  #32 0x0000000001a8efae in event_process_active (base=<optimized out>) at logdevice/external/libevent-2.1.3-alpha/event.c:1599
  #33 ld_event_base_loop (base=0x7f0b7401b000, flags=<optimized out>) at logdevice/external/libevent-2.1.3-alpha/event.c:1819
  #34 0x000000000176b5f1 in facebook::logdevice::EvBaseLegacy::loop (this=0x7f0b74000018) at logdevice/common/libevent/EvBaseLegacy.cpp:58
  #35 0x00000000010d5a79 in facebook::logdevice::EventLoop::run (this=<optimized out>) at logdevice/common/EventLoop.cpp:140
  #36 0x00000000016fa223 in std::function<void ()>::operator()() const (this=0x7f0b90d0c520) at third-party-buck/platform007/build/libgcc/include/c++/trunk/bits/std_function.h:706
  #37 facebook::logdevice::thread_func (arg=0x7f0b90d0c520) at logdevice/common/PThread.cpp:9
  #38 0x00007f0b936136b6 in start_thread (arg=0x7f0b74dff700) at pthread_create.c:465
  #39 0x00007f0b92f2cebf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Reviewed By: gdrane

Differential Revision: D17975422

fbshipit-source-id: 499c6aceb577f859e9d4e71006b86f73537221ab
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants