Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Domain ID not changable on rclcpp::NodeOptions #910

Closed
rotu opened this issue Oct 30, 2019 · 21 comments · Fixed by #1165
Closed

Domain ID not changable on rclcpp::NodeOptions #910

rotu opened this issue Oct 30, 2019 · 21 comments · Fixed by #1165
Assignees
Labels
backlog enhancement New feature or request

Comments

@rotu
Copy link
Contributor

rotu commented Oct 30, 2019

rclcpp::NodeOptions has no way to set the Domain ID on a per-node basis, and it's kludgy to get the domain ID node_options->get_rcl_node_options()->domain_id. This functionality is exposed on rcl_node_options_t, so it's just a deficiency in the c++ API.

@wjwwood wjwwood self-assigned this Nov 14, 2019
@wjwwood wjwwood added enhancement New feature or request help wanted Extra attention is needed labels Nov 14, 2019
@wjwwood
Copy link
Member

wjwwood commented Nov 15, 2019

This makes sense to me, we'd appreciate anyone who has time to contribute this addition to the NodeOptions. We should consider doing it in rclpy at the same time, if it's not already there.

@suab321321
Copy link
Contributor

@wjwwood sir I would like to contribute

@ivanpauno
Copy link
Member

Note: When we finish with ros2/rmw#183, one domain id per node will have no more sense, and it will be one domain id per context.

@wjwwood
Copy link
Member

wjwwood commented Nov 18, 2019

That's a good point @ivanpauno. In that case maybe we should hold off on this until we have a little more clear direction on that set of changes.

@suab321321 it sounds like we'd like to hold off on this for now. Our intention is to follow through with ros2/rmw#183, in which case this issue will no longer make sense. There's a chance we change our mind, in which case this issue would make sense again, but that is unlikely in my opinion.

I added a note to the issue description of ros2/rmw#183 that we should follow up on this issue once we've committed to doing that work or not.

@wjwwood wjwwood removed the help wanted Extra attention is needed label Nov 18, 2019
@suab321321
Copy link
Contributor

@wjwwood okay

@fujitatomoya
Copy link
Collaborator

@wjwwood @ivanpauno

kinda off topic from this thread, but the same goes to rclpy ?

actually rclpy does not have support for ROS_DOMAIN_ID (ros2/rclpy#484). i think that ROS_DOMAIN_ID should be supported via rcl and frontend(rclxxx) can ask the domain id to rcl. (just to reduce redundancy of implementation.)

could you share your thoughts on this?

@ivanpauno
Copy link
Member

actually rclpy does not have support for ROS_DOMAIN_ID (ros2/rclpy#484). i think that ROS_DOMAIN_ID should be supported via rcl and frontend(rclxxx) can ask the domain id to rcl. (just to reduce redundancy of implementation.)

The ROS_DOMAIN_ID logic is actually implemented in rcl, as you suggested.
If it's not working, it must be an error in how rclpy is wrapping rcl stuff.

@fujitatomoya
Copy link
Collaborator

@ivanpauno

The ROS_DOMAIN_ID logic is actually implemented in rcl, as you suggested.

sorry my bad, i should have checked.

https://github.com/ros2/rcl/blob/024c3ee50d9b79a85665efe1c191cc1225026519/rcl/src/rcl/node.c#L262-L281

and the question is why rclcpp has the same code to get the ROS_DOMAIN_ID?

// TODO(wjwwood): reuse rcutils_get_env() to avoid code duplication.
// See also: https://github.com/ros2/rcl/issues/119
size_t
NodeOptions::get_domain_id_from_env() const
{
// Determine the domain id based on the options and the ROS_DOMAIN_ID env variable.
size_t domain_id = std::numeric_limits<size_t>::max();
char * ros_domain_id = nullptr;
const char * env_var = "ROS_DOMAIN_ID";
#ifndef _WIN32
ros_domain_id = getenv(env_var);
#else
size_t ros_domain_id_size;
_dupenv_s(&ros_domain_id, &ros_domain_id_size, env_var);
#endif
if (ros_domain_id) {
uint32_t number = strtoul(ros_domain_id, NULL, 0);
if (number == (std::numeric_limits<uint32_t>::max)()) {
#ifdef _WIN32
// free the ros_domain_id before throwing, if getenv was used on Windows
free(ros_domain_id);
#endif
throw std::runtime_error("failed to interpret ROS_DOMAIN_ID as integral number");
}
domain_id = static_cast<size_t>(number);
#ifdef _WIN32
free(ros_domain_id);
#endif
}
return domain_id;
}

i believe that we could delete the redundant code in rclcpp?
if so, i will make the issue and PR against it.

@ivanpauno
Copy link
Member

and the question is why rclcpp has the same code to get the ROS_DOMAIN_ID?

It's needed just for better introspection. Currently, the default node options have a domain id value of SIZEMAX. When that value is detected in the node creation, it's updated to the value indicated by the env variable. To be able to introspect correctly the domain id if you call get_rcl_node_options before creating a node, the value is also get from the environment in that case.

i believe that we could delete the redundant code in rclcpp?
if so, i will make the issue and PR against it.

Yes, the redundant code can be avoided.
As part of ros2/rcl#515, I'm creating an rcl_domain_id function, that allows you to get the default domain_id from the environment.
Code can easily be reused after that gets merged.

@fujitatomoya
Copy link
Collaborator

@ivanpauno

got it, i will get it done after your PR is merged.

thanks

@fujitatomoya
Copy link
Collaborator

@ivanpauno

i got a question, since Context is introduced and mapped to dds participant, I think that domain id also should be mapped to Context (same as security) not only for NodeOptions.

i am considering the following,

  • expose domain_id to NodeOptions
    default std::numeric_limits<size_t>::max() and ROS_DOMAIN_ID variable is set via get_domain_id_from_env(), but user can overwrite the domain_id if needed. actually rmw_fastrtps/cyclonedds does not use this domain_id via rmw_node_create, so this is for implementation that can support domain_id with node level.
  • expose domain_id to InitOptions
    this is actually effective with dds implementation such as fastrtps or cyclonedds. default std::numeric_limits<size_t>::max() and ROS_DOMAIN_ID variable is set, and user can overwrite the domain_id if needed.

do you have any other thoughts?

@ivanpauno
Copy link
Member

We shouldn't expose nothing in NodeOptions, as that will not continue to be used.
Exposing it in InitOptions is ok, but I would prefer to have a cleaner API (i.e.: not exposing that size_t max means "default").

class InitOptions
{
...
void use_default_domain_id();  // this will take the domain id from the env
void set_domain_id(size_t domain_id); 
size_t get_domain_id() const; 
...
}

@fujitatomoya
Copy link
Collaborator

I do agree with you, so maybe it is better to clean anything related to ROS_DOMAIN_ID stuff from NodeOptions. and we also update docs (if exist) explicitly about domain id with Context for user.

@fujitatomoya
Copy link
Collaborator

@suab321321

sorry i missed it, are you up for this?

@ivanpauno
Copy link
Member

I do agree with you, so maybe it is better to clean anything related to ROS_DOMAIN_ID stuff from NodeOptions. and we also update docs (if exist) explicitly about domain id with Context for user.

If you want to clean that up, it's welcomed too! The only reference too domain_id in node related API is in rmw IIRC.
Exposing the domain id here is not related with that change, so it can be done separately.

@fujitatomoya
Copy link
Collaborator

@ivanpauno

either me or someone else from our team can work on this, you can leave this on us. though, we would require your review for sure.

@aprotyas
Copy link
Member

aprotyas commented May 28, 2022

Looks like the new API to support setting/getting domain IDs targeted post-Foxy distributions.

Does anyone involved in this thread know how to obtain the domain ID from a node in Foxy? The snippet below gives me a runtime error:

size_t
get_domain_id_from_node(
  const rclcpp::Node & node)
{
  const rcl_init_options_t * rcl_init_options = 
    node.get_node_base_interface()->get_context()->get_init_options().get_rcl_init_options();

  std::size_t domain_id;
  // const_cast is safe because `rcl_init_options_get_domain_id` only reads the input structure
  rcl_ret_t ret =
    rcl_init_options_get_domain_id(
    const_cast<rcl_init_options_t *>(rcl_init_options),
    &domain_id);
  if (RCL_RET_OK != ret) {
    std::string err {"Failed to get domain ID from rcl_init_options: "};
    err += std::to_string(domain_id);
    throw std::runtime_error(err);
  }
}
$ ./domain_id_test
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to get domain ID from rcl_init_options: 140276023001360
Aborted (core dumped)

@fujitatomoya
Copy link
Collaborator

ros2/rcl#946 backport foxy has been merged, so it should be working. can we take a look at the stack trace via gdb?

@aprotyas
Copy link
Member

@fujitatomoya I expected it to work too, but maybe you would have some insights from the backtrace:

Output of gdb exec -ex "run" -ex "thread apply all bt" -ex "quit"
$ gdb build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id -ex "run" -ex "thread apply all bt" -ex "quit"

GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id...
Starting program: /home/aprotyas/dev/workspaces/ros2_playground_ws/build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff5723700 (LWP 97916)]
[New Thread 0x7ffff4e99700 (LWP 97917)]
[New Thread 0x7ffff4698700 (LWP 97918)]
[New Thread 0x7ffff3e97700 (LWP 97919)]
[New Thread 0x7ffff3689700 (LWP 97920)]
[New Thread 0x7ffff2e88700 (LWP 97921)]
[New Thread 0x7ffff267a700 (LWP 97922)]
[New Thread 0x7ffff1d79700 (LWP 97923)]
[New Thread 0x7ffff1565700 (LWP 97924)]

Thread 1 "rclcpp_foxy_dom" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50

Thread 10 (Thread 0x7ffff1565700 (LWP 97924)):
#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x7fffe4000ba8) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7fffe4000bb0, cond=0x7fffe4000b80) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x7fffe4000b80, mutex=0x7fffe4000bb0) at pthread_cond_wait.c:647
#3  0x00007ffff7658e30 in std::condition_variable::wait(std::unique_lock<std::mutex>&) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff6f7567d in std::condition_variable::wait<rmw_fastrtps_shared_cpp::__rmw_wait(char const*, rmw_subscriptions_t*, rmw_guard_conditions_t*, rmw_services_t*, rmw_clients_t*, rmw_events_t*, rmw_wait_set_t*, const rmw_time_t*)::<lambda()> >(std::unique_lock<std::mutex> &, rmw_fastrtps_shared_cpp::<lambda()>) (this=0x7fffe4000b80, __lock=..., __p=...) at /usr/include/c++/9/condition_variable:101
#5  0x00007ffff6f75153 in rmw_fastrtps_shared_cpp::__rmw_wait (identifier=0x7ffff701bd00 "rmw_fastrtps_cpp", subscriptions=0x7ffff1564560, guard_conditions=0x7ffff1564570, services=0x0, clients=0x0, events=0x0, wait_set=0x7fffe4000b60, wait_timeout=0x0) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/rmw_wait.cpp:173
#6  0x00007ffff6f33296 in node_listener (context=0x5555555da350) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/ros2/rmw_fastrtps/rmw_fastrtps_shared_cpp/src/listener_thread.cpp:136
#7  0x00007ffff6f34385 in std::__invoke_impl<void, void (*)(rmw_context_t*), rmw_context_t*> (__f=@0x5555556ba8e0: 0x7ffff6f33075 <node_listener(rmw_context_t*)>) at /usr/include/c++/9/bits/invoke.h:60
#8  0x00007ffff6f342d1 in std::__invoke<void (*)(rmw_context_t*), rmw_context_t*> (__fn=@0x5555556ba8e0: 0x7ffff6f33075 <node_listener(rmw_context_t*)>) at /usr/include/c++/9/bits/invoke.h:95
#9  0x00007ffff6f34221 in std::thread::_Invoker<std::tuple<void (*)(rmw_context_t*), rmw_context_t*> >::_M_invoke<0ul, 1ul> (this=0x5555556ba8d8) at /usr/include/c++/9/thread:244
#10 0x00007ffff6f341c3 in std::thread::_Invoker<std::tuple<void (*)(rmw_context_t*), rmw_context_t*> >::operator() (this=0x5555556ba8d8) at /usr/include/c++/9/thread:251
#11 0x00007ffff6f34194 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(rmw_context_t*), rmw_context_t*> > >::_M_run (this=0x5555556ba8d0) at /usr/include/c++/9/thread:195
#12 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#13 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#14 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 9 (Thread 0x7ffff1d79700 (LWP 97923)):
#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x5555555dcd4c) at ../sysdeps/nptl/futex-internal.h:183
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5555555dbc00, cond=0x5555555dcd20) at pthread_cond_wait.c:508
#2  __pthread_cond_wait (cond=0x5555555dcd20, mutex=0x5555555dbc00) at pthread_cond_wait.c:647
#3  0x00007ffff7658e30 in std::condition_variable::wait(std::unique_lock<std::mutex>&) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff64646c5 in std::_V2::condition_variable_any::wait<std::unique_lock<std::recursive_timed_mutex> > (this=0x5555555dcd20, __lock=...) at /usr/include/c++/9/condition_variable:273
#5  0x00007ffff64623d4 in eprosima::fastrtps::rtps::AsyncWriterThread::run (this=0x5555555dcc80) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/resources/AsyncWriterThread.cpp:135
#6  0x00007ffff646501f in std::__invoke_impl<void, void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> (__f=@0x555555623d80: (void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(class eprosima::fastrtps::rtps::AsyncWriterThread * const)) 0x7ffff64622c0 <eprosima::fastrtps::rtps::AsyncWriterThread::run()>, __t=@0x555555623d78: 0x5555555dcc80) at /usr/include/c++/9/bits/invoke.h:73
#7  0x00007ffff6464f39 in std::__invoke<void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> (__fn=@0x555555623d80: (void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(class eprosima::fastrtps::rtps::AsyncWriterThread * const)) 0x7ffff64622c0 <eprosima::fastrtps::rtps::AsyncWriterThread::run()>) at /usr/include/c++/9/bits/invoke.h:95
#8  0x00007ffff6464e89 in std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> >::_M_invoke<0ul, 1ul> (this=0x555555623d78) at /usr/include/c++/9/thread:244
#9  0x00007ffff6464e2b in std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> >::operator() (this=0x555555623d78) at /usr/include/c++/9/thread:251
#10 0x00007ffff6464dfc in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::AsyncWriterThread::*)(), eprosima::fastrtps::rtps::AsyncWriterThread*> > >::_M_run (this=0x555555623d70) at /usr/include/c++/9/thread:195
#11 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#12 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#13 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 8 (Thread 0x7ffff267a700 (LWP 97922)):
#0  0x00007ffff736a678 in futex_abstimed_wait_cancelable (private=128, abstime=0x7ffff2678d20, clockid=0, expected=0, futex_word=0x7ffff2680110) at ../sysdeps/nptl/futex-internal.h:320
#1  do_futex_wait (sem=sem@entry=0x7ffff2680110, abstime=abstime@entry=0x7ffff2678d20, clockid=0) at sem_waitcommon.c:112
#2  0x00007ffff736a7a3 in __new_sem_wait_slow (sem=0x7ffff2680110, abstime=0x7ffff2678d20, clockid=0) at sem_waitcommon.c:184
#3  0x00007ffff6920707 in boost::interprocess::ipcdetail::semaphore_timed_wait (handle=0x7ffff2680110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/posix/semaphore_wrapper.hpp:226
#4  0x00007ffff6920c5d in boost::interprocess::ipcdetail::posix_semaphore::timed_wait (this=0x7ffff2680110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/posix/semaphore.hpp:55
#5  0x00007ffff6920ccf in boost::interprocess::interprocess_semaphore::timed_wait (this=0x7ffff2680110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/interprocess_semaphore.hpp:139
#6  0x00007ffff692143e in eprosima::fastdds::rtps::RobustInterprocessCondition::do_timed_wait (this=0x7ffff267b138, abs_time=..., mut=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/RobustInterprocessCondition.hpp:347
#7  0x00007ffff692fd91 in eprosima::fastdds::rtps::RobustInterprocessCondition::timed_wait<std::unique_lock<boost::interprocess::interprocess_mutex>, eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop(eprosima::fastdds::rtps::MultiProducerConsumerRingBuffer<eprosima::fastdds::rtps::SharedMemGlobal::BufferDescriptor>::Listener&, std::atomic<bool> const&, unsigned int)::{lambda()#1}>(std::unique_lock<boost::interprocess::interprocess_mutex>&, boost::posix_time::ptime const&, eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop(eprosima::fastdds::rtps::MultiProducerConsumerRingBuffer<eprosima::fastdds::rtps::SharedMemGlobal::BufferDescriptor>::Listener&, std::atomic<bool> const&, unsigned int)::{lambda()#1}) (this=0x7ffff267b138, lock=..., abs_time=..., pred=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/RobustInterprocessCondition.hpp:148
#8  0x00007ffff6924fdd in eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop (this=0x555555618d80, listener=..., is_listener_closed=..., listener_index=0) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemGlobal.hpp:549
#9  0x00007ffff692945c in eprosima::fastdds::rtps::SharedMemManager::Listener::pop (this=0x555555618b10) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemManager.hpp:695
#10 0x00007ffff692cdb0 in eprosima::fastdds::rtps::SharedMemChannelResource::Receive (this=0x555555618e50, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:182
#11 0x00007ffff692c928 in eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation (this=0x555555618e50, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:133
#12 0x00007ffff6992a38 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x55555561b818: (void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(class eprosima::fastdds::rtps::SharedMemChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff692c88c <eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x55555561b810: 0x555555618e50) at /usr/include/c++/9/bits/invoke.h:73
#13 0x00007ffff6992503 in std::__invoke<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x55555561b818: (void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(class eprosima::fastdds::rtps::SharedMemChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff692c88c <eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
#14 0x00007ffff699229f in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x55555561b7f8) at /usr/include/c++/9/thread:244
#15 0x00007ffff6991c25 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x55555561b7f8) at /usr/include/c++/9/thread:251
#16 0x00007ffff6990fd6 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x55555561b7f0) at /usr/include/c++/9/thread:195
#17 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#18 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#19 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 7 (Thread 0x7ffff2e88700 (LWP 97921)):
#0  __libc_recvmsg (flags=0, msg=0x7ffff2e86f60, fd=10) at ../sysdeps/unix/sysv/linux/recvmsg.c:28
#1  __libc_recvmsg (fd=10, msg=0x7ffff2e86f60, flags=0) at ../sysdeps/unix/sysv/linux/recvmsg.c:25
#2  0x00007ffff65f70cc in asio::detail::socket_ops::recvfrom (s=10, bufs=0x7ffff2e870a0, count=1, flags=0, addr=0x7ffff2e871d0, addrlen=0x7ffff2e87090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:939
#3  0x00007ffff65f71c5 in asio::detail::socket_ops::sync_recvfrom (s=10, state=32 ' ', bufs=0x7ffff2e870a0, count=1, flags=0, addr=0x7ffff2e871d0, addrlen=0x7ffff2e87090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:961
#4  0x00007ffff65f8bd7 in asio::detail::reactive_socket_service<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555dc0e0, impl=..., buffers=..., sender_endpoint=..., flags=0, ec=...) at /usr/include/asio/detail/reactive_socket_service.hpp:298
#5  0x00007ffff65f8593 in asio::basic_datagram_socket<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555f5150, buffers=..., sender_endpoint=...) at /usr/include/asio/basic_datagram_socket.hpp:858
#6  0x00007ffff65f5491 in eprosima::fastdds::rtps::UDPChannelResource::Receive (this=0x5555555f5110, receive_buffer=0x5555556085e0 "", receive_buffer_capacity=65500, receive_buffer_size=@0x5555555f512c: 0, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:90
#7  0x00007ffff65f522f in eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation (this=0x5555555f5110, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:61
#8  0x00007ffff65f95d5 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x5555555f53d8: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x5555555f53d0: 0x5555555f5110) at /usr/include/c++/9/bits/invoke.h:73
#9  0x00007ffff65f9481 in std::__invoke<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x5555555f53d8: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
#10 0x00007ffff65f9391 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x5555555f53b8) at /usr/include/c++/9/thread:244
#11 0x00007ffff65f9317 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x5555555f53b8) at /usr/include/c++/9/thread:251
#12 0x00007ffff65f92e8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x5555555f53b0) at /usr/include/c++/9/thread:195
#13 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#14 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#15 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 6 (Thread 0x7ffff3689700 (LWP 97920)):
#0  0x00007ffff736a678 in futex_abstimed_wait_cancelable (private=128, abstime=0x7ffff3687d20, clockid=0, expected=0, futex_word=0x7ffff368f110) at ../sysdeps/nptl/futex-internal.h:320
#1  do_futex_wait (sem=sem@entry=0x7ffff368f110, abstime=abstime@entry=0x7ffff3687d20, clockid=0) at sem_waitcommon.c:112
#2  0x00007ffff736a7a3 in __new_sem_wait_slow (sem=0x7ffff368f110, abstime=0x7ffff3687d20, clockid=0) at sem_waitcommon.c:184
#3  0x00007ffff6920707 in boost::interprocess::ipcdetail::semaphore_timed_wait (handle=0x7ffff368f110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/posix/semaphore_wrapper.hpp:226
#4  0x00007ffff6920c5d in boost::interprocess::ipcdetail::posix_semaphore::timed_wait (this=0x7ffff368f110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/posix/semaphore.hpp:55
#5  0x00007ffff6920ccf in boost::interprocess::interprocess_semaphore::timed_wait (this=0x7ffff368f110, abs_time=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/thirdparty/boost/include/boost/interprocess/sync/interprocess_semaphore.hpp:139
#6  0x00007ffff692143e in eprosima::fastdds::rtps::RobustInterprocessCondition::do_timed_wait (this=0x7ffff368a138, abs_time=..., mut=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/RobustInterprocessCondition.hpp:347
#7  0x00007ffff692fd91 in eprosima::fastdds::rtps::RobustInterprocessCondition::timed_wait<std::unique_lock<boost::interprocess::interprocess_mutex>, eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop(eprosima::fastdds::rtps::MultiProducerConsumerRingBuffer<eprosima::fastdds::rtps::SharedMemGlobal::BufferDescriptor>::Listener&, std::atomic<bool> const&, unsigned int)::{lambda()#1}>(std::unique_lock<boost::interprocess::interprocess_mutex>&, boost::posix_time::ptime const&, eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop(eprosima::fastdds::rtps::MultiProducerConsumerRingBuffer<eprosima::fastdds::rtps::SharedMemGlobal::BufferDescriptor>::Listener&, std::atomic<bool> const&, unsigned int)::{lambda()#1}) (this=0x7ffff368a138, lock=..., abs_time=..., pred=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/RobustInterprocessCondition.hpp:148
#8  0x00007ffff6924fdd in eprosima::fastdds::rtps::SharedMemGlobal::Port::wait_pop (this=0x5555555f5260, listener=..., is_listener_closed=..., listener_index=0) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemGlobal.hpp:549
#9  0x00007ffff692945c in eprosima::fastdds::rtps::SharedMemManager::Listener::pop (this=0x5555555f5030) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemManager.hpp:695
#10 0x00007ffff692cdb0 in eprosima::fastdds::rtps::SharedMemChannelResource::Receive (this=0x5555555f53f0, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:182
#11 0x00007ffff692c928 in eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation (this=0x5555555f53f0, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemChannelResource.hpp:133
#12 0x00007ffff6992a38 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x5555555f54a8: (void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(class eprosima::fastdds::rtps::SharedMemChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff692c88c <eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x5555555f54a0: 0x5555555f53f0) at /usr/include/c++/9/bits/invoke.h:73
#13 0x00007ffff6992503 in std::__invoke<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x5555555f54a8: (void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(class eprosima::fastdds::rtps::SharedMemChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff692c88c <eprosima::fastdds::rtps::SharedMemChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
#14 0x00007ffff699229f in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x5555555f5488) at /usr/include/c++/9/thread:244
#15 0x00007ffff6991c25 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x5555555f5488) at /usr/include/c++/9/thread:251
#16 0x00007ffff6990fd6 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::SharedMemChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x5555555f5480) at /usr/include/c++/9/thread:195
#17 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#18 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#19 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 5 (Thread 0x7ffff3e97700 (LWP 97919)):
#0  __libc_recvmsg (flags=0, msg=0x7ffff3e95f60, fd=8) at ../sysdeps/unix/sysv/linux/recvmsg.c:28
#1  __libc_recvmsg (fd=8, msg=0x7ffff3e95f60, flags=0) at ../sysdeps/unix/sysv/linux/recvmsg.c:25
#2  0x00007ffff65f70cc in asio::detail::socket_ops::recvfrom (s=8, bufs=0x7ffff3e960a0, count=1, flags=0, addr=0x7ffff3e961d0, addrlen=0x7ffff3e96090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:939
#3  0x00007ffff65f71c5 in asio::detail::socket_ops::sync_recvfrom (s=8, state=32 ' ', bufs=0x7ffff3e960a0, count=1, flags=0, addr=0x7ffff3e961d0, addrlen=0x7ffff3e96090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:961
#4  0x00007ffff65f8bd7 in asio::detail::reactive_socket_service<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555dc0e0, impl=..., buffers=..., sender_endpoint=..., flags=0, ec=...) at /usr/include/asio/detail/reactive_socket_service.hpp:298
#5  0x00007ffff65f8593 in asio::basic_datagram_socket<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555f4b20, buffers=..., sender_endpoint=...) at /usr/include/asio/basic_datagram_socket.hpp:858
#6  0x00007ffff65f5491 in eprosima::fastdds::rtps::UDPChannelResource::Receive (this=0x5555555f4ae0, receive_buffer=0x5555555f58a0 "", receive_buffer_capacity=65500, receive_buffer_size=@0x5555555f4afc: 0, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:90
#7  0x00007ffff65f522f in eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation (this=0x5555555f4ae0, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:61
#8  0x00007ffff65f95d5 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x5555555e41b8: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x5555555e41b0: 0x5555555f4ae0) at /usr/include/c++/9/bits/invoke.h:73
#9  0x00007ffff65f9481 in std::__invoke<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x5555555e41b8: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
#10 0x00007ffff65f9391 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x5555555e4198) at /usr/include/c++/9/thread:244
#11 0x00007ffff65f9317 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x5555555e4198) at /usr/include/c++/9/thread:251
#12 0x00007ffff65f92e8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x5555555e4190) at /usr/include/c++/9/thread:195
#13 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#14 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#15 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 (Thread 0x7ffff4698700 (LWP 97918)):
#0  __libc_recvmsg (flags=0, msg=0x7ffff4696f60, fd=7) at ../sysdeps/unix/sysv/linux/recvmsg.c:28
#1  __libc_recvmsg (fd=7, msg=0x7ffff4696f60, flags=0) at ../sysdeps/unix/sysv/linux/recvmsg.c:25
#2  0x00007ffff65f70cc in asio::detail::socket_ops::recvfrom (s=7, bufs=0x7ffff46970a0, count=1, flags=0, addr=0x7ffff46971d0, addrlen=0x7ffff4697090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:939
#3  0x00007ffff65f71c5 in asio::detail::socket_ops::sync_recvfrom (s=7, state=32 ' ', bufs=0x7ffff46970a0, count=1, flags=0, addr=0x7ffff46971d0, addrlen=0x7ffff4697090, ec=...) at /usr/include/asio/detail/impl/socket_ops.ipp:961
#4  0x00007ffff65f8bd7 in asio::detail::reactive_socket_service<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555dc0e0, impl=..., buffers=..., sender_endpoint=..., flags=0, ec=...) at /usr/include/asio/detail/reactive_socket_service.hpp:298
#5  0x00007ffff65f8593 in asio::basic_datagram_socket<asio::ip::udp>::receive_from<asio::mutable_buffers_1> (this=0x5555555e3fd0, buffers=..., sender_endpoint=...) at /usr/include/asio/basic_datagram_socket.hpp:858
#6  0x00007ffff65f5491 in eprosima::fastdds::rtps::UDPChannelResource::Receive (this=0x5555555e3f90, receive_buffer=0x5555555e4a40 "RTPS\002\003\001\017\001\017\353}x~Jl\001", receive_buffer_capacity=65500, receive_buffer_size=@0x5555555e3fac: 264, remote_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:90
#7  0x00007ffff65f522f in eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation (this=0x5555555e3f90, input_locator=...) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/UDPChannelResource.cpp:61
#8  0x00007ffff65f95d5 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__f=@0x5555555dc228: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>, __t=@0x5555555dc220: 0x5555555e3f90) at /usr/include/c++/9/bits/invoke.h:73
#9  0x00007ffff65f9481 in std::__invoke<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> (__fn=@0x5555555dc228: (void (eprosima::fastdds::rtps::UDPChannelResource::*)(class eprosima::fastdds::rtps::UDPChannelResource * const, class eprosima::fastrtps::rtps::Locator_t)) 0x7ffff65f518e <eprosima::fastdds::rtps::UDPChannelResource::perform_listen_operation(eprosima::fastrtps::rtps::Locator_t)>) at /usr/include/c++/9/bits/invoke.h:95
#10 0x00007ffff65f9391 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::_M_invoke<0ul, 1ul, 2ul> (this=0x5555555dc208) at /usr/include/c++/9/thread:244
#11 0x00007ffff65f9317 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> >::operator() (this=0x5555555dc208) at /usr/include/c++/9/thread:251
#12 0x00007ffff65f92e8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::UDPChannelResource::*)(eprosima::fastrtps::rtps::Locator_t), eprosima::fastdds::rtps::UDPChannelResource*, eprosima::fastrtps::rtps::Locator_t> > >::_M_run (this=0x5555555dc200) at /usr/include/c++/9/thread:195
#13 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#14 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#15 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 3 (Thread 0x7ffff4e99700 (LWP 97917)):
#0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7ffff4e98470, clockid=<optimized out>, expected=0, futex_word=0x5555555dcb88) at ../sysdeps/nptl/futex-internal.h:320
#1  __pthread_cond_wait_common (abstime=0x7ffff4e98470, clockid=<optimized out>, mutex=0x5555555dc380, cond=0x5555555dcb60) at pthread_cond_wait.c:520
#2  __pthread_cond_timedwait (cond=0x5555555dcb60, mutex=0x5555555dc380, abstime=0x7ffff4e98470) at pthread_cond_wait.c:665
#3  0x00007ffff7d23c8f in __gthread_cond_timedwait (__cond=0x5555555dcb60, __mutex=0x5555555dc380, __abs_timeout=0x7ffff4e98470) at /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:872
#4  0x00007ffff7d29bd4 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dcb60, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:188
#5  0x00007ffff7d27d90 in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dcb60, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:121
#6  0x00007ffff645bb0b in std::_V2::condition_variable_any::wait_until<std::unique_lock<std::timed_mutex>, std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dcb60, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:296
#7  0x00007ffff645a236 in eprosima::fastrtps::rtps::ResourceEvent::event_service (this=0x5555555dcae8) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/resources/ResourceEvent.cpp:182
#8  0x00007ffff646077b in std::__invoke_impl<void, void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> (__f=@0x5555555df430: (void (eprosima::fastrtps::rtps::ResourceEvent::*)(class eprosima::fastrtps::rtps::ResourceEvent * const)) 0x7ffff645a0e0 <eprosima::fastrtps::rtps::ResourceEvent::event_service()>, __t=@0x5555555df428: 0x5555555dcae8) at /usr/include/c++/9/bits/invoke.h:73
#9  0x00007ffff6460695 in std::__invoke<void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> (__fn=@0x5555555df430: (void (eprosima::fastrtps::rtps::ResourceEvent::*)(class eprosima::fastrtps::rtps::ResourceEvent * const)) 0x7ffff645a0e0 <eprosima::fastrtps::rtps::ResourceEvent::event_service()>) at /usr/include/c++/9/bits/invoke.h:95
#10 0x00007ffff64605e5 in std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> >::_M_invoke<0ul, 1ul> (this=0x5555555df428) at /usr/include/c++/9/thread:244
#11 0x00007ffff6460587 in std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> >::operator() (this=0x5555555df428) at /usr/include/c++/9/thread:251
#12 0x00007ffff6460558 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastrtps::rtps::ResourceEvent::*)(), eprosima::fastrtps::rtps::ResourceEvent*> > >::_M_run (this=0x5555555df420) at /usr/include/c++/9/thread:195
#13 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#14 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#15 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7ffff5723700 (LWP 97916)):
#0  futex_abstimed_wait_cancelable (private=<optimized out>, abstime=0x7ffff5722470, clockid=<optimized out>, expected=0, futex_word=0x5555555dc330) at ../sysdeps/nptl/futex-internal.h:320
#1  __pthread_cond_wait_common (abstime=0x7ffff5722470, clockid=<optimized out>, mutex=0x5555555dc338, cond=0x5555555dc308) at pthread_cond_wait.c:520
#2  __pthread_cond_timedwait (cond=0x5555555dc308, mutex=0x5555555dc338, abstime=0x7ffff5722470) at pthread_cond_wait.c:665
#3  0x00007ffff7d23c8f in __gthread_cond_timedwait (__cond=0x5555555dc308, __mutex=0x5555555dc338, __abs_timeout=0x7ffff5722470) at /usr/include/x86_64-linux-gnu/c++/9/bits/gthr-default.h:872
#4  0x00007ffff7d29bd4 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dc308, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:188
#5  0x00007ffff7d27d90 in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x5555555dc308, __lock=..., __atime=...) at /usr/include/c++/9/condition_variable:121
#6  0x00007ffff6934d8c in std::condition_variable::wait_until<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> >, eprosima::fastdds::rtps::SharedMemWatchdog::run()::{lambda()#1}>(std::unique_lock<std::mutex>&, std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > const&, eprosima::fastdds::rtps::SharedMemWatchdog::run()::{lambda()#1}) (this=0x5555555dc308, __lock=..., __atime=..., __p=...) at /usr/include/c++/9/condition_variable:138
#7  0x00007ffff692f0eb in std::condition_variable::wait_for<long, std::ratio<1l, 1000l>, eprosima::fastdds::rtps::SharedMemWatchdog::run()::{lambda()#1}>(std::unique_lock<std::mutex>&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, eprosima::fastdds::rtps::SharedMemWatchdog::run()::{lambda()#1}) (this=0x5555555dc308, __lock=..., __rtime=..., __p=...) at /usr/include/c++/9/condition_variable:166
#8  0x00007ffff69236c4 in eprosima::fastdds::rtps::SharedMemWatchdog::run (this=0x5555555dc2a0) at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/eProsima/Fast-DDS/src/cpp/rtps/transport/shared_mem/SharedMemWatchdog.hpp:128
#9  0x00007ffff6992b20 in std::__invoke_impl<void, void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> (__f=@0x5555555dee80: (void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(class eprosima::fastdds::rtps::SharedMemWatchdog * const)) 0x7ffff692364a <eprosima::fastdds::rtps::SharedMemWatchdog::run()>, __t=@0x5555555dee78: 0x5555555dc2a0) at /usr/include/c++/9/bits/invoke.h:73
#10 0x00007ffff69925c9 in std::__invoke<void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> (__fn=@0x5555555dee80: (void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(class eprosima::fastdds::rtps::SharedMemWatchdog * const)) 0x7ffff692364a <eprosima::fastdds::rtps::SharedMemWatchdog::run()>) at /usr/include/c++/9/bits/invoke.h:95
#11 0x00007ffff69922f1 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> >::_M_invoke<0ul, 1ul> (this=0x5555555dee78) at /usr/include/c++/9/thread:244
#12 0x00007ffff6991c67 in std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> >::operator() (this=0x5555555dee78) at /usr/include/c++/9/thread:251
#13 0x00007ffff6990ffa in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (eprosima::fastdds::rtps::SharedMemWatchdog::*)(), eprosima::fastdds::rtps::SharedMemWatchdog*> > >::_M_run (this=0x5555555dee70) at /usr/include/c++/9/thread:195
#14 0x00007ffff765ede4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#15 0x00007ffff7360609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#16 0x00007ffff749a133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7ffff704ef40 (LWP 97912)):
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff739d859 in __GI_abort () at abort.c:79
#2  0x00007ffff7626911 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff763238c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff76323f7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff76326a9 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x000055555555cda1 in get_domain_id_from_node (node=std::shared_ptr<class rclcpp::Node> (use count 1, weak count 1) = {...}) at /home/aprotyas/dev/workspaces/ros2_playground_ws/src/ros2_playground/rclcpp_foxy_domain_id/src/main.cpp:61
#7  0x000055555555ceef in main () at /home/aprotyas/dev/workspaces/ros2_playground_ws/src/ros2_playground/rclcpp_foxy_domain_id/src/main.cpp:70
A debugging session is active.

	Inferior 1 [process 97912] will be killed.

Quit anyway? (y or n) y

Since it's pretty long, you can access it here too: https://pastebin.ubuntu.com/p/bFjpD3Rsn3/

@wjwwood
Copy link
Member

wjwwood commented May 31, 2022

@aprotyas you should check rcl_get_error_string() when an rcl function fails to see more info. The backtrace isn't useful because it's just showing us where you threw your own exception.

@aprotyas the latest version takes a pointer to const as the first argument:

https://github.com/ros2/rcl/blob/72cee8a9fbda40304ff4f8e7c5d12b6ce6410f8d/rcl/include/rcl/init_options.h#L152

vs foxy:

https://github.com/ros2/rcl/blob/9b0b151a49ed9013c78520e738658c948bd93116/rcl/include/rcl/init_options.h#L150

Maybe there are other backports missing.

@aprotyas
Copy link
Member

aprotyas commented May 31, 2022

@aprotyas you should check rcl_get_error_string() when an rcl function fails to see more info. The backtrace isn't useful because it's just showing us where you threw your own exception.

@wjwwood that's a good point, thanks! I figured out what happened from the rcl_get_error_string() and a Valgrind trace of my test program:

terminate called after throwing an instance of 'std::runtime_error'
  what():  init_options->impl argument is null, at /home/aprotyas/dev/workspaces/ros2_foxy_ws/src/ros2/rcl/rcl/src/rcl/init_options.c:149
Aborted (core dumped)
==98403== Memcheck, a memory error detector
==98403== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==98403== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==98403== Command: ./build/rclcpp_foxy_domain_id/rclcpp_foxy_domain_id
==98403== Parent PID: 1
==98403== 
==98403== Invalid read of size 8
==98403==    at 0x505C2CA: rcl_init_options_get_domain_id (init_options.c:149)
==98403==    by 0x110C4A: main (main.cpp:102)
==98403==  Address 0x5b1bc20 is 0 bytes inside a block of size 8 free'd
==98403==    at 0x483F1CF: operator delete(void*, unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==98403==    by 0x4DDB25D: std::default_delete<rcl_init_options_t>::operator()(rcl_init_options_t*) const (unique_ptr.h:81)
==98403==    by 0x4DDB13B: std::unique_ptr<rcl_init_options_t, std::default_delete<rcl_init_options_t> >::~unique_ptr() (unique_ptr.h:292)
==98403==    by 0x4DDACD5: rclcpp::InitOptions::~InitOptions() (init_options.cpp:78)
==98403==    by 0x110C22: main (main.cpp:98)
==98403==  Block was alloc'd at
==98403==    at 0x483DE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==98403==    by 0x4DDA9E1: rclcpp::InitOptions::InitOptions(rcl_init_options_t const&) (init_options.cpp:34)
==98403==    by 0x4DDAB19: rclcpp::InitOptions::InitOptions(rclcpp::InitOptions const&) (init_options.cpp:44)
==98403==    by 0x4DA088C: rclcpp::Context::get_init_options() (context.cpp:275)
==98403==    by 0x110BFD: main (main.cpp:98)

The problem is this line:

const rcl_init_options_t * rcl_init_options = 
    node.get_node_base_interface()->get_context()->get_init_options().get_rcl_init_options();

get_init_options() invokes an InitOptions copy constructor, which eventually calls:

InitOptions::InitOptions(const rcl_init_options_t & init_options)
: init_options_(new rcl_init_options_t)

Since this constructor allocates new memory, when this line terminates, the rclcpp::InitOptions instance goes out of scope and the memory allocated for the struct rcl_init_options_t is free'd.

Replacing the problematic line with a block that allows me to get domain ID before the InitOptions instance goes out of scope works properly:

const auto init_options = node->get_node_options().context()->get_init_options();                                                                                               
const rcl_init_options_t * rcl_init_options_ptr = __init_options.get_rcl_init_options();

// get_domain_id  

The error makes sense to me, but it was a bit unexpected. Should this behavior be expected?

aprotyas added a commit to ros2/domain_bridge that referenced this issue Jun 1, 2022
Reference: ros2/rclcpp#910 (comment)

Signed-off-by: Abrar Rahman Protyasha <[email protected]>
DensoADAS pushed a commit to DensoADAS/rclcpp that referenced this issue Aug 5, 2022
* Expose the Rosbag2QoS class for use

Signed-off-by: Geoffrey Biggs <[email protected]>
Co-authored-by: Emerson Knapp <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants