Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conflict between rmw_cyclone_cpp and rmw_implementation when building projects that rely on rclcpp #215

Open
Mechazo11 opened this issue Sep 21, 2024 · 8 comments
Labels
bug Something isn't working

Comments

@Mechazo11
Copy link

Version or commit hash

humble-2024.07.0

RMW Implementation

CycloneDDS

Steps to reproduce issue

Dear Space ROS maintainers.

Apologies in advance for the long message but replicating this error and its tentative solution, both requires some convoluted process.

As part of my NASA Space ROS Summer Sprint Challenge 2024 (space-ros/demos PR #64) submission, I demonstrated Open 3D Engine (O3DE) can successfully integrate with Space ROS (any ROS 2 variant to be more exact) via the ROS 2 Gem from O3DE project. ROS 2 Gem depends on the following ROS 2 packages

target_depends_on_ros2_packages(${gem_name}.Static rclcpp builtin_interfaces std_msgs sensor_msgs nav_msgs tf2_ros ackermann_msgs gazebo_msgs)
target_depends_on_ros2_package(${gem_name}.Static control_toolbox 2.2.0 REQUIRED)

However, during my build attempt in Docker, I came across a potential fatal bug. Apparently, there is a conflict between rmw_cyclonedds_cpp and rmw_implementation where it appears to find two instances of rmw_implementation::rmw_implementation. Please see the attached console output. I tried looking up this error online but did not find anything that resembles this exact problem. The closest I found was this: ros2/rmw_implementation#4

Reproducing this error is somewhat challenging as it happens only when compiling an O3DE project with ROS 2 Gem. To help reproduce this error I have created a public repository](https://github.com/Mechazo11/space_ros_rmw_conflict) which will build the rmw_conflict image based on osrf/space_ros base image. Please allot 30GB space and 20-25mins to building the rmw_conflict image. After the image has been built, follow these steps

  • Open a container: docker run -it rmw_conflict /bin/bash

  • Execute the following commands in the container's terminal

source /opt/ros/humble/setup.bash
source $SPACEROS_DIR/install/setup.bash
cmake -B build/linux -G "Ninja Multi-Config" -DLY_DISABLE_TEST_MODULES=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DLY_STRIP_DEBUG_SYMBOLS=ON -DAZ_USE_PHYSX5:BOOL=ON

The error message shown in the provided console output clearly shows rmw_cyclonedds_cpp and rmw_implementation are in odds with each other.

I was able to solve this problem but it involved deleting the Install directory of the base spaceros workspace and rebuilding it. This was done in the following steps

# Clone all space-ros sources
RUN mkdir ${SPACEROS_DIR}/src \
  && vcs import ${SPACEROS_DIR}/src < ${SPACEROS_DIR}/exact.repos

RUN sudo sh -c 'echo "deb http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/ros2-latest.list'
RUN curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -

# Clear Install folder, we will need to rebuild the workspace again
WORKDIR ${SPACEROS_DIR}
RUN sudo rm -rf install/

# Delete problematic packages
WORKDIR ${SPACEROS_DIR}/src
RUN sudo rm -rf rmw_cyclonedds_cpp
RUN sudo rm -rf rmw_implementation  
RUN sudo rm -rf rmw_cyclonedds
RUN sudo rm -rf realtime_tools

WORKDIR ${SPACEROS_DIR}

# Update rosdep
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  sudo apt-get update -y
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  sudo apt-get dist-upgrade -y
RUN rosdep update

# Install them back from apt
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  sudo apt install -y \
  ros-humble-rmw-cyclonedds-cpp \
  ros-humble-rmw-implementation \
  ros-humble-realtime-tools

# Rebuild Space ROS workspace
RUN /bin/bash -c 'source /opt/ros/humble/setup.bash \ 
    && colcon build --packages-ignore test_rmw_implementation'

# Ensure proper permission for all files in SPACEROS_DIR
RUN sudo chown -R ${USERNAME}:${USERNAME} ${SPACEROS_DIR}

The above steps while solving this RMW conflict issue also installs ros-humble-rmw-fastrtps-cpp that then requires to be manually removed.

I am not sure why this patch works but taking a look at the ros2.repos for the base image

cyclonedds:
    type: git
    url: https://github.com/eclipse-cyclonedds/cyclonedds.git
    version: 0.10.4
rmw:
    type: git
    url: https://github.com/ros2/rmw.git
    version: 6.1.2
  rmw_cyclonedds:
    type: git
    url: https://github.com/ros2/rmw_cyclonedds.git
    version: 1.3.4
  rmw_dds_common:
    type: git
    url: https://github.com/ros2/rmw_dds_common.git
    version: 1.6.0
  rmw_implementation:
    type: git
    url: https://github.com/ros2/rmw_implementation.git
    version: 2.8.4

could there be a potential bug in rmw_implementation version 2.8.4 (I checked it corresponds to humble branch for that repo)?

Opening a container of the rmw_conflict image described above and running apt show ros-humble-rmw-implementation gives the following output

Package: ros-humble-rmw-implementation
Version: 2.8.4-1jammy.20240728.212901
Status: install ok installed
Priority: optional
Section: misc
Maintainer: Audrow Nash <[email protected]>
Installed-Size: 117 kB
Depends: libc6 (>= 2.32), libgcc-s1 (>= 3.0), libstdc++6 (>= 5.2), ros-humble-ament-index-cpp, ros-humble-rcpputils, ros-humble-rcutils, ros-humble-rmw-implementation-cmake, ros-humble-ros-workspace, ros-humble-rmw-fastrtps-cpp | ros-humble-rmw-cyclonedds-cpp | ros-humble-rmw-connextdds
Download-Size: unknown
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status
Description: Proxy implementation of the ROS 2 Middleware Interface.

Any insight to this problem will be highly appreciated.

Expected behavior

ROS 2 Gem in O3DE finds the correct RMW without raising this error. Space ROS workspace does not need to be rebuilt and not binary version of the aforementioned packages needs to be installed.

Actual behavior

Install directory of base space ros image needs to be deleted. Multiple packages needs to be manually removed from the src file and then their ROS 2 binary counterpart needs to be installed. After that, when doing a colcon build, test_rmw_implementation package needs to be ignored.

Backtrace or Console output

-- Ros Distro is "humble"
-- Found rclcpp: 16.0.10 (/home/spaceros-user/spaceros/install/rclcpp/share/rclcpp/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found rmw_implementation_cmake: 6.1.2 (/home/spaceros-user/spaceros/install/rmw_implementation_cmake/share/rmw_implementation_cmake/cmake)
-- Found rmw_cyclonedds_cpp: 1.3.4 (/home/spaceros-user/spaceros/install/rmw_cyclonedds_cpp/share/rmw_cyclonedds_cpp/cmake)
-- Using RMW implementation 'rmw_cyclonedds_cpp'
-- Found builtin_interfaces: 1.2.1 (/home/spaceros-user/spaceros/install/builtin_interfaces/share/builtin_interfaces/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found std_msgs: 4.2.4 (/home/spaceros-user/spaceros/install/std_msgs/share/std_msgs/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found sensor_msgs: 4.2.4 (/home/spaceros-user/spaceros/install/sensor_msgs/share/sensor_msgs/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found nav_msgs: 4.2.4 (/home/spaceros-user/spaceros/install/nav_msgs/share/nav_msgs/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found tf2_ros: 0.25.7 (/home/spaceros-user/spaceros/install/tf2_ros/share/tf2_ros/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found rmw_implementation_cmake: 6.1.2 (/home/spaceros-user/spaceros/install/rmw_implementation_cmake/share/rmw_implementation_cmake/cmake)
-- Found rmw_cyclonedds_cpp: 1.3.4 (/home/spaceros-user/spaceros/install/rmw_cyclonedds_cpp/share/rmw_cyclonedds_cpp/cmake)
-- Using RMW implementation 'rmw_cyclonedds_cpp'
CMake Error at /home/spaceros-user/spaceros/install/rmw_implementation/share/rmw_implementation/cmake/rmw_implementation-extras.cmake:34 (add_library):
  add_library cannot create imported target
  "rmw_implementation::rmw_implementation" because another target with the
  same name already exists.
Call Stack (most recent call first):
  /home/spaceros-user/spaceros/install/rmw_implementation/share/rmw_implementation/cmake/rmw_implementationConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/rcl/share/rcl/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/rcl/share/rcl/cmake/rclConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/libstatistics_collector/share/libstatistics_collector/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/libstatistics_collector/share/libstatistics_collector/cmake/libstatistics_collectorConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/rclcpp/share/rclcpp/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/rclcpp/share/rclcpp/cmake/rclcppConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/rclcpp_components/share/rclcpp_components/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/rclcpp_components/share/rclcpp_components/cmake/rclcpp_componentsConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/tf2_ros/share/tf2_ros/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/tf2_ros/share/tf2_ros/cmake/tf2_rosConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/o3de-extras/Gems/ROS2/Code/ros2_target_depends.cmake:8 (find_package)
  /home/spaceros-user/spaceros/o3de-extras/Gems/ROS2/Code/ros2_target_depends.cmake:20 (target_depends_on_ros2_package)
  /home/spaceros-user/spaceros/o3de-extras/Gems/ROS2/Code/CMakeLists.txt:72 (target_depends_on_ros2_packages)


-- Found ackermann_msgs: 2.0.2 (/opt/ros/humble/share/ackermann_msgs/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found gazebo_msgs: 3.7.0 (/opt/ros/humble/share/gazebo_msgs/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found control_toolbox: 3.2.0 (/home/spaceros-user/spaceros/install/control_toolbox/share/control_toolbox/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Found rmw_implementation_cmake: 6.1.2 (/home/spaceros-user/spaceros/install/rmw_implementation_cmake/share/rmw_implementation_cmake/cmake)
-- Found rmw_cyclonedds_cpp: 1.3.4 (/home/spaceros-user/spaceros/install/rmw_cyclonedds_cpp/share/rmw_cyclonedds_cpp/cmake)
-- Using RMW implementation 'rmw_cyclonedds_cpp'
CMake Error at /home/spaceros-user/spaceros/install/rmw_implementation/share/rmw_implementation/cmake/rmw_implementation-extras.cmake:34 (add_library):
  add_library cannot create imported target
  "rmw_implementation::rmw_implementation" because another target with the
  same name already exists.
Call Stack (most recent call first):
  /home/spaceros-user/spaceros/install/rmw_implementation/share/rmw_implementation/cmake/rmw_implementationConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/rcl/share/rcl/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/rcl/share/rcl/cmake/rclConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/libstatistics_collector/share/libstatistics_collector/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/libstatistics_collector/share/libstatistics_collector/cmake/libstatistics_collectorConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/rclcpp/share/rclcpp/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/rclcpp/share/rclcpp/cmake/rclcppConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/install/control_toolbox/share/control_toolbox/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/spaceros-user/spaceros/install/control_toolbox/share/control_toolbox/cmake/control_toolboxConfig.cmake:41 (include)
  /home/spaceros-user/spaceros/o3de-extras/Gems/ROS2/Code/ros2_target_depends.cmake:8 (find_package)
  /home/spaceros-user/spaceros/o3de-extras/Gems/ROS2/Code/CMakeLists.txt:73 (target_depends_on_ros2_package)


-- Found parameter_traits: 0.3.8 (/home/spaceros-user/spaceros/install/parameter_traits/share/parameter_traits/cmake)
-- Found rclcpp_lifecycle: 16.0.10 (/home/spaceros-user/spaceros/install/rclcpp_lifecycle/share/rclcpp_lifecycle/cmake)
-- Downloading package into /home/spaceros-user/.o3de/3rdParty/packages/sdformat-13.5.0-rev2-linux
'/usr/bin/cmake' '-E' 'tar' 'xf' '/home/spaceros-user/.o3de/3rdParty/downloaded_packages/sdformat-13.5.0-rev2-linux.tar.xz'
-- Installed And Validated package at /home/spaceros-user/.o3de/3rdParty/packages/sdformat-13.5.0-rev2-linux - OK
-- Using package /home/spaceros-user/.o3de/3rdParty/packages/sdformat-13.5.0-rev2-linux
-- Found geometry_msgs: 4.2.4 (/home/spaceros-user/spaceros/install/geometry_msgs/share/geometry_msgs/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_introspection_c;rosidl_typesupport_fastrtps_c
-- Found rosidl_adapter: 3.1.5 (/home/spaceros-user/spaceros/install/rosidl_adapter/share/rosidl_adapter/cmake)
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_introspection_cpp;rosidl_typesupport_fastrtps_cpp
-- Downloading package into /home/spaceros-user/.o3de/3rdParty/packages/OpenMesh-8.1-rev3-linux
'/usr/bin/cmake' '-E' 'tar' 'xf' '/home/spaceros-user/.o3de/3rdParty/downloaded_packages/OpenMesh-8.1-rev3-linux.tar.xz'
-- Installed And Validated package at /home/spaceros-user/.o3de/3rdParty/packages/OpenMesh-8.1-rev3-linux - OK
-- Using package /home/spaceros-user/.o3de/3rdParty/packages/OpenMesh-8.1-rev3-linux
-- Configuring incomplete, errors occurred!
@Bckempa
Copy link

Bckempa commented Sep 22, 2024

Hi @Mechazo11, I haven't fully digested this whole issue yet but I already noticed something off that probably isn't helping the situation:

source /opt/ros/humble/setup.bash
source $SPACEROS_DIR/install/setup.bash

Space ROS is a separate ROS distribution and shouldn't be sourced as an overlay on any other ROS install the way it is overlaying humble here. We actually consider it a bug that there are humble components being apt-installed and are working to remove that and ensure we check for it in the future space-ros/docker#187

I suspect, but have not worked through your report enough to confirm, that this issue stems from the build system selecting dependencies from the underlay and overlay in an unsupported way.

The fact that you are apt installing the required packages instead of building any missing ones into a workspace on top of the Space ROS base is only going to further the problem as you are falling back on a different ROS disto's pre-pcakged bins rather than bringing the required deps into Space ROS.

@Mechazo11
Copy link
Author

Mechazo11 commented Sep 22, 2024

@Bckempa thank you very much for looking into this. Yes, I suspected this much why Space ROS preferred to source build. However at that time (and as of today), without the apt like patch, I cannot get O3DE's ROS 2 Gem to build which subsequently breaks the build. What can be the next step in troubleshooting this issue?

Update: I will try to update the Dockerfile for the rmw_conflict image tomorrow to build O3DE's required packages within a separate workspace following the example from openrobotics/space_robots_demo image.

@asimonov
Copy link

@Mechazo11, great writeup. i have to admit i did not read it entirely.

but can i suggest to try something to solve the original issue you face? not sure you have done this, but the following setup may be better:

  • start with space-ros container
  • source spaceros as underlay: source $SPACEROS_DIR/install/setup.bash
  • create your workspace to put the packages that you need to build the Gem, using humble version of those. use rosinstall/vcstool/rosdep route as we do in spaceros containers, but check that rosdep does not bring 'apt install ros-xxx' packages (you can run rosdep with -s option to see what it is running
  • build and source your workspace as overlay of spaceros
  • build the gems and the rest of your application

@Mechazo11
Copy link
Author

Hi @asimonov
No worries, after typing it up, even I was skeptical whether I would have read it myself. But I kept it nonetheless since the problem was confusing to me and caused me a lot of headache when I was trying to do my submission.

I recall trying this route by modifying the space_robots_demo container (demo_ws) but I will try it again by updating the docker file for the rmw_conflict image. Hopefully that will be more in line with Space ROS regulations and we can then proceed to finding out why the conflict occurs.

@Mechazo11
Copy link
Author

Mechazo11 commented Oct 8, 2024

Hi @asimonov and @Bckempa , apologies for the long delay but over the last weekend I was able to follow along the suggested steps from the discussion we had a few weeks earlier.

Even after ensuring I am not using any ros2-humble packages (rosdep install installs a bunch, I remove all of them later on) and correctly overlaying a colcon_ws on top of spaceros_ws base workspace, I am still getting rmw conflict error.

When convenient, can you kindly try out the following docker image to reproduce the error on your end. Please allot about 25GB of disk space.

  • Instructions for building the image
docker pull osrf/space-ros
cd ~/Downloads
git clone https://github.com/Mechazo11/space_ros_rmw_conflict.git
cd space_ros_rmw_conflict/
docker build -t rmw_conflict -f Dockerfile_rmw_conflict .
  • Open a docker container: docker run -it rmw_conflict /bin/bash

  • Execute the following commands

source ${SPACEROS_DIR}/install/setup.bash
source ${COLCON_WS_DIR}/install/setup.bash
cmake -B build/linux -G "Ninja Multi-Config" -DLY_DISABLE_TEST_MODULES=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DLY_STRIP_DEBUG_SYMBOLS=ON -DAZ_USE_PHYSX5:BOOL=ON

Hopefully with some guidance, I should be able to finish this integration and have Open 3D Engine correctly supported in Space ROS (and in extension to any source built derivative of ROS 2).

@Mechazo11
Copy link
Author

Hi @asimonov and @Bckempa

When convenient, would you kindly try out the docker file from the previous comment? I have tried to incorporate all the suggested steps but still saw the same problem. Maybe a fresh perspective would help us resolve this problem.

With best,
@Mechazo11

@asimonov
Copy link

asimonov commented Oct 18, 2024 via email

@Mechazo11
Copy link
Author

@asimonov yes you should see the build fail and reproduce the error shown in the first comment. From there is where I am confused on how to proceed in solving the error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants