No detect output from yolox

[beginningfan]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I'm convinced that this is not my fault but a bug.

Description

I try to start the tensorrt_yolox node. When I set preprocess_on_gpu here to true,the ouptput in /perception/object_recognition/detection/rois0 is empty. When I set it to false, it works well.

Expected behavior

When I launch tensorrt_yolox with preprocess_on_gpu is true,the rois in output is not empty

Actual behavior

Steps to reproduce

use ros2 launch tensorrt_yolox yolox.launch.xml to launch this node

Versions

OS: Ubuntu
ROS2: humble
Autoware: latest

Possible causes

No response

Additional context

No response

[liuXinGangChina]

share the rosbag.

[beginningfan]

The bag was uploaded here https://drive.google.com/file/d/1Tp-cUaLl7m0XHDsvItfPMVkY-dWtGg7j/view?usp=drive_link

[liuXinGangChina]

Hi mits-san, will you help us by assign someone to take a look at this issue

[badai-nguyen]

@beginningfan
Thank you for your raising this issue. However, I tested the latest autoware' tensorrt_yolox on two options of preprocess_on_gpu, both worked well on my side. Could you check around gpu and tensorrt seting and testing again?

preprocess_on_gpu=true:

preprocess_on_gpu=false:

My environment:
OS: Ubuntu
ROS2: humble
Autoware: latest

autoware@npc2201009-ubuntu22:~/autoware$ dpkg -l | grep TensorRT
hi  libnvinfer-dev                                    8.4.2-1+cuda11.6                                  amd64        TensorRT development libraries and headers
hi  libnvinfer-plugin-dev                             8.4.2-1+cuda11.6                                  amd64        TensorRT plugin libraries
hi  libnvinfer-plugin8                                8.4.2-1+cuda11.6                                  amd64        TensorRT plugin libraries
hi  libnvinfer8                                       8.4.2-1+cuda11.6                                  amd64        TensorRT runtime libraries
hi  libnvonnxparsers-dev                              8.4.2-1+cuda11.6                                  amd64        TensorRT ONNX libraries
hi  libnvonnxparsers8                                 8.4.2-1+cuda11.6                                  amd64        TensorRT ONNX libraries
hi  libnvparsers-dev                                  8.4.2-1+cuda11.6                                  amd64        TensorRT parsers libraries
hi  libnvparsers8                                     8.4.2-1+cuda11.6                                  amd64        TensorRT parsers libraries
ii  ros-humble-tensorrt-cmake-module                  0.0.3-1jammy.20240516.164153                      amd64        Exports a CMake module to find TensorRT.
autoware@npc2201009-ubuntu22:~/autoware$ dpkg -l | grep nvinfer
hi  libnvinfer-dev                                    8.4.2-1+cuda11.6                                  amd64        TensorRT development libraries and headers
hi  libnvinfer-plugin-dev                             8.4.2-1+cuda11.6                                  amd64        TensorRT plugin libraries
hi  libnvinfer-plugin8                                8.4.2-1+cuda11.6                                  amd64        TensorRT plugin libraries
hi  libnvinfer8                                       8.4.2-1+cuda11.6                                  amd64        TensorRT runtime libraries

[beginningfan]

@badai-nguyen Thanks for your reply.
I tested it with your environment and it worked well. But I used this docker image before, the TRT version is different from yours. It dosen't work.

root@Dell:/my_autoware# dpkg -l | grep TensorRT
hi  libnvinfer-dev                                    8.6.1.6-1+cuda12.0                      amd64        TensorRT development libraries
hi  libnvinfer-headers-dev                            8.6.1.6-1+cuda12.0                      amd64        TensorRT development headers
hi  libnvinfer-headers-plugin-dev                     8.6.1.6-1+cuda12.0                      amd64        TensorRT plugin headers
hi  libnvinfer-plugin-dev                             8.6.1.6-1+cuda12.0                      amd64        TensorRT plugin libraries
hi  libnvinfer-plugin8                                8.6.1.6-1+cuda12.0                      amd64        TensorRT plugin libraries
hi  libnvinfer8                                       8.6.1.6-1+cuda12.0                      amd64        TensorRT runtime libraries
hi  libnvonnxparsers-dev                              8.6.1.6-1+cuda12.0                      amd64        TensorRT ONNX libraries
hi  libnvonnxparsers8                                 8.6.1.6-1+cuda12.0                      amd64        TensorRT ONNX libraries
hi  libnvparsers-dev                                  8.6.1.6-1+cuda12.0                      amd64        TensorRT parsers libraries
hi  libnvparsers8                                     8.6.1.6-1+cuda12.0                      amd64        TensorRT parsers libraries
ii  ros-humble-tensorrt-cmake-module                  0.0.3-1jammy.20240516.164153            amd64        Exports a CMake module to find TensorRT.

[badai-nguyen]

@beginningfan Thank you for your confirmation.
BTW, is there any error logging when you tested on 8.6.1.6-1+cuda12.0?

[beginningfan]

@badai-nguyen There is no useful information in the log.
I save all output on screen to this tenerrt_yolox.log

[miursh]

I use the same version as @beginningfan. Dai's version is a bit outdated; you should update the NVIDIA-related libraries.

$  dpkg -l | grep TensorRT
hi  libnvinfer-dev                                              8.6.1.6-1+cuda12.0                            amd64        TensorRT development libraries
ii  libnvinfer-headers-dev                                      8.6.1.6-1+cuda12.0                            amd64        TensorRT development headers
ii  libnvinfer-headers-plugin-dev                               8.6.1.6-1+cuda12.0                            amd64        TensorRT plugin headers
hi  libnvinfer-plugin-dev                                       8.6.1.6-1+cuda12.0                            amd64        TensorRT plugin libraries
hi  libnvinfer-plugin8                                          8.6.1.6-1+cuda12.0                            amd64        TensorRT plugin libraries
hi  libnvinfer8                                                 8.6.1.6-1+cuda12.0                            amd64        TensorRT runtime libraries
hi  libnvonnxparsers-dev                                        8.6.1.6-1+cuda12.0                            amd64        TensorRT ONNX libraries
hi  libnvonnxparsers8                                           8.6.1.6-1+cuda12.0                            amd64        TensorRT ONNX libraries
hi  libnvparsers-dev                                            8.6.1.6-1+cuda12.0                            amd64        TensorRT parsers libraries
hi  libnvparsers8                                               8.6.1.6-1+cuda12.0                            amd64        TensorRT parsers libraries
ii  ros-humble-tensorrt-cmake-module                            0.0.3-1jammy.20230919.195325                  amd64        Exports a CMake module to find TensorRT.

As a result, the model (yolox-sPlus-opt-pseudoV2-T4-960x960-T4-seg16cls.onnx) worked without any issues, as shown in this image below.

I'm not sure what is happening but could you try with this PR ?

[beginningfan]

@miursh Thanks for your replay.
I used the cmakelists.txt in the PR you mentioned, it still doesn't work. I tried in two different devices and two different docker image(20240618-devel-cuda-amd64 and 20240708-devel-cuda-amd64), the results are the same. Where did you compile and run this node? Can you try it in one of these two dockers?

[miursh]

@beginningfan
I'm sorry to hear that you're still encountering issues. I compiled and ran the node on my local machine (Ubuntu22.04), not within a Docker container.
I wil try running it in the Docker images you mentioned.

[liuXinGangChina]

hi miura-san，any feedback for “run yolox node in docker(20240618-devel-cuda-amd64 and 20240708-devel-cuda-amd64) ” ？ @miursh

[miursh]

@liuXinGangChina
Sorry for the late response.
I tried with ghcr.io/autowarefoundation/autoware:20240708-devel-cuda-amd64 but the node died with following error

[ERROR] [tensorrt_yolox_node_exe-2]: process has died [pid 89, exit code 127, cmd '/autoware/install/tensorrt_yolox/lib/tensorrt_yolox/tensorrt_yolox_node_exe --ros-args -r __node:=tensorrt_yolox --params-file /tmp/launch_params_ghjqgbg6 --params-file /tmp/launch_params_t05htsl5 -r ~/in/image:=/sensing/camera/camera0/image_rect_color -r ~/out/objects:=/perception/object_recognition/detection/rois0 -r ~/out/mask:=/perception/object_recognition/detection/mask0'].

It seems the build process does not complete successfully...
I'm not familiar with the Docker usage in Autoware, so I'm not sure what to do.
Did you run colcon build inside the Docker container?

[miursh]

@liuXinGangChina @beginningfan
I finally managed to run YOLOX inside the Docker container.
As a result, it seems that there are some values in rois, and it is not always zero.

[storrrrrrrrm]

test on code up to this commit:083ed2f , it works fine

container version:

output: