Does render_stereo work?

[koenvaneijk]

I tried setting the arg of render_stereo to 1, however I got an error that two nodes had the same name (camera_left_link). So I went into core.launch and updated the name to camera_right_link. However, then I got the following error:

[ WARN] [1549799539.366419806, 10.066667472]: Could NOT find required uav to camera transform: "uav/camera/center" passed to lookupTransform argument target_frame does not exist. 
[uav/flightgoggles_ros_bridge-7] process has died [pid 17535, exit code 1, cmd /home/koen/catkin_ws/devel/lib/flightgoggles_ros_bridge/ROSClient __name:=flightgoggles_ros_bridge __log:=/home/koen/.ros/log/4b31e4ca-2d2a-11e9-983e-38d5471a2e1e/uav-flightgoggles_ros_bridge-7.log].
log file: /home/koen/.ros/log/4b31e4ca-2d2a-11e9-983e-38d5471a2e1e/uav-flightgoggles_ros_bridge-7*.log

I then went to the contents of ROSClient.cpp and on line 39 I find the following:

//imagePubRight_ = it_.advertiseCamera("/uav/camera/right/image_rect_color", 1);

I tried un-commenting the line above and publishing the contents in the same manner as the left-camera, but with index of 1 (renderOutput.images[1]), however it appears that the rendering engine is not rendering more than one image.

I could deep-dive in the Unity code from an earlier commit (https://github.com/mit-fast/FlightGoggles/tree/042b4a6f5331fa5bf57ec023df530d167226360a/FlightGoggles), however I'm affraid that this might be an older version and incompatible with the current ROSClient.cpp, which might send me even further down the rabbit hole. Even if I would make changes, I understand that I need some copyrighted Unity assets (rendering effects) to rebuild the binary for the simulator.

I tried subscribing to the CompressedImage in /uav/camera/left/image_rect_color/compressedDepth with Python and reading it in an image buffer with OpenCV. but I just get the following output from the image_transport:

Compressed Depth Image Transport - Compression requires single-channel 32bit-floating point or 16bit raw depth images (input format is: bgr8).

My question is:

• Is there a right camera, or depth image, being rendered by Unity? If so: how can we access it? If not: will it be added?

Looking forward to your help!

Kind regards,
Koen

[Winter-Guerra]

Stereo and depth have been temporarily disabled. It might be enabled in the future for the AlphaPilot challenge, but not guaranteed. If you'd like stereo in the challenge, you should ask Lockheed about it. Best, -Winter

…

On Sun, Feb 10, 2019, 07:22 Koen van Eijk ***@***.***> wrote: I tried setting the arg of render_stereo to 1, however I got an error that two nodes had the same name (camera_left_link). So I went into core.launch and updated the name to camera_right_link. However, then I got the following error: [ WARN] [1549799539.366419806, 10.066667472]: Could NOT find required uav to camera transform: "uav/camera/center" passed to lookupTransform argument target_frame does not exist. [uav/flightgoggles_ros_bridge-7] process has died [pid 17535, exit code 1, cmd /home/koen/catkin_ws/devel/lib/flightgoggles_ros_bridge/ROSClient __name:=flightgoggles_ros_bridge __log:=/home/koen/.ros/log/4b31e4ca-2d2a-11e9-983e-38d5471a2e1e/uav-flightgoggles_ros_bridge-7.log]. log file: /home/koen/.ros/log/4b31e4ca-2d2a-11e9-983e-38d5471a2e1e/uav-flightgoggles_ros_bridge-7*.log I then went to the contents of ROSClient.cpp and on line 39 I find the following: //imagePubRight_ = it_.advertiseCamera("/uav/camera/right/image_rect_color", 1); I tried un-commenting the line above and publishing the contents in the same manner as the left-camera, but with index of 1 ( renderOutput.images[1]), however it appears that the rendering engine is not rendering more than one image. I could deep-dive in the Unity code from an earlier commit ( https://github.com/mit-fast/FlightGoggles/tree/042b4a6f5331fa5bf57ec023df530d167226360a/FlightGoggles), however I'm affraid that this might be an older version and incompatible with the current ROSClient.cpp, which might send me even further down the rabbit hole. Even if I would make changes, I understand that I need some copyrighted Unity assets (rendering effects) to rebuild the binary for the simulator. I tried subscribing to the CompressedImage in /uav/camera/left/image_rect_color/compressedDepth with Python and reading it in an image buffer with OpenCV. but I just get the following output from the image_transport: Compressed Depth Image Transport - Compression requires single-channel 32bit-floating point or 16bit raw depth images (input format is: bgr8). My question is: - Is there a right camera, or depth image, being rendered by Unity? If so: how can we access it? If not: will it be added? Looking forward to your help! Kind regards, Koen — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#32>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOwrl87WAUkvzIoTjhshT7_-FyW3LYzks5vMA7ngaJpZM4azB9L> .

[koenvaneijk]

Hi @Winter-Guerra,

Thank you for your quick reply.

Unfortunate to hear it has been disabled in the 2.0 version of FlightGoggles, we'll have to find a workaround then.

I'll reference this issue from the challenge forum in case anyone else is having the same questions.

Kind regards,
Koen

[varunmurali1]

Closing this issue, please see reply from herox for discussion on the extent of visual processing required.