Groundbased_rgbd_people_detector

[aichim]

@nizar-sallem , @mmunaro

I am playing with the people detector, and the plane selection is really messed up. I click on some points, and get completely random points in the scene. They seem to be rolled around the z-axis by 180 degrees (see the screenshot). But I cannot seem to find the problem in the code. Nizar, can you have a look, the code is super short in apps/main_ground_base_rbgd_people_detector.cpp ?

Thanks guys!

[mmunaro]

Hi Alex,

sorry for the late reply.
It is a weird behavior, It does not happen to me...
I experienced that the points can be badly selected when there is an object more or less on the same direction but nearer than the ground plane point that I am selecting.

Try to click on the ground plane where the point cloud is more dense and be sure that there are no obstacles near there (maybe zooming on the ground plane could help).

Cheers,
Matteo

[mmunaro]

Hi Alex,

I just updated my code with the latest trunk and I experienced the same error you reported!
With trunk version before the 28th of May the clicking on the ground plane points was working, but now something should have been changed somewhere...
The Kinect point cloud seems to be upside down when it is shown by the visualizer with respect to how it was visualized some days ago.

Do you think that Nizar can understand what could be the cause?

Cheers,
Matteo

[mmunaro]

The code is working (the coordinate of the points is correct), but the visualization of the Kinect point cloud or of the clicked points is wrong (z coordinate is inverted). From the image below, it seems that Kinect point cloud is wrongly visualized (points have negative z value). It seems to be caused by a change to the visualizer, but I do not know how to fix it and I do not think I should change my code.

[nizar-sallem]

I agree with the explanation given above by mmunaro seems kinect point clouds are flipped.
I am trying to get a fix for other stuff we noticed not working properly maybe this will be fixed as well. Please be patient just a little more, I am compiling/testing right now, it is a bit long cause my laptop is not super fast.

[mmunaro]

Hi Nizar,
did you manage to understand what is causing this problem?

Cheers,
Matteo

[mmunaro]

Hi Alex and Nizar,
the visualization problem started after this commit:

Merge pull request #92 from nfioraio/master
Better support to point cloud pose setting in PCLVisualizer
789fb08

The last good and bad commits were:
last good: b16dff1
last bad: a49df4c

What was meant to be introduced with this commit?

Cheers,
Matteo

[jspricke]

Hi Matteo,

it doesn't really make sense to identify the merge as the broken commit. Is there some easy test, so I can try it as well?

[mmunaro]

Well, I was using git bisect, but I found some not compiling versions of the code, thus it could not finish...
However, if you run pcl_ground_based_rgbd_people_detector in the bin folder with a Kinect/Xtion attached, you can easily check for the problem:

• if you see the point cloud in the visualizer at the beginning, then the point cloud is correctly visualized
• if you have to turn the visualizer upside down for seeing the point cloud, then there is the problem.

[jspricke]

Hi,

based on Matteos tests I've found that it would be enough ti remove this and the next line to fix this: https://github.com/PointCloudLibrary/pcl/blob/master/visualization/include/pcl/visualization/impl/pcl_visualizer.hpp#L1247. On the other hand, this is part of a bigger change to support the sensor origin fields, so maybe adding an appropriate one would help as well. @nfioraio what's your opinion on this? Can you come up with a solution?

[mamoto]

hi, another issue flip in addPointCloudNormals

[mmunaro]

In my opinion, here, the normals are correctly drawn, but the point cloud is flipped.
It is the same issue as in the people detector: the clicked points and the people bounding boxes are correctly drawn, but the point cloud is flipped.

[mamoto]

i had same issue in the people detector point cloud flipped. did anyone solve this issue ? (*_^)

[jspricke]

Sounds like #92 needs some more work. I'm reverting it for now. Please report back if this fixes all problems.

[mmunaro]

Ok, I will let you know tomorrow, because I cannot test it now.

[mamoto]

it solved!! thank you!!

[jspricke]

@nfioraio in #156:

Dear all, I didn't read all the posts, but my pull was just about applying the sensor pose stored in the pcl::PointCloud objects using the standard VTK transformations. May the 180° rotation be related to some approx error due to the quaternion --> 3x3 matrix conversion? I haven't experienced anything like this on my pc but...

Such a big error in the conversion would be really bad. I would rather guess that some of the point clouds we are trying to visualize don't have the right viewpoint, or no viewpoint at all. Nicola, could you have a look into this? I would really like to have your #92 back in. Thanks!

[nfioraio]

Ok, here I am! I'm not sure I got the problem.

if you see the point cloud in the visualizer at the beginning, then the point cloud is correctly visualized
if you have to turn the visualizer upside down for seeing the point cloud, then there is the problem.

Can you better explain how do you turn the viewer? Do you move it by hands (ie using the mouse) or do you call some function? The sensor origin/orientation is simply copied, but I don't know what exactly happens under the hood. Also, can you check that the sensor origin/orientation are set to some value (no NaN, no e-32 etc.)?

Thx!

[mmunaro]

Hi Nicola,

I mean rotating by hand, moving the mouse.
I experienced this problem when grabbing Kinect point clouds with pcl::OpenNIGrabber.

Apart from the fact that the point cloud changed orientation in the visualizer with that commit, the problem is that it is no more consistent with the PointPickingEvent and the points and rectangles that I plot afterwards.
Have a look at the image I posted above or run pcl_ground_based_rgbd_people_detector in the bin folder with a Kinect/Xtion attached and you should be able to see the problem (if you use a version before the commit reversal).

I am currently recompiling in order to see if the commit reversal solved the problem.

[mmunaro]

Ok, I confirm that reverting the commit solved the visualization issue.
The Kinect point cloud has origin [0 0 0 0] and orientation [0 0 0 1].

[nfioraio]

Thank you for reporting the issue. I'm working on this!

[nfioraio]

I found the same error running the openni_viewer. However, it seems that vtk correctly applied the transformation because the sensor_orientation_ field IS NOT SET TO IDENTITY ROTATION but to (1,0,0,0) instead. See here: https://github.com/PointCloudLibrary/pcl/blob/master/io/src/openni_grabber.cpp#L618

[jspricke]

Good find Nicola! Is that the only place we have this problem? Can you prepare a pull request to fix this? @rbrusu you wrote the line in question, is there any problem with changing it?

[nfioraio]

I commented out those lines and the visualization works fine now. I'll make a pull request for removing those lines, then @mmunaro and @mamoto please try to apply pull #92 and check if everything is fine. By the way, pointclouds acquired and saved with that orientation should be fixed to be visualized in the right way.

[rbrusu]

I believe that is the correct orientation for Kinect data (1,0,0,0). Also, it must have been set by Suat not me.

[mamoto]

I build version 08/07 finished a few minutes ago, i used Asus Xtion and my results look very good, reversal was solved. thank you and I have found a small issues, can not create pcl_people_debug.lib , I had to create this file manually. please fix this.

[mmunaro]

What is pcl_people_debug.lib for? Are you using Windows?

[mamoto]

Yes, I use windows 8 with visual studio 12

[mmunaro]

Maybe you should build in debug mode for generating that kind of file?
Have you tried to do the cmake with:
cmake -DCMAKE_BUILD_TYPE=Debug ..
and then compile the library?

[mamoto]

Yes. I always build type debug. All libraries create automatically without people. I checked properties build setting, it's ok. At the moment, I make it manually. Change dynamic dll to static library and .dll to. Lib

[mmunaro]

The only difference in the CMakeLists.txt with respect to the other modules is the target_link_libraries line.
Could you please add this line at line 46 (after PCL_ADD_LIBRARY) of the CMakeLists.txt of the people library, recompile and see if it works?
target_link_libraries(${LIB_NAME} pcl_common pcl_kdtree pcl_search pcl_features pcl_sample_consensus pcl_filters pcl_io pcl_visualization pcl_geometry pcl_segmentation pcl_octree)

[mamoto]

@mmunaro I did as your guide then recompiled it, but still didn't produce a library file. I have been compared with other modules and didn't found differently. it is difficult to understand.

[nfioraio]

@rbrusu

I believe that is the correct orientation for Kinect data (1,0,0,0).

I really don't understand why we need an extra rotation for kinect data. A reference frame is just a convention and if you say that the kinect points should be rotated, then the user should use such rotated points, instead of the original ones, when using the cloud with others coming from different sources, otherwise, ie you use the original points, the sensor_pose_ would have no meaning.

[mmunaro]

@mamoto: are you following this tutorial? http://pointclouds.org/documentation/tutorials/compiling_pcl_windows.php
It seems that the compiled libraries should be dll by default, but you could uncheck PCL_SHARED_LIBS.
It should be also said that the pcl_people library contains only the hog.cpp file so far. The rest of files are .h and .hpp because they contain templated classes, thus they are not contained within the pcl_people library.

However, I am not familiar with PCL compilation in Windows.
@jspricke: do you think that a pcl_people_debug.lib should be produced?

[jspricke]

@mmunaro: No. As far as I can see, there is nothing special in the people/CMakeLists.txt so it's not our problem.
@mamoto: Can you delete your build dir and try it with a clean checkout?

[mamoto]

Yes, I have followed the steps in the tutorial and create a new build. I think the problem lies in the file hog.cpp, maybe I need to reconsider.

please add #define _USE_MATH_DEFINES at line 43 (#include < math.h >) in hog.h file

[aichim]

I confirm that the visualization problems have gone away with the latest trunk.
Thanks guys!

Closing this as the discussion went off-topic. @mamoto , please open another issue if you are still having trouble.