Street Map Creation with Loop Closure

[TR7]

Describe the problem
i like to generate a map of some streets. i took 500+ pictures with a wide angle camera every 0.5 seconds while walking along streets. when i come back to a street where i have been before, meshroom apparently does not calculate a loop closure.

my question: is there an option to activate loop closing? or another tool or workflow to create such a map?

Screenshots

Dataset
if it helps, i can upload all photos from the dataset. but the matching between the photos looks good.

Desktop (please complete the following and other pertinent information):

• OS: win 10
• Meshroom version: release 2019.2.0

[natowi]

Do you have enough image overlap between start and end of you route?

The problem could also be related to your wide angle camera (distortion).

is there an option to activate loop closing?

By default all images are being matched against each other.

0.5 seconds? When walking, 1-5 seconds may be enough.

[TR7]

By default all images are being matched against each other.
you mean meshroom has by default a loop closure algorithmic? i was not sure..

i have around 30 meter overlap, see screenshot on the right.
And the split of the path is not in the overlaping region of the photos but somewhere else.

I also tried to see if there was a matching problem at the split point. I simply loaded 20 images separately into Meshroom before and after the cut point and there was no problem with the reconstruction. so i was assuming it's a loop closure problem from meshroom.

0.5 seconds? When walking, 1-5 seconds may be enough.
i was not sure because of the current problem, so i decided to go down to 0.5 secs.. just to make sure there is definitly no matching problem. but also same problem with 1+ secs.

[TR7]

The problem could also be related to your wide angle camera (distortion).

thats maybe possible. i'm using one side of a gopro fusion.
here are two successive pictures at the separated location:

[simogasp]

how many images are reconstructed out of the 500+?

[natowi]

Try to make sure you are not visible in the images to avoid false matches. Try to walk in the blind spot.

You could try to convert your fisheye images:
#526

[TR7]

how many images are reconstructed out of the 500+?

537 / 559 Images
But at the "Cut-Area" there are all surrounding images reconstructed.

[TR7]

Try to make sure you are not visible in the images to avoid false matches. Try to walk in the blind spot.

i removed it with a black mask, painted on the image in the region where i was walking. just send you the original images to not confuse you.

and i don't think the matching is the problem because of the test with the 2x20 selected images from the problematic region with in itself it works well

[simogasp]

ok, effectively it could be a problem with the loop closure.
Normally in SfM the loop closure is possible thanks to the global bundle adjustment, but here I'm afraid the drift is too large to be corrected. Or we need to tweak the parameters a bit
Would be possible for you to share the dataset, it is an interesting case fisheye + loop closure?

[TR7]

Would be possible for you to share the dataset, it is an interesting case fisheye + loop closure?
of course! but will take some time today. via PM?

[natowi]

You can send a link to the private mailing list [email protected]

[simogasp]

as u prefer, u can male a wetransfer/fromsmash, if u cannot make it public use a pm

[MikeZski]

Hi,
I did this kind of thing before but not in Meshroom. This kind of aligment needs some special approach. I recommend to use also normal camera to help with proper aligment. Shoot some photos where you want to close the loop. With normal camera shoot mostly the ground. From 1 to 2m is optimal distance between the 360 photos. You don't need to avoid your apperance in the view. Take as much groud as you can. I used tripod that I was holding on my arm so camera was like 1,5m behind me. If you are using your hand hold camera under the hand. I used also timelapse. It is good to make two passes with camera looking at the ground from different angles. Use two sides of your 360 camera and use it as a calibrate rig. Don't point camera like this, it should be rotated 90 degrees, or point it directly to the ground.

[TR7]

ok, effectively it could be a problem with the loop closure.
Normally in SfM the loop closure is possible thanks to the global bundle adjustment, but here I'm afraid the drift is too large to be corrected. Or we need to tweak the parameters a bit

ok, i have tested alot: i calibrated the camera parameters more precisely and locked the parameters for distortion. also i used the rig configuration -> both led to significantly more matching points and more precise SfM. So the drift should now not be too large, because it matches really close:

but something interessting happens:
In SfM only 1048 of the 1110 cameras are "calibrated". but it seems that the cameras were simply removed by the SfM instead of loop closing while they were actually matched geometrically really stable.
What i mean is: when I reconstructed exactly these images (the misssing cameras) separately in Meshroom, there were no problems with SfM.

It seems as if the loop closing here, due to the large graph, would rather play it safe and remove the cameras than make a minimal correction.

maybe the size of the structur-graph is to big to connect the loop?
do you know a parameter for tweaking that?

i found the "localBAdistanceGraph" and "localizer max ransac error" parameters and tried out the whole range but without visible change in the reconstruction

[anon-14]

I was wondering what types of calibrating for the camera you did. I just posted about trying to do some similar work with Google Street Data and how poor my results have been coming out.

[fabiencastan]

i found the "localBAdistanceGraph" and "localizer max ransac error" parameters and tried out the whole range but without visible change in the reconstruction

Have you tried to disable the localBA?
The Local BA is a strategy to reduce the number of cameras considered in the Bundle Adjustment.
localBAdistanceGraph represents the degree of connections between images to reduce the number of cameras considered in the BA. For instance, a degree of 2 between 2 images means that they do not have common matches, but they both have matches to a common intermediate image.