[Godot Physics] Simulating rigidbody composed of multiple collision shapes gets all its coordinates set to NaN

[e344fde6bf]

Godot version:
3.3rc6, 3.3.rc.custom_build.cca2637b9

Couldn't reproduce this on 3.2.3

OS/device including version:
N/A

Issue description:

If you simulate a rigibody that uses multiple collision shapes, after some time all it's position and velocity values will be set to NaN, and the console will spam error messages:

ERROR: set_axis_angle: The axis Vector3 must be normalized.
   At: core/math/basis.cpp:980.
ERROR: instance_set_transform: Condition "Math::is_nan(v.x)" is true.
   At: servers/visual/visual_server_scene.cpp:723.

This only happens using the Godot physics backend.

If you turn on drawing of collision shapes, you can see that it's drawing contact points between the rigid bodies own collision shapes:

Steps to reproduce:

• Create a rigidbody that is composed of multiple collision shapes
• Simulate it for some time and wait for the error messages. The simulation is not deterministic, so it doesn't always happen.

Minimal reproduction project:
rigid-compound-shape-bug.zip

[akien-mga]

I can reproduce the issue, it seems to be have introduced between 3.2.4 beta 5 and 3.2.4 beta 6.

Changelog between the two: a18df71...7e207cf

From that list #44901 seemed like a potential candidate, and indeed disabling 3d/godot_physics/use_bvh in the project settings fixes the regression for this project. CC @lawnjelly @pouleyKetchoupp

[qarmin]

I was able to track it only to

godot/servers/physics/body_sw.cpp

Lines 581 to 582 in 5fdb8b0

	void BodySW::integrate_velocities(real_t p_step) {

Looks that in this function linear_velocity is full of nans and putting this lines with breakpoint should show more information's:

if (Math::is_nan(linear_velocity.x)){
	ERR_FAIL_COND(Math::is_nan(linear_velocity.x)); // Put breakpoint here
}

[qarmin]

Finally I found that nan and inf values are generated because in this line

godot/servers/physics/collision_solver_sat.cpp

Line 339 in 5fdb8b0

real_t dist = -(clip.normal.dot(edge0_A) - clip.d) / den;

den is equal to 0

[lawnjelly]

Sorry it's taken a while to get to this.

It looks like there's a check I've missed in the BVH .. if the calling userdata is the same for 2 colliding objects, the callbacks should not be sent (the userdata being the physics object, the table I guess?), which must be the mechanism used to prevent self collision. This should be easy enough to fix.

The spam from the physics may be this self intersection situation it cannot deal with for some reason.

[akien-mga]

Fixed by #47614.