Have raycast in GDScript be able to intersect with non-collision mesh triangles and return the UV coordinate of the texture, as well as the standard info already present

[leiget]

Describe the project you are working on

A small Forest "walking simulator" where in the player simply walks around and views the scenery.

Describe the problem or limitation you are having in your project

I want to have different stepping sound effects depending on the texture that the player is stepping on. In some cases, I can simply find out the texture being used on the current mesh that is the parent of the StaticBody that was hit with a raycast. However, on some meshes I have a custom shader which has a "mask" texture which mixes 4 textures together. In this case, the current raycast is unable to return a UV position, therefore I am unable to find out exactly what texture is being stepped on.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I want to be able to get the UV position of the players position on the mask texture so I can find out what sound to play.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

# Cast a ray from the nodes 3D position to 1m below the node.
Ray_SpaceState = get_world().get_direct_space_state()`
Ray_From = self.get_global_transform().origin
Ray_To = self.get_global_transform().origin + Vector3(0.0, -1.0, 0.0)
# intersect_ray(
#           from: Vector3, to: Vector3,
#           exclude: Array = [  ], collision_mask: int = 0x7FFFFFFF,
#           collide_with_polys: bool = false, collide_with_bodies: bool = true, collide_with_areas: bool = false
#           )
Ray_Result = Ray_SpaceState.intersect_ray(Ray_From,Ray_To,[self],0x7FFFFFFF,true,false,false)

# Get the UV 
var uv_pos = Ray_Result.uv_position
print(str(uv_pos))

This would print something like (0.23151, 0.4431441)

The returned dictionary might look like this:

- collider_id: The colliding object's ID.
- normal: The object's surface normal at the intersection point.
- position: The intersection point.
- uv_position: The UV1 position at the intersection point.
- uv_position_2: The UV2 position at the intersection point.
- rid: The intersecting object's RID.
- shape: The shape index of the colliding shape.

If this enhancement will not be used often, can it be worked around with a few lines of script?

This will likely be used often, as it is often used piece of information.
Yes, it can be used the same way that a "regular" ray cast is done in GDScript.

Is there a reason why this should be core and not an add-on in the asset library?

Because the data that needs to be accessed in order to accomplish this doesn't seem to be available in GDscript. Also, performance would suffer greatly if not written in C++.

[jitspoe]

I'd actually like to have this from a tools perspective. Being able to click on meshes to spawn objects, paint textures, etc., without having to make collision for them would be great. I actually did something like this by exposing the intersect_ray function from the editor gizmos to GDScript, but it requires a camera and might only work in editor. I should probably do a pull request for that, at least, and maybe later a version that doesn't require the camera can be added.

This would be great for addons like @HungryProton's scatter.

[PauloVBettio]

Any workarounds or progress on this?

[Calinou]

godotengine/godot#56597 may help expose something like this to the user, but it doesn't retrieve UV to my knowledge.

[elim2g]

Would also like to know what a workaround may be. Godot's collision normals are so poor that I've resorted to raycasting on the collision points generated from collide_shape against a trimesh. Unfortunately that also gives smoothed, interpolated (rubbish) normals so I've precomputed a list of normals against the faces of the TriMesh. However there seems to be no way to then raycast into a mesh and retrieve its face so I can use my LUT to get an accurate, usable normal. Crawling through all these threads is making me awfully tired. Forgive the rant but it's honestly unreal how difficult it is to get a decent surface normal out of this engine.

Anyway, would like to know how to get a mesh's face at runtime. Cheers.

[Calinou]

Would also like to know what a workaround may be. Godot's collision normals are so poor that I've resorted to raycasting on the collision points generated from collide_shape against a trimesh. Unfortunately that also gives smoothed, interpolated (rubbish) normals so I've precomputed a list of normals against the faces of the TriMesh.

Smoothed normals are a Bullet issue, which should go away if you switch the 3D physics engine to GodotPhysics in the project settings: godotengine/godot#28032

If you have to stick to Bullet, decreasing the collision margin can also help mitigate this.

[elim2g]

Sure. I've had a lot of issues trying GodotPhysics (specifically with CylinderShape) but instead of polluting this issue with my nonsense I'll open issues with repro projects. Apologies for the nonconstructive rambling.

[thismarty]

This is a great proposal. I would add to it having raycast hits return the barycenter of the mesh triangle hit, which can augment having UVW data (as per the OP's request) as well as afford better surface tracking. Note that this is a well-used feature in Unity, for instance, where it has been implemented since version 1.0. Love to see it added to Godot!

[Zireael07]

@thismarty Barycenter of triangle is actually being implemented in #71233

[jitspoe]

Never got around to making a pull request for it, but these changes on my personal repo allow for using the intersect_ray function from gdscript:
jitspoe/godot@733ea0d
jitspoe/godot@514dadc

Doesn't cover the UV coordinates, but allows for a lot more functionality from tools.

[DigitOtter]

I made a godot module to determine the mesh triangle that's under the mouse cursor:
https://github.com/DigitOtter/gd_module_triangle_ray_select.

I think the module can be used to access UV coordinates at a mesh-ray intersection point. I added an (untested) example function to a project at https://github.com/DigitOtter/gd_triangle_ray_select_example (_get_uv_coordinates in main.gd).

The module accesses the vertex buffer after skeleton and blendshape deformations have been applied, so the ray intersection test works on the the same model as what the player sees. I'm using a compute shader to iterate over the mesh triangles, which should make it somewhat efficient.

The module is still a work-in-progress. It only works with the Vulkan rendering backend, and it requires a patch to the Godot source code so that it can access the vertex buffers after skeleton and blendshape deformations.