Add Transform Helper and Posterize VisualShader nodes

[paddy-exe]

Describe the project you are working on

Godot Engine and VFX

Describe the problem or limitation you are having in your project

There are several Nodes that are commonly used in other engines that makes creating specific VFX easier.
Furthermore some aspects of the Visual Shader system such as transforming between coordinate spaces are not VFX-Artist friendly.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I propose the following Visual Shader Nodes for better support:

• Transform Helper Node (name open for discussion)
  Unity example:
  
• Posterize (Quantize Node)
  

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The Transform Helper Node will be different if you are in the Vertex or fragment stage and will change the available space conversions accordingly. The underlying code will be switched out depending on a drop-down options like World Space -> View Space or View Space -> Local Space etc.

For the Posterization node here is the code:

vec4 posterize(vec4 col, int steps) {
	return (floor(col * float(steps)) / float(steps - 1));
}

If this enhancement will not be used often, can it be worked around with a few lines of script?

Not as easily for the Transform node but the Posterize node could be

Is there a reason why this should be core and not an add-on in the asset library?

Ease of use for VFX artists

[LiveTrower]

I'll leave these functions here in case they're helpful. The functions perform each of the conversions between local, world, view, and tangent spaces.

//tbn_local = mat3(TANGENT, -BINORMAL, NORMAL);
//tbn_world = mat3(mat3(MODEL_MATRIX) * TANGENT, mat3(MODEL_MATRIX) * -BINORMAL, mat3(MODEL_MATRIX) * NORMAL);
//tbn_view = mat3(TANGENT, -BINORMAL, NORMAL); in fragment()

vec3 local_to_world(mat4 model_matrix, vec3 x){
	return (model_matrix * vec4(x, 1.0)).xyz;
}

vec3 local_to_view(mat4 view_matrix, vec3 x){
	return (view_matrix * vec4(x, 1.0)).xyz;
}

vec3 local_to_tangent(mat3 tbn_local, vec3 x){
	return normalize(x * tbn_local);
}

vec3 world_to_local(mat4 inv_model_matrix, vec3 x){
	return (inv_model_matrix * vec4(x, 1.0)).xyz;
}

vec3 world_to_view(mat4 view_matrix, vec3 x){
	return (view_matrix * vec4(x, 1.0)).xyz;
}

vec3 world_to_tangent(mat3 tbn_world, vec3 x){
	return normalize(x * tbn_world);
}

vec3 view_to_local(mat4 model_matrix, mat4 inv_view_matrix, vec3 x){
	return (inverse(model_matrix) * (inv_view_matrix * vec4(x, 1.0))).xyz;
}

vec3 view_to_world(mat4 inv_view_matrix, vec3 x){
	return (inv_view_matrix * vec4(x, 1.0)).xyz;
}

vec3 view_to_tangent(mat3 tbn_view, vec3 x){
	return normalize(x * tbn_view);
}

vec3 tangent_to_local(mat3 tbn_local, vec3 x){
	return normalize(tbn_local * x);
}

vec3 tangent_to_world(mat3 tbn_world, vec3 x){
	return normalize(tbn_world * x);
}

vec3 tangent_to_view(mat3 tbn_view, vec3 x){
	return normalize(tbn_view * x);
}

Note: I might have made a mistake in some of the conversions so be careful.

[tetrapod00]

The transform helper is implemented in godotengine/godot#97215. Only model, world, view, and clip for now. No tangent space yet.

[LiveTrower]

@tetrapod00 I have been working on conversions to tangent space, and I've seen different approaches taken by other engines:

Approach in Unity:

Unity performs conversions to tangent space based on world space, according to the documentation and the code it generates:

The TBN matrix is formed using the tangent, bitangent, and normal in world space, so the conversions would be something like this:
Object -> World -> Tangent
World -> Tangent
View -> World -> Tangent
Screen -> View -> World -> Tangent
Absolute World -> World -> Tangent

Tangent -> World -> Object
Tangent -> World
Tangent -> World -> View
Tangent -> World -> View -> Screen
Tangent -> World -> Absolute World

The vector to be converted needs to be transformed to world space before converting it to tangent space.

Approach in Unreal:

Unreal is not clear on how it handles tangent space, but based on my tests, it seems to be done in local space:

Unreal_tangent_space.mp4

Thus, the TBN matrix is formed using the tangent, bitangent, and normal in local space, and the conversions would be something like:
Object -> Tangent
World -> Object -> Tangent
View -> World -> Object -> Tangent
Camera -> View -> World -> Object -> Tangent

...

As I said, I'm not sure since Unreal doesn't have this part well documented; I'm just assuming based on what I see.

Possible approach in Godot:

Since #97215 has a very similar approach to Unity, we could adopt its approach. However, Godot has a somewhat problematic detail, which is that by default, in the vertex shader, the inputs VERTEX, NORMAL, BINORMAL, and NORMAL are in local space. But Godot has an option called "world_vertex_coords" that converts all these inputs to world space. This means we would need to detect if the user has this option enabled, and based on that, either perform the local-to-world conversion or directly pass the vectors.

The TBN matrix can be created in the vertex function and passed to fragment and light through a varying, for example:

shader_type spatial;

varying mat3 TBN;

void vertex(){
   #if world_vertex_coords is not enabled
   vec3 t = (MODEL_MATRIX * vec4(TANGENT, 0.0)).xyz;
   vec3 b = (MODEL_MATRIX * vec4(BINORMAL, 0.0)).xyz;
   vec3 n = (MODEL_MATRIX * vec4(NORMAL, 0.0)).xyz;
   TBN = mat3(normalize(t), normalize(-b), normalize(n));

   #if world_vertex_coords is enabled
   TBN = mat3(TANGENT, -BINORMAL, NORMAL);
}

Unity performs many peculiar calculations to transform between spaces, and to achieve similar results in Godot, I applied some concepts they use:
Type Position
In Unity, when converting to tangent space in position mode, they don't simply apply w = 1.0, but rather subtract the world-space position from the vector before converting to tangent space. Conversely, when reversing from tangent space, they add the position back after reversing from tangent space.

Type Direction
In Unity, when converting to tangent space in direction mode, they don't simply apply w = 0.0, but rather convert the matrices to 3x3, and by default, they always normalize the result for better performance, although they offer an option to disable this.

There are several other details like the condition all(isfinite(_Transform_Out_1_Vector3)), which I have no idea what it does, along with other specifics.

Fortunately, I was able to replicate the results from Unity quite closely. Here's my implementation(https://github.com/LiveTrower/godot/blob/tests/scene/resources/visual_shader_nodes.cpp#L2824-L3120) I hope it was clear and helpful.

Note: I couldn't find a way to detect the "world_vertex_coords" option.

[tetrapod00]

@LiveTrower Thanks for looking into this! If we want to add tangent space as an option, this looks like a pretty good implementation (though I haven't looked too deeply yet).

However, I think implementing tangent space conversions should wait until there is a clear user need. The current conversions definitely are needed; I implemented it because I saw confusion about the current node in a help chat. But I'm not sure how much demand there is for a tangent space conversion in visual shaders. I've seen people asking for it in text shaders, though.

Also, ideally the existence of tangent space, and how it is meant to be used, should be more documented in the text shader docs first. Currently tangent space is only mentioned in the docs three times, all talking about ANISOTROPY or BaseMaterial3D.anisotropy_enabled.

If it turns out there is a need for tangent space, I think it can be done as a quick followup to the current VectorCoordinateTransform node. As currently implemented, it can easily be extended in the future to add tangent space (with the code you've already written). Alternately, since all the transformations to and from tangent space include a Tangent to World or World to Tangent conversion, we may want to make the Tangent space conversion node a separate, dedicated node.

[LiveTrower]

@tetrapod00 In my experience, I have seen that tangent space is used in many shaders in Unity and Unreal. In the case of Godot, its use is quite rare, and this could be due to what you mentioned: the lack of documentation and possibly the lack of experience among many Godot users. Additionally, Godot also uses tangent space in parallax occlusion mapping since this effect requires it, so we could conclude that tangent space is not something that should be overlooked as it can be vital for certain effects.

However, it would be better if the rendering team reviewed this case.

Since you mentioned making it a specialized node, I've been thinking about clip space because it is just as complex as tangent space, given that it can have several uses. In your PR, I see that when you try to convert from clip space to world space or clip space to local space, it doesn't give the expected result:

The result it gives

The result it should give

Now, from what I saw, the correct way is to divide like this: vec3 ndc = (clipSpacePos.xyz / clipSpacePos.w); instead of only passing the xyz axes. The result is closer to what we are looking for but remains inconsistent.

ClipToWorldIssue.mp4

I was researching and couldn't find how to solve this problem, as it happens due to data loss when converting to clip space. Unity has a mysterious function that solves this problem, but it is very difficult to decipher.

Note: It seems I found something that could help here

[tetrapod00]

Ah, you're right. That means that for correct clip space conversions, the node will need vec4 inputs and outputs when using clip space, which complicates the current clean design of always converting vec3 to vec3... Maybe the current design where every space can be converted to every other space needs to be reconsidered.