Raise brush has steps

[Norodix]

Describe the bug

When applying the raise brush to a terrain it creates weird step patterns.

I first noticed it when the terrain became unexpectedly bumpy after editing, but the issue is most visible when using a small opacity brush on the same very small area for a few seconds. (see screenshot).

I suspected the reason is the small resolution brush but it seems that is not the case. I edited the raise shader to not use the bursh image but a gradient coded in the shader and I still run into the same issue.

Another weird artifact that I noticed is when trying to mess with this raise shader is that I cannot seem to be able to adjust the height by very small amounts. When I tried to add the following code for debug purposes, the height did not change at all, until I raised the brush_value to 0.125.

void fragment() {
	float brush_value = u_factor * u_opacity * texture(TEXTURE, UV).r;
	brush_value = 0.1;
	float src_h = texture(u_src_texture, get_src_uv(SCREEN_UV)).r;
	float h = src_h + brush_value;
	COLOR = vec4(h, 0.0, 0.0, 1.0);
}

I suspect this is related.

To Reproduce
Steps to reproduce the behavior:

1. Create a Hterrain
2. Grab the raise brush
3. set the size to about 200 and the opacity to 4
4. Try to slowly draw on the terrain

Expected behavior
The plugin produces a smooth surface without steps.

Screenshots

The hills were made with 20, 4 and 2 as opacity setting (left to right).

Environment
System:
Host: pop-os Kernel: 5.19.0-76051900-generic x86_64 bits: 64
Desktop: GNOME 42.3.1 Distro: Pop!_OS 22.04 LTS

Godot v3.5.stable
Plugin version: 1.6.1
GLES3

[Zylann]

This is likely to be expected. Terrain heights currently use half-precision floats (16-bit), so while it is enough to represent decent hills, doing a lot of very faint additive edits in the same place is prone to producing artifacts. Because very small values + done many times makes precision errors accumulate a lot more. Also, the higher you go, the lower the precision. There is no workaround for this currently, maybe apart from changing heightmap format, but that's currently not supported. I'd like to do that in the future.
For now maybe you can try using the smooth brush after creating the broad shape?

[Norodix]

I'm not sure that is the root cause of the issue. It might be but when I do the same operation on a sloped surface the steps dont appear at constant heights, they only appear radially.
So I think the intermediate values can be stored in this format.
What do you think?

[Zylann]

You already tested using a brush with full precision, so there aren't many possibilities left.
I can also confirm that the issue is more pronounced the higher you sculpt, which is more evidence that heightmap precision is the problem.

Doing many small strokes in the same spot accumulates precision errors because each stroke reads the existing height and puts it back in. If the change is too small to reach the smallest increment possible, it will not get raised. After a lot of iterations, the effect shows up as rings, because of the brush's falloff.

The issue can be mostly worked around with some smoothing:

[Norodix]

I dont see a reason why the height parameter's precision would be affected by the size of the brush.

Is there a reason the plugin is using half precision floats? Can this be changed relatively easily in the code to test if the problem is solved by 32bit floats?

[Zylann]

I dont see a reason why the height parameter's precision would be affected by the size of the brush.

The height of the ground you paint on has an effect (but not on the size of the brush (?)) because the higher a height is, the lower precision becomes. It's like floating point precision loss when getting far from the origin, same deal here.

Is there a reason the plugin is using half precision floats? Can this be changed relatively easily in the code to test if the problem is solved by 32bit floats?

It was good enough for a long time, and it uses twice as less memory than full floats. It is really all you need in a large variety of cases. This specific painting case doesnt grant doubling memory usage in-game, though indeed it is annoying.
I suppose it is relatively easy to replace all occurrences of RH in the plugin to RF, but 1) it will double memory usage in all the cases that were doing fine with RH, 2) it will break compatibility, 3) gives no choice.

At some point I considered using RGB8. 24-bit fixed-point is more precise, has the same precision at all heights, supports higher heights and fits in a standard PNG, although it needs conversion when accessed. Besides, if necessary, in the future it could be changed to RGBA8 where A stores additional information in a single fetch (but that's just an idea). It was also a good candidate for drivers not supporting float textures, although GLES2 is no longer being supported in Godot 4. Also I'm not sure about filtering.
Eventually conversion choices could be given on top of that, including 32-bit floats.

[Norodix]

I tried changing the image format to Image.RF (see image format). I might have missed something so take this with a grain of salt but look at the screenshot.

I marked 2 points. If the red point was not possible to represent due to floating point precision issues, how could the green point be represented? Why would it produce nice slopes along the steps?

If I understand correctly the brush is applied by using a background sprite, adding the brush sprite on top of it and rendering it to a viewport. It feels a bit like the brush sprite has a different format with only 8bit color, but I cannot find where the sprites texture's format is set. (or if can be specified at all).

[Zylann]

Unfortunately viewports use half-precision floats too (another reason why RGB8 could have been better). You'd need them to use full floats. The plugin uses viewports to make brushes fast. It isn't possible to increase them to full floats, so actually it might not be possible to do right now.

If the red point was not possible to represent due to floating point precision issues, how could the green point be represented? Why would it produce nice slopes along the steps?

As I said again, these points CAN be represented the way you expected them, the problem is that they are the result of many accumulations of small, similar errors of precision done on half-precision floats.

It feels a bit like the brush sprite has a different format with only 8bit color

Brushes use half-precision too I think, thats why they use EXR. But changing them to float would not have much impact. Feel free to check their format by printing it from the image I guess. It would be great if i was wrong about the rest, but AFAIK brushes are at least 16-bit. Also, even if they were 8-bit, that doesn't explain the fact precision degrades the higher you paint (which is rather explained by the format of the modified viewport data).

[Calinou]

Could dithering be added to make banding less visible? This would cause visible bumps to appear in terrain though.

The plugin uses viewports to make brushes fast. It isn't possible to increase them to full floats, so actually it might not be possible to do right now.

Viewports can use 32 bpc precision in 3.x, but not in 4.0 yet.

[Zylann]

Dithering helps when you render something once at low precision. Here it's an accumulation, so with dithering it would probably result in noise.

godotengine/godot#51708

Ah, so maybe this is the last bit to change in addition to using RF for textures.

[Norodix]

This absolutely solved the problem in my case!
Bumps on the right were made before the change, the big one on the left was after. (I had to reload the plugin)

Maybe doing the edits with full precision floats and storing the result with half precision as currently done could be implemented? The terrain did not feel jagged, only the edits so maybe this could be a useful change until the bigger work is started with the more detailed solution you mentioned (RGBA8, configurable precision, etc).

[Norodix]

Actually the original problem I had was smoothing didn't seem to work very well. That is how I found this issue and I was convinced it is related. I reported this because this was more obvious and reproducible.

On this image you can see that the peak on the right (with the brush on it) was smoothed with full float precision. It is nice and smooth. In the foreground you can see the peak that was smoothed with half precision has visible artifacts.

[Zylann]

It's probably a matter of degree, I havent seen these artifacts before. I noticed some imperfections sometimes, but for terrains, this wasn't a big deal since it's very rare for ground to be perfectly smooth. With textures and grass it's even less noticeable. Maybe they tend to also occur more with low opacities or high heights.

[Norodix]

I think it must be because of the higher heights, this previous picture was done with max opacity.

I noticed it because our character is a dog, so the character's up direction should match the terrain. Because of this it was bobbing up and down stupidly on a bumpy terrain.

Anyway, I'm happy to use the plugin modified for our use-case, so thank you for the help with figuring this out!
Do you think it is worth trying to move the edits to 32 bit floats for a less intrusive fix, or this was a weird use-case and not worth to modify the master branch because of it?

[Zylann]

Do you think it is worth trying to move the edits to 32 bit floats for a less intrusive fix

Unfortunately I think doing that is not possible at the moment, each edit gets stored back into the heightmap, they dont stay in the viewport. The viewport is only the size of the brush and can move arbitrarily.

What's more likely to happen is the ability to convert between formats, so you could work entirely using 32-bit floats, and convert at the end if desired. New terrains will default to 32, while old ones will remain 16, with option to convert.

[Norodix]

As far as I can tell, this has been done in v4, so it can be closed.
Thanks for the great addon again!