Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Addons: Add TRAAPassNode. #29636

Merged
merged 17 commits into from
Oct 15, 2024
Merged

Addons: Add TRAAPassNode. #29636

merged 17 commits into from
Oct 15, 2024

Conversation

Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Oct 13, 2024

Fixed #14050.
Related issue: #29295.

Description

This PR rewrites TAARenderPass as TRAAPassNode for WebGPURenderer.

It is now a full TRAA approach meaning it uses reprojection based on a velocity pass. TRAAPassNode also implements color clamping based on neighborhood sampling as well as flicker reduction based on luminance weighing. The implementation could be further enhanced e.g. by variance clipping, subpixel sampling or post sharpening but for now it's a good start.

There are some open issues:

  • WebGL is not supported yet since the pass needs the ability to copy between render targets. A few days ago @gkjohnson coincidentally filed the respective issue: WebGLRenderer: Allow for copying data between render targets using "copyTextureToTexture" #29612.
  • Like any other TRAA implementation, TRAAPassNode uses jitter to produce a frame with a slight offset which is used for accumulation. However, the jitter should only affect the beauty pass. Right now, the pass injects a non-modified projection matrix into VelocityNode to get accurate results. However, depth and any other additional MRT output should not receive jitter as well.

To solve 2, one solution would be an unjittered pre-pass for velocity, depth, normals etc.. The beauty is then produced with a separate pass that incorporates the jitter. That means however only beauty will receive the AA, depth and normals not. I'm not sure how other engines handle this aspect but I think it would be one way to go.

Copy link

github-actions bot commented Oct 13, 2024

📦 Bundle size

Full ESM build, minified and gzipped.

Before After Diff
WebGL 690.29
171.01
690.29
171.01
+0 B
+0 B
WebGPU 815.35
219.78
816.05
219.94
+703 B
+164 B
WebGPU Nodes 814.86
219.65
815.56
219.81
+703 B
+165 B

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

Before After Diff
WebGL 463.31
111.89
463.31
111.89
+0 B
+0 B
WebGPU 538.09
145.28
538.61
145.41
+524 B
+127 B
WebGPU Nodes 494.2
135.15
494.73
135.28
+524 B
+128 B

@sunag
Copy link
Collaborator

sunag commented Oct 14, 2024

MRT support is not complete. TRAAPassNode uses MRT to produce the beauty and velocity pass but it does not honor any existing MRT settings from the app. So it's not possible e.g. to render normal or metalness data yet.

Hmm.. could you give me an example?

@Mugen87
Copy link
Collaborator Author

Mugen87 commented Oct 14, 2024

I have refactored the MRT handling a bit so TRAAPassNode is now more similar to SSAAPassNode. Meaning it has no internal MRT setting anymore but uses the one from the app.

However, because of the intermediate sample render target that gets a jitter applied, the result isn't correct. We can't use the same approach like in SSAAPassNode where we resolve all targets via blending. I guess we need two passes like mentioned above:

  • Use an overwrite material to render the original MRT setup without jitter (prepass).
  • Render beauty and depth with jitter (this depth is only needed in the next step).
  • Resolve and copy the result into output.

I've implemented this via 172c2ee but unfortunately, I get a Uncaught RangeError: Maximum call stack size exceeded runtime error in updateBefore() 😟 .

@Mugen87
Copy link
Collaborator Author

Mugen87 commented Oct 14, 2024

Reverting the prepass approach for now. I have the feeling we need a different approach like configuring what MRT outputs should be affected jitter and which not. In this way, we would not need any updates to VelocityNode.

@Mugen87
Copy link
Collaborator Author

Mugen87 commented Oct 15, 2024

I'll go ahead and merge since adding full MRT support might take a while until we figure out a solution. In the meanwhile, TRAAPassNode works in WebGPU and WebGL so it can already be included in certain FX setups.

@Mugen87 Mugen87 merged commit b9144b9 into mrdoob:dev Oct 15, 2024
12 checks passed
@Mugen87 Mugen87 added this to the r170 milestone Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Temporal Reprojection Anti-Aliasing (TRAA)
2 participants