Restore energy loss in indirect (IBL) specular lighting via multiscattering approximation

[jsantell]

Currently, for metal surfaces (MeshPhysicsMaterial, MeshStandardMaterial), as the roughness of the material increases, the surface darkens if lit via IBL+PMREM (possibly #7489?). For fully metal surfaces (conductors), all incoming irradiance should equal outgoing radiance (F0=1) -- like a mirror (metalness=1, roughness=0). Even if the metal is rough (metalness=1, roughness=1), the same amount of energy should be reflected off of the material.

There is the white furnace test, which has a uniformly lit enironment with fully metal, white spheres with varying degrees of roughness. As these surfaces are fully metal, all incoming light must be reflected, and should be indistinguishable from the environment. If the spheres are visible, then some energy is either lost or gained.

Currently, energy is lost as metal surfaces increase in roughness:

BRDFs only account for a single scattering of light -- for rough, metal surfaces, light will bounce around microfacets after the initial scattering before reaching the viewer, which isn't accounted for in BRDF models.

via Filament's beautiful docs

There are several sampling and preprocessing methods that aid in real-time approximation of multiscattering light, but ultimately, I was able to get Fdez-Agüera's method (A Multiple-Scattering Microfacet
Model for Real-Time Image-based Lighting) working well, and used a DFG approximation from Filament instead of a DFG LUT .

Above, on the left is r100, on the right is this branch -- from top to bottom, metal surface with increasing roughness, rough surface with increasing metalness, metal surface with increasing roughness, nonmetal increasing rough surface). Much more energy is retained on rough, metal objects with this change.

Above is the single-scattering specular indirect lighting only (left), only the multiscattering (center), and both (combined). Currently in r100, only the single-scattering is visible.

Perhaps the least exciting image, above is the furnace test with these changes. All incoming light equals the outgoing light, making the spheres invisible.

Now a handful of questions:

• Should this be MeshPhysicalMaterial only?
• What should the conditions for this be? Currently, it's defined( USE_ENVMAP ) && defined( ENVMAP_TYPE_CUBE_UV ), but mostly I'm checking to see if a radiance map exists.
• Needs a good correctness check
• Performance check (seems good, but needs a rigorous test)
• Is the DFG approximation valid?
• Do we need to separate the indirect specular and diffuse lighting functions? They're combined here to save computations.
• Maybe this should only be on desktop, depending on perf

[WestLangley]

Sweet. Thanks for this!

Should this be MeshPhysicalMaterial only?

We'll see. It depends on the performance ramifications.

Your remaining questions are basically rhetorical...

If you want to refactor the shader code, that would be fine with me. I find the #define RE_Direct chunks, etc., somewhat confusing.

[mrdoob]

@bhouston @spidersharma03 do these changes look good to you?

[jsantell]

Some perf testing, it looks like this might be slighty slightly less performant than the current implementation, but practically the same. Haven't given too much of an eye to optimization yet. It'd be helpful if someone could check it out on iOS.

Test case, 64 objects mostly filling the screen

• https://indirect-perf.glitch.me/ (for testing PR)
• https://indirect-perf.glitch.me/?r100 (for testing current r100 build)

Mobile Results

device	browser	current FPS	proposed FPS
Pixel 2	Chrome r71	~59	~59
Pixel 2	Firefox v64	~57	~57
Pixel 1	Chrome r71	~39	~38
Pixel 1	Firefox v64	~47	~46

Linux Laptop, 800x800

browser	DPR	current FPS	proposed FPS
Firefox v64	1	~58	~58
Firefox v64	2	~32	~30
Firefox v64	3	~19	~19
Chromium v71	2	~44	~43

If you want to refactor the shader code, that would be fine with me. I find the #define RE_Direct chunks, etc., somewhat confusing.

Sounds great, I can take care in a follow up

[WestLangley]

FWIW: iPhone 6, Safari: ~48 fps current; ~42 fps proposed.

[donmccurdy]

iPhone SE, Safari: 33fps before and after.

[bhouston]

Is there any way this could be precalculated into the ibl map, even in an approximate fashion just to maintain energy?

…

On Sat, Jan 26, 2019, 3:54 PM Don McCurdy ***@***.*** wrote: iPhone SE, Safari: 33fps before and after. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15644 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAj6_Y5Ow25EnqG9rE62eM14HNvsq4Glks5vHMCcgaJpZM4aT26j> .

[looeee]

Moto G6 (Adreno 506): 6fps before and after

[jsantell]

Updated PR with preserveEnergy boolean on MeshPhysicalMaterial that #define PRESERVE_ENERGY's when set with a PMREM so it could possibly be more opt in. Also consolidated the integrated BRDF generation code which was mostly what the current indirect specular was generating.

Also as a quick double check, this is what the DFG approximation looks like generated in integrateBRDF:

[mrdoob]

Shouldn't preserveEnergy be true by default? The improved output seems worth the slight loss of fps.

[WestLangley]

Shouldn't preserveEnergy be true by default? The improved output seems worth the slight loss of fps.

I'd hardwire it, then, and remove the preserveEnergy flag.

[jsantell]

Shouldn't preserveEnergy be true by default? The improved output seems worth the slight loss of fps.

Wasn't sure about the (small) performance delta, and figured this would be the best way for opt in -- happy to remove the flag if should be enabled for all MeshPhysicalMaterials with PMREMs -- which maybe MeshPhysicalMaterial over MeshStandardMaterial is sufficient opt-in for potentially higher cost for more accurate renderings

[mrdoob]

I think this should apply to both MeshStandardMaterial and MeshPhysicalMaterial and be enabled by default. Good looking by default is best. If people need extra ms they could opt-out.

[jsantell]

I think this should apply to both MeshStandardMaterial and MeshPhysicalMaterial and be enabled by default. Good looking by default is best. If people need extra ms they could opt-out.

SGTM -- should it continue being a boolean for folks to disable?

[mrdoob]

Is there any other reason for that boolean apart from a few extra ms? If not, I would remove the boolean altogether (and avoid complexity).

[jsantell]

@mrdoob that's the only reason, and maybe my lack of confidence with physics 😄

[jsantell]

@selfshadow saw your comment right before I force pushed latest changes, so sorry if I blew it away! I did update the coefficient to 1/21 however. Thanks!

@mrdoob Removed preserveEnergy property and enable multiscatter for both Standard/Physical if ENVMAP_TYPE_CUBE_UV defined

[jsantell]

Updated the styling of the glsl strings 👍

[mrdoob]

Thanks!

[mrdoob]

webgl_materials_standard looks definitely much better.

Before:

https://raw.githack.com/mrdoob/three.js/r100/examples/webgl_materials_standard.html

After:

https://raw.githack.com/mrdoob/three.js/dev/examples/webgl_materials_standard.html

However, webgl_materials_envmaps_exr and webgl_materials_envmaps_hdr now have some sort of dark fresnel?

Before:

https://raw.githack.com/mrdoob/three.js/r100/examples/webgl_materials_envmaps_exr.html

After:

https://raw.githack.com/mrdoob/three.js/dev/examples/webgl_materials_envmaps_exr.html

Before:

https://raw.githack.com/mrdoob/three.js/r100/examples/webgl_materials_envmaps_hdr.html

After:

https://raw.githack.com/mrdoob/three.js/dev/examples/webgl_materials_envmaps_hdr.html

[jsantell]

@mrdoob noted, will take a look!

[WestLangley]

webgl_materials_standard looks definitely much better.

What exactly are you seeing? I see no visible difference. Maybe I am missing something...

I have only seen significant visible difference in real-world examples (that is, non-furnace tests) for rough-metallic surfaces. Maybe @jsantell has seen otherwise...

[mrdoob]

webgl_materials_standard looks definitely much better.

What exactly are you seeing? I see no visible difference. Maybe I am missing something...

If you open both links in different tabs and switch between one and the other you'll see the new version has more light details.

[looeee]

The difference is extremely subtle. Using www.diffchecker.com/image-diff on webgl_materials_standard:

[jsantell]

webgl_materials_standard looks definitely much better.

What exactly are you seeing? I see no visible difference. Maybe I am missing something...

I have only seen significant visible difference in real-world examples (that is, non-furnace tests) for rough-metallic surfaces. Maybe @jsantell has seen otherwise...

The only lighting that's changed is an extra specular lobe for metal surfaces so the change is most noticeable on very rough metallic surfaces, where previously the specular light was lost

[jsantell]

Potential fix for the dark fresnel border: #15673

[Mugen87]

@jsantell I think we should note in the migration guide that the rendering of MeshPhysicsMaterial/ MeshStandardMaterial slightly changed with R101. How would you phrase the most important aspects of the PR?

[jsantell]

@Mugen87 to take a crack at it, "MeshStandardMaterial and MeshPhysicalMaterial now preserve energy for IBL lighting, resulting in brighter, more accurate colors for metallic materials with high roughness values when lit via a map generated by PMREMGenerator."

[Mugen87]

Perfect! I've added your suggestion to the guide.

[mrdoob]

@looeee

The difference is extremely subtle.

I guess my eyes can see extremely subtle things at this point 😅