Proposal: Allow drop shadows to respect z-depth

[kmelmon]

Proposal: [Allow drop shadows to respect z-depth]

This proposes a mechanism for allowing apps to control the "fake depth" applied by drop shadows by using today's z-depth.

Abstract
When ThemeShadow was first introduced, it came with the concept of depth. Internally, projected shadows are rendered by the DWM as a 2.5D primitive, and use the z-depth as an input. The greater the z-depth, the bigger the shadow.

The guidance we’ve been giving customers is to set UIElement.Translation.Z to control ThemeShadow’s depth. But any 3D transform, including an old style RenderTransform will also work. Also, nesting elements with z-depth accumulates and the total depth is used.

Starting in Cobalt, XAML introduced a new implementation for ThemeShadow which draws simpler shadows as "drop shadows" (think of it like a bitmap that draws with the content). With drop shadows, the DWM no longer draws shadows at all. Instead XAML creates a SpriteVisual and sets a NineGrid as its brush. Although these shadows do have the concept of depth, this simpler depth is not 2.5D, and is essentially just a simple mapping to insets on the NineGrid brush. For the purposes of this doc, we’ll call this treatment of depth a “fake depth”, as it’s not tied in any way to 3D perspective.

The drop shadow rendering code does not respect z-depth at all, and there is currently no public API for controlling the fake depth. This document proposes a solution.

The ideal solution would allow app code to continue working with no changes. The proposal is a slight compromise but will honor most of today’s behavior. To summarize the proposal:

• Drop shadows will respect any non-animating Translation.Z value set on the element[MC1][KM2] with a ThemeShadow set on it.
• Animating Translation.Z will not animate fake depth.
• Depending on customer needs, we’ll consider adding a mechanism for a drop shadow to animate its opacity, allowing apps to animate the drop shadow into view (typically when it first enters the live tree). Several options being considered:
  o Add a new ThemeShadow.Opacity property
  o Auto-detect animation of Translation.Z and map this animation to an Opacity animation
  The rest of this document describes today’s design, outlines how it will move to respect non-animating Translation.Z values, and outline other changes needed for a complete implementation.

Overview of drop shadow parts:
High level summary of today’s drop shadow rendering code:

• XAML creates an offscreen Composition Visual. This visual is rendered in the DWM, it draws up to 2 Composition drop shadows. The “recipe” from design is implemented into this captured visual tree (more details below).
• XAML captures this offscreen Visual with a CompositionVisualSurface and uses it as the Source for a CompositionNineGridBrush.
• XAML creates a SpriteVisual, sets its Brush to the above captured CompositionNineGridBrush and puts this SpriteVisual into the live visual tree.

Recipe
Design provided a “recipe” for shadow drawing. Here’s a list of the parts of the recipe that are derived from depth:

Part	Purpose/Value	Notes
Elevation (aka “fake depth”)	8, 16, or 64	Elevation can be set via private API: default: 16 Tooltip: 8 ContentDialog: 64
Ambient Shadow	Draws border around control	Only used at high elevation
Opacity	Discrete value, depends on Elevation	Uses conditionals
Color	Function of Opacity
Directional Shadow	Draws shadow	Always Used
Opacity	Discrete value, depends on Elevation	Uses conditionals
BlurRadius	Same as Elevation
Y offset	Function of Elevation
Color	Function of Opacity
Insets	Function of Blur Radius of both shadows and Y offset	Uses max

Offscreen Visual
Here’s an XML description of the offscreen Visual:

<ContainerVisual>               <==Contains both Ambient and Directional Shadows
    <Clip = GeometricClip>     <==Clips ShapeVisuals down to just shadow  -->
    <LayerVisual>                     <==Ambient Shadow 
        <Shadow = DropShadow>
        <ShapeVisual>
           <CompositionSpriteShape>
               <Geometry = CompositionRoundedRectangleGeometry>
               <FillBrush = CompositionColorBrush(opaque black)>  <==Provides source of shadow pixels
           </CompositionSpriteShape>
        </ShapeVisual>
    </LayerVisual>
    <LayerVisual>    <==Directional Shadow
        <Shadow = DropShadow> 
        <ShapeVisual>
            <CompositionSpriteShape>
                <Geometry = CompositionRoundedRectangleGeometry>
                <FillBrush = CompositionColorBrush(opaque black)>  <==Provides source of shadow pixels
           </CompositionSpriteShape>
        </ShapeVisual>
    </LayerVisual>
</ContainerVisual>

Composition properties
Here’s a table showing all the Composition properties in this tree that are derived from depth:

Part/Property	Purpose/Value	Notes
RoundedRectangleGeometry.Size	Function of AmbientBlurRadius, DirectionalBlurRadius, and CornerRadius	Uses max and ceiling
Ambient ShapeVisual	Draws rounded rectangle for ambient shadow	Not [MC1] [KM2] present at low elevations
Offset	Function of Recipe.Insets
Size	Function of Recipe.Insets
Ambient LayerVisual	Draws drop shadow for ambient shadow	Not present at low elevations
Size	Function of Recipe.Insets
DropShadow.BlurRadius	Function of Recipe.AmbientBlurRadius
DropShadow.Color	Function of Recipe.AmbientColor
Directional ShapeVisual	Draws rounded rectangle for directional shadow	Always present
Offset	Function of Recipe.Insets
Size	Function of Recipe.Insets
Directional LayerVisual	Draws drop shadow for directional shadow	Always present
Size	Function of Recipe.Insets
DropShadow.BlurRadius	Function of Recipe.DirectionalBlurRadius
DropShadow.Color	Function of Recipe.DirectionalColor
ContainerVisual	Holds all the visuals
Size	Function of Recipe.Insets
Clip	Function of Recipe.Insets	Uses D2D Geometry which isn’t a Composition object
VisualSurface	Captures ContainerVisual
SourceSize	Function of Recipe.Insets
RealizationSize	Function of Recipe.Insets
NineGridBrush	Holds VisualSurface
Insets	Function of Insets

The proposal here is to feed any non-animating value for Translation.Z (on the element with a ThemeShadow set only) into Recipe instances, and keep all the code above unchanged. This has the consequence that animating Translation.Z will have no effect on fake depth. Only the “final value” of Translation.Z will be respected. An alternative design that would respect animating fake depth would be to convert all of the above property values into ExpressionAnimations, but there are several problems with this:

1. The biggest problem is that the ContainerVisual’s Clip is generated by D2D. This geometry is not independently animatable, and would require re-generation on every frame of the animation. This is a non-starter. We’d need to explore other ways of achieving the same effect.
2. Dev Cost: There are a large number of properties that would need to change to be driven by ExpressionAnimations. A number of these use conditionals and other functions (max, ceiling, etc), so converting to ExpressionAnimations would be a complex, time consuming task.
3. Performance: The current design just uses static values for all of the above properties, converting all these properties to ExpressionAnimation-driven values would carry a perf cost of computing all the animating values in the DWM. It is likely not prohibitively expensive but certainly more expensive than today’s model.
   Conclusion: drop shadow rendering is best left as UI-thread-only. Ideas for animating fake depth are explored below (see “Replacement for animating depth”).

Other limitations of this proposal worth calling out:

• The older Transform3D [MC5][KM6]property will be ignored for the purposes of drop shadows. Apps must use Translation.Z to set the fake depth of a drop shadow.
• The Translation.Z value on ancestor elements will be ignored (ie accumulation of depth is not supported). Translation.Z must be set on the same element that has a Shadow set.

NineGridBrush Caching
Each unique NineGridBrush is cached for performance reasons. The cache is a map that uses the Recipe instance as the key. Thus Elevation must be a known, constant value in order to cache the brush. If depth is animating, there is no way to cache the brush. We could create a separate NineGridBrush during animations and switch to a cached one when the animation completes, but this adds more complexity, perf cost, and thus strengthens the case for ignoring animating depth values.
The proposal here is to keep this code as is and rely on Recipe receiving only non-animating Elevation values.

Windowed Popups
The bounds of the HWND for windowed Popups is made larger to accommodate a drop shadow. This area is transparent and the drop shadow visual is positioned in this transparent area.

Currently, for the sake of simplicity, the HWND bounds is always inflated by 64 DIPs on each edge to account for a maximum shadow inset of 64. There are no known customers that want to set a depth > 64 and that depth produces a very large shadow, it doesn’t seem useful to go any larger. So the proposal is to keep this as is.

The max fake depth of 64 will be enforced as well in the shadow rendering code – the proposal is cap Elevation to 64 as we compute the Elevation value for a Recipe instance based on an incoming Translation.Z value.

Replacement for animating depth
Because animating fake depth itself cannot be easily supported, we may want some other way for apps to animate the shadow into view when a shadow-casting element is first put into the live tree. This is the common scenario for both in-box controls as well as certain WinUI controls (eg TeachingTip).

We’ll need to talk to customers and determine if there is a need for this capability. Part of this discussion will determine if an opacity animation is sufficient, or if what customers really want is to animate fake depth. If customers actually need to animate fake depth, a compromise solution we could consider is playing a scale animation on the shadow SpriteVisual instead of animating fake depth. Note though that animating the scale may not achieve an acceptable visual quality, so would need to be prototyped first.

For the purposes of this doc, assuming animating opacity is sufficient, we have several options to consider:

Option 1: Add ThemeShadow.Opacity property
This option adds a new public ThemeShadow.Opacity property, which will control the opacity of the shadow only. With this property, controls can animate Shadow.Opacity from 0 to 1 as a replacement for today’s behavior, which animates Translation.Z from 0 to the final depth when setting up the visual tree for a control. It should be a straightforward branch in controls which already animate Translation.Z (if the ThemeShadow.Opacity API is available, use it, otherwise fallback to animating Translation.Z). For in-box controls this happens in one central function. WinUI does not animate Translation.Z on any of its controls, so no changes are needed to WinUI controls.

Option 2: Auto-detect
This option detects when an animation is playing on Translation.Z and maps this animation to an opacity animation. In-box controls currently animate Translation.Z from 0 to its final value, so a heuristic that would work well would be to normalize Translation.Z into a range of [0,1] and play an animation on opacity that reads the Translation.Z animation’s current and final values, mapping those to the normalized range, and playing an equivalent opacity animation.

For example if a control animated Translation.Z from 0 to 16, this would produce an opacity animation from 0 to 1.

ElevationHelper Changes
ElevationHelper currently does several things for projected shadows:

1. It creates a ThemeShadow and sets it on the target element
2. It sets Translation.Z on the target element, set to the final value
   a. This code also handles the nested case and computes an accumulated Z.
3. It plays an independent animation on Translation.Z, animating from 0 to the target depth

Currently if drop shadows are turned on, ElevationHelper only does #1, and skips setting/animating Translation.Z.
The proposal is to modify this behavior for the drop shadows code path only, as follows:
a) Do not compute nested depth. Nesting only applies for projected shadows as in that model, the receivers also have a depth, which is impossible to replicate with drop shadows
b) Play a ThemeShadow.Opacity animation from 0 to 1 instead of animating Translation.Z.

Private ThemeShadow.DropShadowDepth API no longer needed
Currently, ThemeShadow has a private API to control fake depth. It’s used by Tooltip and ContentDialog only. This API will no longer be necessary with this proposal and will be removed.

Windowed Popup bounds need to stay tight around shadow
Windowed Popups currently have the behavior that pointer input cannot pass through the shadow area to an app that lies underneath. We currently have special code that classifies shadow depth as "Small/Medium/Large" and shrinks the bounds down according to a hard-coded set of values in these 3 categories. With depth becoming controlled by a public API, we'll need to generalize the solution for this.

Note that we are considering changing the design to use shadows drawn by the system, which would also fix this.

Links
https://docs.microsoft.com/en-us/windows/uwp/design/layout/depth-shadow
https://docs.microsoft.com/en-us/uwp/api/windows.ui.xaml.media.themeshadow?view=winrt-19041

[mdtauk]

At present I believe the only way to set Z-,depth is through code behind.

X and Y translations can be found in Xaml.

With this prototyping, would it be possible to consider adding an Elevation value in Xaml, which is either an int between the minimum and maximum shadow depth values in the design docs or an enumeration of set values?

1, 2, 3, 4, 8, 16, 20, 24, 32, 48, 60, 64

For example?