Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Elements: deferred upgrade until displayable #6480

Open
zcorpan opened this issue Mar 11, 2021 · 47 comments
Open

Custom Elements: deferred upgrade until displayable #6480

zcorpan opened this issue Mar 11, 2021 · 47 comments
Labels
topic: custom elements Relates to custom elements (as defined in DOM and HTML)

Comments

@zcorpan
Copy link
Member

zcorpan commented Mar 11, 2021

This was first written by @dvoytenko in https://docs.google.com/document/d/1KNVZXlw-wygoNwZz0m_105ISBSjN1Q-sKh1bkw4CgPw/edit# , moving it here as suggested by @chrishtr to facilitate broader discussion. cc @justinfagnani @mfreed7

I'll repeat the existing open comments in the doc below.

Background

Custom Elements API is a mature Web spec that allows DOM elements to be upgraded with custom styling, markup and behavior. It’s a convenient and performant solution for many use cases. However, this spec lacks any notion of “lazy instantiation”. The browser upgrades a defined element as soon as it’s parsed. This could be a bottleneck for applications that rely heavily on custom elements. This document explores an idea of allowing a natural lazy-upgrade mechanism for custom elements.

Goal

Extend the Custom Elements spec to enable lazy upgrade and instantiation of defined custom elements. The expectation is that it will reduce initial CPU and memory usage of a page that relies on custom elements.

Proposal

The proposed solution is to defer upgrade of custom elements that are not displayed on-screen or likely to be on-screen soon, and/or have CSS which make them invisible without a change to DOM state. Let’s call these conditions displayable.

Examples of non-displayable elements:

  1. An element with display:none style or hidden attribute.
  2. An element with visibility:collapse when space is removed (tables and flex children).
  3. An undistributed light DOM child of a shadow host.
  4. An element inside an inactive content-visibility:auto|hidden subtree.
  5. An element is not attached to a document.

By default a custom element upgrade will be unchanged. However, the new APIs will allow an element to opt into the deferred upgrade mode. Such an element will not be immediately upgraded by the browser. Instead they’d be queued up and upgraded when it becomes displayable.

Proposal details

/1/ The custom element class can include static methods to indicate that it desires deferred-upgrade behavior:

class MyElementClass extends HTMLElement {
  static deferredUpgrade(element: Element): boolean;
  static deferredUpgradeSelf(element: Element): boolean;
}

When absent, the deferredUpgrade is assumed to return false, i.e. the current upgrade behavior.

See the “Deferred upgrade of self” section for more information on deferredUpgradeSelf.

/2/ The parser will recognize a “deferred-upgrade” element, but will not immediately upgrade it. Instead it will schedule the upgrade for later, when the style tree is known and thus display, content-visibility, and other relevant values have been computed in respective DOM subtrees. These are “pending-upgrade” elements. The queue of “pending-upgrade” elements is checked when the relevant styles change and, if becomes displayable, the respective elements are upgraded. Ideally they would be upgraded respecting the CPU resources to avoid long tasks.

/4/ The new CustomElementRegistry.whenUpgraded() API will be added, since whenDefined can no longer be used as an indicator that an element has been upgraded:

customElements.whenUpgraded(element: Element):Promise

/5/ The existing CustomElementRegistry.upgrade() API can be changed slightly to force upgrade for “pending-upgrade” elements. Its semantics are already mostly compatible. However, one change we could apply: it should return a promise for when the upgrade is complete.

customElements.upgrade(element: Element):Promise

/6/ A stretch goal: provide :upgraded pseudo-class/state in addition to the existing :defined. This state will be set when the element has upgraded. See upgrade() and whenUpgraded() above.

/7/ A stretch goal: provide an additional callback to the Custom Element class, in addition to the connectedCallback and others - the upgradeCallback. It’s not uncommon for an actual upgrade to be asynchronous. This callback will return a promise that will return when the upgrade has completed. See whenUpgrade() and upgrade() methods.

upgradeCallback():Promise|undefined

/8/ A stretch goal: provide an additional attribute-backed property to disable deferred build behavior.

<my-element upgrade="eager|auto">...</my-element>

It’s a stretch goal because with the proposed deferredUpgrade(Element) API, the callback can check this attribute and return the appropriate response. However, it’d be beneficial to be consistent. And additionally, the browser can handle mutations for this attribute/property automatically.

Deferred upgrade of “self”

This section is mainly relevant to how this proposal works with the content-visibility: hidden. The content-visibility applies to an element’s DOM subtree - not the element itself. However, for many custom elements, if the contents is not rendered, it doesn’t make sense to upgrade the element itself. For such elements, it’d be also beneficial for overall performance to defer their upgrade. This is why the proposal includes the deferredUpgradeSelf() method.

@zcorpan zcorpan added the topic: custom elements Relates to custom elements (as defined in DOM and HTML) label Mar 11, 2021
@zcorpan
Copy link
Member Author

zcorpan commented Mar 11, 2021

Extend the Custom Elements spec to enable lazy upgrade and instantiation of defined custom elements.

@justinfagnani commented:

It would be awesome to incorporate lazy loading, like in: WICG/webcomponents#782

A component definition could provide a loader function and defaults for what triggers loading and upgrades. An attribute could customize triggers per instance.

@zcorpan
Copy link
Member Author

zcorpan commented Mar 11, 2021

The expectation is that it will reduce initial CPU and memory usage of a page that relies on custom elements.

@zcorpan commented:

Are there expected other impacts that are worth noting? What is the impact on accessibility? Page search?

@zcorpan
Copy link
Member Author

zcorpan commented Mar 11, 2021

The proposed solution is to defer upgrade of custom elements that are not displayed on-screen or likely to be on-screen soon, and/or have CSS which make them invisible without a change to DOM state.

@zcorpan commented:

Is this intended to use the same logic as lazy-loaded images/iframes to determine when to upgrade?

@zcorpan
Copy link
Member Author

zcorpan commented Mar 11, 2021

/6/ A stretch goal: provide :upgraded pseudo-class/state in addition to the existing :defined.

@mfreed7 commented:

Is this really a stretch goal? It seems like you'd really need this if you're using deferred-upgrade custom elements, to be able to keep non-upgraded elements from being displayed, for example. Akin to :defined's uses today.

@dvoytenko replied:

Without it it's basically a special class (e.g. "x-upgraded") and:

x-element:not(.x-upgraded) {
/* the element cannot be fully shown yet */
}

A pseudo class would be better, of course.

@mfreed7 replied:

Icky - then the custom element has to muck with its own classes to signal that it's been upgraded, and the page has to coordinate with its stylesheet to watch for the same .x-upgraded class. But yeah I guess.

@dvoytenko replied:

Yeah. Definitely a very-nice-to-have. I'll be happy to set it to must-have :)

@zcorpan
Copy link
Member Author

zcorpan commented Mar 11, 2021

/6/ A stretch goal: provide :upgraded pseudo-class/state in addition to the existing :defined.

@zcorpan commented:

Have you considered changing :defined to not match not-yet-upgraded custom elements? Or do you want to style differently in the 3 states?

@zcorpan
Copy link
Member Author

zcorpan commented Mar 11, 2021

/8/ A stretch goal: provide an additional attribute-backed property to disable deferred build behavior.

@dvoytenko commented:

@mfreed7 another idea that I wanted to run by you.

@mfreed7 commented:

Just to make sure I understand this # 8, this is an alternative declarative way to opt in to deferred upgrades, without the custom element needing to return true from deferredUpgrade()? If so, who would "win" in the case the custom element explicitly returns false from deferredUpgrade, but the author puts "upgrade=auto" (which I assume is the equivalent of returning true)?

@zcorpan
Copy link
Member Author

zcorpan commented Mar 11, 2021

/8/ A stretch goal: provide an additional attribute-backed property to disable deferred build behavior.

@zcorpan commented:

Why upgrade="auto" instead of upgrade="lazy" (which would be more consistent with img/iframe loading="lazy", if I understand the proposal correctly)?

@zcorpan
Copy link
Member Author

zcorpan commented Mar 11, 2021

<my-element upgrade="eager|auto">...</my-element>

@justinfagnani commented:

It would be really great to consider various upgrade triggers besides visibility.

Wiz in particular gets huge benefits from deferring the loading of component definitions until certain user interactions have occurred, specified with the jsaction attribute.

Maybe that could be folded in here where this attribute can contain a microsyntax to specify what events to upgrade on:

upgrade="on:click visibility on:hover"

@domenic
Copy link
Member

domenic commented Mar 11, 2021

Thanks for porting this over, and I like how it's evolved since I last saw it. In particular I love treating all types of visibility on equal footing, instead of special-casing content-visbiility.

/7/ A stretch goal: provide an additional callback to the Custom Element class, in addition to the connectedCallback and others - the upgradeCallback. It’s not uncommon for an actual upgrade to be asynchronous.

I disagree with this goal. Upgrade is always synchronous; it's just the process of establishing the right prototype chain, which the browser does automatically.

There could be some separate lifecycle phase, such as "hydration" or "initialization" or similar, which is element-defined and async. But "upgrade" has a very specific meaning and it's synchronous.

Deferred upgrade of “self”

This section doesn't really make sense to me. It seems like it needs more integration with the "Proposal" section.


I can't really follow all the pasted-in comments so I'll trust you to summarize if there's any proposed changes that come out of them.

@mfreed7 mfreed7 added the agenda+ To be discussed at a triage meeting label Apr 23, 2021
@mfreed7
Copy link
Contributor

mfreed7 commented Apr 23, 2021

I've added this to the agenda for the next triage meeting.

@chrishtr fyi

@annevk
Copy link
Member

annevk commented May 7, 2021

Some thoughts:

  1. Having to still call deferredUpgrade() when parsing seems very sub-optimal. If the goal here is to execute less script while parsing, this should be something static that can be cached at define-time.
  2. As mentioned during the triage meeting, it's unclear how the upgrade happens, how to avoid FOUC during that process, and also how to avoid missing loss of frames if there's too much script to execute for the CPU.

@annevk annevk removed the agenda+ To be discussed at a triage meeting label May 7, 2021
@emilio
Copy link
Contributor

emilio commented May 7, 2021

Yeah, this seems weird, since 99% of the time the upgrade will change rendering itself (either by creating or mutating dom).

@dvoytenko
Copy link

@annevk

Having to still call deferredUpgrade() when parsing seems very sub-optimal. If the goal here is to execute less script while parsing, this should be something static that can be cached at define-time.

There could certainly be a version of this where deferredUpgrade flag is static, though this could force more elements not to use it and do immediate upgrade. But what's really suboptimal is to force upgrade of all elements as soon as they are parsed regardless of whether they are used.

As mentioned during the triage meeting, it's unclear how the upgrade happens, how to avoid FOUC during that process, and also how to avoid missing loss of frames if there's too much script to execute for the CPU.

Take, for instance, the content-visibility: auto. Let's assume there's a block further down the viewport with the content-visibility: auto. Its content is not rendered. This proposal says that the upgrade of custom elements within it can also be deferred. When this block comes closer to the viewport (e.g. within 3 viewports), the content-visibility:auto will instruct the block to start rendering, and at this time the deferred elements can be upgraded. This block must already define some sort of content size or a size estimate, so the layout shift should from this should be minimal. And, hopefully, by the time the user naturally brings this block into the viewport, all the work will be done.

@chrishtr
Copy link
Contributor

chrishtr commented May 7, 2021

Yeah, this seems weird, since 99% of the time the upgrade will change rendering itself (either by creating or mutating dom).

Hi Emilio, could you clarify what you find weird? Is it the whole idea of delayed upgrades, or something specific about the proposal in terms of custom elements?

It seems quite natural to me to have a platform primitive that allows components to start their work only as they come near the viewport. content-visibility does this for UA rendering tasks, and the script equivalent is the next step. Sites these days do a lot of "rendering" in script. Or even if it isn't "rendering" but adding event handlers and so on, there is a natural progressive enhancement that can be delayed until the component comes near the screen, thereby improving performance without sacrificing user experience.

Another way to think of it is that this is providing ways in the platform to easily achieve virtualization of content without a script-based polyfill (which is what many sites do today). Some sites even polyfill delayed custom element upgrades by using "stub" custom element class definitions that un-stub based on IntersectionObserver callback timing.

Having this feature in the platform will:

  • Increase typical performance of sites, since it'll be much easier to virtualize content without breaking something in the site experience
  • Improve accessibility and UA features in general, since more content will be expressed via declarative DOM instead of hidden in virtual DOM / script data
  • Make it easier for UAs to optimize further in the future

To the point about upgrade-changing-rendering: as you say, the custom element upgrade is likely to change rendering state for itself during its upgrade. This will indeed mean that enough time will have to be reserved to avoid visual artifacts. But I view that as the cost of script-based content virtualization techniques. And we could imagine further enhancements to a delayed upgrade feature, such as platform APIs that say "delay, but do the work if there is idle time in which to do it", or specifying the desired distance from the viewport after which to start the upgrade process.

@chrishtr
Copy link
Contributor

chrishtr commented May 7, 2021

  1. As mentioned during the triage meeting, it's unclear how the upgrade happens, how to avoid FOUC during that process, and also how to avoid missing loss of frames if there's too much script to execute for the CPU.

I agree that all of these are concerns to be thought through carefully.

  • Re "how the upgrade happens" (I think you are referring to the timing of it relative the update-the-rendering steps. I'll reply to that now, please let me know if your concern was different than that):

TL;DR I think we can follow the same method as we ended up with in content-visibility. See below for some detailed notes.

There we defined the notion of skipped, which means "descendant of a content-visibility ancestor which is hidden, or auto and far enough offscreen". The skipped state updates after step 12 of update-the-rendering (*), in other words at IntersectionObserver timing. Note 3 in the spec I linked to here says that this means that if content becomes unskipped, it in general will not fully render until the subsequent frame, just as an IntersectionObserver callback doesn't fire until the update-the-rendering event loop task that led to it finishes.

The steps would be:

Update-the-rendering for frame N:

  1. Update rendering state (style, layout)
  2. Observe that the IntersectionObserver shows the element does not skip its contents. Schedule an event loop task to run upgrades.
  3. Present frame N to the screen.

Subsequent event loop task:

  1. [New with this proposal] Run custom element upgrades for all delayed custom elements in the contents that are not skipped.

Update-the-rendering for frame N+1:

  1. Update rendering state (style, layout). Includes any updates caused by the upgrades.
  2. Present frame N+1 to the screen.

However, observe note 4 in the spec, which basically says that if the first time an element is considered for rendering (**) it's found to be not skipped, the above steps happen synchronously with the frame. In the case of the proposal being discussed here, it'd be this timing:

Update-the-rendering for frame M:

  1. Update rendering state (style, layout)
  2. Observe that the IntersectionObserver shows the element does not skip its contents.
  3. [New with this proposal] Run custom element upgrades for all delayed custom elements in the contents that are not skipped.
  4. Go to step 1.
  5. Present frame M to the screen.

Note that step 3 is at the same timing as ResizeObsever, and both features can and do cause a re-render. However, just as with ResizeObserver, there is now a potential for an infinite loop because script could mutate anything. I think we'll need to apply a similar restriction as we do for ResizeObserver, in the sense of requiring the depth in the DOM of the unskipping IntersectionObserver to keep increasing. If it doesn't we'd fall back to the async callback mentioned above.

For cases that are not content-visibility, in particular display:none, I'd think that the sync, same-frame approach mentioned just above would be appropriate.

  • Re FOUC / visual artifacts, too much CPU:

The second example above shows how to avoid a situation of FOUC / visual artifacts for elements rendered for the first time. In testing content-visibility with partners, we found that this is the primary case that was problematic in earlier iterations.

In other situations, such as scrolling, the CPU/FOUC concern is addressed by setting the margin of the (internal) IntersectionObserver to be a few viewports, which gives time for the UA to render. Now that there would be more script involved, that margin may need to be increased in some cases, or developers given more control over increasing it.

(*) I think the spec may have gotten out of date in the numbering w.r.t the HTML, spec, will address that.
(**) The first time after being inserted into the DOM, or reattached in a new location in the DOM.

@emilio
Copy link
Contributor

emilio commented May 8, 2021

Hi Emilio, could you clarify what you find weird? Is it the whole idea of delayed upgrades, or something specific about the proposal in terms of custom elements?

I guess when I wrote this I imagined this as something that would run directly from the styling process, but your comment made me realize this is more subtle (more on the "IntersectionObserver" / "loading=lazy" camp, if I understand correctly). Is that the kind of model you had in mind? If so, that seems fine.

I read your response above and it seems reasonable to do something like ResizeObserver does. I wrote this before reading it, so leaving here hidden, in case there's something useful (but I think I reached a similar mental model for how this should work, please correct me if I'm wrong):

So for stuff that's just outside of the viewport, and assuming that stuff takes some reasonable size when not upgraded (which is something the page would need to take care of) then this works. But for stuff that's actually not rendered at all it is not quite clear to me how this would work without FOUC. Think something is display: none and turns to display: block. You don't know the final display value until you've done styling, and you don't know its position until you've done layout.

If you do this based on IntersectionObserver, then you get "FOUC", because the observer will report the change once the rendering update is done, right?

Otherwise... at which point do you check whether this is "displayable enough"? Presumably after layout? Or just after styling? In any case, if the element is "displayable", then you need to run an arbitrary amount of content script, which might in turn invalidate all sort of style and make either more or less stuff displayable. So we need some sort of infinite loop prevention like ResizeObserver has.

Anyhow, my main concern at this point, then, is that with a naive implementation of what you describe above there seems to be potential for a lot of wasted styling / layout work. In particular, thinking of nested components, they'll upgrade once-at-a-time, which means that the cost of rendering is (at best) double. E.g., the parent component becomes "displayable", and its upgrade creates a shadow root with more components, which are not rendered yet, so they're not "displayable", so we layout and discover they actually are, and they get upgraded and create more components, and so on...

That sort of pattern seems like it'd be pretty common (think, showing a <dialog>, or any other hidden element really), and it'd be a lot more inefficient to render everything. So I think whatever model we come up with here should probably not be some automatic magic that applies to all custom elements, but something more subtle that maybe applies to a handful of components on a given page, or something like that... But of course people will use it everywhere anyways, so we should probably find a way to make it fast-by-default.

@chrishtr
Copy link
Contributor

chrishtr commented May 8, 2021

I guess when I wrote this I imagined this as something that would run directly from the styling process, but your comment made me realize this is more subtle (more on the "IntersectionObserver" / "loading=lazy" camp, if I understand correctly). Is that the kind of model you had in mind? If so, that seems fine.

Thanks for the quick response! Yes, it's the IntersectionObserver/loading=lazy mode. I agree the other one is unworkable. :)

I read your response above and it seems reasonable to do something like ResizeObserver does. I wrote this before reading it, so leaving here hidden, in case there's something useful (but I think I reached a similar mental model for how this should work, please correct me if I'm wrong):

So for stuff that's just outside of the viewport, and assuming that stuff takes some reasonable size when not upgraded (which is something the page would need to take care of) then this works. But for stuff that's actually not rendered at all it is not quite clear to me how this would work without FOUC. Think something is display: none and turns to display: block. You don't know the final display value until you've done styling, and you don't know its position until you've done layout.

If you do this based on IntersectionObserver, then you get "FOUC", because the observer will report the change once the rendering update is done, right?

I agree that display:none, or nested content-visibility descendants of skipped content-visibility contents, don't have a known viewport-relative position, so we can't directly upgrade their custom elements based on scroll position. We'll have to synchronously upgrade display:none content once it becomes non-display:none. Likewise, nested content-visibility: auto elements will be handled in once they become unskipped, but this will not I think draw across multiple frames. See worked example below.

Anyhow, my main concern at this point, then, is that with a naive implementation of what you describe above there seems to be potential for a lot of wasted styling / layout work. In particular, thinking of nested components, they'll upgrade once-at-a-time, which means that the cost of rendering is (at best) double. E.g., the parent component becomes "displayable", and its upgrade creates a shadow root with more components, which are not rendered yet, so they're not "displayable", so we layout and discover they actually are, and they get upgraded and create more components, and so on...

Nested content-visibility will cause extra rounds of re-rendering within the same frame as they get discovered. This will increase the cost of rendering that frame somewhat, as compared with having upgraded them and made them visible all in one go. For example, consider nested custom elements like this:

<custom-a style="content-visibility: auto">
  <custom-b style="content-visibility: auto">
    <custom-c style="content-visibility: auto">
    </custom-c>
  </custom-b>
</custom-a>

Without delayed custom element upgrades, the sequence of actions would be:

  1. Upgrade a, b and c's custom elements.
  2. Render a, b and c.
  3. Scroll to show a.

Now it'll be:

  1. Scroll to show a.
  2. As a gets close to the viewport, this frame is rendered:
  • Render outer content plus content-a:
  • a is unskipped, and therefore upgraded
  • Render outer content plus content-a and content-b:
  • b is unskipped, and therefore upgraded
  • Render outer content plus content-a and content-b and content-c:
  • c is unskipped, and therefore upgraded

It all happens in one frame due to the "first-render of content" rule in note 4 of the spec. Also note that I'm presuming we aren't so unlucky that b and c end up just past the IntersectionObserver threshold.

This naively looks like three expensive renders with interleaved script, but in a good implementation the second and third renders are much cheaper, and the cost is not much higher than just re-rendering content-a and content-b. The reason we know it won't be very costly is that content-visibility guarantees style, layout and paint containment, which strictly limits rendering costs.

I'd say the total rendering time is likely about O(d^2 * n), where n is the number of DOM nodes and d is the nesting depth of custom elements. It's d^2 because each of the rendering passes are likely to have to walk down the tree to find the next thing to render, or walk up it to set dirty bits.

That sort of pattern seems like it'd be pretty common (think, showing a <dialog>, or any other hidden element really), and it'd be a lot more inefficient to render everything. So I think whatever model we come up with here should probably not be some automatic magic that applies to all custom elements, but something more subtle that maybe applies to a handful of components on a given page, or something like that... But of course people will use it everywhere anyways, so we should probably find a way to make it fast-by-default.

I agree that this cascading would be bad if it resulted in multiple rendered frames or big additional costs, but based on my analysis above it looks like that won't happen.

@emilio
Copy link
Contributor

emilio commented May 8, 2021

This naively looks like three expensive renders with interleaved script, but in a good implementation the second and third renders are much cheaper, and the cost is not much higher than just re-rendering content-a and content-b. The reason we know it won't be very costly is that content-visibility guarantees style, layout and paint containment, which strictly limits rendering costs.

But upgrading custom elements would more often than not change styles (due to the shadow tree rules affecting the host etc), right? So it's not as cheap as regular content-visibility, at best it also requires multiple rounds of style resolution, which is a bit concerning.

@chrishtr
Copy link
Contributor

chrishtr commented May 8, 2021

But upgrading custom elements would more often than not change styles (due to the shadow tree rules affecting the host etc), right?

Re host element: good point. Yes, the host element's styles could change, so that particular element may be have an extra style recalc after upgrade. But no other elements in the parent tree scope will change style, unless the custom element goes out of its way to do something terrible like injecting style sheets into the parent tree scope.

The host element may also be laid out a second time (due to host styling in the shadow DOM), and this may cause changes to outer layout. Likewise, the removal of contain:size when unskipping the contents of the custom element will cause sizing of the subtree to affect the outer layout.

So it's not as cheap as regular content-visibility, at best it also requires multiple rounds of style resolution, which is a bit concerning.

Agree, it's not as cheap as content-visibility. However, the multiple-style recalc overhead is limited to just the host elements.
Repeated layout impacts are potentially larger though.

One thing we could consider doing is to specify that any delayed upgrades trigger recursively. E.g. if you have the custom-c inside custom-b inside custom-a example I mentioned earlier, if custom-a upgrades, we also upgrade custom-b and custom-c at the same time, even if we're not they will be come unskipped, and don't know if they are display:none. This would mean there is only one round of some extra style and layout.

@rniwa
Copy link

rniwa commented May 10, 2021

It is strange to me that deferring upgrade is tied to the entire class of custom elements. In lazy imaging loading, for example, we added a content attribute on img elements themselves to delay loading. Imagine someone wrote a custom element to be used across many different parts of a web site or web app. It seems rather superficial to assume that all documents that load this component would want deferred upgrade semantics. Even within a single document, I'd imagine some instances of a custom element may want to get upgraded immediately. For all these reasons, I'm skeptical of the proposed design.

It is also strange that the proposal seems to make :defined apply to deferred custom elements that have not yet been upgraded since whether :defined applies or not is currently tied to the custom element state. This will certainly require introducing new custom element state but I'm not certain that is desirable. The primary use case of :defined is to detect custom elements that have already been upgraded. What are use cases for matching custom elements that haven't been upgraded but their definitions are available? If a script wanted to find such elements, it can simply check customElements.get(~) instead.

It also seems like the proposed semantics can be easily polyfilled if scoped custom element registries were added since whatever custom elements that want to defer upgrading can simply create a new registry and not define any custom elements in it.

@atotic
Copy link

atotic commented May 18, 2021

There are browser features that assume DOM represents all of page's content.

A) hash links: browser will search DOM to find an Element matching hash-link.
B) text fragments: browser will search DOM for text specified in fragment directive.
C) find in page: browser will query DOM for search text.

Frameworks have already implemented variants of lazy loading (ex: infinite scroll). The lack of interoperability with above features has been an issue (ex Notify page that "find in page" is being used, w3c Find-in-page API proposal ).

Indiscriminately using deferred upgrade for performace benefit would be a hazard for developers. Most do not test the above features specifically, they just work. Deferred upgrade would break them.

I do not have a strong intuition on how to solve this. The obvious solutions are:

  1. Ignore the problem
    Users and developers trade faster first paint times for occasional annoyance.

  2. Upgrade all custom elements before any of the DOM searching features run.
    Pros: everything works just like it did before, except for the possible slowdown.
    Cons: hash links/text fragments/find-in-page are all used often. Might cause many pages that "ran fine on my machine" to be much slower when reached over ex: hash links.

  3. Limit deferred upgrade to elements without searchable text, or reachable hash links (including children)
    Cons: this might make deferred upgrade off limits to most elements.

  4. Some sophisticated architecture where developers can provide text/ids information without doing full element upgrade.

@dvoytenko
Copy link

@atotic I think the proposal here should fit very well and be fairly error-proof. To start with display:none elements are already fully ignored for hash fragments, text fragments, or find in page behaviors. As far as content-visibility: auto concerned - there are very specific proposals in that area that address find in page and text fragments. There was the hidden-matchable proposal that's currently going through a redesign, but conceptually it should have a good fit here.

In a way this proposal fixes the issues like infinite scroll and other lazy loading concepts that you mention.

@atotic
Copy link

atotic commented May 19, 2021

@dvoytenko you are correct, the only scenario where "text is not in DOM" would present problems is content-visibility:auto.

I've looked over the hidden-matchable proposal. It assumes that DOM contains all the relevant text.

Custom elements deferred upgrade combined with "content-visibility:auto" is the only combination of features that results in "Incomplete DOM until upgrade". Do any of the current proposals address this scenario?

@chrishtr
Copy link
Contributor

Custom elements deferred upgrade combined with "content-visibility:auto" is the only combination of features that results in "Incomplete DOM until upgrade". Do any of the current proposals address this scenario?

In many cases, the searchable content is in light DOM that is slotted into the custom element. Therefore it's there in the DOM. See below for an example.

<custom-element>
  <div id=slotted>Slotted content</div>
</custom-element>

Right now, there is no such thing as a "registered but not upgraded" custom element (because there is no deferred upgrade mechanism). There is only "not registered" and "upgraded". Here is what happens in those situations:

Not registered:

<custom-element> will be treated the same as the default styling of a <div>. Slotted content will be visible (unless it has a style sheet saying otherwise, such as via :defined or another other mechanism; this is equivalent to display:none).

Therefore the content will be searchable by find-in-page etc.

Upgraded:

Content is searchable, because the slotted content is distributed.

The new state is: "registered".

What should happen? I think there are two cases:

a. Shadow DOM is attached (via declarative shadow DOM)
b. Shadow DOM is not attached

Case b is potentially bad, because undistributed content is like display:none in terms of searchability. However, if there is no shadow DOM then the element will behave like a

so we're all good. Right?

@atotic
Copy link

atotic commented May 20, 2021

In many cases, the searchable content is in light DOM that is slotted into the custom element.

If the content is in light DOM, there is no problem. The issue only exists if text nodes are created by custom element. For example:

<my-button title="View results">

If <my-button> was at the bottom of the page, user might want to search for that button by title.

<my-article src="article.txt">

These are not common use cases, but web developers have a habbit of doing things we did not anticipate, and expecting it to work. It'd be nice if we had an answer. It might be as simple as "All your text should be inside light DOM"

@chrishtr
Copy link
Contributor

These are not common use cases, but web developers have a habbit of doing things we did not anticipate, and expecting it to work. It'd be nice if we had an answer. It might be as simple as "All your text should be inside light DOM"

I think the answer is:

  • For situations where searchable text nodes need to be in shadow DOM: declarative shadow DOM
  • For situations where searchable text nodes are materialized light DOM created by script: server-side rendered HTML

@jridgewell
Copy link

The issue only exists if text nodes are created by custom element. For example:

Note, too, that the proposed design is configurable per Custom Element subclass (pending discussion about how static this needs to be, see #6480 (comment)). If a <my-article> needs to be fully upgraded to be useable for text search, the CE can have static deferredUpgrade() { return false } (or exclude the method entirely), and it'll be upgraded immediately like today's elements are.

For CEs that don't need to be upgraded to be searchable (either through light-dom SSR or declarative shadow), they can use static deferredUpgrade() { return true }.

@frank-dspeed
Copy link

The main problem with lazy loading and init is that there may or may not be additional networks requests which are async again so out of my view there is already lazy loading when you for example write a async connectedCallback that applys lazy stuff on the element (this)

@dvoytenko
Copy link

dvoytenko commented Sep 10, 2021

There are a few comments here about how, in a completely generic case, the deferred upgrade could lead to bad results. However, to stress, this feature is designed to be completely opt-in, either by a component publisher, or a component user, or both. We can tweak the proposal to make it more restricting in this case. The goal is to enable the components that are ideally prerendered and free of other side-effects from taking up too many resources before they are shown. Notice that the way this proposal is currently defined, to be deferred these components must be either in display:none or contain: ~strict/content-visibility: auto|hidden regions.

@atotic
Copy link

atotic commented Sep 10, 2021

@frank-dspeed wrote:

The main problem with lazy loading and init is that there may or may not be additional networks requests

Do you mean that your custom element loads additional network data after being connected? An example would help.

@dvoyenko wrote:

deferred upgrade could lead to bad results .... feature is designed to be completely opt-in

Deferred upgrade is a tool that can be used to optimize page loading.

The optimal, intended use, would be for a page to defer loading of costly non-displayable custom elements. Optimal use would also avoid nested deferred loads, because nested loads cause multiple layout passes, and make loading performance worse, not better.

Like many performance optimization tools, it needs to be used carefully. To decide whether custom element should be loaded lazily, one should know:
A) element's constructor/connectedCallback cost.
B) whether any of its children or parents are lazily loaded. This, in general, is only known by the page author.

Which leads to an interesting conclusion: B) implies that the page author is best equipped to make the decision whether a custom element is lazily loaded or not.

@jridgewell
Copy link

B) is easily handled by the static deferredUpgrade(element: Element) call signature, because the method receives the element. The deferred upgrade could then be exposed as an attribute to the the page author. Eg, loading=lazy/loading=eager attribute:

class XFoo {
  static deferredUpgrade(element) {
    return !element.hasAttribute('loading') || element.getAttribute('loading') === 'lazy';
  }
}

The proposed API is flexible enough to overcome all the bad cases mentioned in this thread, which is part of what Dima is trying to stress.

@atotic
Copy link

atotic commented Sep 10, 2021

@jridgewell wrote:

B) is easily handled by the static deferredUpgrade

I am not seeing how deferredUpgrade method would handle B) "whether any of its children or parents are lazily loaded".
Parent chain might be available, but the shadowDOM children have not been created yet at this point.

@jridgewell
Copy link

You missed the second half of your B) "This, in general, is only known by the page author.". deferredUpgrade allows you to expose any API you want to the author, and the author can have final control over the deferred upgrade.

@frank-dspeed
Copy link

frank-dspeed commented Sep 12, 2021

@atotic Here is a example custom-element that defferes everything that it does exempt executing the upgrade as soon as the element is created

/**
 * Returns a Promise that resolves as soon as the 
 * Element comes first time into viewPort
 * @param {HTMLElement} el 
 * @returns {Promise<void>}
 */
const onDisplayAble = (el) => {
    return new Promise( (resolve) =>{
        const root = null;
        const threshold = 0.1 // set offset 0.1 means trigger if atleast 10% of element in viewport
        const observerConfig = { root, threshold };
        const observer = new window.IntersectionObserver(([entry]) => {
            const isDisplayAble = entry.isIntersecting;
            if (isDisplayAble) {
              resolve();
            }
        }, observerConfig);
        observer.observe(el);
    });
}

const startAsyncUpgrade = async (el) => {
    // do something with the element after setTimeout
    // do something with the element after listiningForEvents
    // do something with the element after networkRequests
    onDisplayAble(el).then(()=> { /* do something with el */ })
}

customElements.define('my-app', class extends HTMLElement {
    connectedCallback() {
        startAsyncUpgrade(this);
    }
})

Bonus Polyfill for the observer part

/**
 * Returns a Promise that resolves as soon as the 
 * Element comes first time into viewPort
 * @param {HTMLElement} el 
 * @returns {Promise<void>}
 */
const onDisplayAblePolyfill = (el) => {
    return new Promise((resolve) => {
        const handler = (el) => {
            const rect = el.getBoundingClientRect();
            const isDisplayAble = rect.top >= 0 &&
                rect.left >= 0 &&
                rect.bottom <= (window.innerHeight || document.documentElement.clientHeight) && 
                rect.right <= (window.innerWidth || document.documentElement.clientWidth)

            if (isDisplayAble) {
                resolve()
            }
        };

        if (window.addEventListener) {
            addEventListener('DOMContentLoaded', handler, false);
            addEventListener('load', handler, false);
            addEventListener('scroll', handler, false);
            addEventListener('resize', handler, false);
        } else if (window.attachEvent)  {
            attachEvent('onDOMContentLoaded', handler); // Internet Explorer 9+ :(
            attachEvent('onload', handler);
            attachEvent('onscroll', handler);
            attachEvent('onresize', handler);
        }

    })
}

Conclusion

customElements are simple pre defined selectors thats it. So they are similar to a on insert event and so should be Sync.

@atotic
Copy link

atotic commented Sep 13, 2021

You missed the second half of your B)

Oh, I missed the place where you agree that page author being in best position to decide. We are fully in agreement.

If page author being decision whether a custom element is lazily loaded or not, the next question is:
"How does page author specify that custom elements can be lazily loaded"?

The proposal is for custom element class to include static method:

static deferredUpgrade(element: Element): boolean;

How would page author do this, if they do not have control over the element implementation?

What do you think about:

  1. Element author is able to opt out of deferred upgrade. This allows authors of elements that would break with deferred upgrades to opt out.
class MyElement {
  static allowDeferredUpgrade(Element* element): boolean;
}
  1. Page author designates elements for deferred upgrade using CSS property, or an HTML attribute. We should pick one. I think CSS is more author-friendly, because attributes need to be applied to each element separately.
   my-element {
      custom-element-upgrade: eager | lazy;
   }

@jridgewell
Copy link

jridgewell commented Sep 13, 2021

  1. Element author is able to opt out of deferred upgrade. This allows authors of elements that would break with deferred upgrades to opt out.

Isn't this the same as the author using a custom subclass of a library's custom element?

class MyElement extends LibElement {
  // Override `LibElement.deferredUpgrade` for my use case
  static deferredUpgrade() { return false }
}
customElements.define('my-element', MyElement);
  1. Page author designates elements for deferred upgrade using CSS property

A new CSS rule to opt-into deferred behavior would be fine with me.

@emilio
Copy link
Contributor

emilio commented Sep 14, 2021

Making element upgrade depend on a CSS property doesn't seem like a good idea to me. That would cause style updates before every custom element upgrade.

@atotic
Copy link

atotic commented Sep 14, 2021

That would cause style updates before every custom element upgrade.

How? custom-element-upgrade CSS property would not change.

custom-element-upgrade would default to eager, not inherited.

@emilio
Copy link
Contributor

emilio commented Sep 14, 2021

How? custom-element-upgrade CSS property would not change.

What guarantees that? But anyhow the case I'm concerned about is not the property dynamically changing.

When insert an element into the DOM, you don't style that element until you need to (either because of CSSOM APIs, or because you want to render the page to the screen). However in order to know whether the element needs to be upgraded synchronously, you need to style the element right on insertion, individually (because it might be eager). And because of how inheritance works you also need to update style of all its flattened tree ancestors.

So it'd be a massive performance blunder, unless I'm missing something.

@atotic
Copy link

atotic commented Sep 14, 2021

Thanks for the long explanation, @emilio. That settles the issue, CSS is not suitable for marking elements for deferred upgrade, because upgrades might happen before styles are available.

I've just moved from Layout to DOM team, and appreciate your patience with my newbie questions.

@dvoytenko
Copy link

To reduce dependencies of upgrade decision making, IMHO the two options I see working best are:

a. A upgrade=auto|lazy attribute.
b. A callback API on the CustomElement class as the original proposal suggests.

The (a) might end up better since it will give more control to the page author. It's also similar to loading=lazy for img.

@chrishtr
Copy link
Contributor

chrishtr commented Jan 27, 2022

I discussed some more with @atotic today. A summary of our current thinking is below.

  • Use cases: there are two key use cases that look important to consider:
    a. A custom element that is very expensive to load+render (think: a chart backed by a canvas or iframe)
    b. A section of content/custom element that has a large number of custom elements under it that don't all need to be upgraded

  • An HTML attribute looks best, to avoid the computation performance problems of a CSS property when inserting into the DOM

  • It is best for the page author to control the deferred-ness of custom elements rather than the custom element author, for the reasons @rniwa mentioned (the page author is the one who knows the UX that is being actually shown to the user).

Therefore I propose an HTML attribute:

upgrade=lazy: upgrade self and all descendants only when self becomes near the viewport, except descendant upgrade=lazy element subtrees. It's essentially the same logic as skipping for content-visibility:auto subtrees: upgrade only when self becomes relevant to the user.

@chrishtr
Copy link
Contributor

chrishtr commented Jan 27, 2022

Regarding the question of why add this attribute rather than a polyfill from developers: this is a method of a developer quickly and effectively improving performance of their existing pages, without having to optimize it from the inside-out. It's similar to the reasons why content-visibility is easy to adopt on existing pages and yield substantial rendering performance benefits.

In addition, I expect a polyfill will be less performant, because it still requires executing javascript that can otherwise be more easily deferred.

@chrishtr
Copy link
Contributor

Another one to consider is an upgrade-content=lazy attribute which would apply only to descendants but not self; this might be useful to avoid boilerplate for a lot of sibling custom elements in a component such as a carousel.

@frank-dspeed
Copy link

frank-dspeed commented Jan 27, 2022

i still see this in the domain of the component author who needs to choose if he can implement his algos async or sync then he can offer to the user a Observed Attribut or even none observed and check on connectedCallback if the attribut is set to lazy or something else

and this way give the page author the power to choose to lazy load or what ever loading algo is supported by the specific component implementation

i for example code really complex WebRTC Components with a lot of interactions with multimedia elements and also workers and audio worklets a lot of processing and all that also remote

for example codec selectors and all that. I really know out of production what edge cases are there for example registering audioWorklet processors async then establish signalingChannels async and so on.

a single component that does only one thing can have easy 1600 loc (lines of code) and i do long one liners.

When you for example create a simple Component nothing complex like i do with a lot of async dependencies there is never a need to do async on a single component level.

the css auto property to lazy render already deffers the execution of the connectedCallback

Maybe edge case to consider

i create often customElements as container they assign connectionPromises as propertys to the component so that all nested components get the propertys from the used parent component.

when a developer (page author) now lazy loads that everything inside it will break when that upgrade would happen async all components would not be able to grab the propertys. even the ones that are not loaded.

also everything that uses it will break we now need to handle enforced sync upgrade as edge case.

@justinfagnani
Copy link

justinfagnani commented Jan 27, 2022

I like the direction this is going @chrishtr

I'd like to reiterate my desire to see this considered within a broader context of lazy loading/upgrading use cases, especially since lazy loading is rapidly becoming a primary feature of frameworks like Wiz (Google internal), Astro, Qwik, and work my team is doing.

The two general axes these frameworks sit on are:

  1. What triggers cause the lazy behavior to activate. Some trigger on visibility, some on interaction (which requires buffering & replaying events - a very large topic on it's own).
  2. Which work is lazy: some only lazily upgrade/hydrate, some lazily load code.

I know event buffering is too large of a topic to cover here, but I wonder if we could consider how these attributes would interact with WICG/webcomponents#782. Specifically, if a developer wanted to load and upgrade a component lazily, would upgrade=lazy be paired with some lazy registration call to the custom elements registry?

IOW, what happens if we have:

<script type="module">
  customElements.lazyDefine('x-foo', async () => (await import('./x-foo.js')).XFoo);
</script>
<x-foo upgrade="lazy"></x-foo>

Does that make sense? Would you need both, or could the definition have an option for loading on content visibility rather than element creation, eliminating the need for the attribute?

@frank-dspeed
Copy link

frank-dspeed commented Jan 28, 2022

@justinfagnani i see lazy define as near impossible at last the element constructors need to get finished else it will break this

class myEl extends HTMLElement {}
new myEl() // illegal constructor
customElements.define('my-el', myEl)
new myEl() // works will produce my-el tag

(as your also working on scoped registries only for your interrest scoped registries also breaks that)

new myEl can only work when the HTMLElement and constructor gets completed via the customElement Register Call

ok after more thinking sure it is possible but then we introduce extra code that is needed to watch when what defines what

i see always abstraction over abstraction that makes me code more and more to handle the new introduced cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: custom elements Relates to custom elements (as defined in DOM and HTML)
Development

No branches or pull requests