[cssom-view] Proposal: add Element.isVisible[toDocument] to detect if element is visible in the document.

[vmpstr]

I filed a proposal on whatwg/html#7379 but cross-posting it here, since cssom-view is likely a good place for speccing it if it is a good idea.

The issue above has a full description of the proposal, but the summary is that I would like to propose adding Element.isRendered() function to detect whether an element is rendered. Specifically, if it is display: none or in such a subtree, or in a subtree of a content-visibility: hidden element, then isRendered() on that element would return false. Otherwise it would return true

(please see the other issue for discussions of whether things like images that haven't loaded be unrendered as well).

[emilio]

Sounds reasonable though it'd be extremely weird that isRendered would return anything that doesn't match https://html.spec.whatwg.org/#being-rendered

[Loirooriol]

What about display: contents? Related: whatwg/html#1837

[vmpstr]

I think having it match https://html.spec.whatwg.org/#being-rendered is reasonable. I'm going to see if we can amend "being rendered" somehow to include content-visibility stuff <_<

[vmpstr]

After some more discussion here, would you support isRendered being something like "returns true if the element is rendered and not skipped"

The problem is that we would really like this function to return false for content-visibility: hidden subtrees. But it's challenging to amend the being-rendered spec to say that elements in content-visibility: hidden subtrees are not rendered, since it's not entirely true. They may not be rendered, but the rendering will be updated for any kind of queries that require the rendering to be updated.

Would it help if the name is not isRendered but something else (although exactly what I don't know)

[Kaiido]

I may be bikeshedding here, but I have the feeling that this method should be called something else.
I fear that by calling this Element.isRendered authors will think it's just a getter with no side-effects. However no matter how it's optimized behind the scene, it will at least force a synchronous reflow like any of these.
So if one were to e.g change the DOM while looping over a list of elements themselves nested deep in a complex tree doing

for (const elem of list) {
  if (elem.isRendered()) {
    elem.before(newElement());
  }
}

the styles would get recalculated for the whole page at every iteration of the loop with sensible performance costs.

I am really not good at naming, but maybe something like getRenderingStatus() would make it more obvious that something will happen when calling it, and maybe it could also return different values like "not-rendered", "skipped", "rendered" if the distinction is useful?

[chrishtr]

I propose we change to Element.isVisible, meaning conceptually "visible to the document" (not "visible to the user" - IntersectionObserver would be needed for that).

Definition:

• Is rendered
• Has a used style of visibility of visible
• Is not in the subtree of an element that skip its contents

I think this is better, because it is (a) clearly defined in terms of the spec, and (b) solves various use cases we've encountered well. Use cases include:

• Detect whether content is skipped (in order to avoid forcing layout, such as Ability to detect whether an element is rendered. WICG/display-locking#112)
• Know when an element is visible (e.g. via puppeteer or similar frameworks (also see here))

[josepharhar]

Know when an element is visible (e.g. via puppeteer or similar frameworks (also see here))

Here is some puppeteer code and some playwright code that could use this feature to work properly with content-visibility, as well as a playwright issue where someone ran into this shortcoming due to content-visibility in <details>.

[emilio]

I propose we change to Element.isVisible, meaning conceptually "visible to the document" (not "visible to the user" - IntersectionObserver would be needed for that).

I'm a bit concerned about people expecting the later rather than the former... Though I can't come up with a particularly better name right away...

Another possible concern is that the API would have to update style synchronously, which might defeat the optimizations it's trying to introduce...

[SebastianZ]

I'm a bit concerned about people expecting the later rather than the former... Though I can't come up with a particularly better name right away...

Me too. It could be simply isVisibleToDocument if that isn't too long. Alternatively, maybe isAvailable?

Sebastian

[chrishtr]

Re name: I agree isVisible is not a perfect name. isVisibleToDocument is descriptive of what it is, that sounds good to me.

Re sync style: this is definitely something to make sure we get right. The is-rendered and visibility:visible parts will require sync style recalc, but only if the element is not skipped (i.e not in a skipped content-visibility subtree). If the element is skipped, then we can detect that with a sync style update up to ancestors that skip their contents, and then observe that the element is in the subtree with a tree walk. (And if style were already up-to-date, then of course the above computations would potentially be free or mostly free.)

Worst-case performance summary:

• Forces style for non-skipped parts of the document
• Requires an ancestor tree walk for skipped elements up to a non-skipped ancestor

(p.s. I updated the third bullet point of my proposal a few comments back to be accurate about skipped elements)

[vmpstr]

Worst-case performance summary:

• Forces style for non-skipped parts of the document
• Requires an ancestor tree walk for skipped elements up to a non-skipped ancestor

I agree. Also note that even if we tie this function only to being rendered, the style update (of non-skipped parts) would still be required. I think the extra ancestor walk is a small price to pay to let developers reliably check and avoid computation in non-document-visible trees.

Bikeshed: maybe isDocumentVisible instead of isVisibleToDocument?

[smfr]

isVisibleToDocument is an odd term (why is the document looking at elements?) that might make sense for implementors, but as an author I wouldn't understand it.

If this is about whether getBoundingClientRect() will trigger more work, why not use a name that communicate that more clearly?

[chrishtr]

isVisibleToDocument is an odd term (why is the document looking at elements?) that might make sense for implementors, but as an author I wouldn't understand it.

How about isVisibleInDocument ?

If this is about whether getBoundingClientRect() will trigger more work, why not use a name that communicate that more clearly?

There are two known use cases:

1. Detect if the element is actually visible, e.g. for testing in puppeteer/playwright or similar. Joey linked to examples above.
2. Avoid forcing rendering in a skipped content-visibility. e.g. sites would wrap potentially expensive code such as getBoundingClientRect in an isVisibleInDocument.

[css-meeting-bot]

The CSS Working Group just discussed add Element.isVisible[toDocument], and agreed to the following:

• RESOLVED: Work on solving these use cases for an isVisible API and revisit in a future call

The full IRC log of that discussion

<fantasai> Topic: add Element.isVisible[toDocument]
<fantasai> github: https://github.com//issues/6850
<fantasai> chrishtr: ...
<fantasai> chrishtr: Need to do this to determine that things properly displayed
<Rossen_> q?
<Rossen_> ack florian_irc
<fantasai> chrishtr: currently use approximations that are increasingly incorrect as we add more advanced rendering features
<fantasai> chrishtr: When content is skipped using content-visiblity, protects code so that don't accidentally force-???
<fantasai> chrishtr: We've seen this problem when adopting content-visibility
<fantasai> chrishtr: Proposed definition is that box has to be "rendered" per HTML, i.e. has a box
<fantasai> chrishtr: 'visibility' is 'visible'
<fantasai> chrishtr: not in the skipped subtree of content-visibility
<fantasai> smfr: opacity:0 is visible, right?
<fantasai> chrishtr: yes, and clipped out counts as visible
<fantasai> smfr: so not really visible to the user, but some more subtle definition
<fantasai> chrishtr: Also scrolled off-screen considered visible
<lea> Seems like there's a lot of disagreement about the exact definition, what if this accepted a dictionary so that authors can define the type of "visibility" they need?
<fantasai> chrishtr: so seems like need name to be slightly different, e.g. isVisibletoDocument
<lea> Also, not sure how `isVisibleToDocument()` adds clarity over `isVisible()`, it mainly seems more verbose
<fantasai> Rossen_: I wanted clarification on whether or not isVisible state here means that the box of the element is rendered inside the viewport or not
<fantasai> Rossen_: Sounds like the answer is not necessarily?
<fantasai> Rossen_: Seems like the definition is box of the element could potentially be viewable
<fantasai> Rossen_: So, like, *can* be visible
<Rossen_> q?
<fantasai> lea: It seems there's a lot of uncertainty about the exact definition of this function
<fantasai> lea: couldn't we have options so that authors can decide what type of visiblity they care about?
<fantasai> lea: because clearly a single definition is not serving all use cases
<Rossen_> ack lea
<fantasai> lea: Also, isVisibleToDocument() doesn't seem to really add clarity, just adds verbosity
<fantasai> <fantasai> +1
<fantasai> chrishtr: Not sure I understand use cses?
<fantasai> lea: Some use cases need to know whether visible in viewport, some just need to know if there's a box, and some need to know if contents are rendered
<fantasai> lea: so different use cases
<fantasai> chrishtr: We did consult with reps from Pupeteer and Clearwrite
<dholbert> q+
<fantasai> chrishtr: and they thought the definition was good for their use case
<fantasai> chrishtr: and meets the needs of the content-visibility partners we have
<florian_irc> q+
<fantasai> lea: As a web dev, testing whether a box is visible is something web developers have needed and use frequently
<florian_irc> q-
<fantasai> lea: adding isVisible that is a specific type of rendering for a specific use case
<smfr> +1 to lea’s suggestion
<fantasai> lea: if we're adding this new function, can't we add an ability that a majority of the authors need?
<fantasai> +1 to lea
<Rossen_> ack dholbert
<fantasai> dholbert: I'm curious whether it makes sense to include visiblity: hidden in the definition
<fantasai> dholbert: seems different category of visiblity than content-visiblity and display:none
<fantasai> dholbert: visibility:hidden, you still have to compute layout with it
<fantasai> dholbert: and authors are fully in control of it
<lea> Potentially useful, as this has historically been used *a lot* by authors: https://api.jquery.com/visible-selector/#visible1
<fantasai> chrishtr: Maybe it makes sense to be more specific about what the function does, and maybe introduce multiple function
<smfr> q+
<fantasai> chrishtr: e.g. has a box and not ??
<fantasai> chrishtr: visiblity can be checked directly with existing methods
<lea> (jQuery implements as a selector, but it was used in the same way, to test visibility of a given element, for a certain definition of visibility)
<fantasai> dholbert: opacity:0, as smfr mentioned; and then content covered up by other content
<fantasai> chrishtr: so the only one is has a box and is not skipped
<fantasai> chrishtr: I think that would meet the needs
<fantasai> dholbert: only issue is that isVisible is a problematic name
<lea> also, would be nice if it addressed use cases like https://stackoverflow.com/questions/123999/how-can-i-tell-if-a-dom-element-is-visible-in-the-current-viewport
<Rossen_> a11y tools have been requesting such behavior for a very long time... yet I don't see that as a strong use case
<florian_irc> fantasai: I suggest "isDisplayed", cause that's got more to do with what we are doing
<florian_irc> fantasai: here, visibility variants isn't quite right
<Rossen_> q
<Rossen_> ack fantasai
<fantasai> fantasai: displayed links to the idea of 'display' and the box
<fantasai> s/box/box generation/
<fantasai> smfr: I like lea's idea of a more fine-grained approach
<fantasai> smfr: could imagine 2 versions
<fantasai> smfr: a function isVisible() takes a dictionary, which properties interested in
<fantasai> smfr: another is returns a dictionary, which says which types of visibility involved in
<fantasai> smfr: first one is more performant, don't have to calculate types that aren't needed
<fantasai> smfr: ...
<dbaron> (leaving the meeting now)
<fantasai> s/.../if more fine-grained API, that removes ambiguity or confusion around things that affect visibility/
<Rossen_> ack smfr
<Rossen_> ack lea
<fantasai> lea: Pasted a linkto jQuery API for similar functionality
<fantasai> lea: They've done a lot of work to address use cases
<fantasai> lea: would be good to take prior art into account
<fantasai> lea: seems they define as whether any layout boxes
<fantasai> lea: but seems definition changed at one point
<fantasai> lea: also linked to stack overflow, other places, ppl want to know whether an element is visible in the viewport
<fantasai> lea: would be useful if it could do that as well
<fantasai> lea: right now the way to test this is to create an intersection observer
<fantasai> lea: but it's awkward to write code like that, so would be nice if there was a function that would just tell you
<fantasai> lea: unsure if isDisplayed is more clear, would go with isVisible because shorter
<vmpstr> q+
<Rossen_> q+
<fantasai> fantasai: just to clarify, I think if we're doing something general then isVisible is fine, but if limiting to whether the box is generated isDisplayed makes sense to me
<fantasai> vmpstr: If going with dictionary approach, would dictionary be things like test viewport visibility, test display none, test ???
<Rossen_> ack vmpstr
<fantasai> vmpstr: how would that work?
<fantasai> lea: I think we should center around use cases rather than specific properties, so that it is more future proof
<tantek> +1 leaverou, decide on the funtionality first, and then the name can follow
<fantasai> lea: I propose we resolve to work on it, and go back to the drawing board and revisit later after redesigning
<bradk> 👍
<fantasai> chrishtr: sgtm
<tantek> +1 leaverou on all that. center around use-cases, revisit after redesigning
<fantasai> Rossen_: Seems like we went down this path of content-visibility and trying to chunk how much content is processed and address perf gains, which is a good effort
<fantasai> Rossen_: now we're trying to contain the damage
<fantasai> Rossen_: ... things that were otherwise easily discoverable
<fantasai> Rossen_: This type of functionality has been requested by accessibility tools for a long long time
<fantasai> Rossen_: to recognize whether something is visible, can it be visible, where is it visible
<fantasai> Rossen_: They have frameworks working around these problems
<fantasai> Rossen_: trying to reverse-engineer engine's decisions and results
<fantasai> Rossen_: so I don't see a11y considered anywhere in this issue, and would recommend we take these use cases as well
<fantasai> Rossen_: in addition to ones described by Lea and Simon
<Rossen_> q?
<fantasai> Rossen_: Any other feedback to this issue?
<fantasai> chrishtr: resolving as Lea suggested that use cases make sense and go back to issue makes sense
<fantasai> chrishtr: Just want to make sure that WG has consensus that these use cases are worth solving
<lea> +1 on the use cases being worth solving
<lea> (that was the first part of the resolution I proposed :)
<fantasai> Proposed Resolution: Work on solving these use cases for an isVisible API
<fantasai> RESOLVED: Work on solving these use cases for an isVisible API and revisit in a future call

[chrishtr]

Prior art:

• jQuery :visible. (Note: meaning: "has a box and width or height greater than zero". CSS visibility or opacity, as well as clipping, are ignored.)

Types of visibility:

1. Has a box ("is rendered" according to HTML spec terminology)
2. Is not skipped by content-visibility ancestors
3. Non-zero opacity
4. Full opacity (opacity:1)
5. visibility:visible
6. Not clipped out by a clip or filter
7. Occluded by some other element
8. Not scrolled out of the viewport

Proposal:

Pass an options dictionary to a new isVisible API, which returns a dictionary of results. By default, use cases 1 and 2 will be queried. Others would be opt-in via the options dictionary.

The justification for including use cases 1 and 2 by default is that there is no other way to do it at present, and these are the minimum requirements to be even possibly visible. The jQuery API also does this (plus a bit more).

// Element.isVisible(opts) -> resultDict

my_element.isVisible({}) -> {hasBox: true/false, isSkipped: true/false}

// possible additions:
my_element.isVisible({trackOpacity: true} -> {hasBox: true/false, isSkipped: true/false, isFullyOpaque: true/false, hasNonZeroOpacity: true/false}
my_element.isVisible{trackVisibilityCSS: true}-> {hasBox: true/false, isSkipped: true/false, visibilityVisible: true/false}
my_element.isVisible({trackViewport: true} -> {hasBox: true/false, isSkipped: true/false, isInViewport: true/false}
my_element.isVisible({trackOcclusion: true} -> {hasBox: true/false, isSkipped: true/false, isOccluded: true/false}

trackOcclusion would use the same algorithm as IOv2.This handles use case 4+6+7 together.

IMO we should not add trackOcclusion, trackVisibility, or trackViewport or trackOpacity unless there is strong demand. Most/all of the use cases can already be solved with IntersectionObsever or checking computed styles, which in the case of IntersectionObserver are more performant.

But this design allows them to be added in the future.

[chrishtr]

@LeaVerou could you check out my proposal above? How does it sound?

[chrishtr]

Here's some accessibility prior art, via @aleventhal.

Chromium already implements an IsVisible value on accessibility objects internally, and exposes it to AT tools directly.

The implementation is equivalent to the pseudocode below. The difference between this and my proposal above is that it checks for aria-hidden and inert.

isVisible() { 
  return !isHiddenViaStyle() && !IsAriaHidden() && !IsInert();
}

isHiddenViaStyle() { 
  return isDisplayNone() || isVisibilityHidden() || isContentVisibilityHidden() // *not* content-visibility: auto
}

isVisibilityHidden() {
 return computedStyleOfElementIsVisibiltyHidden()
}

isAriaHidden() {
  return selfOrAncestorHasAriaHidden()
}

isInert() {
  return selfOrAncestorIsInert()
}

[chrishtr]

I think inert and aria-hidden should also be optional attributes.

my_element.isVisible({trackInert: true} -> {hasBox: true/false, isSkipped: true/false, isInert: true/false}
my_element.isVisible({trackAriaHidden: true} -> {hasBox: true/false, isSkipped: true/false, isAriaHidden: true/false}

[dlibby-]

From a conversation with the Narrator/UIA folks at Microsoft, their perspective is that content that is not visible to a sighted user should not be included in the accessibility tree. This determination is performed by the user agent (and specifically in Chromium by the pseudocode Chris referenced above). Therefore, they don't see the need for any such API at the platform accessibility API level.

However, one part of the proposal that I think I'm missing - what would isVisible() return for an element in a content-visibility: auto subtree where contents are skipped? Per #5857, these elements are included in the accessibility tree (with the caveat that this may expose more than the user will see once the contents are activated). If that is the case, then perhaps an isVisible-like API for ATs could be useful - but given the principles around the decision to include that content, I don't know that we'd want to advise ATs to try and handle it separately than their existing logic.

[aleventhal]

I don't think the Narrator/UIA statement is detailed enough. What about content that's scrolled off? That's not visible to a sighted user, so shouldn't be included? What about content that's behind other content in the z-order? Same Q. What about content that's positioned offscreen (often done intentionally as a technique to provide additional instructions to screen reader users). IMO, what browsers do now is useful and works well with screen reader expectations today. Not sure if there's a need to revisit any of this right now TBH. Regarding content-visibility: auto, it needs to be exposed so that screen reader users can navigate by heading, etc. Unless someone sees a way around that. Aaron

…

On Fri, Feb 4, 2022 at 3:31 PM Daniel Libby ***@***.***> wrote: From a conversation with the Narrator/UIA folks at Microsoft, their perspective is that content that is not visible to a sighted user should not be included in the accessibility tree. This determination is performed by the user agent (and specifically in Chromium by the pseudocode Chris referenced above). Therefore, they don't see the need for any such API at the platform accessibility API level. However, one part of the proposal that I think I'm missing - what would isVisible() return for an element in a content-visibility: auto subtree where contents are skipped? Per #5857 <#5857>, these elements are included in the accessibility tree (with the caveat that this may expose more than the user will see once the contents are activated). If that is the case, then perhaps an isVisible-like API for ATs could be useful - but given the principles around the decision to include that content, I don't know that we'd want to advise ATs to try and handle it separately than their existing logic. — Reply to this email directly, view it on GitHub <#6850 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKQAZUK6KKVWIALRN7ROIDUZQZTFANCNFSM5JFHLXJQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[chrishtr]

However, one part of the proposal that I think I'm missing - what would isVisible() return for an element in a content-visibility: auto subtree where contents are skipped?

Good question. I think skipped descendants of a content-visibility: auto element should return true for isVisible, since if the user scrolled to them, they are indeed visible. And it also aligns with the accessibility code I inlined above, which is a good sign.

Also note that calling isVisible will force the UA to do style computations on content-visibility:auto subtrees, which I think is fine, because it forces it also for all other content, except for content-visibility:hidden subtrees. If there is strong demand, we could add another method that is "Is this element inside a skipped content-visibility:auto ancestor?", but I don't know of a strong use case for that situation at present. I think it's better for the auto case for the UA to give a correct answer than to try to optimize performance, given the auto content will be displayed to the user after being scrolled onscreen anyway.