Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add live region semantics flag #5512

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions lib/ui/semantics.dart
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,7 @@ class SemanticsFlag {
static const int _kScopesRouteIndex= 1 << 11;
static const int _kNamesRouteIndex = 1 << 12;
static const int _kIsHiddenIndex = 1 << 13;
static const int _kIsLiveRegionIndex = 1 << 14;

const SemanticsFlag._(this.index);

Expand Down Expand Up @@ -366,6 +367,21 @@ class SemanticsFlag {
/// used to implement accessibility scrolling on iOS.
static const SemanticsFlag isHidden = const SemanticsFlag._(_kIsHiddenIndex);

/// Whether the semantics node is a live region.
///
/// A live region indicates that this semantics node will update its
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds misleading to me. Any semantics node may update its semantics label without being a liveRegion, no? liveRegion just tells a11y systems that a it may be relevant to inform the user about a changed label, no?

/// semantics label. Platforms are free to use this information to
/// make polite updates to the user to inform them of this.
///
/// On Android, TalkBack will make a polite announcement of the first and
/// subsequent updates to the label of this node. This flag is not currently
/// supported on iOS.
///
/// An example of a live region is a [SnackBar] widget. When it appears
/// on the screen it may be difficult to focus to read the value. A live
/// region causes a polite announcement to be generated automatically.
static const SemanticsFlag isLiveRegion = const SemanticsFlag._(_kIsLiveRegionIndex);

/// The possible semantics flags.
///
/// The map's key is the [index] of the flag and the value is the flag itself.
Expand All @@ -384,6 +400,7 @@ class SemanticsFlag {
_kScopesRouteIndex: scopesRoute,
_kNamesRouteIndex: namesRoute,
_kIsHiddenIndex: isHidden,
_kIsLiveRegionIndex: isLiveRegion,
};

@override
Expand Down Expand Up @@ -417,6 +434,8 @@ class SemanticsFlag {
return 'SemanticsFlag.namesRoute';
case _kIsHiddenIndex:
return 'SemanticsFlag.isHidden';
case _kIsLiveRegionIndex:
return 'SemanticsFlag.isLiveRegion';
}
return null;
}
Expand Down
1 change: 1 addition & 0 deletions lib/ui/semantics/semantics_node.h
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ enum class SemanticsFlags : int32_t {
kScopesRoute = 1 << 11,
kNamesRoute = 1 << 12,
kIsHidden = 1 << 13,
kIsLiveRegion = 1 << 14,
};

struct SemanticsNode {
Expand Down
37 changes: 34 additions & 3 deletions shell/platform/android/io/flutter/view/AccessibilityBridge.java
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ class AccessibilityBridge extends AccessibilityNodeProvider implements BasicMess
private static final int ROOT_NODE_ID = 0;

private Map<Integer, SemanticsObject> mObjects;
private Map<Integer, String> mLiveRegions;
private final FlutterView mOwner;
private boolean mAccessibilityEnabled = false;
private SemanticsObject mA11yFocusedObject;
Expand Down Expand Up @@ -91,7 +92,8 @@ enum Flag {
IS_OBSCURED(1 << 10),
SCOPES_ROUTE(1 << 11),
NAMES_ROUTE(1 << 12),
IS_HIDDEN(1 << 13);
IS_HIDDEN(1 << 13),
IS_LIVE_REGION(1 << 14);

Flag(int value) {
this.value = value;
Expand All @@ -104,6 +106,7 @@ enum Flag {
assert owner != null;
mOwner = owner;
mObjects = new HashMap<Integer, SemanticsObject>();
mLiveRegions = new HashMap<Integer, String>();
previousRoutes = new ArrayList<>();
mFlutterAccessibilityChannel = new BasicMessageChannel<>(owner, "flutter/accessibility",
StandardMessageCodec.INSTANCE);
Expand Down Expand Up @@ -238,7 +241,20 @@ public AccessibilityNodeInfo createAccessibilityNodeInfo(int virtualViewId) {
result.addAction(AccessibilityNodeInfo.ACTION_SCROLL_BACKWARD);
}
}

if (object.hasFlag(Flag.IS_LIVE_REGION)) {
result.setLiveRegion(View.ACCESSIBILITY_LIVE_REGION_POLITE);
String priorLabelHint = mLiveRegions.get(object.id);
String labelHint = object.getLabelHint();
// It is very important that this is added to mLiveRegions before sending the accessibility event
// below.
mLiveRegions.put(object.id, labelHint);
if (priorLabelHint == null || !priorLabelHint.equals(labelHint)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a little surprised to see this get send here and not in updateSemantics with the other events. This means, we only send the event if Android actually happens to ask us for the semantics of a particular node, which might not be guaranteed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this is odd. In my experiments I found that (at least on a pixel) no notifications are produced if this event is sent before the framework calls this method. during seems to be no problem

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did some more investigation and found that there was another error that was masking this. Fixed and moved to a better location

sendAccessibilityEvent(object.id, AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED);
}
} else if (object.hadFlag(Flag.IS_LIVE_REGION)) {
mLiveRegions.remove(object.id);
}

boolean hasCheckedState = object.hasFlag(Flag.HAS_CHECKED_STATE);
result.setCheckable(hasCheckedState);
if (hasCheckedState) {
Expand Down Expand Up @@ -499,7 +515,7 @@ void updateSemantics(ByteBuffer buffer, String[] strings) {
rootObject.updateRecursively(identity, visitedObjects, false);
rootObject.collectRoutes(newRoutes);
}

// Dispatch a TYPE_WINDOW_STATE_CHANGED event if the most recent route id changed from the
// previously cached route id.
SemanticsObject lastAdded = null;
Expand Down Expand Up @@ -705,6 +721,7 @@ private void createWindowChangeEvent(SemanticsObject route) {
private void willRemoveSemanticsObject(SemanticsObject object) {
assert mObjects.containsKey(object.id);
assert mObjects.get(object.id) == object;
mLiveRegions.remove(object.id);
object.parent = null;
if (mA11yFocusedObject == object) {
sendAccessibilityEvent(mA11yFocusedObject.id, AccessibilityEvent.TYPE_VIEW_ACCESSIBILITY_FOCUS_CLEARED);
Expand Down Expand Up @@ -1061,6 +1078,20 @@ private float max(float a, float b, float c, float d) {
return Math.max(a, Math.max(b, Math.max(c, d)));
}

private String getLabelHint() {
StringBuilder sb = new StringBuilder();
if (label != null) {
sb.append(label);
}
if (sb.length() > 0) {
sb.append(", ");
}
if (hint != null) {
sb.append(hint);
}
return sb.length() > 0 ? sb.toString() : null;
}

private String getValueLabelHint() {
StringBuilder sb = new StringBuilder();
String[] array = { value, label, hint };
Expand Down