Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[web] Render PlatformViews with SLOT tags. #25747

Merged
merged 37 commits into from
May 21, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
6ea7e48
Wrap all the contents of the glass-pane in a shadow root.
ditman Apr 7, 2021
61ed861
Make analyzer happy
ditman Apr 8, 2021
5e1720d
Tweak semantics_test so it is shadow-DOM aware.
ditman Apr 10, 2021
7b76403
Render all text_editing primitives inside the new shadow root.
ditman Apr 13, 2021
0d11719
Tweak text_editing_test so it is shadow-DOM aware.
ditman Apr 13, 2021
b5b4ec8
dartfmt -w .
ditman Apr 13, 2021
149debf
Do not select flt-glass-pane all the time. Reuse the reference inside .
ditman Apr 13, 2021
a1de790
Don't mess with the dart version in tests
ditman Apr 13, 2021
6daf5c6
Revert fanciness
ditman Apr 13, 2021
e51ad1d
Ensure the glass-pane and its shadow root exist before testing.
ditman Apr 13, 2021
bb97136
Make dom_renderer_test shadow-root aware.
ditman Apr 13, 2021
1ade522
Do not use flt-glass-pane selectors across the code. Expose the shado…
ditman Apr 13, 2021
9e9820d
Make canvas_golden_test shadow-DOM aware.
ditman Apr 13, 2021
172fddc
Add a root Node parameter to text measurement, and pass the correct o…
ditman Apr 15, 2021
da4317e
[wip] Platform views in slots!
ditman Apr 17, 2021
6412db7
Remove flag to enable/disable feature. Cleanup unused code.
ditman Apr 24, 2021
6a9258a
Ensure all positioning happens in the slot container.
ditman Apr 27, 2021
0d41b5c
Make content height:100% if it isn't set by the user. Warn the user.
ditman Apr 27, 2021
a57a5c3
Prevent re-creating the same view multiple times in the CK renderer. …
ditman Apr 28, 2021
63370d5
Make flt-glass-pane tag name configurable.
ditman Apr 28, 2021
bb68314
Noop _compositeWithParams for disposed views, so all the disposed vie…
ditman Apr 28, 2021
1203eb3
Adapt tests to look for PlatformViews in their new locations.
ditman Apr 28, 2021
15bc192
Volkswagen the framework tests (replicate old behavior of returning a…
ditman Apr 29, 2021
8c5a13a
Decouple rendering of platform view contents and slots lifecycle, sim…
ditman Apr 29, 2021
3c55ce8
Remove unused var.
ditman Apr 30, 2021
38d3191
Add unit tests for the new platform_views classes.
ditman May 4, 2021
6397962
Appease analyzer
ditman May 5, 2021
314e246
Make semantics/text_field_test Shadow-DOM aware.
ditman May 12, 2021
518b2e2
Add licenses for new files.
ditman May 12, 2021
ea60e8c
Address PR feedback.
ditman May 13, 2021
d060e90
Fix embedded_views.dart imports.
ditman May 14, 2021
4bca48c
Use Object? as the type for the ParameterizedPlatformViewFactory, ins…
ditman May 17, 2021
3d819ca
Separate slot rendering from platform view content management.
ditman May 17, 2021
4959eea
Add ViewClipChain class to encapsulate the clip root, the contents, a…
ditman May 17, 2021
8d0bac1
Fix semantics_test after rebase.
ditman May 17, 2021
1697d17
Ensure user supplied content is correctly sized.
ditman May 17, 2021
f374b1f
Update Dartdoc so it is aware of the new createPlatformViewSlot funct…
ditman May 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ci/licenses_golden/licenses_flutter
Original file line number Diff line number Diff line change
Expand Up @@ -543,6 +543,9 @@ FILE: ../../../flutter/lib/web_ui/lib/src/engine/onscreen_logging.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/picture.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/platform_dispatcher.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/platform_views.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/platform_views/content_manager.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/platform_views/message_handler.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/platform_views/slots.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/plugins.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/pointer_binding.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/pointer_converter.dart
Expand Down
8 changes: 8 additions & 0 deletions lib/web_ui/lib/src/engine.dart
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,9 @@ part 'engine/onscreen_logging.dart';
part 'engine/picture.dart';
part 'engine/platform_dispatcher.dart';
part 'engine/platform_views.dart';
part 'engine/platform_views/content_manager.dart';
part 'engine/platform_views/message_handler.dart';
part 'engine/platform_views/slots.dart';
part 'engine/profiler.dart';
part 'engine/rrect_renderer.dart';
part 'engine/semantics/accessibility.dart';
Expand Down Expand Up @@ -413,6 +416,11 @@ class NullTreeSanitizer implements html.NodeTreeSanitizer {
void sanitizeTree(html.Node node) {}
}

/// The shared instance of PlatformViewManager shared across the engine to handle
/// rendering of PlatformViews into the web app.
/// TODO(dit): How to make this overridable from tests?
final PlatformViewManager platformViewManager = PlatformViewManager();

/// Converts a matrix represented using [Float64List] to one represented using
/// [Float32List].
///
Expand Down
186 changes: 67 additions & 119 deletions lib/web_ui/lib/src/engine/canvaskit/embedded_views.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,11 @@
// found in the LICENSE file.

import 'dart:html' as html;
import 'dart:typed_data';

import 'package:ui/src/engine.dart' show window, NullTreeSanitizer;
import 'package:ui/src/engine.dart' show window, NullTreeSanitizer, platformViewManager, createPlatformViewSlot;
import 'package:ui/ui.dart' as ui;

import '../html/path_to_svg_clip.dart';
import '../services.dart';
import '../util.dart';
import '../vector_math.dart';
import 'canvas.dart';
Expand Down Expand Up @@ -37,11 +35,13 @@ class HtmlViewEmbedder {
final Map<int, EmbeddedViewParams> _currentCompositionParams =
<int, EmbeddedViewParams>{};

/// The HTML element associated with the given view id.
final Map<int?, html.Element> _views = <int?, html.Element>{};

/// The root view in the stack of mutator elements for the view id.
final Map<int?, html.Element?> _rootViews = <int?, html.Element?>{};
/// The clip chain for a view Id.
///
/// This contains:
/// * The root view in the stack of mutator elements for the view id.
/// * The slot view in the stack (what shows the actual platform view contents).
/// * The number of clipping elements used last time the view was composited.
final Map<int, ViewClipChain> _viewClipChains = <int, ViewClipChain>{};
ditman marked this conversation as resolved.
Show resolved Hide resolved

/// Surfaces used to draw on top of platform views, keyed by platform view ID.
///
Expand All @@ -51,18 +51,12 @@ class HtmlViewEmbedder {
/// The views that need to be recomposited into the scene on the next frame.
final Set<int> _viewsToRecomposite = <int>{};

/// The views that need to be disposed of on the next frame.
final Set<int> _viewsToDispose = <int>{};

/// The list of view ids that should be composited, in order.
List<int> _compositionOrder = <int>[];

/// The most recent composition order.
List<int> _activeCompositionOrder = <int>[];

/// The number of clipping elements used last time the view was composited.
Map<int, int> _clipCount = <int, int>{};

/// The size of the frame, in physical pixels.
ui.Size _frameSize = ui.window.physicalSize;

Expand All @@ -74,76 +68,6 @@ class HtmlViewEmbedder {
_frameSize = size;
}

void handlePlatformViewCall(
ByteData? data,
ui.PlatformMessageResponseCallback? callback,
) {
const MethodCodec codec = StandardMethodCodec();
final MethodCall decoded = codec.decodeMethodCall(data);

switch (decoded.method) {
case 'create':
_create(decoded, callback);
return;
case 'dispose':
_dispose(decoded, callback!);
return;
}
callback!(null);
}

void _create(
MethodCall methodCall, ui.PlatformMessageResponseCallback? callback) {
final Map<dynamic, dynamic> args = methodCall.arguments;
final int? viewId = args['id'];
final String? viewType = args['viewType'];
const MethodCodec codec = StandardMethodCodec();

if (_views[viewId] != null) {
callback!(codec.encodeErrorEnvelope(
code: 'recreating_view',
message: 'trying to create an already created view',
details: 'view id: $viewId',
));
return;
}

final ui.PlatformViewFactory? factory =
ui.platformViewRegistry.registeredFactories[viewType];
if (factory == null) {
callback!(codec.encodeErrorEnvelope(
code: 'unregistered_view_type',
message: 'trying to create a view with an unregistered type',
details: 'unregistered view type: $viewType',
));
return;
}

// TODO(het): Support creation parameters.
html.Element embeddedView = factory(viewId!);
_views[viewId] = embeddedView;

_rootViews[viewId] = embeddedView;

callback!(codec.encodeSuccessEnvelope(null));
}

void _dispose(
MethodCall methodCall, ui.PlatformMessageResponseCallback callback) {
final int? viewId = methodCall.arguments;
const MethodCodec codec = StandardMethodCodec();
if (viewId == null || !_views.containsKey(viewId)) {
callback(codec.encodeErrorEnvelope(
code: 'unknown_view',
message: 'trying to dispose an unknown view',
details: 'view id: $viewId',
));
return;
}
_viewsToDispose.add(viewId);
callback(codec.encodeSuccessEnvelope(null));
}

List<CkCanvas> getCurrentCanvases() {
final List<CkCanvas> canvases = <CkCanvas>[];
for (int i = 0; i < _compositionOrder.length; i++) {
Expand Down Expand Up @@ -179,28 +103,39 @@ class HtmlViewEmbedder {
}

void _compositeWithParams(int viewId, EmbeddedViewParams params) {
final html.Element platformView = _views[viewId]!;
platformView.style.width = '${params.size.width}px';
platformView.style.height = '${params.size.height}px';
platformView.style.position = 'absolute';
// If we haven't seen this viewId yet, cache it for clips/transforms.
ViewClipChain clipChain = _viewClipChains.putIfAbsent(viewId, () {
return ViewClipChain(view: createPlatformViewSlot(viewId));
});

// <flt-scene-host> disables pointer events. Reenable them here because the
// underlying platform view would want to handle the pointer events.
platformView.style.pointerEvents = 'auto';
html.Element slot = clipChain.slot;

// See `apply()` in the PersistedPlatformView class for the HTML version
// of this code.
slot.style
..width = '${params.size.width}px'
..height = '${params.size.height}px'
..position = 'absolute';

// Recompute the position in the DOM of the `slot` element...
final int currentClippingCount = _countClips(params.mutators);
final int? previousClippingCount = _clipCount[viewId];
final int previousClippingCount = clipChain.clipCount;
if (currentClippingCount != previousClippingCount) {
_clipCount[viewId] = currentClippingCount;
html.Element oldPlatformViewRoot = _rootViews[viewId]!;
html.Element? newPlatformViewRoot = _reconstructClipViewsChain(
html.Element oldPlatformViewRoot = clipChain.root;
html.Element newPlatformViewRoot = _reconstructClipViewsChain(
currentClippingCount,
platformView,
slot,
oldPlatformViewRoot,
);
_rootViews[viewId] = newPlatformViewRoot;
// Store the updated root element, and clip count
clipChain.updateClipChain(
root: newPlatformViewRoot,
clipCount: currentClippingCount,
);
}
_applyMutators(params.mutators, platformView, viewId);

// Apply mutators to the slot
_applyMutators(params.mutators, slot, viewId);
}

int _countClips(MutatorsStack mutators) {
Expand All @@ -213,7 +148,7 @@ class HtmlViewEmbedder {
return clipCount;
}

html.Element? _reconstructClipViewsChain(
html.Element _reconstructClipViewsChain(
int numClips,
html.Element platformView,
html.Element headClipView,
Expand Down Expand Up @@ -386,8 +321,6 @@ class HtmlViewEmbedder {
}

void submitFrame() {
disposeViews();

for (int i = 0; i < _compositionOrder.length; i++) {
int viewId = _compositionOrder[i];
_ensureOverlayInitialized(viewId);
Expand All @@ -412,28 +345,26 @@ class HtmlViewEmbedder {
int viewId = _compositionOrder[i];

if (assertionsEnabled) {
if (!_views.containsKey(viewId)) {
if (!platformViewManager.knowsViewId(viewId)) {
debugInvalidViewIds ??= <int>[];
debugInvalidViewIds.add(viewId);
continue;
}
}

unusedViews.remove(viewId);
html.Element platformViewRoot = _rootViews[viewId]!;
html.Element platformViewRoot = _viewClipChains[viewId]!.root;
html.Element overlay = _overlays[viewId]!.htmlElement;
platformViewRoot.remove();
skiaSceneHost!.append(platformViewRoot);
overlay.remove();
skiaSceneHost!.append(overlay);
_activeCompositionOrder.add(viewId);
}

_compositionOrder.clear();

for (final int unusedViewId in unusedViews) {
_releaseOverlay(unusedViewId);
_rootViews[unusedViewId]?.remove();
}
disposeViews(unusedViews);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this new code will actually dispose of "disposed" views. In the old code we kept a list of views which were disposed, and then later we disposed them. In this code we are only disposing views which are trying to be composited into the scene but were never registered. In the old code we would delete disposed views whether or not they are being actively composited this frame.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, the thing is that in the old code this was coupled with the creation/destruction of the actual content of the platform view in the DOM.

Now, slots are "lightweight" so they can be destroyed/recreated per frame without impacting performance, and the actual platform view (with the heavy iframe) will still get destroyed when the framework signals a disposal.

Can you think of some example where this code might be leaking unused views?


if (assertionsEnabled) {
if (debugInvalidViewIds != null && debugInvalidViewIds.isNotEmpty) {
Expand All @@ -445,24 +376,18 @@ class HtmlViewEmbedder {
}
}

void disposeViews() {
if (_viewsToDispose.isEmpty) {
return;
}

for (final int viewId in _viewsToDispose) {
final html.Element rootView = _rootViews[viewId]!;
rootView.remove();
_views.remove(viewId);
_rootViews.remove(viewId);
void disposeViews(Set<int> viewsToDispose) {
for (final int viewId in viewsToDispose) {
// Remove viewId from the _viewClipChains Map, and then from the DOM.
ViewClipChain clipChain = _viewClipChains.remove(viewId)!;
clipChain.root.remove();
// More cleanup
_releaseOverlay(viewId);
_currentCompositionParams.remove(viewId);
_clipCount.remove(viewId);
_viewsToRecomposite.remove(viewId);
_cleanUpClipDefs(viewId);
_svgClipDefs.remove(viewId);
}
_viewsToDispose.clear();
}

void _releaseOverlay(int viewId) {
Expand Down Expand Up @@ -499,6 +424,29 @@ class HtmlViewEmbedder {
}
}

/// Represents a Clip Chain (for a view).
///
/// Objects of this class contain:
/// * The root view in the stack of mutator elements for the view id.
/// * The slot view in the stack (the actual contents of the platform view).
/// * The number of clipping elements used last time the view was composited.
class ViewClipChain {
html.Element _root;
html.Element _slot;
int _clipCount = -1;

ViewClipChain({required html.Element view}) : this._root = view, this._slot = view;

html.Element get root => _root;
html.Element get slot => _slot;
int get clipCount => _clipCount;

void updateClipChain({required html.Element root, required int clipCount}) {
_root = root;
_clipCount = clipCount;
}
}

/// Caches surfaces used to overlay platform views.
class OverlayCache {
static const int kDefaultCacheSize = 5;
Expand Down
Loading