<video> element sizing behavior: support for 'object-fit' and 'overflow' styles

[aoberoi]

Right now the MediaStreamRenderer will always render video by scaling it down so that the entire frame fits within the <video> element (preserving aspect ratio and centering) and fills the rest of the area with black pixels. This is equivalent to the standards when the <video> has the default styles applied (specifically object-fit: contain).

I'm interested in support for the object-fit: cover style. This behavior is different in that the video frame is scaled up to as large as it needs to grow in order to draw in all the pixels inside the video element (preserving aspect ratio and centering). There are no black pixels.

With that style alone, it actually draws video outside of the bounds of the video element. I believe a common use case is to pair this with overflow: hidden on some parent in the DOM tree so that the video frame is "cropped" to the bounds of a certain size. I understand that implementing this in the MediaStreamRenderer would mean you would have to observe every parent of the video element up the DOM tree and that would probably be too slow. I want to suggest that we only aim to implement the correct overflow: hidden behavior for its direct parent.

[aoberoi]

This is the outcome of the discussion in aoberoi/cordova-plugin-opentokjs#5

[aoberoi]

Good test case: http://jsbin.com/yaheka/edit?html,css,js,output

[ibc]

Excellent report, and a "must have" for the plugin. I will do my best effort on this subject next week once I get some spare time.

[ibc]

Guys I'm working on this and can already detect the CSS "object-fit: cover" but cannot figure out how to translate that at the native UIView level.

I've open a thread in discuss-webrtc: https://groups.google.com/forum/#!topic/discuss-webrtc/FnSOHqvFbEw

[ibc]

All the values of object-fit have been implemented. I hope we do not need to also observe parent's overflow: hidden.

[aoberoi]

@ibc: wow, super fast! thanks for that. i'm working on some other problems in the layout for the opentok layer, but i'll pull in the latest the dependency soon. i'll report back my results.

in terms of overflow: hidden, i do think its a very common usage, but i can see that the implementation for this will be more complex. maybe create another issue to track interest from anyone else who wants to see that implemented? that could help inform whether its worth the effort.

[ibc]

@aoberoi since you mention usage of object-fit: cover (already implemented in the cordova-plugin-iosrtc), are you aware of a related bug in Chrome?:

• https://code.google.com/p/chromium/issues/detail?id=400829
• http://jsfiddle.net/73dyw/4/

Regarding the overflow: hidden in parent element(s): I anticipate that the logic for this would be really hard. Currently I use two UIView in the plugin, one which fills the video element (black background) and another one that holds the video. Having to show just a fragment of the video (and also a fragment of the video element) would require yet another UIView and lot of complexity to properly clip and position the views.

May you please share an usage example so I can understand the goal better?

[aoberoi]

@ibc i was not aware of that chrome bug, but i have starred it. thanks for pointing that out. i'll try to get more of my team members to star it as well.

if the expected behavior is as they describe, then we will not need overflow: hidden on the parent elements in order to achieve the desired outcome.

[ibc]

It is clearly a bug in Chrome. Just open the same link in Firefox (it would also work in Safari if you replace the OGG video with any MP4 video or other format supported by Safari).

Basically the cordova-plugin-iosrtc behaves better than Chrome in regards to the CSS object-fit property XD.

[aoberoi]