Update pan, tilt and zoom constraints #218

eehakkin · 2020-03-20T15:14:34Z

Our model of thinking has been :
The Pan-Tilt-Zoom permission can only be requested (and only shown in the prompt) if the currently connected/selected camera supports it.

Camera permission without PTZ (obtained using an older version of the user agent, or with another camera) is not implicitly upgraded to Camera permission with PTZ (even if the hardware supports it). The permission has to be re-requested.

In w3ctag/design-reviews#358, TAG wants to make sure that from the web app’s perspective: Pan-Tilt-Zoom needs to be explicitly requested as an extension to video capture permission.

To facilitate the previous requirements we want to add a new CameraDevicePermissionDescriptor dictionary introduced in w3c/permissions/pull/204 to differentiate between the two cases and to modify media capture so that using a pan, tilt or zoom constrainable property prompts for a {name: "camera", panTiltZoom: true} permission and that it is possible to request elevated permission prompt during getUserMedia call in order to avoid double permission prompt ({name: "camera"} during getUserMedia and {name: "camera", panTiltZoom: true} during applyConstraints).

This allows the following kind of code:

navigator.mediaDevices.getUserMedia({
    video: {
        height: { ideal:  720, min: 240, max: 1080 },
        width:  { ideal: 1280, min: 320, max: 1920 },
        // These empty constraints are to make getUserMedia
        // to request permission to use a permission descriptor
        // {name: "camera", deviceId: deviceId, panTiltZoom: true}
        // instead of normal {name: "camera", deviceId: deviceId}
        // so that subsequent applyConstraints call do not have to
        // prompt for  new permissions.
        pan:    true,  // or {}
        tilt:   {},    // or true
        zoom:   true   // or {}
    }
})
.then(async mediaStream => {
    const video = document.querySelector('video');
    video.srcObject = mediaStream;
    await sleep(1000);  // crbug.com/711524
    const track = mediaStream.getVideoTracks()[0];
    const capabilities = track.getCapabilities();
    const settings = track.getSettings();
    if ('pan' in capabilities) {
        const input = document.getElementById('pan-range');
        input.min = capabilities.pan.min;
        input.max = capabilities.pan.max;
        input.step = capabilities.pan.step;
        input.value = settings.pan;
        input.oninput = function(event) {
            // Applying pan (or tilt or zoom) constraint
            // requests permission to use a permission descriptor
            // {name: "camera", deviceId: deviceId, panTiltZoom: true}
            // which is already granted during successful getUserMedia
            // call above thus this applyConstraints call does not prompt for
            // any new permissions.
            track.applyConstraints({advanced: [{pan: event.target.value}]});
        };
        input.hidden = false;
    }
    if ('tilt' in capabilities) {
        // ditto
    }
    if ('zoom' in capabilities) {
        // ditto
    }
});

For API, it would of course be possible to combine {video: {pan: true, tilt: true, zoom: true}} to {video: {panTiltZoom: true}} but in that case one had to use navigator.mediaDevices.getUserMedia({video: {panTiltZoom: true, zoom: {min: 2}}}) instead of navigator.mediaDevices.getUserMedia({video: {zoom: {min: 2}}}) or similar if one wants to use pan, tilt or zoom constraints in getUserMedia. With separate pan, tilt and zoom (instead of a combined panTiltZoom) it is also easier to expand to roll if pan/tilt/roll/zoom cameras ever become more widespread. But please share your opinions.

/cc @riju

riju · 2020-03-20T15:26:56Z

@eehakkin : Let's edit this PR to add the JS example in Examples Section

riju · 2020-03-20T15:31:12Z

@beaufortfrancois , @reillyeon, @kenchris PTAL

reillyeon

This looks good to me.

index.bs

beaufortfrancois · 2020-03-23T07:43:54Z

index.bs

+    ConstrainDouble              sharpness;
+
+    ConstrainDouble              focusDistance;
+    (boolean or ConstrainDouble) pan;


Can you walk through the rationale of this boolean or ConstrainDouble?
Why introducing this pattern now?

My rationale is to keep similarity with audio/video. So while it is possible to say navigator.mediaDevices.getUserMedia({video: {}}), it is maybe clearer to say navigator.mediaDevices.getUserMedia({video: true}). Similarly while should be possible to say navigator.mediaDevices.getUserMedia({video: {pan: {}, zoom: {}}}), it might be clearer to say navigator.mediaDevices.getUserMedia({video: {pan: true, zoom: true}}).

I agree with you about the simplicity. It looks much better.
I'm wondering why this pattern wasn't used before for other constraints than audio/video. Maybe there's something we're missing.

@YellowDoge maybe would know

@beaufortfrancois no particular reason, IIRC this spec predates the major clean up and homogeneisation of the constraints about 3-4y ago...? So perhaps it was just due an overhaul. IMHO this looks better & we should create implementation bugs.

beaufortfrancois · 2020-03-25T06:51:28Z

index.bs

+    ConstrainDouble              sharpness;
+
+    ConstrainDouble              focusDistance;
+    (boolean or ConstrainDouble) pan;


@YellowDoge maybe would know

beaufortfrancois · 2020-03-25T06:56:39Z

index.bs

+          input.oninput = function(event) {
+            track.applyConstraints({advanced: [{tilt: event.target.value}]});
+          };
+          input.parentElement.hidden = false;


I think we can remove the hidden part as it doesn't add that much value in the context of an example in a spec.

Note that it wasn't present in the pan example section

beaufortfrancois

LGTM with an updated codepen example that reflects new pan and tilt constraints.

beaufortfrancois · 2020-03-30T05:43:17Z

index.bs

@@ -835,7 +847,7 @@ A {{Point2D}} represents a location in a two dimensional space. The origin of co
  Slightly modified versions of these examples can be found in e.g. <a href="https://codepen.io/collection/nLeobQ/">this codepen collection</a>.
 </div>

-## Update camera zoom and {{takePhoto()}} ## {#example1}
+## Update camera pan, tilt and zoom and {{takePhoto()}} ## {#example1}

 <div class="note">
  The following example can also be found in e.g. <a href="https://codepen.io/miguelao/pen/BLPzKx/left?editors=1010">this codepen</a> with minimal modifications.


I believe we can now link to an updated sample

riju

Thanks @eehakkin and @beaufortfrancois !

jan-ivar · 2020-04-06T20:56:50Z

@eehakkin Was the reason for the new true semantics to be able to acquire the camera without altering the current pan, tilt and zoom values? Otherwise why not just:

navigator.mediaDevices.getUserMedia({video: {pan: 0, tilt: 0, zoom: 1}});

eehakkin · 2020-04-07T09:41:07Z

@eehakkin Was the reason for the new true semantics to be able to acquire the camera without altering the current pan, tilt and zoom values?

Yes, exactly that. So with the new true semantics it is possible to acquire a pan-tilt-zoom camera (instead of a static camera) during getUserMedia without altering the current pan, tilt and zoom values and then later pan, tilt or zoom the acquired camera using applyConstraints without a new prompt (as there is no need to elevate the permission from a normal camera permission to a pan-tilt-zoom camera permission as the latter one was requested from the beginning).

eehakkin mentioned this pull request Mar 20, 2020

Add CameraDevicePermissionDescriptor for 'camera' permission w3c/permissions#204

Merged

reillyeon approved these changes Mar 20, 2020

View reviewed changes

beaufortfrancois reviewed Mar 23, 2020

View reviewed changes

eehakkin force-pushed the master branch 3 times, most recently from c4140e4 to 6f55db9 Compare March 24, 2020 16:32

beaufortfrancois reviewed Mar 25, 2020

View reviewed changes

eehakkin force-pushed the master branch from 6f55db9 to a03627f Compare March 27, 2020 16:26

beaufortfrancois approved these changes Mar 30, 2020

View reviewed changes

Update pan, tilt and zoom constraints

3e3b6a7

eehakkin force-pushed the master branch from a03627f to 3e3b6a7 Compare March 30, 2020 15:21

riju approved these changes Mar 30, 2020

View reviewed changes

riju merged commit f362399 into w3c:master Mar 30, 2020

eehakkin deleted the master branch March 30, 2020 19:02

riju added the PTZ Pan-Tilt-Zoom label Apr 27, 2020

beaufortfrancois mentioned this pull request Apr 28, 2020

The (bool or ConstrainDouble) case for pan/tilt/zoom is unclear #225

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update pan, tilt and zoom constraints #218

Update pan, tilt and zoom constraints #218

eehakkin commented Mar 20, 2020 •

edited by riju

Loading

riju commented Mar 20, 2020 •

edited

Loading

riju commented Mar 20, 2020

reillyeon left a comment

beaufortfrancois Mar 23, 2020

eehakkin Mar 23, 2020

beaufortfrancois Mar 23, 2020 •

edited

Loading

beaufortfrancois Mar 25, 2020

yell0wd0g Mar 30, 2020

beaufortfrancois Mar 30, 2020

beaufortfrancois Mar 25, 2020

beaufortfrancois Mar 25, 2020

beaufortfrancois Mar 25, 2020

beaufortfrancois left a comment

beaufortfrancois Mar 30, 2020

riju left a comment

jan-ivar commented Apr 6, 2020

eehakkin commented Apr 7, 2020

Update pan, tilt and zoom constraints #218

Update pan, tilt and zoom constraints #218

Conversation

eehakkin commented Mar 20, 2020 • edited by riju Loading

riju commented Mar 20, 2020 • edited Loading

riju commented Mar 20, 2020

reillyeon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

beaufortfrancois Mar 23, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

beaufortfrancois left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

riju left a comment

Choose a reason for hiding this comment

jan-ivar commented Apr 6, 2020

eehakkin commented Apr 7, 2020

eehakkin commented Mar 20, 2020 •

edited by riju

Loading

riju commented Mar 20, 2020 •

edited

Loading

beaufortfrancois Mar 23, 2020 •

edited

Loading