Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate jitterBufferTarget as candidate addition #2953

Merged
merged 2 commits into from
Apr 11, 2024
Merged

Integrate jitterBufferTarget as candidate addition #2953

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
24b121e
Integrate jitterBufferTarget as candidate addition
dontcallmedom Mar 29, 2024
2670d2f
Update references to PR in amendments list
dontcallmedom Mar 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 54 additions & 1 deletion amendments.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -819,13 +819,66 @@
"id": 42
}
],
"create-receiver-algo": [
"rtcrtpreceiver-receivecodecs": [
{
"description": "Redefine SendCodecs and ReceiveCodecs",
"pr": 2935,
"type": "addition",
"status": "candidate",
"id": 41
}
],
"rtcrtpreceiver-laststablestatereceivecodecs": [
{
"description": "Add control for the receiver's jitter buffer",
"type": "addition",
"status": "candidate",
"difftype": "append",
"pr": 2953,
"id": 44,
"tests": [
"webrtc/RTCRtpReceiver-jitterBufferTarget.html",
"webrtc/RTCRtpReceiver-audio-jitterBufferTarget-stats.https.html",
"webrtc/RTCRtpReceiver-video-jitterBufferTarget-stats.html"
],
"testUpdates": [
"web-platform-tests/wpt#45427"
]
}
],
"webidl-rtcrtpreceiver": [
{
"description": "Add control for the receiver's jitter buffer",
"type": "addition",
"status": "candidate",
"pr": 2953,
"id": 44,
"tests": [
"webrtc/RTCRtpReceiver-jitterBufferTarget.html",
"webrtc/RTCRtpReceiver-audio-jitterBufferTarget-stats.https.html",
"webrtc/RTCRtpReceiver-video-jitterBufferTarget-stats.html"
],
"testUpdates": [
"web-platform-tests/wpt#45427"
]
}
],
"rtcrtpreceiver-attributes-transport": [
{
"description": "Add control for the receiver's jitter buffer",
"type": "addition",
"status": "candidate",
"difftype": "append",
"pr": 2953,
"id": 44,
"tests": [
"webrtc/RTCRtpReceiver-jitterBufferTarget.html",
"webrtc/RTCRtpReceiver-audio-jitterBufferTarget-stats.https.html",
"webrtc/RTCRtpReceiver-video-jitterBufferTarget-stats.html"
],
"testUpdates": [
"web-platform-tests/wpt#45427"
]
}
]
}
8 changes: 4 additions & 4 deletions base-rec.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10192,14 +10192,14 @@ <h3 id="x5-3-rtcrtpreceiver-interface"><bdi class="secno">5.3 </bdi>
internal slot and initialize it to an empty list.
</p>
</li>
<li class="needs-tests">
<li class="needs-tests" id="rtcrtpreceiver-receivecodecs">
<p>
Let <var>receiver</var> have a <dfn data-dfn-type="idl" id="dfn-receivecodecs">[[ReceiveCodecs]]</dfn>
internal slot, representing a list of <a data-link-type="idl" href="#dom-rtcrtpcodecparameters" class="internalDFN" id="ref-for-dom-rtcrtpcodecparameters-6"><code><code>RTCRtpCodecParameters</code></code></a>
dictionaries, and initialized to an empty list.
</p>
</li>
<li class="needs-tests">
<li class="needs-tests" id="rtcrtpreceiver-laststablestatereceivecodecs">
<p>
Let <var>receiver</var> have a
<dfn data-dfn-type="idl" id="dfn-laststablestatereceivecodecs">[[LastStableStateReceiveCodecs]]</dfn> internal slot and
Expand All @@ -10213,7 +10213,7 @@ <h3 id="x5-3-rtcrtpreceiver-interface"><bdi class="secno">5.3 </bdi>
</li>
</ol>
<div>
<pre class="idl has-tests def" data-tests="idlharness.https.window.js" id="webidl-436594269"><span class="idlHeader"><a class="self-link" href="#webidl-436594269">WebIDL</a></span><span data-idl="" class="idlInterface" id="idl-def-rtcrtpreceiver" data-title="RTCRtpReceiver">[<span class="extAttr"><a data-type="extended-attribute" href="https://heycam.github.io/webidl/#Exposed">Exposed</a>=<a data-type="interface" href="https://html.spec.whatwg.org/multipage/window-object.html#window">Window</a></span>]
<pre class="idl has-tests def" data-tests="idlharness.https.window.js" id="webidl-rtcrtpreceiver"><span class="idlHeader"><a class="self-link" href="#webidl-436594269">WebIDL</a></span><span data-idl="" class="idlInterface" id="idl-def-rtcrtpreceiver" data-title="RTCRtpReceiver">[<span class="extAttr"><a data-type="extended-attribute" href="https://heycam.github.io/webidl/#Exposed">Exposed</a>=<a data-type="interface" href="https://html.spec.whatwg.org/multipage/window-object.html#window">Window</a></span>]
interface <a class="internalDFN idlID" data-link-type="interface" href="#dom-rtcrtpreceiver" id="ref-for-dom-rtcrtpreceiver-25"><code>RTCRtpReceiver</code></a> {<span data-idl="" class="idlAttribute" id="idl-def-rtcrtpreceiver-track" data-title="track" data-dfn-for="RTCRtpReceiver">
readonly attribute<span class="idlType"> <a data-type="interface" href="https://www.w3.org/TR/mediacapture-streams/#dom-mediastreamtrack">MediaStreamTrack</a></span> <a class="internalDFN idlName" data-link-type="attribute" href="#dom-rtpreceiver-track" id="ref-for-dom-rtpreceiver-track-1"><code>track</code></a>;</span><span data-idl="" class="idlAttribute" id="idl-def-rtcrtpreceiver-transport" data-title="transport" data-dfn-for="RTCRtpReceiver">
readonly attribute<span class="idlType"> <a href="#dom-rtcdtlstransport" class="internalDFN" data-link-type="idl" id="ref-for-dom-rtcdtlstransport-14"><code>RTCDtlsTransport</code></a>?</span> <a class="internalDFN idlName" data-link-type="attribute" href="#dom-rtcrtpreceiver-transport" id="ref-for-dom-rtcrtpreceiver-transport-1"><code>transport</code></a>;</span><span data-idl="" class="idlMethod" id="idl-def-rtcrtpreceiver-getcapabilities-kind" data-title="getCapabilities" data-dfn-for="RTCRtpReceiver">
Expand Down Expand Up @@ -10249,7 +10249,7 @@ <h4 id="attributes-10">
<dt>
<dfn data-idl="attribute" data-export="" data-dfn-type="attribute" id="dom-rtcrtpreceiver-transport" data-title="transport" data-dfn-for="RTCRtpReceiver" data-type="RTCDtlsTransport" data-lt="transport" data-local-lt="RTCRtpReceiver.transport"><code>transport</code></dfn> of type <span class="idlAttrType"><a data-link-type="idl" href="#dom-rtcdtlstransport" class="internalDFN" id="ref-for-dom-rtcdtlstransport-15"><code><code>RTCDtlsTransport</code></code></a></span>, readonly, nullable
</dt>
<dd>
<dd id="rtcrtpreceiver-attributes-transport">
<p>
The <a data-link-type="idl" href="#dom-rtcrtpreceiver-transport" class="internalDFN" id="ref-for-dom-rtcrtpreceiver-transport-2"><code><code>transport</code></code></a> attribute is the transport over which media
for the receiver's <a data-link-type="idl" href="#dom-rtpreceiver-track" class="internalDFN" id="ref-for-dom-rtpreceiver-track-5"><code><code>track</code></code></a> is received in
Expand Down
121 changes: 117 additions & 4 deletions webrtc.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10274,7 +10274,7 @@ <h3>
internal slot and initialize it to an empty list.
</p>
</li>
<li>
<li id="rtcrtpreceiver-receivecodecs">
<p>
Let <var>receiver</var> have a <dfn data-dfn-for="RTCRtpReceiver">[[\ReceiveCodecs]]</dfn>
internal slot, representing a list of [=tuple=]s, each containing a {{RTCRtpCodecParameters}}
Expand All @@ -10283,21 +10283,27 @@ <h3>
set in an implementation defined manner.
</p>
</li>
<li>
<li id="rtcrtpreceiver-laststablestatereceivecodecs">
<p>
Let <var>receiver</var> have a
<dfn data-dfn-for="RTCRtpReceiver">[[\LastStableStateReceiveCodecs]]</dfn> internal slot and
initialize it to an empty list.
</p>
</li>
<li class="add-to-rtcrtpreceiver-laststablestatereceivecodecs" data-test="webrtc/RTCRtpReceiver-jitterBufferTarget.html">
<p>
Let <var>receiver</var> have a <dfn class="export" data-dfn-for="RTCRtpReceiver">[[\JitterBufferTarget]]</dfn>
internal slot initialized to <code>null</code>.
</p>
</li>
<li class="no-test-needed">
<p>
Return <var>receiver</var>.
</p>
</li>
</ol>
<div>
<pre class="idl" data-tests="idlharness.https.window.js">[Exposed=Window]
<pre id="webidl-rtcrtpreceiver" class="idl" data-tests="idlharness.https.window.js">[Exposed=Window]
interface RTCRtpReceiver {
readonly attribute MediaStreamTrack track;
readonly attribute RTCDtlsTransport? transport;
Expand All @@ -10306,6 +10312,7 @@ <h3>
sequence&lt;RTCRtpContributingSource&gt; getContributingSources();
sequence&lt;RTCRtpSynchronizationSource&gt; getSynchronizationSources();
Promise&lt;RTCStatsReport&gt; getStats();
attribute DOMHighResTimeStamp? jitterBufferTarget;
};</pre>
<section>
<h2>
Expand Down Expand Up @@ -10335,7 +10342,7 @@ <h2>
<dfn data-idl="">transport</dfn> of type <span class=
"idlAttrType">{{RTCDtlsTransport}}</span>, readonly, nullable
</dt>
<dd>
<dd id="rtcrtpreceiver-attributes-transport">
<p>
The {{transport}} attribute is the transport over which media
for the receiver's {{RTCRtpReceiver/track}} is received in
Expand All @@ -10350,6 +10357,112 @@ <h2>
{{RTCRtpReceiver/[[ReceiverTransport]]}} slot.
</p>
</dd>
<dt class="add-to-rtcrtpreceiver-attributes-transport"><dfn>jitterBufferTarget</dfn> of type {{DOMHighResTimeStamp}}, nullable
<dd data-tests="RTCRtpReceiver-jitterBufferTarget.html,RTCRtpReceiver-audio-jitterBufferTarget-stats.https.html,RTCRtpReceiver-video-jitterBufferTarget-stats.html" class="add-to-rtcrtpreceiver-attributes-transport">
<p>This attribute allows the application to specify a target duration
of time in milliseconds of media for the {{RTCRtpReceiver}}'s jitter
buffer to hold. This influences the amount of buffering done by the
<a>user agent</a>, which in turn affects retransmissions and packet loss
recovery. Altering the target value allows applications to control the
tradeoff between playout delay and the risk of running out of audio or
video frames due to network jitter.

<p>The <a>user agent</a> MUST have a <dfn>minimum allowed target</dfn> and a
<dfn>maximum allowed target</dfn> reflecting what the <a>user agent</a> is
able or willing to provide based on network conditions and memory
constraints, which can change at any time.</p>
<div class="note">
<p>This is a target value. The resulting change in delay can be gradually
observed over time. The receiver's average jitter buffer delay can be
measured as the delta
{{RTCInboundRtpStreamStats/jitterBufferDelay}} divided by the delta
{{RTCInboundRtpStreamStats/jitterBufferEmittedCount}}.
</p>
<p>
An average delay is expected even if DTX is used. For example, if
DTX is used and packets start flowing after silence, larger targets can
influence the <a>user agent</a> to buffer these packets rather than
playing them out.
</p>
</div>
<p>On getting, this attribute MUST return the value of the
{{RTCRtpReceiver/[[JitterBufferTarget]]}} internal slot.</p>
<p>On setting, the <a>user agent</a> MUST run the following steps:</p>
<ol class=algorithm>
<li>
<p>Let <var>receiver</var> be the
{{RTCRtpReceiver}} object on which the setter is
invoked.</p>
</li>
<li>
<p>Let <var>target</var> be the argument to the setter.</p>
</li>
<li>
<p>If <var>target</var> is negative or larger than 4000 milliseconds, then
[=exception/throw=] a {{RangeError}}.</p>
</li>
<li>
<p>Set <var>receiver</var>'s {{RTCRtpReceiver/[[JitterBufferTarget]]}}
to <var>target</var>.</p>
</li>
<li>
<p>Let <var>track</var> be <var>receiver</var>'s
{{RTCRtpReceiver/[[ReceiverTrack]]}}.</p>
</li>
<li>
<p>In parallel, begin executing the following steps:</p>
<ol>
<li>
<p>Update the underlying system about the new <var>target</var>,
or that there is no application preference if <var>target</var> is
<code>null</code>.</p>
<p>
If <var>track</var> is synchronized with another
{{RTCRtpReceiver}}'s track for
<a data-cite="RFC5888#section-7">audio/video synchronization</a>,
then the <a>user agent</a> SHOULD use the larger of the two receivers'
{{RTCRtpReceiver/[[JitterBufferTarget]]}} for both receivers.
</p>
<p>
When the underlying system is applying a jitter buffer target, it will
continuously make sure that the actual jitter buffer target is clamped
within the <a>minimum allowed target</a> and <a>maximum allowed
target</a>.
<p class="note">
If the <a>user agent</a> ends up using a target different from the
requested one (e.g. due to network conditions or physical memory
constraints), this is not reflected in the
{{RTCRtpReceiver/[[JitterBufferTarget]]}} internal slot.
</p>
</p>
</li>
<li>
<p>Modifying the jitter buffer target of the underlying system SHOULD
affect the internal audio or video buffering gradually in order not
to hurt user experience. Audio samples or video frames SHOULD
be accelerated or decelerated before playout, similarly to how
it is done for
<a data-cite="RFC5888#section-7">
audio/video synchronization</a> or in response to congestion
control.</p>
<p>The acceleration or deceleration rate may vary depending on
network conditions or the type of audio received (e.g. speech
or background noise). It MAY take several seconds to achieve 1
second of buffering but SHOULD not take more than 30 seconds
assuming packets are being received. The speed MAY be
different for audio and video.</p>
<p class="note">
For audio, acceleration and deceleration can be measured
with {{RTCInboundRtpStreamStats/insertedSamplesForDeceleration}}
and {{RTCInboundRtpStreamStats/removedSamplesForAcceleration}}.
For video, this may result in the same frame being rendered
multiple times or frames may be dropped.
</p>
</li>
</ol>
</li>
</ol>
</dd>
</dl>
</section>
<section>
Expand Down
Loading