Rewrite RTC ICE and DTLS transport tests with alternative dependencies #9424

soareschen · 2018-02-07T10:07:50Z

This PR rewrites the tests for RTCDtlsTransport and RTCIceTransport to have separate tests for getting the transports from both RTP sender/receiver and SCTP transport. The common parts of the assertions are factored out into separate functions so that they can be tested in multiple test cases.

I have tried to write the tests as dependency-neutral as possible, and test this specifically on Microsoft Edge 42.17083. However there are still some issues that cause the tests to still fail on Edge:

The RTCPeerConnection() constructor on Edge is non compliant and require minimally new RTCPeerConnection({ iceServers: [] }) to work. On my Windows machine none of the WPT WebRTC tests that use new RTCPeerConnection() would pass. I made local modification to all constructors so that the tests can at least run on Edge locally.
Edge only supports the legacy addStream() method and not addTrack() and addTransceiver(). I have specifically added a addTrackOrTransceiver() helper function that calls either addTransceiver(), addTrack(), or even addStream() so that we can continue testing the higher level features.
Edge does not implement getSenders() and getReceivers(). This makes it impossible to retrieve the RTCRtpSender and RTCRtpReceiver instances so that we can get the DTLS and ICE transport objects. The workaround of getting hold of an RTCRtpSender using addTrack() also does not work, because Edge only implements addStream() which returns only undefined.

So despite my best effort I am unable to tweak the tests to pass on any of the current browsers. :( (perhaps the methodology also doesn't feel right)

If there are internal builds of browsers that have implemented the new features, please do run the tests and feedback on the test result is appreciated.

w3c-bots · 2018-02-07T10:35:55Z

Build PASSED

Started: 2018-02-13 11:02:25
Finished: 2018-02-13 11:36:47

Failing Jobs

chrome:dev
safari:11.0
MicrosoftEdge:16.16299

View more information about this build on:

nils-ohlmeier · 2018-02-09T23:02:06Z

Firefox hasn't implemented any of the transport objects. So these will continue to fail in the foreseeable future.

fippo

including adapter.js will get you an Edge RTCPeerConnection which supports addTrack and ice transports as well as dtls transports. It comes with its own quirks though (e.g. icetransport.statechange is still named icestatechange). More inline...

fippo · 2018-02-12T06:43:04Z

webrtc/RTCTransport-helper.js

+
+  for (const sender of senders) {
+    dtlsTransports.add(sender.transport);
+    dtlsTransports.add(sender.rtcpTransport);


rtcp-mux is optional, this will be unset set quite often. Which afaics would throw an exception further down.

Hmm in the section for Set the RTCSessionDescription, the sender.transport and sender.rtcpTransport are always set together. Though I guess we can ignore the fact here and write test elsewhere for transport and rtcpTransport must both be set.

" instead let rtcpTransport equal transport." -- right, its not null. Still, this isn't relevant to check here.

I have changed the test to continue if it can get at least one RTCDtlsTransport from the sender/receiver.

fippo · 2018-02-12T06:43:45Z

webrtc/RTCTransport-helper.js

+
+  for (const dtlsTransport of dtlsTransports) {
+    assert_true(dtlsTransport instanceof RTCDtlsTransport,
+      'Expect sctp.transport to be an RTCDtlsTransport');


no sctp in here

fippo · 2018-02-12T06:51:54Z

webrtc/RTCTransport-helper.js

+    doSignalingHandshake
+*/
+
+// Wait for a connecting dtlsTransport to transist


transist into connected state -> become connected

fippo · 2018-02-12T07:05:12Z

webrtc/RTCTransport-helper.js

+    'Expect pc to have at least one sender');
+
+  assert_greater_than(receivers.length, 0,
+    'Expect pc to have at least one receiver');


this is only going to hold in the transceiver model which is not what is tested here. Also senders and receivers share the same transport so taking both doesn't offer much value.
If you want to keep this requirement then I'd suggest adding a remote stream

I have changed the test to continue if it can get at least one RTCDtlsTransport from the senders/receivers.

fippo · 2018-02-12T07:18:44Z

webrtc/RTCIceTransport.html


-      const candidatePair1 = iceTransport1.getSelectedCandidatePair();
-      const candidatePair2 = iceTransport2.getSelectedCandidatePair();
+      return Promise.all(iceTransports.map(


this is much more complicated than waiting for pc1.iceConnectionState. Same goes for the dtls variant (which could look at connectionstate)

That's a good idea. I have changed it to wait for pc.connectionState to become connected.

fippo · 2018-02-12T08:03:41Z

webrtc/RTCIceTransport.html

-      'Expect candidate parameter to be non-null after data channels are connected');
+  function validateIceCandidateParameter(param) {
+   assert_not_equals(param, null,
+     'Expect candidate parameter to be non-null after data channels are connected');


this is no longer related to data channels. Here and elsewhere

fippo · 2018-02-12T08:04:46Z

webrtc/RTCIceTransport.html

@@ -114,77 +70,90 @@
    assert_true(candidatePair.remote instanceof RTCIceCandidate,
      'Expect candidatePair.remote to be instance of RTCIceCandidate');

-    validateCandidateParameter(iceTransport.getLocalParameters());
-    validateCandidateParameter(iceTransport.getRemoteParameters());
+    validateIceCandidateParameter(iceTransport.getLocalParameters());


there are the ice transport parameters, not candidate parameters.

fippo · 2018-02-12T08:07:13Z

webrtc/RTCPeerConnection-helper.js

+  } else if (typeof pc.addTrack === 'function') {
+    // Note: The mediaStream argument is optional in the spec, but we require
+    // it for now so that we can test older versions of Firefox that only
+    // implements addTrack with compulsary mediaStream argument.


fippo · 2018-02-12T08:07:36Z

webrtc/RTCPeerConnection-helper.js

@@ -418,6 +418,31 @@ function getUserMediaTracksAndStreams(count, type = 'audio') {
  });
 }

+// A dependency agnostic way of adding track depending
+// on the supported method on a browser. Uses addTransceiver(track)
+// first if available, otherwise use addTrack(track, mediaStream).


or falls back to the legacy addStream method (which has been removed from the spec).

alvestrand · 2018-11-03T03:48:51Z

webrtc/RTCDtlsTransport-getRemoteCertificates.html

-        const certs = dtlsTransport.getRemoteCertificates();
+    return createDtlsTransportsFromSenderReceiver(pc1, pc2)
+    .then(([dtlsTransports1, dtlsTransports2]) => {
+      const dtlsTransports = [...dtlsTransports1, ... dtlsTransports2];


Please use spaces around the ... operator.

alvestrand

A number of things that I don't think are very readable. In particular, mixing state-changing (connecting PCs) and info-fetching (extracting the DTLS and ICE transports) makes the tests hard to read and maintain.

alvestrand · 2018-11-03T03:55:14Z

webrtc/RTCDtlsTransport-getRemoteCertificates.html

+  }
+
+  function validateConnectingDtlsTransport(dtlsTransport) {
+    if (dtlsTransport.state !== 'connected') {


If state is connected, this function will succeed. If that's intentional, add a comment.

alvestrand · 2018-11-03T03:58:20Z

webrtc/RTCPeerConnection-helper.js

@@ -418,6 +418,28 @@ function getUserMediaTracksAndStreams(count, type = 'audio') {
  });
 }

+// A dependency agnostic way of adding track depending


English: "of adding track" => "to add a track"

alvestrand · 2018-11-03T04:00:14Z

webrtc/RTCPeerConnection-helper.js

@@ -418,6 +418,28 @@ function getUserMediaTracksAndStreams(count, type = 'audio') {
  });
 }

+// A dependency agnostic way of adding track depending
+// on the supported method on a browser. Uses addTransceiver(track)


Rewrite this to be more readable:

If addTransceiver(track) is available, use that.

If addtrack(track, mediastream) is available, use that.

Otherwise, use addStream(mediastream).

alvestrand · 2018-11-03T04:01:21Z

webrtc/RTCPeerConnection-helper.js

+    // support addTrack or addTransceiver.
+    return pc.addStream(mediaStream)
+  } else {
+    throw new Error('This test requires either addTrack or addTransceiver implemented');


Make the error correct - "This test requires one of addTrack, addTransceiver or addStream to be implemented". We can remove addStream from the error message at the same time as removing it from the code.

alvestrand · 2018-11-03T04:03:21Z

webrtc/RTCTransport-helper.js

+// is "connected", all RTCIceTransports and RTCDtlsTransports are in the
+// "connected", "completed" or "closed" state and at least one of them is in the
+// "connected" or "completed" state.
+function waitConnectingPc(pc) {


This name is not obvious. Can we call it "waitForConnectedState"?

alvestrand · 2018-11-03T04:09:40Z

webrtc/RTCTransport-helper.js

+  return dtlsTransports;
+}
+
+function getIceTransportFromDtlsTransport(dtlsTransport) {


This function is so small, I don't think it should be a function.

You can replace it with dtlsTransport.transport (dtlsTransport.iceTransport after the change agreed to in Lyon goes through).

alvestrand · 2018-11-03T04:13:50Z

webrtc/RTCTransport-helper.js

+  return [...iceTransports];
+}
+
+// Create DTLS transports by creating data channels


English: "Create a pair of DTLS transports by creating a data channel from pc1 to pc2. Returns an array containing two RTCDtlsTransports, obtained from each PC via pc.sctp."

alvestrand · 2018-11-03T04:14:31Z

webrtc/RTCTransport-helper.js

+}
+
+// Create DTLS transports by adding tracks and connecting
+// two peer connections. Returns an array of two array of


As above, but also "Returns an array of two arrays of RTCDtlsTransports."

alvestrand · 2018-11-03T04:16:53Z

webrtc/RTCTransport-helper.js

+// and connecting two peer connections. Returns an
+// array of two RTCDtlsTransports, which obtained
+// from RTCSctpTransport in pc.sctp.
+function createDtlsTransportsFromSctp(pc1, pc2) {


Suggest renaming this function "connectPeerConnectionsAndReturnDtlsTransports".

I'd really prefer to not have this function, but instead have "connectPeerConnnections" and the (existing) getDtlsTransportFromSctpTransport called from the test functions - merging these two disparate things makes the test harder to read.

alvestrand · 2018-11-03T04:17:29Z

webrtc/RTCTransport-helper.js

+  });
+}
+
+function createIceTransportsFromSctp(pc1, pc2) {


These two functions also, I'd prefer that did not exist.

soareschen · 2018-11-03T07:13:47Z

Closing this with #13886 as continuation.

soareschen added 4 commits February 6, 2018 18:23

Decouple dependencies for RTCIceTransport tests

be4523d

Rewrite ICE and DTLS transport tests with alternative dependencies

e2f8cdb

Add in comments

029c0b6

Refactor RTCIceTransport tests

1650556

wpt-pr-bot added webrtc wg-webrtc labels Feb 7, 2018

wpt-pr-bot requested review from agouaillard, alvestrand, dontcallmedom, guidou, henbos, phoglund, rwaldron, snuggs and youennf February 7, 2018 10:07

fippo reviewed Feb 13, 2018

View reviewed changes

soareschen added 2 commits February 13, 2018 18:04

Fix review feedback

c618204

Remove console.warn to pass lint check

a57c501

alvestrand reviewed Nov 3, 2018

View reviewed changes

soareschen mentioned this pull request Nov 3, 2018

Rewrite RTC ICE and DTLS transport tests with alternative dependencies (Rebased) #13886

Closed

soareschen closed this Nov 3, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rewrite RTC ICE and DTLS transport tests with alternative dependencies #9424

Rewrite RTC ICE and DTLS transport tests with alternative dependencies #9424

soareschen commented Feb 7, 2018 •

edited by wpt-pr-bot

Loading

w3c-bots commented Feb 7, 2018 •

edited

Loading

nils-ohlmeier commented Feb 9, 2018

fippo left a comment

fippo Feb 12, 2018

soareschen Feb 13, 2018

fippo Feb 13, 2018

soareschen Feb 13, 2018

fippo Feb 12, 2018

fippo Feb 12, 2018

fippo Feb 12, 2018

soareschen Feb 13, 2018

fippo Feb 12, 2018

soareschen Feb 13, 2018

fippo Feb 12, 2018

fippo Feb 12, 2018

fippo Feb 12, 2018

fippo Feb 12, 2018

alvestrand Nov 3, 2018

alvestrand left a comment

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

alvestrand Nov 3, 2018

soareschen commented Nov 3, 2018

Rewrite RTC ICE and DTLS transport tests with alternative dependencies #9424

Rewrite RTC ICE and DTLS transport tests with alternative dependencies #9424

Conversation

soareschen commented Feb 7, 2018 • edited by wpt-pr-bot Loading

w3c-bots commented Feb 7, 2018 • edited Loading

Build PASSED

Failing Jobs

nils-ohlmeier commented Feb 9, 2018

fippo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvestrand left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soareschen commented Nov 3, 2018

soareschen commented Feb 7, 2018 •

edited by wpt-pr-bot

Loading

w3c-bots commented Feb 7, 2018 •

edited

Loading