Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ICE end-of-candidates messages #2622

Merged
merged 3 commits into from
Aug 26, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 28 additions & 19 deletions src/webrtc/call.ts
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ export class MatrixCall extends TypedEventEmitter<CallEvent, CallEventHandlerMap
// possible
private candidateSendQueue: Array<RTCIceCandidate> = [];
private candidateSendTries = 0;
private sentEndOfCandidates = false;
private candidatesEnded = false;
private feeds: Array<CallFeed> = [];
private usermediaSenders: Array<RTCRtpSender> = [];
private screensharingSenders: Array<RTCRtpSender> = [];
Expand Down Expand Up @@ -1597,6 +1597,11 @@ export class MatrixCall extends TypedEventEmitter<CallEvent, CallEventHandlerMap
*/
private gotLocalIceCandidate = (event: RTCPeerConnectionIceEvent): Promise<void> => {
if (event.candidate) {
if (this.candidatesEnded) {
logger.warn("Got candidate after candidates have ended - ignoring!");
return;
}

logger.debug(
"Call " + this.callId + " got local ICE " + event.candidate.sdpMid + " candidate: " +
event.candidate.candidate,
Expand All @@ -1606,29 +1611,18 @@ export class MatrixCall extends TypedEventEmitter<CallEvent, CallEventHandlerMap

// As with the offer, note we need to make a copy of this object, not
// pass the original: that broke in Chrome ~m43.
if (event.candidate.candidate !== '' || !this.sentEndOfCandidates) {
if (event.candidate.candidate === '') {
this.queueCandidate(null);
} else {
this.queueCandidate(event.candidate);

if (event.candidate.candidate === '') this.sentEndOfCandidates = true;
}
}
};

private onIceGatheringStateChange = (event: Event): void => {
logger.debug(`Call ${this.callId} ice gathering state changed to ${this.peerConn.iceGatheringState}`);
if (this.peerConn.iceGatheringState === 'complete' && !this.sentEndOfCandidates) {
// If we didn't get an empty-string candidate to signal the end of candidates,
// create one ourselves now gathering has finished.
// We cast because the interface lists all the properties as required but we
// only want to send 'candidate'
// XXX: We probably want to send either sdpMid or sdpMLineIndex, as it's not strictly
// correct to have a candidate that lacks both of these. We'd have to figure out what
// previous candidates had been sent with and copy them.
const c = {
candidate: '',
} as RTCIceCandidate;
this.queueCandidate(c);
this.sentEndOfCandidates = true;
if (this.peerConn.iceGatheringState === 'complete') {
this.queueCandidate(null);
}
};

Expand Down Expand Up @@ -2247,7 +2241,12 @@ export class MatrixCall extends TypedEventEmitter<CallEvent, CallEventHandlerMap
}
}

private queueCandidate(content: RTCIceCandidate): void {
/**
* Queue a candidate to be sent
* @param content The candidate to queue up, or null if candidates have finished being generated
* and end-of-candidates should be signalled
*/
private queueCandidate(content: RTCIceCandidate | null): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could avoid the step with null and set candidatesEnded right at the start?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean here - at the start of what?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking we would set candidatesEnded = true right after receiving the last candidate and avoid calling queueCandidate() with null though since that might require a bit more work (if even possible) and you're on holiday, we can go with this as is

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I see - we need to schedule it to be sent too though, so this seemed the easiest way of re-using that code.

// We partially de-trickle candidates by waiting for `delay` before sending them
// amalgamated, in order to avoid sending too many m.call.candidates events and hitting
// rate limits in Matrix.
Expand All @@ -2257,7 +2256,11 @@ export class MatrixCall extends TypedEventEmitter<CallEvent, CallEventHandlerMap
// currently proposes as the way to indicate that candidate gathering is complete.
// This will hopefully be changed to an explicit rather than implicit notification
// shortly.
this.candidateSendQueue.push(content);
if (content) {
this.candidateSendQueue.push(content);
} else {
this.candidatesEnded = true;
}

// Don't send the ICE candidates yet if the call is in the ringing state: this
// means we tried to pick (ie. started generating candidates) and then failed to
Expand Down Expand Up @@ -2446,6 +2449,12 @@ export class MatrixCall extends TypedEventEmitter<CallEvent, CallEventHandlerMap
this.candidateSendQueue = [];
++this.candidateSendTries;
const content = { candidates: candidates.map(candidate => candidate.toJSON()) };
if (this.candidatesEnded) {
// If there are no more candidates, signal this by adding an empty string candidate
content.candidates.push({
candidate: '',
});
}
logger.debug(`Call ${this.callId} attempting to send ${candidates.length} candidates`);
try {
await this.sendVoipEvent(EventType.CallCandidates, content);
Expand Down