From 78bfa258657a99fdb4c1ca381fbb2b462b7108f0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Thu, 22 Jul 2021 14:42:26 +0200 Subject: [PATCH 01/22] MSC: Muting in VoIP calls MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/0000-muting.md | 76 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 proposals/0000-muting.md diff --git a/proposals/0000-muting.md b/proposals/0000-muting.md new file mode 100644 index 00000000000..741293dc460 --- /dev/null +++ b/proposals/0000-muting.md @@ -0,0 +1,76 @@ +# MSC0000: Muting in VoIP calls + +During VoIP calls, it is common for a user to mute their microphone/camera. +Ideally, the other side should be able to see that the opponent's camera is +muted, so that it could reflect this in the UI (e.g. show the user's avatar +instead of their camera feed). We would also want the changes in the mutes state +to be quick. + +Using pure WebRTC there are two ways to do muting and both have their issues: + ++ Disabling the corresponding track - there will be no way to know the track is + muted from the opponent's perspective. On the other hand, this is almost + instantaneous. ++ Setting the corresponding track as `recvonly`/`inactive` - this is problematic + since it leads to re-negotiation which takes time. This could also conflict + with holding (as defined in + [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). + +## Proposal + +This MSC proposes a solution to this by extending the `SDPStreamMetadata` (see +[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) by the following +fields: + ++ `audio_muted` - a boolean indicating the current audio mute state ++ `video_muted` - a boolean indicating the current video mute state + +This MSC also adds a new call event `m.call.sdp_stream_metadata_changed`, which +has the common VoIP fields as specified in +[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) (`version`, +`call_id`, `party_id`) and a `sdp_stream_metadata` object which is the same +thing as `sdp_stream_metadata` in `m.call.negotiate`, `m.call.invite` and +`m.call.answer`. The client sends this event the when `sdp_stream_metadata` has +changed but no negotiation is required (e.g. the user mutes their +camera/microphone). + +If a client sends an event with some of the fields missing the previous state +should be assumed. If there is no previous state all tracks should be assumed as +*not* muted. If a track is missing from a stream that track should be considered +muted. + +### Example + +```JSON +{ + "type": "m.call.sdp_stream_metadata_changed", + "room_id": "!roomId", + "content": { + "call_id": "1414213562373095", + "party_id": "1732050807568877", + "sdp_stream_metadata": { + "2311546231" { + "purpose": "m.usermedia", + "audio_muted:": false, + "video_muted": true, + } + }, + "version": "1", + }, +} +``` + +## Potential issues + +When the user mutes their camera, the client will keep sending black pixels +which will waste bandwidth. + +## Security considerations + +None that I can think of. + +## Unstable prefix + +|Release |Development | +|-------------|------------------------------------------------------| +|`m.call.mute`|`org.matrix.msc0000.call.sdp_stream_metadata_changed` | From a3d1abc403e89e6cb6ebdbcf296f0d1ead86b743 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Thu, 22 Jul 2021 14:44:43 +0200 Subject: [PATCH 02/22] Update MSC number MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/{0000-muting.md => 3291-muting.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{0000-muting.md => 3291-muting.md} (98%) diff --git a/proposals/0000-muting.md b/proposals/3291-muting.md similarity index 98% rename from proposals/0000-muting.md rename to proposals/3291-muting.md index 741293dc460..c6c9823c1f2 100644 --- a/proposals/0000-muting.md +++ b/proposals/3291-muting.md @@ -1,4 +1,4 @@ -# MSC0000: Muting in VoIP calls +# MSC3291: Muting in VoIP calls During VoIP calls, it is common for a user to mute their microphone/camera. Ideally, the other side should be able to see that the opponent's camera is From 57a6330f994f4244de871664296e6c9e81c83804 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Thu, 22 Jul 2021 14:46:59 +0200 Subject: [PATCH 03/22] Missing : MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index c6c9823c1f2..dd49991b718 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -49,7 +49,7 @@ muted. "call_id": "1414213562373095", "party_id": "1732050807568877", "sdp_stream_metadata": { - "2311546231" { + "2311546231": { "purpose": "m.usermedia", "audio_muted:": false, "video_muted": true, From 7658c4a7b13e4da92af8523a675236013b6504fa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Thu, 22 Jul 2021 14:57:42 +0200 Subject: [PATCH 04/22] Fix MSC number in prefix MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index dd49991b718..b27b553b359 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -73,4 +73,4 @@ None that I can think of. |Release |Development | |-------------|------------------------------------------------------| -|`m.call.mute`|`org.matrix.msc0000.call.sdp_stream_metadata_changed` | +|`m.call.mute`|`org.matrix.msc3291.call.sdp_stream_metadata_changed` | From 2130df95a729e36ce58e45a5f1c4dfea43b740de Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Fri, 23 Jul 2021 07:55:56 +0200 Subject: [PATCH 05/22] Fix unstable prefix table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index b27b553b359..0a1dc1af59f 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -71,6 +71,6 @@ None that I can think of. ## Unstable prefix -|Release |Development | -|-------------|------------------------------------------------------| -|`m.call.mute`|`org.matrix.msc3291.call.sdp_stream_metadata_changed` | +|Release |Development | +|------------------------------------|------------------------------------------------------| +|`m.call.sdp_stream_metadata_changed`|`org.matrix.msc3291.call.sdp_stream_metadata_changed` | From 3a9701d1d8a0efd747e12a8fb98fb349ccf6a742 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Fri, 23 Jul 2021 08:34:00 +0200 Subject: [PATCH 06/22] Reword pottential issues MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 0a1dc1af59f..01d60f02e4a 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -62,8 +62,9 @@ muted. ## Potential issues -When the user mutes their camera, the client will keep sending black pixels -which will waste bandwidth. +When the user mutes their camera, the client will meaningless which will waste +bandwidth. + ## Security considerations From d809c3b62e00f216e9fbca486f2a17a112769389 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Fri, 23 Jul 2021 08:50:40 +0200 Subject: [PATCH 07/22] Add an Alternatives section MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 52 ++++++++++++++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 10 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 01d60f02e4a..e55d5d4b0d4 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -8,19 +8,17 @@ to be quick. Using pure WebRTC there are two ways to do muting and both have their issues: -+ Disabling the corresponding track - there will be no way to know the track is - muted from the opponent's perspective. On the other hand, this is almost - instantaneous. -+ Setting the corresponding track as `recvonly`/`inactive` - this is problematic - since it leads to re-negotiation which takes time. This could also conflict - with holding (as defined in - [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). ++ Disabling the corresponding track ++ Setting the corresponding track as `recvonly`/`inactive` + +The Alternatives section describes the issues with using these alone. ## Proposal -This MSC proposes a solution to this by extending the `SDPStreamMetadata` (see -[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) by the following -fields: +This MSC proposes that clients disable muted tracks (as is almost instantaneous) +and extends the `SDPStreamMetadata` object (see +[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) to allow +indicating the mute state to the other side using the following fields: + `audio_muted` - a boolean indicating the current audio mute state + `video_muted` - a boolean indicating the current video mute state @@ -65,6 +63,40 @@ muted. When the user mutes their camera, the client will meaningless which will waste bandwidth. +## Alternatives + +### Only disabling the corresponding track + +This is the solution that some clients (e.g. Element) use at the moment. While +this is almost instantaneous, it doesn't allow the other side to know the +opponent's mute state. This leads to the opponent showing a black screen for a +muted video track and not doing anything for a muted audio track which is bad +for UX. + +### Setting the corresponding track as `recvonly`/`inactive` + +While this would be beneficial for low bandwidth connections, it takes time. The +delay might be acceptable for video but isn't for audio (with which your would +assume an instantaneous mute state change). This is also problematic since there +could be a confusion with holding (as defined in +[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). + +### Using a separate event for muting + +While this might feel clearer initially, it doesn't have much real benefit. The +mute state is in fact a meta information about the stream and using +`sdp_stream_metadata` is also more flexible for cases where the user joins a +call already muted. It is also more flexible in general and would be useful if +we ever decided to do what is described in the next section. + +### A combination of disabling tracks, `sdp_stream_metadata` and SDP + +An option would be using the current method in combination with setting the +corresponding track as `recvonly`/`inactive`. Along with this clients would need +to set the mute state in `sdp_stream_metadata` to avoid conflicts with holding. +While this solution might be the most flexible solution as it would allow +clients to choose between bandwidth and a mute state change delay for each +track, it would be harder to implement and feels generally disjointed. ## Security considerations From 67168b35b3ed89aae669399a92396db16cd35e4d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 26 Jul 2021 15:25:14 +0200 Subject: [PATCH 08/22] Remove trailing comma Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index e55d5d4b0d4..f1d1a2eace7 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -54,7 +54,7 @@ muted. } }, "version": "1", - }, + } } ``` From ec24d9593f3b6292721928ed82d7ce9ab87aab68 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 26 Jul 2021 15:25:57 +0200 Subject: [PATCH 09/22] Fix a typo Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index f1d1a2eace7..c64f3bd3d61 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -76,7 +76,7 @@ for UX. ### Setting the corresponding track as `recvonly`/`inactive` While this would be beneficial for low bandwidth connections, it takes time. The -delay might be acceptable for video but isn't for audio (with which your would +delay might be acceptable for video but isn't for audio (with which you would assume an instantaneous mute state change). This is also problematic since there could be a confusion with holding (as defined in [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). From 0c019088dc11d34b723db18a7568ddfc8efc892a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 26 Jul 2021 15:29:00 +0200 Subject: [PATCH 10/22] Fix missing word MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index c64f3bd3d61..36633fbbacf 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -60,8 +60,8 @@ muted. ## Potential issues -When the user mutes their camera, the client will meaningless which will waste -bandwidth. +When the user mutes their camera, the client will keep sending meaningless data +which will waste bandwidth. ## Alternatives From cc8b73ae846f590d4a3f09ed1586397e9ea39d0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 26 Jul 2021 15:30:30 +0200 Subject: [PATCH 11/22] Link to MSC for holding MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 36633fbbacf..30c5155dc86 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -93,7 +93,8 @@ we ever decided to do what is described in the next section. An option would be using the current method in combination with setting the corresponding track as `recvonly`/`inactive`. Along with this clients would need -to set the mute state in `sdp_stream_metadata` to avoid conflicts with holding. +to set the mute state in `sdp_stream_metadata` to avoid conflicts with holding +(as defined in [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746)). While this solution might be the most flexible solution as it would allow clients to choose between bandwidth and a mute state change delay for each track, it would be harder to implement and feels generally disjointed. From f1ac61f94264af05b70dc5fb71de6238e063615f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Tue, 27 Jul 2021 13:41:43 +0200 Subject: [PATCH 12/22] Update unstable prefixes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 30c5155dc86..0bb9fbb6045 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -105,6 +105,10 @@ None that I can think of. ## Unstable prefix -|Release |Development | -|------------------------------------|------------------------------------------------------| -|`m.call.sdp_stream_metadata_changed`|`org.matrix.msc3291.call.sdp_stream_metadata_changed` | +|Release |Development | +|------------------------------------|---------------------------------------------| +|`m.call.sdp_stream_metadata_changed`|`org.matrix.call.sdp_stream_metadata_changed`| +|`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` | + +We use an unstable prefix for `sdp_stream_metadata` to match +[MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077). From 29e3c6d1a263ee628f989dac5a65289609670e65 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Tue, 10 Jan 2023 21:47:33 +0100 Subject: [PATCH 13/22] Simplify things MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 0bb9fbb6045..48a7942e522 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -32,10 +32,7 @@ thing as `sdp_stream_metadata` in `m.call.negotiate`, `m.call.invite` and changed but no negotiation is required (e.g. the user mutes their camera/microphone). -If a client sends an event with some of the fields missing the previous state -should be assumed. If there is no previous state all tracks should be assumed as -*not* muted. If a track is missing from a stream that track should be considered -muted. +All tracks should be assumed unmuted unless specified otherwise. ### Example From 5372b7f709d527d9c9a57ecbd9c401cbccde8783 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Thu, 11 May 2023 09:21:34 +0200 Subject: [PATCH 14/22] Be explicit about deps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 48a7942e522..9073e895caa 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -100,6 +100,10 @@ track, it would be harder to implement and feels generally disjointed. None that I can think of. +## Dependencies + ++ [MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077) + ## Unstable prefix |Release |Development | From 00da3d347e006c27706b92acef14dda3dff7afc6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Thu, 11 May 2023 09:52:48 +0200 Subject: [PATCH 15/22] Be clearer about how things work MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 9073e895caa..449bfd2a99f 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -15,8 +15,7 @@ The Alternatives section describes the issues with using these alone. ## Proposal -This MSC proposes that clients disable muted tracks (as is almost instantaneous) -and extends the `SDPStreamMetadata` object (see +This MSC proposes extending the `SDPStreamMetadata` object (see [MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) to allow indicating the mute state to the other side using the following fields: @@ -27,13 +26,20 @@ This MSC also adds a new call event `m.call.sdp_stream_metadata_changed`, which has the common VoIP fields as specified in [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) (`version`, `call_id`, `party_id`) and a `sdp_stream_metadata` object which is the same -thing as `sdp_stream_metadata` in `m.call.negotiate`, `m.call.invite` and +thing as `sdp_stream_metadata` in `m.call.negotiate`, `m.call.invite` and `m.call.answer`. The client sends this event the when `sdp_stream_metadata` has changed but no negotiation is required (e.g. the user mutes their camera/microphone). All tracks should be assumed unmuted unless specified otherwise. +It is suggested `video_muted` (but not required) should determine if a user sees +video but `audio_muted` only determines if a user is shown a mute indication +rather than actually muting the audio track on the receiving side. This is +suggested because if the track is muted using the WebRTC `disabled` property, +there can be delay between the audio track getting unmuted and the client +receiving the `m.call.sdp_stream_metadata_changed` event. + ### Example ```JSON @@ -41,20 +47,25 @@ All tracks should be assumed unmuted unless specified otherwise. "type": "m.call.sdp_stream_metadata_changed", "room_id": "!roomId", "content": { + "version": "1", "call_id": "1414213562373095", "party_id": "1732050807568877", "sdp_stream_metadata": { "2311546231": { "purpose": "m.usermedia", - "audio_muted:": false, - "video_muted": true, + "audio_muted:": true, + "video_muted": true } - }, - "version": "1", + } } } ``` +This event indicates that both audio and video are muted. It is suggested the +video track of stream `2311546231` should be hidden in the UI (probably replaced +by an avatar). It also suggests the UI should show an indication that the audio +track is muted but the client should not mute the audio on the receiving side. + ## Potential issues When the user mutes their camera, the client will keep sending meaningless data From fb8f957c6b367d79a8ebdd96f93de37cd7f50b09 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sun, 11 Jun 2023 16:10:00 +0200 Subject: [PATCH 16/22] Update proposals/3291-muting.md Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 449bfd2a99f..585d9062eca 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -27,7 +27,7 @@ has the common VoIP fields as specified in [MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) (`version`, `call_id`, `party_id`) and a `sdp_stream_metadata` object which is the same thing as `sdp_stream_metadata` in `m.call.negotiate`, `m.call.invite` and -`m.call.answer`. The client sends this event the when `sdp_stream_metadata` has +`m.call.answer`. The client sends this event when the `sdp_stream_metadata` has changed but no negotiation is required (e.g. the user mutes their camera/microphone). From c97f6504d06443893327e52cb8875bc5ca22852c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sun, 11 Jun 2023 16:12:55 +0200 Subject: [PATCH 17/22] `disabled` -> `enabled` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 585d9062eca..170eab22e9d 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -36,7 +36,7 @@ All tracks should be assumed unmuted unless specified otherwise. It is suggested `video_muted` (but not required) should determine if a user sees video but `audio_muted` only determines if a user is shown a mute indication rather than actually muting the audio track on the receiving side. This is -suggested because if the track is muted using the WebRTC `disabled` property, +suggested because if the track is muted using the WebRTC `enabled` property, there can be delay between the audio track getting unmuted and the client receiving the `m.call.sdp_stream_metadata_changed` event. From 1aa2cd414389d25df5ffe144cfcdbf9a1525cb47 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sun, 11 Jun 2023 16:17:08 +0200 Subject: [PATCH 18/22] Fix client mention MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 170eab22e9d..958b24a847d 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -75,8 +75,8 @@ which will waste bandwidth. ### Only disabling the corresponding track -This is the solution that some clients (e.g. Element) use at the moment. While -this is almost instantaneous, it doesn't allow the other side to know the +This is the solution that some clients (e.g. Element Android) use at the moment. +While this is almost instantaneous, it doesn't allow the other side to know the opponent's mute state. This leads to the opponent showing a black screen for a muted video track and not doing anything for a muted audio track which is bad for UX. From dec84a89f6314c5f7015466adeec81c6d0b95562 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sun, 11 Jun 2023 16:25:35 +0200 Subject: [PATCH 19/22] Improve explaination MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3291-muting.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 958b24a847d..54f2f8457d6 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -33,12 +33,16 @@ camera/microphone). All tracks should be assumed unmuted unless specified otherwise. -It is suggested `video_muted` (but not required) should determine if a user sees -video but `audio_muted` only determines if a user is shown a mute indication -rather than actually muting the audio track on the receiving side. This is -suggested because if the track is muted using the WebRTC `enabled` property, -there can be delay between the audio track getting unmuted and the client -receiving the `m.call.sdp_stream_metadata_changed` event. +Clients are recommended to not mute the audio of WebRTC tracks locally when a +stream has the `audio_muted` field set to `false`. This is because when the +other user unmutes themselves, there may be a slight delay between their client +sending audio and the `m.call.sdp_stream_metadata_changed` event arriving. If +`enabled` is set to `false`, then any audio sent in between those two events +will not be heard. The other user will still stop transmitting audio once they +mute on their side, so no audio is sent without the user's knowledge. + +The same suggestion does not apply to `video_muted` - there clients _should_ +mute video locally, so that the receiving side doesn't see black video. ### Example From 56d35419c553070eb13ff59f2bd03a8c49f4cd37 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 3 Jul 2023 16:35:00 +0200 Subject: [PATCH 20/22] Be more precise Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 54f2f8457d6..9e3b6fa9c78 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -72,7 +72,7 @@ track is muted but the client should not mute the audio on the receiving side. ## Potential issues -When the user mutes their camera, the client will keep sending meaningless data +When the user mutes their camera, some browsers may keep sending meaningless data which will waste bandwidth. ## Alternatives From 8905378cc03a03b64085608dee599df0fe4ffa4a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 3 Jul 2023 16:39:59 +0200 Subject: [PATCH 21/22] Fix negation Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 9e3b6fa9c78..3b6e54c162e 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -34,7 +34,7 @@ camera/microphone). All tracks should be assumed unmuted unless specified otherwise. Clients are recommended to not mute the audio of WebRTC tracks locally when a -stream has the `audio_muted` field set to `false`. This is because when the +incoming stream has the `audio_muted` field set to `true`. This is because when the other user unmutes themselves, there may be a slight delay between their client sending audio and the `m.call.sdp_stream_metadata_changed` event arriving. If `enabled` is set to `false`, then any audio sent in between those two events From 4da6f86bb94280ec144c96dfcf0e9ef35cc20d3d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Tue, 4 Jul 2023 16:03:03 +0200 Subject: [PATCH 22/22] Don't refer to something that doesn't exist Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3291-muting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3291-muting.md b/proposals/3291-muting.md index 3b6e54c162e..6fd114d5466 100644 --- a/proposals/3291-muting.md +++ b/proposals/3291-muting.md @@ -15,7 +15,7 @@ The Alternatives section describes the issues with using these alone. ## Proposal -This MSC proposes extending the `SDPStreamMetadata` object (see +This MSC proposes extending the `sdp_stream_metadata` object (see [MSC3077](https://github.com/matrix-org/matrix-doc/pull/3077)) to allow indicating the mute state to the other side using the following fields: