Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

webrtc: add advanced audio settings #2434

Merged
merged 2 commits into from
Nov 8, 2022

Conversation

MrAnno
Copy link
Contributor

@MrAnno MrAnno commented Jun 5, 2022

autoGainControl, echoCancellation, and noiseSuppression are audio
processing options that are usually enabled by default on WebRTC input
tracks.

This PR adds the possibility to enable/disable them, as they can be
undesirable in some cases (audiophile use cases). For example, one might
want to stream electronic dance music, which is basically noise, so it
should not be suppressed in that specific case.

Note that these are not exact settings, they are set as "ideal" in order
not to break anything on devices where those constraints are not
implemented.

Related to matrix-org/matrix-react-sdk#8759

Type: task


Here's what your changelog entry will look like:

✨ Features

  • webrtc: add advanced audio settings (#2434). Contributed by @MrAnno.

@MrAnno MrAnno marked this pull request as ready for review June 5, 2022 21:14
@MrAnno MrAnno requested a review from a team as a code owner June 5, 2022 21:14
@github-actions github-actions bot added the T-Task Tasks for the team like planning label Jun 5, 2022
@MrAnno MrAnno force-pushed the audio-processing-settings branch from fa40eea to 28f353d Compare June 5, 2022 21:17
@SimonBrandner SimonBrandner added T-Enhancement and removed T-Task Tasks for the team like planning labels Jun 6, 2022
Copy link
Contributor

@SimonBrandner SimonBrandner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks sane - thank you!

src/webrtc/mediaHandler.ts Outdated Show resolved Hide resolved
@MrAnno MrAnno force-pushed the audio-processing-settings branch from 28f353d to 39e2a88 Compare June 6, 2022 15:52
src/webrtc/mediaHandler.ts Show resolved Hide resolved
@vi
Copy link

vi commented Sep 16, 2022

Are this one and element-hq/element-web#6278 waiting for matrix-org/matrix-react-sdk#8759?

@robintown
Copy link
Member

Yes, they are

@SimonBrandner SimonBrandner enabled auto-merge (squash) November 7, 2022 15:22
@SimonBrandner
Copy link
Contributor

Arghhh, it looks like this is going to need some tests written - is there any chance you could do that?

@MrAnno
Copy link
Contributor Author

MrAnno commented Nov 7, 2022

Sure, I'll try to add some tests soon. :)

auto-merge was automatically disabled November 8, 2022 15:00

Head branch was pushed to by a user without write access

@MrAnno MrAnno requested a review from a team as a code owner November 8, 2022 15:00
Copy link
Contributor

@SimonBrandner SimonBrandner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome - thanks!

@SimonBrandner SimonBrandner enabled auto-merge (squash) November 8, 2022 15:08
@SimonBrandner
Copy link
Contributor

If you could please merge develop, it would be awesome! Sorry for the merge conflicts!

@MrAnno
Copy link
Contributor Author

MrAnno commented Nov 8, 2022

@SimonBrandner Not my prettiest unit test, but I wrote a few.

I just realized that I was working on top of an old branch. Since then, MediaHandler got a lot of nice unit tests, so I should not have written them from scratch.

Let me reimplement those tests on top of the current develop branch.

autoGainControl, echoCancellation, and noiseSuppression are audio
processing options that are usually enabled by default on WebRTC input
tracks.

This commit adds the possibility to enable/disable them, as they can be
undesirable in some cases (audiophile use cases). For example, one might
want to stream electronic dance music, which is basically noise, so it
should not be suppressed in that specific case.

Note that these are not exact settings, they are set as "ideal" in order
not to break anything on devices where those constraints are not
implemented.

Signed-off-by: László Várady <[email protected]>
auto-merge was automatically disabled November 8, 2022 15:51

Head branch was pushed to by a user without write access

@MrAnno
Copy link
Contributor Author

MrAnno commented Nov 8, 2022

Done.

@SimonBrandner SimonBrandner enabled auto-merge (squash) November 8, 2022 15:56
@SimonBrandner SimonBrandner merged commit a584324 into matrix-org:develop Nov 8, 2022
su-ex added a commit to SchildiChat/matrix-js-sdk that referenced this pull request Nov 25, 2022
* Make calls go back to 'connecting' state when media lost ([\matrix-org#2880](matrix-org#2880)).
* Add ability to send unthreaded receipt ([\matrix-org#2878](matrix-org#2878)).
* Add way to abort search requests ([\matrix-org#2877](matrix-org#2877)).
* sliding sync: add custom room subscriptions support ([\matrix-org#2834](matrix-org#2834)).
* webrtc: add advanced audio settings ([\matrix-org#2434](matrix-org#2434)). Contributed by @MrAnno.
* Add support for group calls using MSC3401 ([\matrix-org#2553](matrix-org#2553)).
* Make the js-sdk conform to tsc --strict ([\matrix-org#2835](matrix-org#2835)). Fixes matrix-org#2112 matrix-org#2116 and matrix-org#2124.
* Let leave requests outlive the window ([\matrix-org#2815](matrix-org#2815)). Fixes element-hq/element-call#639.
* Add event and message capabilities to RoomWidgetClient ([\matrix-org#2797](matrix-org#2797)).
* Misc fixes for group call widgets ([\matrix-org#2657](matrix-org#2657)).
* Support nested Matrix clients via the widget API ([\matrix-org#2473](matrix-org#2473)).
* Set max average bitrate on PTT calls ([\matrix-org#2499](matrix-org#2499)). Fixes element-hq/element-call#440.
* Add config option for e2e group call signalling ([\matrix-org#2492](matrix-org#2492)).
* Enable DTX on audio tracks in calls ([\matrix-org#2482](matrix-org#2482)).
* Don't ignore call member events with a distant future expiration date ([\matrix-org#2466](matrix-org#2466)).
* Expire call member state events after 1 hour ([\matrix-org#2446](matrix-org#2446)).
* Emit unknown device errors for group call participants without e2e ([\matrix-org#2447](matrix-org#2447)).
* Mute disconnected peers in PTT mode ([\matrix-org#2421](matrix-org#2421)).
* Add support for sending encrypted to-device events with OLM ([\matrix-org#2322](matrix-org#2322)). Contributed by @robertlong.
* Support for PTT group call mode ([\matrix-org#2338](matrix-org#2338)).
* Fix registration add phone number not working ([\matrix-org#2876](matrix-org#2876)). Contributed by @bagvand.
* Use an underride rule for Element Call notifications ([\matrix-org#2873](matrix-org#2873)). Fixes element-hq/element-web#23691.
* Fixes unwanted highlight notifications with encrypted threads ([\matrix-org#2862](matrix-org#2862)).
* Extra insurance that we don't mix events in the wrong timelines - v2 ([\matrix-org#2856](matrix-org#2856)). Contributed by @MadLittleMods.
* Hide pending events in thread timelines ([\matrix-org#2843](matrix-org#2843)). Fixes element-hq/element-web#23684.
* Fix pagination token tracking for mixed room timelines ([\matrix-org#2855](matrix-org#2855)). Fixes element-hq/element-web#23695.
* Extra insurance that we don't mix events in the wrong timelines ([\matrix-org#2848](matrix-org#2848)). Contributed by @MadLittleMods.
* Do not freeze state in `initialiseState()` ([\matrix-org#2846](matrix-org#2846)).
* Don't remove our own member for a split second when entering a call ([\matrix-org#2844](matrix-org#2844)).
* Resolve races between `initLocalCallFeed` and `leave` ([\matrix-org#2826](matrix-org#2826)).
* Add throwOnFail to groupCall.setScreensharingEnabled ([\matrix-org#2787](matrix-org#2787)).
* Fix connectivity regressions ([\matrix-org#2780](matrix-org#2780)).
* Fix screenshare failing after several attempts ([\matrix-org#2771](matrix-org#2771)). Fixes element-hq/element-call#625.
* Don't block muting/unmuting on network requests ([\matrix-org#2754](matrix-org#2754)). Fixes element-hq/element-call#592.
* Fix ICE restarts ([\matrix-org#2702](matrix-org#2702)).
* Target widget actions at a specific room ([\matrix-org#2670](matrix-org#2670)).
* Add tests for ice candidate sending ([\matrix-org#2674](matrix-org#2674)).
* Prevent exception when muting ([\matrix-org#2667](matrix-org#2667)). Fixes element-hq/element-call#578.
* Fix race in creating calls ([\matrix-org#2662](matrix-org#2662)).
* Add client.waitUntilRoomReadyForGroupCalls() ([\matrix-org#2641](matrix-org#2641)).
* Wait for client to start syncing before making group calls ([\matrix-org#2632](matrix-org#2632)). Fixes matrix-org#2589.
* Add GroupCallEventHandlerEvent.Room ([\matrix-org#2631](matrix-org#2631)).
* Add missing events from reemitter to GroupCall ([\matrix-org#2527](matrix-org#2527)). Contributed by @toger5.
* Prevent double mute status changed events ([\matrix-org#2502](matrix-org#2502)).
* Don't mute the remote side immediately in PTT calls ([\matrix-org#2487](matrix-org#2487)). Fixes element-hq/element-call#425.
* Fix some MatrixCall leaks and use a shared AudioContext ([\matrix-org#2484](matrix-org#2484)). Fixes element-hq/element-call#412.
* Don't block muting on determining whether the device exists ([\matrix-org#2461](matrix-org#2461)).
* Only clone streams on Safari ([\matrix-org#2450](matrix-org#2450)). Fixes element-hq/element-call#267.
* Set PTT mode on call correctly ([\matrix-org#2445](matrix-org#2445)). Fixes element-hq/element-call#382.
* Wait for mute event to send in PTT mode ([\matrix-org#2401](matrix-org#2401)).
* Handle other members having no e2e keys ([\matrix-org#2383](matrix-org#2383)). Fixes element-hq/element-call#338.
* Fix races when muting/unmuting ([\matrix-org#2370](matrix-org#2370)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants