Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let’s Encrypt certificates for TURNS are rejected #1533

Open
rajil opened this issue Jun 23, 2020 · 24 comments
Open

Let’s Encrypt certificates for TURNS are rejected #1533

rajil opened this issue Jun 23, 2020 · 24 comments
Labels

Comments

@rajil
Copy link

rajil commented Jun 23, 2020

I tried dev 0.23 build from https://buildkite.com/matrix-dot-org/riotx-android/builds/2964

It has the same TURNS error reported previously, here and here. The coturn is checked it with is 4.5.1.2.

The coturn config looks like so,


listening-port=3478
tls-listening-port=5349
alt-listening-port=3479
alt-tls-listening-port=5350
listening-ip=192.168.1.10
external-ip=xx.xx.xx.xx
min-port=49152
max-port=65535
verbose
fingerprint
use-auth-secret
static-auth-secret=xx
realm=turn.mydomain.com
userdb=/var/lib/turnserver/turndb
stale-nonce=600
cert=/etc/letsencrypt/live/mydomain.com/fullchain.pem
pkey=/etc/letsencrypt/live/mydomain.com/privkey.pem
cipher-list="ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AES:RSA+3DES:!ADH:!AECDH:!MD5"
no-multicast-peers
mobility
cli-password=xx
no-tlsv1
no-tlsv1_1
@532910
Copy link
Contributor

532910 commented Aug 1, 2020

I believe this is a security issue!

@hex-m
Copy link

hex-m commented Dec 2, 2020

Our friends over a Jitsi seem to have a similar problem. (issue, discussion)

@a1ad
Copy link

a1ad commented Jan 20, 2021

yeah, same here: unknown CA

@dhaminitinAltran
Copy link

facing the same issue has anyone got a solution yet?

@buzzdx
Copy link

buzzdx commented Mar 16, 2021

i'm having the same problem. synapse and coturn are both set up to use turns. when i try to make a call from element android app to another android app on a different network the turnserver logs look like this:

turnserver[14012]: 860: IPv4. tcp or tls connected to: a.b.c.d:58145 turnserver[14012]: 860: session 001000000000000028: TLS/TCP socket disconnected: a.b.c.d.5:58145 turnserver[14012]: 860: session 001000000000000028: usage: realm=<my_realms>, username=<>, rp=0, rb=0, sp=0, sb=0 turnserver[14012]: 860: session 001000000000000028: closed (2nd stage), user <> realm <my_realms> origin <>, local h.i.j.k:5349, remote a.b.c.d, reason: TLS/TCP socket buffer operation error (callback)

when i switch the turn_uris to begin with turn: instead of turns: it works flawlessly from any device to any device (phone to app, app to phone, all over different networks).

as soon as tls is used with the android app it does not work anymore. this seems to be a real problem as there are more than one discussions about it on github and other sites.

coturn has access to the certs, which are let's encrypt certs, as many others described.

this may be off topic, but there's also a problem with the android app where you cannot verify anything if your first login after account creation is made with the android app. had to use desktop/web app to do the first login, after that all works as expected.

would be really great if somebody could solve this, or point to a solution or potential upcoming fix. thanks.

edit: is it possible that this is the same problem we are seeing here?
https://community.jitsi.org/t/android-app-rejecting-lets-encrypt-certificates-for-turns/29468

quote of the final post on that page:

The problem – as I understand it – is on the client side:

If the Android app connects to the webserver (nginx) via https it verifies the server certificate using the list of trusted root CAs
that is built into the Android OS. Let’s Encrypt certificates are verified successfully.
However, if the app cannot establish a connection to the videobridge directly and needs to fallback to the turnserver a “different
part” of the Jitsi Meet Android app is used to establish the turns:// connection: This connection is apparently handled by a webrtc
library (which seems to be included in the Jitsi App as a native library via react-webrtc). And the webrtc library does not use the
list of trusted CAs which is provided by the Android OS. Instead, another list of trusted CAs is compiled into this library and only
the CAs contained in this hardcoded list are used to verify the certificate that the app receives from the turn server.
Unfortunately, this hardcoded list of CAs does not include the trust anchor that’s needed to trust a certificate issued by Let’s
Encrypt.

@buzzdx
Copy link

buzzdx commented Mar 24, 2021

It seems that what was mentioned in my previous post is correct. I switched to ZeroSSL certificates as replacements for Let's Encrypt certificates, and without any other change, Element Android was immediately able to connect voice and video calls using turns across different networks. So the problem is indeed that for some reason, maybe that webrtc library having an incomplete list of trusted CAs as mentioned above, Element Android does not accept Let's Encrypt certificates. The Jitsi devs have an open ticket where they stated that for now they will add Let's Encrypt to the webrtc-lib's trusted ca list. Maybe someone could do the same for Element Android?

@asimons04
Copy link

Can confirm using ZeroSSL certs with coturn is a workaround to avoid having to fallback to no SSL. Just make sure you combine your cert + ca bundle into a fullchain for coturn, though.

@buzzdx
Copy link

buzzdx commented May 14, 2021

thanks for confirming the workaround works asimons04. now is there anything we can do to make the devs aware of the problem and possible solution? i tried email contact but they couldn't help, from what i understand they do not give support for self hosting.
they told me: Please also submit debug logs from Element Android after you have created your issue and link them together.
I'm not really sure how to do that, pretty new to github. if anybody here knows , maybe you could submit the needed data?
thanks

@tleydxdy
Copy link

fwiw the issue eventually comes down to this https://bugs.chromium.org/p/webrtc/issues/detail?id=11710 which is part of the webrtc dependency.

@richvdh richvdh changed the title TURNS (TURN with TLS/DTLS) socket buffer operation error Let’s Encrypt certificates for TURNS are rejected Dec 14, 2021
@AndrewFerr
Copy link
Member

Note that this issue affects matrix.org's TURN server too, which makes sense given that it uses a Let's Encrypt certificate.

From my testing, this issue prevents calls to connect if all of the following conditions are met:

  • at least one client is Element Android (I haven't tested with the iOS client)
  • call traffic is required to go through the TURN server (which I tested by having one client behind a corporate VPN that apparently blocks direct p2p traffic)
  • only turns connections are available (which I tested by blocking traffic on ports 3478-3479)

Under these conditions, not only did calls between users on my own homeserver fail (which uses a Let's Encrypt cert for its TURN server), but calls between matrix.org accounts failed to connect, as well.

DMRobertson pushed a commit to matrix-org/synapse that referenced this issue Jan 17, 2022
* Warn against using Let's Encrypt certs for encrypted TURN

This helps to avoid client-side issues:
* element-hq/element-android#1533
* element-hq/element-ios#2712

Signed-off-by: Andrew Ferrazzutti <[email protected]>
@progserega
Copy link

Switch to ZeroSSL is not allow for all. Some countries is banned:

1

@xuhdev
Copy link

xuhdev commented May 4, 2022

fwiw the issue eventually comes down to this https://bugs.chromium.org/p/webrtc/issues/detail?id=11710 which is part of the webrtc dependency.

Nice catch, and a comment is suggesting that users of the library should not use their truststore at all: https://bugs.chromium.org/p/webrtc/issues/detail?id=11710#c11

@mrPjer
Copy link

mrPjer commented Jan 22, 2023

An alternative to ZeroSSL is also the Namecheap PositiveSSL certificate - https://www.namecheap.com/security/ssl-certificates/comodo/positivessl/.

Tried it out and it works.

@karolyi
Copy link

karolyi commented Jan 23, 2023

I just had a WebRTC call via matrix and by looking at the remote candidates (my buddy was using element android) it seems that letsencrypt certificates work. My own coturn server which offers TURNS, started to show up as a remote candidate with both IPv4 and IPv6 addresses.

The call was built in the end with my Librewolf using my TURN server, that talked to directly his IPv6 address. Call quality was good.

Can someone confirm?

@bessw
Copy link

bessw commented Jan 23, 2023

I can confirm. Just tried it and first one mobile could start calls, the other not, but after signing out (make sure you have a key backup) and logging in again it worked on that mobile, too.

We tested with or without VPN and mobile data or WIFI:

  • Element Android - Element Android
  • Element Android - Element Desktop

@karolyi
Copy link

karolyi commented Jan 23, 2023

Sounds good.

Now the only question remains, what's the underlying cause of it suddenly working? Is something in LetsEncrypt's cross-signers changed or is this a security hole?

@theAeon
Copy link

theAeon commented Jan 24, 2023

Could just mean that google finally got around to fixing the WebRTC inbuilt certs.

@maximushugus
Copy link

I can confirm the problem is still here for me. I self host synapse and coturn, and I use Letsencrypt certificate.
Using TURN (not encrypted) working without any problem, I can make call behind nat from Android Element app, from Firefox or Google Chrome.
But as soon as I use TURNS (encrypted) with letsencrypt certificate, I'm not able to make call behind NAT

I do not want to use another certificate provider.
Is there any solution ?
Why Element doesn't trust Letsencrypt ?

Thanks

@Josue-T
Copy link

Josue-T commented Mar 9, 2024

Hello,

Tested between element web and element android with this following synapse config and it looks like working correctly

turn_uris: [ "turns:matrix.domain.tld:<port>", "turns:matrix.domain.tld:<alt_port>" ]

Checked with wireshark and I saw a DTLS negotiation so I think it's working with let's encrypt.

So I think the issue is fixed.

@532910
Copy link
Contributor

532910 commented Mar 10, 2024

@Josue-T: im.vector.app or io.element.android.x and which version?

@532910
Copy link
Contributor

532910 commented Mar 10, 2024

can confirm im.vector.app works with turns tcp

@Josue-T
Copy link

Josue-T commented Mar 10, 2024

@Josue-T: im.vector.app or io.element.android.x and which version?

With im.vector.app from fdroid on /e/ OS.

@maximushugus
Copy link

maximushugus commented Mar 17, 2024

I still have the problem with im.vector.app installed directly from this github repository (v1.6.12) (on a degoogle lineageOS phone) : if one (or both) of the 2 devices involved in a call is behind NAT, and using TURNS with a Let's Encrypt certificate, the call doesn't work. If I switch to TURN (not encrypted), with the same condition, it does work.
My TURN server is working correctly even with TURNS with a certificate from Let's Encrypt, because i also use it for other programs without any problem.

For people who managed to make it work, are you sure you are using TURNS with Let's Encrypt, behind a NAT ?

@ketan-vijayvargiya
Copy link

I too think this issue is now resolved.

This is what I have in my NixOS config:

    coturn = {
      enable = true;
      use-auth-secret = true;
      static-auth-secret-file = config.age.secrets.coturnStaticAuthSecret.path;

      realm = coturnRealm;

      no-cli = true;
      secure-stun = true;
      cert = "${config.security.acme.certs."${coturnRealm}".directory}/fullchain.pem";
      pkey = "${config.security.acme.certs."${coturnRealm}".directory}/key.pem";
    };

    matrix-synapse = {
      enable = true;

      extraConfigFiles = [ config.age.secrets.coturnStaticAuthSecretForSynapse.path ];

      settings = {
        // Bunch of other stuff.

        turn_uris = [ "turns:${coturnRealm}?transport=udp" "turns:${coturnRealm}?transport=tcp" ];
        turn_allow_guests = false;
      };
    };

I and spouse use Element for Android and iOS respectively, and we are able to call each other.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests