MQTT SSL possible?

[Atalonica]

I've been trying for a long time with a lot of things but I can't get it to work, I don't even know if it is possible.
I would like to connect to a MQTT broker via TLS/SSL. I'm using EthernetLarge, SSLClient and pubsubclient with a SAMD21 board. No compile problems, but when I try to connect this appears:

(SSLClient)(SSL_INFO)(connect): Base client connected!
(SSLClient)(SSL_INFO)(m_run_until): m_run changed state:
(SSLClient)(SSL_INFO)(m_run_until): State:
RECVREC
(SSLClient)(SSL_INFO)(m_run_until): Expected bytes count:
(SSLClient)(SSL_INFO)(m_run_until): 5
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 9791
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 9791
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 9791
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 9791
(SSLClient)(SSL_WARN)(m_run_until): Terminating because the ssl engine closed
(SSLClient)(SSL_ERROR)(m_start_ssl): Failed to initlalize the SSL layer
(SSLClient)(SSL_ERROR)(m_print_br_error): Chain could not be linked to a trust anchor.
[MQTT](KO!): -2
(SSLClient)(SSL_ERROR)(connect): Cannot have two connections at the same time! Please create another SSLClient instance.

And the last SSL_ERROR message keeps repeating. On my server side a new connection appears but nothing else (mosquitto).

Part of the Arduino code:
(...)
EthernetClient ethClient;
SSLClient ethClientSSL(ethClient, TAs, (size_t)TAs_NUM, A6, 1, SSLClient::SSL_INFO);
PubSubClient client(mqttServer, 8883, callback, ethClientSSL);
(...)
client.connect(clientId, usr, pass, willTopic, willQoS, willRetain, willMessage)

Any help will be very appreciated.

[prototypicalpro]

Hello! It looks like you provided incorrect trust anchors. Trust anchors provide a means for the client (SSLClient) to verify that the server is who we think they are, and must be generated based off of cryptographic information provided by the server you are connecting to. Without the correct trust anchor corresponding to the server you are attempting to connect to, BearSSL (and SSLClient by extension) cannot verify the server, and immediately closes the connection as a result. You can read more about what trust anchors are and why SSLClient needs them here.

I assume you're setting up your own Mosquito instance, which means you probably went through TLS setup instructions that look something like this. If this is the case, you can generate a trust anchor using the python script here on the CA certificate you generated to create a trust anchor for your server. To do this, you'll need to download a copy of this repository and follow the python setup instructions at the top of SSLClient/tools/pycert_bearssl/pycert_bearssl.py. Once python is setup correctly, you can use the command pycert_bearssl.py convert --no-search myca.pem to create a header file containing the correct trust anchor from the CA certificate you created for your Mosquito server.

If you are using someone else's Mosquito server, the process will be the same except you will need to find the CA certificate they are using. I would be surprised if this isn't available on the internet somewhere, so I would try that first. You may end up needing to email some people if not.

I hope that answers your question! Let me know if you continue to encounter issues.

[Atalonica]

Hello, thanks for the reply. Now I'm almost sure that the certificate / header file part is OK, but still I can't get it to work.
The serial output has changed a little though:
(SSLClient)(SSL_INFO)(connect): Base client connected!
(SSLClient)(SSL_INFO)(m_run_until): m_run changed state:
(SSLClient)(SSL_INFO)(m_run_until): State:
RECVREC
(SSLClient)(SSL_INFO)(m_run_until): Expected bytes count:
(SSLClient)(SSL_INFO)(m_run_until): 5
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_INFO)(m_update_engine): Memory:
(SSLClient)(SSL_INFO)(m_update_engine): 18351
(SSLClient)(SSL_ERROR)(m_update_engine): Error writing to m_client
(SSLClient)(SSL_ERROR)(m_update_engine): 1
(SSLClient)(SSL_WARN)(m_run_until): Terminating with write error:
(SSLClient)(SSL_WARN)(m_run_until): 4
(SSLClient)(SSL_ERROR)(m_start_ssl): Failed to initlalize the SSL layer
(SSLClient)(SSL_ERROR)(m_print_br_error): Unknown error code: 0
[MQTT](KO!): -2
(SSLClient)(SSL_ERROR)(connected): Not connected because write error is set
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL

For the Mosquitto part (yes it is my own instance, well the Home Assistant add-on), it reports:
New connection from X.X.X.X on port 8883.
1582716363: OpenSSL Error: error:1417C0C7:SSL routines:tls_process_client_certificate:peer did not return a certificate

Any hints of what can be causing those error? Thanks anyway.

[prototypicalpro]

You appear to be missing a client certificate. MQTT uses mTLS instead of regular TLS, which requires that the client present authentication information as well as the server. Since SSLClient did not present any authentication information, Mosquito cannot verify the connection, and closes it immediately. You can read more about mTLS and client certificates here.

Configuring SSLClient with a client certificate is tricky right now. You're welcome to try and figure it out yourself (relevant function is here), however if you're willing to wait a couple of weeks I can clean up the API and update you when it's ready? I've been putting off rewriting this code for awhile, and I may as well do it now.

[Atalonica]

Got it. I may read the documentation but I'm pretty sure I won't figure it.
It would be awesome to have a working example with MQTT connecting to Mosquitto using the PubSubClient library like those ESP8266 ones. Anyway hope you get the update. Thank you!

[prototypicalpro]

Alright, I've reworked the mTLS API so it should be much simpler to use. You'll need a client certificate and private key (I'm not sure how to do that with Mosquito, but it probably involves openssl), and the latest version of SSLClient from master branch. Once you have those, you can use the SSLClientParamters API to parse your certificate and key at the top of your sketch:

const char* my_cert = "-----BEGIN CERTIFICATE-----\r\n <insert certificate bytes> \r\n-----END CERTIFICATE-----";
const char* my_key = "-----BEGIN EC/RSA PRIVATE KEY-----\r\n <insert key bytes> \r\n-----BEGIN EC/RSA PRIVATE KEY-----";

SSLClientParamters mTLS = SSLClientParamters::fromPEM(my_cert, sizeof my_cert, my_key, sizeof my_key);

Given this, you can enable mTLS with SSLClient by calling the SSLClient::setMutualAuthParams before you connect:

SSLClient my_client(...)
...
void setup() {
	...
	my_client.setMutualAuthParams(mTLS);
	...
}

I've tested these changes myself, however I'm going to wait on formally releasing them until they are confirmed working with your setup.

Good luck, and let me know how it goes!

[Atalonica]

Hello nice work on implementing it so fast. Unfortunately I can't get it working.
I have no compilation issues (SSLClient.h should be included before certificates.h, my fault).

• I did use this to generate ca.crt, server.crt and server.key and configured them on Mosquitto (server side).

• I did use this to generate client.crt and client.key and placed its contents to both char pointers, exactly like you wrote above (with whitespaces after and before \r\n, see below):
  const char* my_cert = "-----BEGIN CERTIFICATE-----\r\n MIIDPDC...mSg== \r\n-----END CERTIFICATE-----"
const char* my_key * ""-----BEGIN RSA PRIVATE KEY-----\r\n MIIEpAI...rrA== \r\n-----END RSA PRIVATE KEY-----"
  I tried putting the X.duckdns.org domain both for the CommonName of the 'server' and 'CA' files and only for the 'server' files. But nothing.

• I assume the certificates.h file should be generated from ca.crt (or am I wrong?).

I get the same output as before.

Some lines of my .ino file:
SSLClientParameters mTLS = SSLClientParameters::fromPEM(my_cert, sizeof(my_cert), my_key, sizeof(my_key));
EthernetClient ethClient;
SSLClient ethClientSSL(ethClient, TAs, (size_t)TAs_NUM, A6, 1, SSLClient::SSL_INFO);
PubSubClient client(mqttServer, 12345, callback, ethClientSSL);
On setup(): ethClientSSL.setMutualAuthParams(mTLS);

I have a random high port forwarded to 8883 (X.duckdns.org:12345 -> PrivIP_runningHASSio:8883) on my router.

Am I doing anything obviusly wrong?

PD: I think I found a typo here --no-search should be --no-verify right?
Thanks!

[prototypicalpro]

The certificate and private key string should not contain any spaces, sorry for the miscommunication. In other words, your final certificate string should look like this:

const char my_cert[] = "-----BEGIN CERTIFICATE-----\r\nMIIB+DCCAZ+gAwIBAgIUQjqvUXZ0PnFiFDIEb6WW2ojMjeUwCgYIKoZIzj0EAwIw\r\nWTELMAkGA1UEBhMCVVMxDzANBgNVBAgMBk9yZWdvbjEZMBcGA1UECgwQT3BlbiBT\r\nZW5zaW5nIExhYjEeMBwGA1UEAwwVTG9vbURCIFJvb3QgQXV0aG9yaXR5MB4XDTIw\r\nMDIyODE4NTgzNVoXDTIwMDkxNTE4NTgzNVowXDELMAkGA1UEBhMCVVMxDzANBgNV\r\nBAgMBk9yZWdvbjEZMBcGA1UECgwQT3BlbiBTZW5zaW5nIExhYjEhMB8GA1UEAwwY\r\nU3Bvb2wgQ2xpZW50IENlcnRpZmljYXRlMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcD\r\nQgAEBgE+VNc6N0GFl9w50BdeGYNGofUeERKgpMx9DUX4V26TQs0XuQ6Sgmpt3a11\r\nFKtQ5h4P7rpl0L5SapAK2fiNnaNCMEAwHQYDVR0OBBYEFB+ug0aetyp47/xSEHWB\r\nO9o6n6ZwMB8GA1UdIwQYMBaAFFZgt0LzPASch9rtMzdyqNSeoj8+MAoGCCqGSM49\r\nBAMCA0cAMEQCICCTm2vzMn0lTMtIjm95MrUMm9SZbWU+XJQyl9jt5Fm4AiAfqUF9\r\nACFM5mpSBu6MDhCDGKG/KyhP/oAs8mHxDGwgng==\r\n-----END CERTIFICATE-----";

The same goes for your private key. This format follows PEM certificate formatting, allowing BearSSL to parse it correctly.

In addition, the variable type of my_cert and my_key must be const char[], and cannot be const char*. This distinction is important because the sizeof function returns different things for the different types (see https://stackoverflow.com/questions/7903551/when-to-use-const-char-and-when-to-use-const-char).

Everything else looks good! Try fixing your cert and key and see if it changes the error ;).

Thanks for pointing out the typo! I'll fix it asap.

[Atalonica]

Awesome work! Thank you for all the tips, now its working flawlessly.
I noticed (I know it's the same) but the cert and key could be written like this so that the user can see the whole content.
Thanks again!

[prototypicalpro]

Great! I'm glad it works. I'll go ahead and close this issue!

Thanks for the tip, I'll incorporate it into the documentation on mTLS with SSLClient eventually.

[prototypicalpro]

Could I have a stripped down version of your sketch for an SSLClient example of using mTLS? Thanks!

[Atalonica]

Yes, of course. Sorry for the delay, I've been sick:

// Increase MQTT_MAX_PACKET_SIZE (PubSubClient.h)
#include <SPI.h>
#include <EthernetLarge.h>
#include <SSLClient.h>
#include "certificates.h" // Generated from ca.crt and python script.
#include <PubSubClient.h>

const char my_cert[] = (...)
const char my_key[] = (...)

byte mac[] = { 0x12, 0x34, 0x56, 0x78, 0x9A, 0xBC }; // Custom MAC?
const char* mqttServer = "mosquittobroker.example"; // Broker adress / IP
IPAdress ip (192, 168, 1, 2); // Custom client static IP

const char* clientId = "clientId ";
const char* usr = "username";
const char* pass = "password";
const char* willTopic = "home/LWT";
const char* willMessage = "Offline";
const char* gatewayWentOnline = "Online";
const uint8_t willQoS = 1;
const boolean willRetain = true;

void callback(char* topicPointer, byte* payload, unsigned int payloadLength){ ... };

EthernetClient ethClient;
SSLClient ethClientSSL(ethClient, TAs, (size_t)TAs_NUM, A6, 1);
PubSubClient mqttClient(mqttServer, 8883, callback, ethClientSSL);

void ethernetReconnect() {
ethClientSSL.setMutualAuthParams(mTLS);
Ethernet.init(5);
Ethernet.begin(mac, ip);
}

boolean mqttReconnect() {
if (mqttClient.connect(clientId, usr, pass, willTopic, willQoS, willRetain, willMessage)) {
mqttClient.publish(willTopic, gatewayWentOnline, willRetain);
mqttClient.subscribe("home/RFM69GW/commands/+/+");
}
return mqttClient.connected();
}

void setup(){
randomSeed(analogRead(A6));
ethClientSSL.setMutualAuthParams(mTLS);
// We could initialize Ethernet here...
}

void loop(){
if (!mqttClient.connected() || Ethernet.linkStatus() != LinkON) {
ethernetReconnect();
delay(1500);
if (!mqttReconnect() && Ethernet.linkStatus() == LinkON) {
delay(1500);
}
} else {
mqttClient.loop();
// CODE
}
}

[pablofr918]

Hi @prototypicalpro! I'm trying to use SSLClient with GSM connection, but I'm new in the world of SSL/TLS and there are a lot of things that I don't understand.

So in my case, I want to make a TLS MQTT connection to a broker, test.mosquitto.org, for example, but making the internet connection trhough GPRS. The parts until I get Network connection through the Modem are done, so I can get internet, but the following step is to connect to the broker, but there is where I get lost.

Do I need trust anchors, client/server certificates and private keys, or everything? and how to get them in order to put them in the code?

Thanks for the shared information.

[codenicks]

Hi, I have the same problem, when I run the program, it connects but is not sending data and after that I get these errors:
"(SSLClient)(SSL_ERROR)(connected): Not connected because write error is set
(SSLClient)(SSL_ERROR)(m_print_ssl_error): SSL_CLIENT_WRITE_FAIL"
Any help would be appreciated. Thankyou!