quic+locks

[elyograg]

I was informed about the excessive locking problems in openssl 3.x on the haproxy mailing list. All indications are that the openssl project is ignoring the problem. The info I have available says that openssl 3.x is many times slower under heavy load than 1.1.x because of the excessive locking.

This project had a +quic+locks variant for 3.1.0, but at this time I do not see such a variant for 3.1.2. I made an attempt at testing for a performance difference between 1.1.x+quic, 3.0.x+quic, and 3.1.0+quic+locks. I did not see any real difference, so I suspect my testing methodology was not correct.

What can you tell me about the current state of fixing locking issues in either this repo or the upstream openssl repo?

My installs of haproxy do not see enough traffic for the locking to present a problem, but it interests me. I don't have an understanding of the code involved.

[tmshort]

A number of fixes were migrated into upstream openssl in the openssl-3.1 and master branches (and possibly some in the openssl-3.0 branch). These were all made relatively recently, so the releases that came out this week should have them (except those that are exclusive to the master branch). There were a number of discussions on the OpenSSL mailing lists and in appropriate PRs. Fundamentally, access to some structures (in the provider) where locked more often than they needed to be. You really needed to have a multi-threaded application running OpenSSL at high data throughput.

You could try the openssl-3.1.2-quic1 release to see if there's any improvement. You really won't see much of anything in a single-threaded, or low throughput, application.

[elyograg]

I'm upgraded to 3.1.2+quic on my haproxy installs.

I wrote a Java program to pound haproxy with requests from multiple threads for a super simple page served by haproxy directly rather than forwarded to a backend. It didn't show the expected difference between the versions ... 3.0 (I think it was 3.0.8) was slightly faster than 1.1 (1.1.1t I think) ... which was very surprising given what I was told on the haproxy mailing list.

As I said, my web pages normally see so little traffic that I am never going to have a problem with it, but the whole thing fascinates me, so I am keeping an eye on the problem.

[elyograg]

I don't expect to be able to really contribute to fixing the locking issues that have been found. I'm eagerly awaiting testing results from people who know how to properly create a test for it, because my test results seem to indicate that my test wasn't valid.

Also wanting to see where things are going to end up in the quic/http3 landscape. From what I can see, so far this project is the best option currently available for enabling H3 support. I hope openssl leverages work done by this project and others in a spirit of respectful cooperation.

[tmshort]

The upstream project is going in a different direction in 3.2 with their own QUIC API and implementation. The APIs will not be compatible with Chromium QUIC, and it will take a few releases before upstream has a complete implementation, as they are focusing on the client in 3.2.
Because of this, it may not be possible to add these QUIC APIs to OpenSSL 3.2. There may be another approach that can be taken (I'm in discussion with some of the OpenSSL folks about it), but it might have to wait until 3.2 is closer to release (and I actually have time to make it a reality). It would be a separate quictls project that (hopefully) would work with vanilla OpenSSL 3.2.
That being said, BoringSSL and Chromium QUIC API surfaces are malleable and have no backwards compatibility guarantees.

[elyograg]

My primary interest is QUIC is for haproxy. I'm an end user, not on the haproxy development team.

I am guessing that haproxy will need to spend a while refactoring their QUIC code to work with openssl 3.2.

Getting back to this issue ... is openssl working to eliminate the locking overhead introduced in 3.0, which the 3.1.0+quic+locks branch here seemed to be tackling? My testing environment for the locking overhead didn't actually reveal the overhead, which I am attributing to issues in my test methodology.

[tmshort]

As far as I know, upstream has submitted various fixes; based on risk, to the openssl-3.1 and openssl-3.0 branches. Not everything made it to openssl-3.0.