Enable buffer pooling settings with a SSL configurable option #5316
Conversation
|
@cescoffier and @vietj PTAL I've added some asserts to make it existing tests to fail and make sure the invariant I'm assuming for SSL on vertx are correct. |
|
@cescoffier this is key for both https and vertx sql client - cause the latter will rely heavily on it for the reactive sql drivers (both mysql and postgres AFAIK) - @tsegismont AFAIK vertx sql driver should rely on |
|
After talking with @cescoffier I'm changing this back to use a specific option to enable this feature. |
franz1981
force-pushed
the
4.x_unified_allocator_fix_ssl
branch
from
2dc914b
to
0fca2ca
Compare
franz1981
changed the title
|
PTAL @vietj I've added some test (I could improve thsi further) - but it is not clear to me yet the role of https://vertx.io/docs/apidocs/io/vertx/core/http/HttpClientOptions.html#setSsl-boolean- And indeed it seems that can you help me to understand what's |
|
@tsegismont i.e. Here I need users to set https://vertx.io/docs/apidocs/io/vertx/core/net/TCPSSLOptions.html#setSsl-boolean- and https://vertx.io/docs/apidocs/io/vertx/core/net/TCPSSLOptions.html#setSslEngineOptions-io.vertx.core.net.SSLEngineOptions- (passing a JDK SSL option which enable heap buffer pooling), in order to made vertx to do the "right" thing and avoid quarkusio/quarkus#41880 (comment) to happen. |
You can set options with
In Quarkus, you have to customize pool creation, I think. |
|
Wdyt @cescoffier ? It is ok enough? Thanks @tsegismont |
|
@cescoffier can you validate this one ? |
|
@vietj can you paste the additional test you found in our call, bud? |
|
I think you want to look at NetTest with NetSocketInternal |
|
Drafting till fixing the latest tests |
franz1981
force-pushed
the
4.x_unified_allocator_fix_ssl
branch
from
53d55ab
to
a3393ac
Compare
|
many thanks @vietj and @cescoffier for this hard round of review: you were right, there's something terrible hidden on SSL... In a3393ac I've put some comment for some shocking outcome, and I think we need to have a sync before moving this forward - which BTW is not enabled by default eh |
|
While working on this PR I've noticed something odd, exactly 2 things:
And it refers to https://github.com/openjdk/jdk21/blob/890adb6410dab4606a4f26a942aed02fb2f55387/src/java.base/share/classes/com/sun/crypto/provider/GCTR.java#L212 which shows, for |
@cescoffier could you elaborate on what your concerns are for native compilation? Looking at the changes I don't expect any drastic changes. Trying the change with Quarkus' {
"types": {
"total": 16096,
"reflection": 4455,
"jni": 62,
"reachable": 14093
},
"methods": {
"foreign_downcalls": -1,
"total": 125514,
"reflection": 3555,
"jni": 55,
"reachable": 72175
},
"classes": {
"total": 16096,
"reflection": 4455,
"jni": 62,
"reachable": 14093
},
"fields": {
"total": 36087,
"reflection": 138,
"jni": 67,
"reachable": 21524
}
}vs {
"types": {
"total": 16075,
"reflection": 4451,
"jni": 62,
"reachable": 14072
},
"methods": {
"foreign_downcalls": -1,
"total": 125400,
"reflection": 3550,
"jni": 55,
"reachable": 72092
},
"classes": {
"total": 16075,
"reflection": 4451,
"jni": 62,
"reachable": 14072
},
"fields": {
"total": 36038,
"reflection": 138,
"jni": 67,
"reachable": 21480
}
}
with 4.5.10 which is reflected to an increase of 36kb in the native executable size. However, I suspect that this IT test doesn't enable SSL. Any hints how I could/should test this with SSL enabled? |
|
@zakkak you can use the vertx-http integration tests - it has TLS tests (gRPC has some TLS tests too). My concern is that in the past, such a change broke the native compilation because the classes were not on the classpath or because loading the class brought many other concerns and initialization issues (in addition to making the overall executable bigger). This should not be the case here, but because if we merge, it will land in the Quarkus LTS, we need to be sure we will not shoot ourselves in the foot. |
|
Thank you @cescoffier. I am appending the results from TLDR: LGTM. vertx-core 4.x (tip of vert.x without this PR) "types": {
"total": 14411,
"reflection": 4023,
"jni": 62,
"reachable": 12484
},
"methods": {
"foreign_downcalls": -1,
"total": 113204,
"reflection": 3337,
"jni": 55,
"reachable": 64071
},
"fields": {
"total": 33089,
"reflection": 133,
"jni": 67,
"reachable": 19472
}this PR "types": {
"total": 14419,
"reflection": 4023,
"jni": 62,
"reachable": 12491
},
"methods": {
"foreign_downcalls": -1,
"total": 113234,
"reflection": 3338,
"jni": 55,
"reachable": 64081
},
"fields": {
"total": 33097,
"reflection": 133,
"jni": 67,
"reachable": 19479
}this PR with pooled heap buffers enabled "types": {
"total": 14410,
"reflection": 4023,
"jni": 62,
"reachable": 12483
},
"methods": {
"foreign_downcalls": -1,
"total": 113201,
"reflection": 3337,
"jni": 55,
"reachable": 64064
},
"fields": {
"total": 33085,
"reflection": 133,
"jni": 67,
"reachable": 19471
}To enable the pooled heap buffers I patched Quarkus with: diff --git a/extensions/vertx-http/runtime/src/main/java/io/quarkus/vertx/http/runtime/options/HttpServerOptionsUtils.java b/extensions/vertx-http/runtime/src/main/java/io/quarkus/vertx/http/runtime/options/HttpServerOptionsUtils.java
index 36fe46a9d3a..872d79a85cb 100644
--- a/extensions/vertx-http/runtime/src/main/java/io/quarkus/vertx/http/runtime/options/HttpServerOptionsUtils.java
+++ b/extensions/vertx-http/runtime/src/main/java/io/quarkus/vertx/http/runtime/options/HttpServerOptionsUtils.java
@@ -82,6 +82,7 @@ public static HttpServerOptions createSslOptions(HttpBuildTimeConfig buildTimeCo
serverOptions.setAlpnVersions(Arrays.asList(HttpVersion.HTTP_2, HttpVersion.HTTP_1_1));
}
}
+ serverOptions.setSslEngineOptions(new JdkSSLEngineOptions().setPooledHeapBuffers(true));
setIdleTimeout(httpConfiguration, serverOptions);
TlsConfiguration bucket = getTlsConfiguration(httpConfiguration.tlsConfigurationName, registry); |
|
Thanks @zakkak ! All good for me then! |
|
so @vietj here we are: this can be moved forward and I'll create the relevant issues on the quarkus repo for the missing bits to make it available to use there. I strongly suggest @vietj that vertx users which relies on JDK SSL will enable this feature as well, TBH, as performance with JDK SSL is terrible without it... |
|
thanks for the contrib @franz1981 |
SSL JDK is very memory allocation intensive, see netty/netty#14208: by default Vertx is using unpooled heap buffers of 16 KB for each interaction with SSL/TSL, which can lead to increase significantly the memory footprint.
In order to fix it I've created a new ssl configuration property (which should become the default in vertx 5) to enable pooling of heap buffers using JDK SSL, while keeping the existing behaviour by default.