Skip to content

Commit

Permalink
[#522] Update to Hono 2.5.0
Browse files Browse the repository at this point in the history 
Changed configuration to run Apache Kafka in KRaft mode.

Fixes #522
  • Loading branch information
@sophokles73
sophokles73 committed Jan 28, 2024
1 parent 93b0e84 commit ea0b606
Show file tree
Hide file tree
Showing 8 changed files with 125 additions and 105 deletions.
8 changes: 4 additions & 4 deletions charts/hono/Chart.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2019, 2023 Contributors to the Eclipse Foundation
# Copyright (c) 2019 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
Expand All @@ -15,9 +15,9 @@ name: hono
description: |
Eclipse Hono™ provides remote service interfaces for connecting large numbers of IoT devices to a back end and
interacting with them in a uniform way regardless of the device communication protocol.
version: 2.5.6
version: 2.5.7
# Version of Hono being deployed by the chart
appVersion: 2.4.0
appVersion: 2.5.0
keywords:
- iot-chart
- IoT
Expand Down Expand Up @@ -50,5 +50,5 @@ dependencies:
condition: "mongodb.createInstance"
- name: "kafka"
repository: "oci://registry-1.docker.io/bitnamicharts"
version: "^21.x"
version: "^26.8"
condition: "kafkaMessagingClusterExample.enabled"
23 changes: 13 additions & 10 deletions charts/hono/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,19 +48,17 @@ Kubernetes *Services*. The following command lists all services and their endpoi
kubectl get service -n hono
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
eclipse-hono-adapter-amqp LoadBalancer 10.99.197.79 127.0.0.1 5672:32672/TCP,5671:32671/TCP 2m30s
eclipse-hono-adapter-http LoadBalancer 10.102.247.45 127.0.0.1 8080:30080/TCP,8443:30443/TCP 2m29s
eclipse-hono-adapter-mqtt LoadBalancer 10.98.68.57 127.0.0.1 1883:31883/TCP,8883:30883/TCP 2m29s
eclipse-hono-kafka ClusterIP 10.104.176.12 <none> 9092/TCP 2m30s
eclipse-hono-kafka-0-external LoadBalancer 10.98.132.252 127.0.0.1 9094:32094/TCP 2m29s
eclipse-hono-kafka-headless ClusterIP None <none> 9092/TCP,9093/TCP 2m30s
eclipse-hono-service-auth ClusterIP 10.99.220.217 <none> 5671/TCP 2m29s
eclipse-hono-adapter-amqp LoadBalancer 10.99.197.79 127.0.0.1 5671:32671/TCP 2m30s
eclipse-hono-adapter-http LoadBalancer 10.102.247.45 127.0.0.1 8443:30443/TCP 2m29s
eclipse-hono-adapter-mqtt LoadBalancer 10.98.68.57 127.0.0.1 8883:30883/TCP 2m29s
eclipse-hono-kafka ClusterIP 10.104.176.12 <none> 9092/TCP,9095/TCP 2m30s
eclipse-hono-kafka-controller0-external LoadBalancer 10.98.132.252 127.0.0.1 9094:32094/TCP 2m29s
eclipse-hono-kafka-controller-headless ClusterIP None <none> 9094/TCP,9092/TCP,9093/TCP 2m30s
eclipse-hono-service-auth ClusterIP 10.99.220.217 <none> 5671/TCP,8088/TCP 2m29s
eclipse-hono-service-command-router ClusterIP 10.98.52.92 <none> 5671/TCP 2m29s
eclipse-hono-service-device-registry ClusterIP 10.109.46.233 <none> 5671/TCP,8080/TCP,8443/TCP 2m29s
eclipse-hono-service-device-registry-ext LoadBalancer 10.97.217.173 127.0.0.1 28080:31080/TCP,28443:31443/TCP 2m29s
eclipse-hono-service-device-registry-ext LoadBalancer 10.97.217.173 127.0.0.1 28443:31443/TCP 2m29s
eclipse-hono-service-device-registry-headless ClusterIP None <none> <none> 2m30s
eclipse-hono-zookeeper ClusterIP 10.104.9.153 <none> 2181/TCP,2888/TCP,3888/TCP 2m29s
eclipse-hono-zookeeper-headless ClusterIP None <none> 2181/TCP,2888/TCP,3888/TCP 2m30s
```
The listing above has been retrieved from a Minikube cluster that emulates a load balancer via the `minikube tunnel`
Expand Down Expand Up @@ -103,6 +101,11 @@ The command removes all the Kubernetes components associated with the chart and
## Release Notes
### 2.5.7
* Use Hono 2.5.0 container images.
* Update bitnami/kafka chart to version 26.8.x which uses Kafka 3.6 in Kraft mode.
### 2.5.6
* Update bitnami/kafka chart to version 21.x which uses Kafka 3.4.
Expand Down
19 changes: 13 additions & 6 deletions charts/hono/ci/datagrid-values.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2020, 2022 Contributors to the Eclipse Foundation
# Copyright (c) 2020 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
Expand All @@ -26,11 +26,18 @@ kafka:
externalAccess:
autoDiscovery:
enabled: false
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32094"
broker:
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32194"
controller:
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32094"
serviceAccount:
create: false
rbac:
Expand Down
19 changes: 13 additions & 6 deletions charts/hono/ci/mongodb-based-device-registry-values.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2020, 2022 Contributors to the Eclipse Foundation
# Copyright (c) 2020 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
Expand Down Expand Up @@ -32,11 +32,18 @@ kafka:
externalAccess:
autoDiscovery:
enabled: false
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32094"
broker:
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32194"
controller:
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32094"
serviceAccount:
create: false
rbac:
Expand Down
32 changes: 26 additions & 6 deletions charts/hono/ci/quarkus-native-images-values.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2021, 2022 Contributors to the Eclipse Foundation
# Copyright (c) 2021 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
Expand Down Expand Up @@ -27,11 +27,18 @@ kafka:
externalAccess:
autoDiscovery:
enabled: false
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32094"
broker:
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32194"
controller:
service:
type: "NodePort"
# length of the array must match replicaCount
nodePorts:
- "32094"
serviceAccount:
create: false
rbac:
Expand Down Expand Up @@ -61,6 +68,19 @@ commandRouterService:
cpu:
memory: "120Mi"

deviceRegistryExample:
embeddedJdbcDeviceRegistry:
imageName: "eclipse/hono-service-device-registry-jdbc-native"
cmdLineArgs:
- "-Xmx160m"
resources:
requests:
cpu:
memory: "120Mi"
limits:
cpu:
memory: "120Mi"

adapters:
amqp:
enabled: false
Expand Down
11 changes: 4 additions & 7 deletions charts/hono/templates/NOTES.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ NAME {{ "READY STATUS RESTARTS AGE"
{{ include "hono.fullname" . }}-dispatch-router-6c77dc78bd-hjn4l 1/1 Running 0 5m51s
{{- end }}
{{- if ( has "kafka" .Values.messagingNetworkTypes ) }}
{{ include "hono.fullname" . }}-kafka-0 1/1 Running 0 5m51s
{{ include "hono.fullname" . }}-kafka-controller-0 1/1 Running 0 5m51s
{{- end }}
{{- if all .Values.deviceRegistryExample.enabled ( eq .Values.deviceRegistryExample.type "mongodb" ) }}
{{ include "hono.fullname" . }}-monogodb-server-77ffb85c97-78554 1/1 Running 0 5m51s
Expand All @@ -47,9 +47,6 @@ NAME {{ "READY STATUS RESTARTS AGE"
{{ include "hono.fullname" . }}-service-device-registry-64d645dd9c-q9c5q 1/1 Running 0 5m51s
{{- end }}
{{- end }}
{{- if ( has "kafka" .Values.messagingNetworkTypes ) }}
{{ include "hono.fullname" . }}-zookeeper-0 1/1 Running 0 5m51s
{{- end }}

Once all pods have reached the READY state, you can start using Hono.

Expand All @@ -58,15 +55,15 @@ https://eclipse.dev/hono/docs/getting-started/

{{- if ( has "kafka" .Values.messagingNetworkTypes ) }}
{{- if .Values.kafkaMessagingClusterExample.enabled }}
{{- $isKafkaTlsEnabled := ( eq .Values.kafka.auth.clientProtocol "sasl_tls" ) }}
{{- $isKafkaTlsEnabled := ( eq .Values.kafka.listeners.external.protocol "SASL_SSL" ) }}
Hono is configured with an example Kafka cluster for messaging. Clients can connect to the
example Kafka cluster using the configuration properties below.
Please refer to the guide mentioned above for details on how to determine the values for
$KAFKA_IP{{ if $isKafkaTlsEnabled }} and $KAFKA_TRUSTSTORE_PATH{{ end }}.

bootstrap.servers=$KAFKA_IP:{{ .Values.kafka.externalAccess.service.ports.external }}
bootstrap.servers=$KAFKA_IP:{{ .Values.kafka.externalAccess.controller.service.ports.external }}
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="{{ first .Values.kafka.auth.sasl.jaas.clientUsers }}" password="{{ first .Values.kafka.auth.sasl.jaas.clientPasswords }}";
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="{{ first .Values.kafka.sasl.client.users }}" password="{{ first .Values.kafka.sasl.client.passwords }}";
{{- if $isKafkaTlsEnabled }}
security.protocol=SASL_SSL
ssl.endpoint.identification.algorithm=""
Expand Down
20 changes: 10 additions & 10 deletions charts/hono/templates/_helpers.tpl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -286,21 +286,21 @@ The scope passed in is expected to be a dict with keys
kafka:
{{- if .dot.Values.kafkaMessagingClusterExample.enabled }}
commonClientConfig:
{{- $bootstrapServers := printf "%[1]s-0.%[1]s-headless:%d" ( include "common.names.fullname" .dot.Subcharts.kafka ) ( .dot.Values.kafka.service.ports.client | int ) }}
{{- $bootstrapServers := printf "%[1]s-%[2]s-controller-headless:%d" .dot.Release.Name .dot.Values.kafka.nameOverride ( .dot.Values.kafka.service.ports.client | int ) }}
bootstrap.servers: {{ $bootstrapServers | quote }}
{{- if eq .dot.Values.kafka.auth.clientProtocol "sasl_tls" }}
{{- if eq .dot.Values.kafka.listeners.client.protocol "SASL_SSL" }}
security.protocol: "SASL_SSL"
sasl.mechanism: "SCRAM-SHA-512"
sasl.jaas.config: "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"{{ first .dot.Values.kafka.auth.sasl.jaas.clientUsers }}\" password=\"{{ first .dot.Values.kafka.auth.sasl.jaas.clientPasswords }}\";"
sasl.jaas.config: "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"{{ first .dot.Values.kafka.sasl.client.users }}\" password=\"{{ first .dot.Values.kafka.sasl.client.passwords }}\";"
ssl.truststore.type: "PEM"
ssl.truststore.location: "/opt/hono/tls/ca.crt"
ssl.endpoint.identification.algorithm: "" # Disables hostname verification. Don't do this in productive setups!
{{- else if eq .dot.Values.kafka.auth.clientProtocol "sasl" }}
{{- else if eq .dot.Values.kafka.listeners.client.protocol "SASL_PLAINTEXT" }}
security.protocol: "SASL_PLAINTEXT"
sasl.mechanism: "SCRAM-SHA-512"
sasl.jaas.config: "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"{{ first .dot.Values.kafka.auth.sasl.jaas.clientUsers }}\" password=\"{{ first .dot.Values.kafka.auth.sasl.jaas.clientPasswords }}\";"
sasl.jaas.config: "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"{{ first .dot.Values.kafka.sasl.client.users }}\" password=\"{{ first .dot.Values.kafka.sasl.client.passwords }}\";"
{{- else }}
{{- required ".Values.kafka.auth.clientProtocol has unsupported value" nil }}
{{- required ".Values.kafka.listeners.client.protocol has unsupported value" nil }}
{{- end }}
{{- else }}
{{- $bootstrapServers := dig "kafkaMessagingSpec" "commonClientConfig" "bootstrap.servers" "" . }}
Expand All @@ -326,11 +326,11 @@ The scope passed in is expected to be a dict with keys
{{- define "hono.kafkaConfigCheck" -}}
{{- if and (has "kafka" .dot.Values.messagingNetworkTypes) .dot.Values.kafkaMessagingClusterExample.enabled }}
{{- if .dot.Values.useLoadBalancer }}
{{- if not (eq .dot.Values.kafka.externalAccess.service.type "LoadBalancer") }}
{{- required ".Values.kafka.externalAccess.service.type MUST be 'LoadBalancer' if .Values.useLoadBalancer is true" nil }}
{{- if not (eq .dot.Values.kafka.externalAccess.broker.service.type "LoadBalancer") }}
{{- required ".Values.kafka.externalAccess.broker.service.type MUST be 'LoadBalancer' if .Values.useLoadBalancer is true" nil }}
{{- end }}
{{- else if not (eq .dot.Values.kafka.externalAccess.service.type "NodePort") }}
{{- required ".Values.kafka.externalAccess.service.type MUST be 'NodePort' if .Values.useLoadBalancer is false" nil }}
{{- else if not (eq .dot.Values.kafka.externalAccess.broker.service.type "NodePort") }}
{{- required ".Values.kafka.externalAccess.broker.service.type MUST be 'NodePort' if .Values.useLoadBalancer is false" nil }}
{{- end }}
{{- end }}
{{- end }}
Expand Down
98 changes: 42 additions & 56 deletions charts/hono/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1724,82 +1724,68 @@ kafka:
# Set to false for productive setups. Topic management then needs to be provided externally.
autoCreateTopicsEnable: true
deleteTopicEnable: true
auth:
clientProtocol: "sasl_tls"
sasl:
jaas:
clientUsers:
- "hono"
clientPasswords:
- "hono-secret"
zookeeperUser: "zookeeperUser"
zookeeperPassword: "zookeeperPassword"
tls:
type: "pem"
pemChainIncluded: true
existingSecrets:
# Evaluated in kafka chart via tpl; kafka chart full name used here!
- '{{ include "hono.fullname" . }}-example-keys'

service:
ports:
client: 9092
internal: 9093
external: 9094
broker:
automountServiceAccountToken: true

controller:
replicaCount: 1
automountServiceAccountToken: true
extraConfig: |
group.initial.rebalance.delay.ms=0
offsets.topic.replication.factor=1
transaction.state.log.min.isr=1
transaction.state.log.replication.factor=1
listeners:
client:
protocol: "SASL_SSL"
external:
protocol: "SASL_SSL"

sasl:
client:
users:
- "hono"
passwords:
- "hono-secret"

tls:
type: "PEM"
pemChainIncluded: false
# Evaluated in kafka chart via tpl; kafka chart full name used here!
existingSecret: '{{ include "hono.fullname" . }}-example-keys'

replicaCount: 1
# Do not write data to a PersistentVolume by default.
# When setting this to true, make sure to also set "kafka.zookeeper.persistence.enabled" to true and
# make sure that Kafka is stopped cleanly using SIGTERM _before_ Zookeeper is stopped.
# Otherwise you will most likely run into issues like
# https://stackoverflow.com/questions/39759071/error-while-starting-kafka-broker
persistence:
enabled: false
# persistence:
# enabled: false
# Expose the Kafka service to be accessed from outside the cluster (LoadBalancer service).
# To use service type NodePort instead of LoadBalancer, refer to
# https://github.com/bitnami/charts/tree/40bf90ef4a52969b9ed8acca3d8a67e90d800673/bitnami/kafka#accessing-kafka-brokers-from-outside-the-cluster
externalAccess:
enabled: true
autoDiscovery:
enabled: true
service:
type: "LoadBalancer"
ports:
external: 9094
# length of the array must match replicaCount
nodePorts:
- 32094
broker:
service:
type: "LoadBalancer"
ports:
external: 9094
controller:
service:
type: "LoadBalancer"
ports:
external: 9094
serviceAccount:
create: true
rbac:
# Note that this could require creating RBAC rules, for more information refer to
# https://github.com/bitnami/charts/tree/master/bitnami/kafka#accessing-kafka-brokers-from-outside-the-cluster
create: true
# limit Kafka JVM's memory consumption
heapOpts: "-Xms620M -Xmx620M"
resources:
requests:
memory: "800Mi"
limits:
memory: "800Mi"

zookeeper:
persistence:
enabled: false
auth:
client:
enabled: true
clientUser: "zookeeperUser"
clientPassword: "zookeeperPassword"
serverUsers: "zookeeperUser"
serverPasswords: "zookeeperPassword"
# limit Zookeeper JVM's memory consumption
heapSize: 400
resources:
requests:
memory: "256Mi"
limits:
memory: "500Mi"

# amqpMessagingNetworkExample contains properties for configuring an example AMQP network
# to be used for messaging if "messagingNetworkTypes" contains "amqp"
Expand Down

0 comments on commit ea0b606

Please sign in to comment.