bundle creation dependency cycle ?

[joelmarty]

The 1.5.0 (at least) version of the library has the KafkaConsumerBundle constructor accepting a list of topics and a rebalance listener.

My understanding of the consumer rebalance listener is that it is a mechanism that allows the consumer to be warned of the rebalancing event so it has an opportunity to commit currently processing offsets.
That means that the rebalance listener needs a Consumer instance. Which it cannot have at that point since it is the bundle that is being created that can provide such instance.

I can circumvent by having a setter on the rebalancer listener for the consumer instance but that is confusing.

Kind of same remark for the topics argument, we don't have access to the configuration in the bootstrap() step so we cannot load the topics from configuration at that point.

[natalie-zamani]

Noted on the use of the rebalance listener. We'll want to make some changes to the API to allow that, as that sounds like a hole today.

There was another discussion of having topics in configs here #17

[natalie-zamani]

@joelmarty I did some more digging. A consumer rebalance listener needs to be provided at the time subscribe() is called on the consumer, which we leave up to users to do. So I don't think our APIs should include rebalance listeners at all.

Let me know if you disagree, otherwise I'll close this in several days.

[joelmarty]

It's good for me, I submitted this ticket because I wanted to try it out and noticed that. In the end I cannot use it because we use a custom serialization with Avro and when I saw #13 I abandonned there.

[natalie-zamani]

There’s nothing preventing the usage of serialization formats like Avro or protobuf today, there are just not pre-created serializer/deserializer factories to allow configuring it entirely via yaml.