-
Notifications
You must be signed in to change notification settings - Fork 646
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add feature to request consumer group information #435
base: master
Are you sure you want to change the base?
Add feature to request consumer group information #435
Conversation
It looks like @Ledostuff hasn't signed our Contributor License Agreement, yet.
You can read and sign our full Contributor License Agreement here. Once you've signed reply with Appreciation of efforts, clabot |
[clabot:check] |
@confluentinc It looks like @Ledostuff just signed our Contributor License Agreement. 👍 Always at your service, clabot |
2591e63
to
bddeb50
Compare
bddeb50
to
2454312
Compare
@Ledostuff PR build is failing. Could you check? |
The build failed:
|
This PR shouldn't need to depend on ZooKeeper. We should be able to use the public AdminClient. What's the security model here? How do we decide who can and cannot access admin APIs? |
@gAmUssA Thank you for review. I`ll fix build and push changes asap. |
@cmccabe Thank you for review. |
2454312
to
5d2c237
Compare
Why is this information needed for REST proxy? If it is needed, then we should at least think about providing it through the AdminClient.
This PR seems to give rest-proxy users a lot of powerful abilities to access administrative APIs. I'm not sure that rest-proxy is the right place to manage the Kafka cluster. If we did want to do this we would need to figure out how to give certain users the ability to use the admin APIs and block everyone else. For example, users should not be able to list groups unless they have DESCRIBE permission on the group, or DESCRIBE permission on the Cluster resource. To be honest, I'm not sure this is the right direction to go-- at least not without a lot more discussion. |
@cmccabe Thank you for the answer.
We are only show information about groups who consumes messges from clusters topics. We don`t try to change settings or do any other work with cluster.
I don't see any problem about that. Because we allow users to consume messages through the rest-proxy. But why do we can't show information about consumer groups? Even if we allow users DESCRIBE permission on groups or DESCRIBE permission on the Cluster resource .
I didn't found similar method in the API of AdminClient. If you show me alternative method to get information about brokers in cluster with security protocol, I'll repair code in this PR gladly.) |
fcaa5d5
to
cc021ea
Compare
Hey @Ledostuff, thanks for the PR! I haven't had time to give this a proper look yet but I wanted to ask - do we need ZooKeeper to access the cluster information only ( The consumer information should be fetchable through the adminClient only, correct? There is a question of whether that information should be limited to consumers managed by this proxy instance or consumers in the Kafka Cluster.
|
cc021ea
to
870947f
Compare
No, we don't. This is only valid for the case where authentication and acl's are off OR we elevate kafka-rest principal's resource permissions like READ/DESCRIBE on cluster/group/topic, depending on the operation type we want. We are free to choose which ACLs to give to kafka-rest's principal, should consumer groups be exposed or not. That is, users are free to choose their own security model.
That's right, unless we provide DESCRIBE permission on the Cluster resource. Viewing consumer group offsets or listing consumer groups is just a feature that can be enabled or disabled, according to a security model adapted by the user. |
Hi @rootex-, That's true. I agree with exposing consumer group information. |
Not to be annoying, but... is there still interest in this? I'd welcome having a list of consumer groups available via REST proxy. |
Yes, there is still a huge interest in this feature. Could we merge this pull request? |
870947f
to
21627e9
Compare
Hello there! Please review this PR. Thanks) |
Hi @stanislavkozlovski, |
Repair retrieving topic-partition subscription information restricted by topic
…y topic name Add some documentation
Final version
9c1f5b7
to
016d0fe
Compare
Hello, @stanislavkozlovski :) |
Hellooo. One month has already passed since last comment. PLease, review this pull-request. |
|
Add feature to request consumer group information(start, end offsets, lags, consumer-instance-id, consumer-instance-ip)
Added rest paths:
Add unit and integration tests