Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Blocker: customUserAcls - Plan command - Marks the action for every entry on the state.yaml as "REMOVE". #77

Open
jaykatti opened this issue Aug 27, 2021 · 14 comments
Open

Blocker: customUserAcls - Plan command - Marks the action for every entry on the state.yaml as "REMOVE". #77

jaykatti opened this issue Aug 27, 2021 · 14 comments

Comments

@jaykatti
Copy link

I have generated a current state.yaml for my existing cluster.
I changed some couple of users the permission as "DENY". So I was expecting the "plan" command to show me those many "updates", instead, it is including those records in the plan json file and marks everything else for "REMOVE".

Kindly advise at the earliest.

This currently is not allowing me to deploy onto our server, so we can include this into our process permanently.

Thanks,
Jay.
@jaykatti
Copy link
Author

I am using the latest version 0.2.15.
This seems to be a complete blocker. Can this please be taken-up on priority ?

@jaykatti jaykatti changed the title customUserAcls - Plan command - Marks the action for every entry on the state.yaml as "REMOVE". Blocker: customUserAcls - Plan command - Marks the action for every entry on the state.yaml as "REMOVE". Aug 27, 2021
@devshawn
Copy link
Owner

Hi @jaykatti, can you provide an example state.yaml file to help reproduce?

Do note that ACLs do not update, they only add/remove. (so a change to an ACL should have one create and one delete).

@jaykatti
Copy link
Author

jaykatti commented Aug 31, 2021 •
edited
Loading

Hello Shawn @devshawn , Thanks for your response !

For the below sample state.yaml, if I have to change the permission for describe_configs_topic to "DENY". I change the permission to "DENY" for that block and then run the kafka-gitops command for validate, plan.
When I run the plan, I see that, the plan indicates "Delete" or "Remvove" for all the ACL's present in my system and not just the ones in the yaml file.

customUserAcls:
User:kattijay:
describe_configs_topic:
name: topic1
pattern: LITERAL
host: ""
permission: ALLOW
type: TOPIC
operation: DESCRIBE_CONFIGS
describe_topic:
name: topic1
pattern: LITERAL
host: ""
permission: ALLOW
type: TOPIC
operation: DESCRIBE
write_topic:
name: topic1
pattern: LITERAL
host: ""
permission: ALLOW
type: TOPIC
operation: WRITE
read_topic:
name: topic1
pattern: LITERAL
host: ""
permission: ALLOW
type: TOPIC
operation: READ

@jaykatti
Copy link
Author

host above is * for all the records.

@jaykatti
Copy link
Author

Hello @devshawn , Any further update for me on this please ?

@jaykatti
Copy link
Author

Hi @jaykatti, can you provide an example state.yaml file to help reproduce?

Do note that ACLs do not update, they only add/remove. (so a change to an ACL should have one create and one delete).

I have generated a yaml (initial state) for my existing cluster. Now, this includes a large list of users and ACL's. I will use this as my base configuration and start modifying it based on the requirement. Either update an ACL or delete or Add a new one.
Say, I want to add a new ACL to the existing list, How do I go about this ? I do not want any of my existing ACL's to be deleted.

Kindly advise at the earliest as I need to get this wrapped up and start using it on our cluster.

Thanks,
Jay.

@devshawn
Copy link
Owner

Hi @jaykatti,

You have to have every ACL listed in your state file. So you need to take your existing cluster and make an initial state file with all topics / ACLs until you run a plan and it says there are no changes. Then, you can start adding/removing ACLs and it will only have the changes you want.

Or am I misunderstanding the situation?

@jaykatti
Copy link
Author

jaykatti commented Sep 21, 2021 via email

@fireydagostino
Copy link

Hi @jaykatti

You're stating a customUserAcls in the above text - do you have a matching users in your state.yaml file as well?

Every customUserAcls and customServiceAcls is connected to a matching users and service respectively.

If you're only claiming the customUserAcls in your state file without the matching users definition, then the plan will disregard the customUserAcls listed in your state file.

For example, if I were to add a customUserAcls named Access_Bob then I'd have in my state file:

users:
  Access_Bob:
    principal: User:Bob
customUserAcls:
  Access_Bob:
    read-kstream-topics:
      name: kstream-
      type: TOPIC
      pattern: PREFIXED
      host: "*"
      operation: READ
      permission: ALLOW

@jaykatti
Copy link
Author

Hi @jaykatti

You're stating a customUserAcls in the above text - do you have a matching users in your state.yaml file as well?

Every customUserAcls and customServiceAcls is connected to a matching users and service respectively.

If you're only claiming the customUserAcls in your state file without the matching users definition, then the plan will disregard the customUserAcls listed in your state file.

For example, if I were to add a customUserAcls named Access_Bob then I'd have in my state file:

users:
  Access_Bob:
    principal: User:Bob
customUserAcls:
  Access_Bob:
    read-kstream-topics:
      name: kstream-
      type: TOPIC
      pattern: PREFIXED
      host: "*"
      operation: READ
      permission: ALLOW

Oh !! Okay, Let me try that.

Thanks !, I will keep you posted.

@jaykatti jaykatti reopened this Sep 28, 2021
@jaykatti
Copy link
Author

Sorry, closed by mistake

@jaykatti
Copy link
Author

Hi @jaykatti
You're stating a customUserAcls in the above text - do you have a matching users in your state.yaml file as well?
Every customUserAcls and customServiceAcls is connected to a matching users and service respectively.
If you're only claiming the customUserAcls in your state file without the matching users definition, then the plan will disregard the customUserAcls listed in your state file.
For example, if I were to add a customUserAcls named Access_Bob then I'd have in my state file:

users:
  Access_Bob:
    principal: User:Bob
customUserAcls:
  Access_Bob:
    read-kstream-topics:
      name: kstream-
      type: TOPIC
      pattern: PREFIXED
      host: "*"
      operation: READ
      permission: ALLOW

Oh !! Okay, Let me try that.

Thanks !, I will keep you posted.

Hello @fireydagostino @devshawn

I tried with that and it seems to be working. So many thanks !

Do we also need to list all the topics as part of the state.yaml ?
I see, when I run the plan, it is marking the topics for deletion, even though, there are ACL's for those topics.
Kindly confirm !

Thanks,
Jay.

@fireydagostino
Copy link

fireydagostino commented Sep 28, 2021 •
edited
Loading

Yeah @jaykatti

The state file literally claims everything that should be present within the current state of your Kafka cluster.

Any topics/ACLs that are on Kafka, but not defined for in your state will be privy to deletion upon execution - and respectively the opposite for topics/Acls defined in your state but not present on Kafka.

Your state.yaml file will need

  • default options / blacklisted topic prefixes
  • topics
  • users + customUserAcl
  • services + customServiceAcl

In my production environment, we have some of these separated into specific directories/files pertaining to different teams/technologies and then utilize yq4 to merge all the information into the state.yaml file.

@jaykatti
Copy link
Author

jaykatti commented Sep 28, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@devshawn @fireydagostino @jaykatti