Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 Update Config generation to use inClusterConfig #10729

Merged
merged 3 commits into from
Aug 2, 2024

Conversation

AbdullahAlShaad
Copy link
Contributor

@AbdullahAlShaad AbdullahAlShaad commented Jun 6, 2024

What this PR does / why we need it:

Currently, we cannot use clusterctl binary from inside the pod because it can not find the cluster. This PR updates config generation to use GetConfigOrDie() method from controller-runtime so that clusterctl can discover the cluster even kubeconfig is not provided.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #6286

/area clusterctl
--> area/clusterctl

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-area PR is missing an area label labels Jun 6, 2024
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jun 6, 2024
@AbdullahAlShaad AbdullahAlShaad changed the title Update Config generation to use inClusterConfig 🌱 Update Config generation to use inClusterConfig Jun 6, 2024
}
return nil, err
}
restConfig := ctrl.GetConfigOrDie()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the issue with this is, the underlying function tries to use a global kubeconfig flag, which we don't set or use.

Instead I think what we should do is use rest.InCLusterConfig() in the error case as fallback.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. Anyway, though it would take a lot of refactoring, wouldn't it be better to use kubeconfig flag?

@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 7, 2024
}
restConfig, err = rest.InClusterConfig()
if err != nil {
return nil, err
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's return an error that expresses that both the kubeconfig and the in cluster config didn't work (previosu error from l.146 + something for in cluster

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the error message to "invalid kubeconfig file and failed to create inClusterConfig as well"

@sbueringer
Copy link
Member

sbueringer commented Jun 10, 2024

I think this should be okay, but would be good to get confirmation from someone more familiar with clusterctl

Also not sure if we want to make this behavior opt-in with some additional flag or something (if we think we have to safeguard this a bit)

Signed-off-by: AbdullahAlShaad <[email protected]>
if strings.HasPrefix(err.Error(), "invalid configuration:") {
return nil, errors.New(strings.Replace(err.Error(), "invalid configuration:", "invalid kubeconfig file; clusterctl requires a valid kubeconfig file to connect to the management cluster:", 1))
if !strings.HasPrefix(err.Error(), "invalid configuration:") {
return nil, err
Copy link
Member

@enxebre enxebre Jun 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason to not use controller runtime builtin GetConfig() which handles the fallback for you? i.e.
https://github.com/kubernetes-sigs/controller-runtime/blob/1f5b39fa59d15fae78e521c9c9f2acabbbb3ea17/pkg/client/config/config.go#L60-L75

ctrconfig "sigs.k8s.io/controller-runtime/pkg/client/config"
restConfig, err := ctrconfig.GetConfig()

cc @fabriziopandini

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because GetConfig() uses a global "kubeconfig" flag which clusterctl does not set.

@sbueringer
Copy link
Member

cc @fabriziopandini

@fabriziopandini
Copy link
Member

/lgtm
the change is localized and it should not have side effects.

I defer to others when to merge this and if to cherry-pick (or backport). My two cents is to avoid both unless there are specific needs.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 2, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: c6ae222a8723175b32e841445f3cd565718fd954

@sbueringer
Copy link
Member

Alrighty. Let's merge it

I"ll cherry-pick into 1.8, because there is no 1.8 release yet, but not further back (otherwise folks would have to wait until December to get this change)

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 2, 2024
@sbueringer
Copy link
Member

/cherry-pick release-1.8

@k8s-infra-cherrypick-robot

@sbueringer: once the present PR merges, I will cherry-pick it on top of release-1.8 in a new PR and assign it to you.

In response to this:

/cherry-pick release-1.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@sbueringer
Copy link
Member

/aera clusterctl

@sbueringer sbueringer added the area/clusterctl Issues or PRs related to clusterctl label Aug 2, 2024
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/needs-area PR is missing an area label label Aug 2, 2024
@k8s-ci-robot k8s-ci-robot merged commit 3327ef7 into kubernetes-sigs:main Aug 2, 2024
21 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.9 milestone Aug 2, 2024
@k8s-infra-cherrypick-robot

@sbueringer: new pull request created: #11006

In response to this:

/cherry-pick release-1.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/clusterctl Issues or PRs related to clusterctl cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

clusterctl inside cluster in pod cannot find management cluster
7 participants