Kustomize transformer to change image registries - to support image mirror #1033
Comments
|
Issue-Label Bot is automatically applying the labels:
Please mark this comment with 👍 or 👎 to give our bot feedback! |
jlewi
pushed a commit
to jlewi/kfctl
that referenced
this issue
Jun 1, 2020
…ependencies
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you add an annotation "image-prefix.kubeflow.org" that contains a map of image
prefixes to change e.g.
```
annotations:
"image-prefix.kubeflow.org": '{"quay.io/jetstack": "gcr.io/$(PROJECT)/quay.io/jetstack", "metacontroller": "gcr.io/$(PROJECT)/metacontroller"}'
```
* The annotation can be added however you want; e.g. using kustomize's annotations transformer
ii) You use kpt to appy the function e.g.
```
kpt fn run $(BUILD_DIR) --image=gcr.io/kubeflow-images-public/kustomize-fns/image-prefix@sha256:7c5b8e0834fc06a2a254307046a32266b0f22256ab34fc12e4beda827b02b628
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
apply it to unhydrated packages assuming the annotations have already been applied.
* In follow on PRs to kubeflow/manifests we can add the annotation "image-prefix.kubeflow.org" and use a kpt setter to make it easy to consistently set across multiple images.
* The current implementation of kfctl mirror was not looking
at dependent packages that were out of tree. This change
causes the mirror command to look at the resources directive
inside the kustomization.yaml and follow those links.
* Fix kubeflow#340 kfctl mirror shouldn't hard code the directory to search
for images but instead allow it to be provided via a flag.
* Refactor the utility to pretty print files so we can reuse it across
tests
* gofmt remove-namespace
jlewi
pushed a commit
to jlewi/kfctl
that referenced
this issue
Jun 2, 2020
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
jlewi
pushed a commit
to jlewi/kfctl
that referenced
this issue
Jun 4, 2020
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
jlewi
pushed a commit
to jlewi/kfctl
that referenced
this issue
Jun 4, 2020
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
k8s-ci-robot
pushed a commit
to kubeflow/kfctl
that referenced
this issue
Jun 4, 2020
* kustomize fn to transform images
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
* Trigger kustomize-fns unittests.
* Revert go.mod changes.
* Address comments.
|
This is fixed by #345 which adds a kpt function to perform the transformation. |
vpavlin
pushed a commit
to vpavlin/kfctl
that referenced
this issue
Jul 10, 2020
* kustomize fn to transform images
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
* Trigger kustomize-fns unittests.
* Revert go.mod changes.
* Address comments.
vpavlin
pushed a commit
to vpavlin/kfctl
that referenced
this issue
Jul 20, 2020
* kustomize fn to transform images
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
* Trigger kustomize-fns unittests.
* Revert go.mod changes.
* Address comments.
vpavlin
pushed a commit
to vpavlin/kfctl
that referenced
this issue
Jul 22, 2020
* kustomize fn to transform images
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
* Trigger kustomize-fns unittests.
* Revert go.mod changes.
* Address comments.
vpavlin
pushed a commit
to vpavlin/kfctl
that referenced
this issue
Jul 22, 2020
* kustomize fn to transform images
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
* Trigger kustomize-fns unittests.
* Revert go.mod changes.
* Address comments.
crobby
pushed a commit
to crobby/kfctl
that referenced
this issue
Feb 25, 2021
* kustomize fn to transform images
* This PR defines a kustomize/kpt function to transform images based on their prefix.
* This is an alternative to using kfctl mirror overwrite
* This works as follows.
i) you define a function config like the following
```
apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
kind: ImagePrefix
metadata:
name: use-mirror-images-gcr
annotations:
config.kubernetes.io/function: |
container:
image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
spec:
imageMappings:
- src: quay.io/jetstack
dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
- src: gcr.io/kubeflow-images-public
dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
```
ii) You use kpt to appy the function e.g.
```
kpt fn ${DIR}
```
* Typically you would apply this to the hydrated manfiests
* The transform will look for any objects with `image-prefix.kubeflow.org` annotation
and then map the image prefixes specified in that annotation
* The kpt function has a number of advantages over the kfctl functionality
* In particular the transform configuration (i.e. the annotation) doesn't need to change
when a user upgrades and the image tags change. Since the transform is applied after
hydration we would end up applying the transform to any docker images
* `kpt fns` are more scalable then baking all the functionality into a single go binary
* It also looks like this functionality will be upstreamed into kustomize
* kubeflow/manifests#1033 proposed a similar transformer for images.
* This is different only in we are using kpt/kustomize functions rather than kustomize transformations
* kpt functions are more flexible because its YAML in YAML out; so you could just as easily
* Trigger kustomize-fns unittests.
* Revert go.mod changes.
* Address comments.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We'd like to use kustomize to rewrite the registries of certain images. We currently do this with a custom logic in kfctl (see kubeflow/kfctl#3210).
A strawman would be to add a custom transformer that would take a kustomization.yaml like the following
When applied the transformer would then iterate over all K8s resources and apply the transformation to the image fields as necessary.
This is what our kfctl go binary is doing today to support pulling the Kubeflow images from a mirror.
The limitation of the current approach is that kfctl does this by rewriting the kustomization.yaml file and adding an image transform entry for every image. The disadvantage of this approach is that for upgrades, if someone updates the base package they have to rerun kfctl to update all the images. Whereas using a kustomize transformer they would just have to run kustomize build.
We should file an issue on kustomize to see if there is any interest in making this a built in feature.
The text was updated successfully, but these errors were encountered: