kustomize fn to transform images (#345)

* kustomize fn to transform images

* This PR defines a kustomize/kpt function to transform images based on their prefix.

  * This is an alternative to using kfctl mirror overwrite

  * This works as follows.

    i) you define a function config like the following

       ```
        apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
     kind: ImagePrefix
     metadata:
       name: use-mirror-images-gcr
       annotations:
         config.kubernetes.io/function: |
           container:
             image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
     spec:
       imageMappings:
       - src: quay.io/jetstack
         dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
       - src: gcr.io/kubeflow-images-public
         dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
       ```

   ii) You use kpt to appy the function e.g.

       ```
       kpt fn ${DIR}
       ```

       * Typically you would apply this to the hydrated manfiests
       * The transform will look for any objects with `image-prefix.kubeflow.org` annotation
         and then map the image prefixes specified in that annotation

* The kpt function has a number of advantages over the kfctl functionality

  * In particular the transform configuration (i.e. the annotation) doesn't need to change
    when a user upgrades and the image tags change. Since the transform is applied after
    hydration we would end up applying the transform to any docker images

  * `kpt fns` are more scalable then baking all the functionality into a single go binary

    * It also looks like this functionality will be upstreamed into kustomize

* kubeflow/manifests#1033 proposed a similar transformer for images.

  * This is different only in we are using kpt/kustomize functions rather than kustomize transformations
  * kpt functions are more flexible because its YAML in YAML out; so you could just as easily

* Trigger kustomize-fns unittests.

* Revert go.mod changes.

* Address comments.

kustomize-fns/.gitignore

@@ -0,0 +1,3 @@
+# Prevent people from checking in the result of go build ./
+kustomize-fns
+remove-namespace/remove-namespace

kustomize-fns/Dockerfile

@@ -0,0 +1,21 @@
+# Copyright 2019 The Kubernetes Authors.
+# SPDX-License-Identifier: Apache-2.0
+
+# TODO(jlewi): We end up pulling go.mod for all of kfctl which pulls in a lot of dependencies
+# which slows down builds. That's pretty unnecessary. The only code shared with kfctl is
+# some test utilities in particular utils.PrintDiff it might be worthmaking the test package
+# its own go module so we don't have to pull in so many dependencies just to build
+# our kustomize functions.
+
+FROM golang:1.13-stretch
+ENV CGO_ENABLED=0
+WORKDIR /go/src/
+COPY go.mod .
+COPY go.sum .
+RUN go mod download
+COPY . .
+RUN go build -v -o /usr/local/bin/config-function ./
+
+FROM alpine:latest
+COPY --from=0 /usr/local/bin/config-function /usr/local/bin/config-function
+CMD ["config-function"]

kustomize-fns/OWNERS

@@ -1,3 +1,2 @@
-
 approvers:
   - jlewi

kustomize-fns/README.md

@@ -1,13 +1,78 @@
-# Kustomize FNs
+# kpt/kustomize FNs
 
-This directory contains kustomize functions ala
-https://github.com/kubernetes-sigs/kustomize/tree/master/functions/examples/injection-tshirt-sizes
+This directory contains [kpt functions](https://googlecontainertools.github.io/kpt/reference/fn/)
+for apply various transformations to your YAMLs.
 
-Functions are away to define custom transformations
-that can be applied via kustomize.
+These kustomizations are intended to be applied using kpt. It looks like support for functions
+is being upstreamed into kustomize and currently has alpha support.
 
-## Warning this is experimental
+## Usage 
 
-The use of kustomize functions is still experimental.
+The [kpt docs](https://googlecontainertools.github.io/kpt/reference/fn/run/#examples) describe
+many ways to invoke kpt functions. For production the most common use will be the 
+[declarative method](https://googlecontainertools.github.io/kpt/reference/fn/run/#declaratively-run-one-or-more-functions).
 
-The use of kustomize functions is still considered Alpha in kustomize.
+1. In the directory you want to apply the transform define a config specifying the transform
+
+   * Depending on the transform this will either be a [ConfigMap](https://googlecontainertools.github.io/kpt/reference/fn/run/#declaratively-run-one-or-more-functions)
+     or a custom resource
+
+   * In both cases the annotation `config.kubernetes.io/function` specifies the
+     docker image to use for the config.
+
+   * As an example for the image prefix transform we define a config like the following
+
+     ```
+     apiVersion: v1alpha1 # Define a transform to change all the image prefixes to use images from a different registry
+     kind: ImagePrefix
+     metadata:
+       name: use-mirror-images-gcr
+       annotations:
+         config.kubernetes.io/function: |
+           container:
+             image: gcr.io/kubeflow-images-public/kpt-fns:v1.0-rc.3-58-g616f986-dirty
+     spec:
+       imageMappings:
+       - src: quay.io/jetstack
+         dest: gcr.io/gcp-private-dev/jetstack # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
+       - src: gcr.io/kubeflow-images-public
+         dest: gcr.io/gcp-private-dev # {"type":"string","x-kustomize":{"setBy":"kpt","partialSetters":[{"name":"gcloud.core.project","value":"gcp-private-dev"}]}}
+     ```
+     
+1. Run kpt fn
+
+   ```
+   kpt fn ${DIR}
+   ```  
+
+   * ${DIR} will be the directory containing the YAML files to process and should include
+     the YAMLs configuring the functions
+
+## Development
+
+During development it can be useful to run your transforms without building a docker image or
+run them in a debugger.
+
+To support this our main binary `kustomize-fns` includes a `debug` subcommand which
+allows you to specify the path of a YAML file containing a `ResourceList` defining
+the resources and functions to apply. You can produce that file using kpt; e.g.
+
+```
+  kpt fn source ${DIR} --function-config=${FN_CFG} 
+```
+
+ * *DIR* should be the directory containing the YAML files defining the resources to process
+ * *FN_CFG* should be a path to a YAML file specifying one or more functions to apply
+
+
+You can use skaffold to build the docker image
+
+```
+skaffold build --kube-context=${KUBE_CONTEXT}
+```
+
+You can build the binary by doing
+
+```
+go build .
+```

kustomize-fns/go.mod

@@ -0,0 +1,10 @@
+module github.com/kubeflow/kfctl/kustomize-fns
+
+go 1.13
+
+require (
+	github.com/pkg/errors v0.8.0
+	github.com/spf13/cobra v1.0.0
+	k8s.io/apimachinery v0.18.3
+	sigs.k8s.io/kustomize/kyaml v0.1.13
+)