Commits on Oct 10, 2019

1. Add "volume" PipelineResource 🔊 …

   This will allow copying content either into or out of a `TaskRun`,
   either to an existing volume or a newly created volume. The immediate
   use case is for copying a pipeline's workspace to be made available as
   the input for another pipeline's workspace without needing to deal
   with uploading everything to a bucket. The volume, whether already
   existing or created, will not be deleted at the end of the
   `PipelineRun`, unlike the artifact storage PVC.
   
   The Volume resource is a sub-type of the general Storage resource.
   
   Since this type will require the creation of a PVC to function (to be
   configurable later), this commit adds a Setup interface that
   PipelineResources can implement if they need to do setup that involves
   instantiating objects in Kube. This could be a place to later add
   features like caching, and also is the sort of design we'd expect once
   PipelineResources are extensible (PipelineResources will be free to do
   whatever setup they need).
   
   The behavior of this volume resource is:
   1. For inputs, copy data _from_ the PVC to the workspace path
   2. For outputs, copy data _to_ the PVC from the workspace path
   
   If a user does want to control where the data is copied from, they can:
   1. Add a step that copies from the location they want to copy from on
      disk to /workspace/whatever
   2. Use the "targetPath" argument in the PipelineResource to control the
      location the data is copied to (still relative to targetPath
      https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#controlling-where-resources-are-mounted)
   3. Use `path` https://github.com/tektoncd/pipeline/blob/master/docs/resources.md#overriding-where-resources-are-copied-from
      (tbd if we want to keep this feature post PVC)
   
   The underlying PVC will need to be created by the Task reonciler, if
   only a TaskRun is being used, or by the PipelineRun reconciler if a
   Pipeline is being used. The PipelineRun reconciler cannot delegate this
   to the TaskRun reconciler b/c when two different reconcilers create PVCs
   and Tekton is running on a regional GKE cluster, they can get created in
   different zones, resulting in a pod that tries to use both being
   unschedulable.
   
   In order to actually schedule a pod using two volume resources, we had
   to:
   - Use a storage class that can be scheduled in a GKE regional cluster
     https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/regional-pd
   - Either use the same storage class for the PVC attached automatically
     for input/output linking or don't use the PVC (chose the latter!)
   
   This commit removes automatic PVC copying for input output linking of
   the VolumeResource b/c since it itself is a PVC, there is no need to
   copy between an intermediate PVCs. This makes it simpler to make a Task
   using the VolumeResource schedulable, removes redundant copying, and
   removes a side effect where if a VolumeResources output was linked to an
   input, the Task with the input would see _only_ the changes made by the
   output and none of the other contents of the PVC.
   
   Also removing the docs on the `paths` param (i.e. "overriding where
   resources are copied from") because it was implemented such that it
   would only work in the output -> input linking PVC case and can't
   actually be used by users and it will be removed in tektoncd#1284.
   
   fixes tektoncd#1062
   
   Co-authored-by: Dan Lorenc <[email protected]>
   Co-authored-by: Christie Wilson <[email protected]>

   

   3 people committed Oct 10, 2019
   Configuration menu

   • View commit details
   • Copy full SHA for f2d564e
   • Browse repository at this point

   Copy the full SHA

   f2d564e View commit details

   Browse the repository at this point in the history