Upgrade 1.4.1 Apps: pipelines

[bryanpaget]

Overview

EPIC: Kubeflow Upgrade Planning

Component	Local Manifests Path	Upstream	Initial Work
pipeline	apps/pipeline	v1.4.1

Adjustments

• 1.4.1

  • Upgrade Argo Workflow CRDS v3.2.6 -> v3.3.9
  • Upgrade ml-pipeline image v1.7.0 -> v1.8.4
  • Upgrade argoproj image v3.2.6 -> v3.3.9

• 1.3.1

  • Installed Argo Workflow CRDS v3.2.6 and additional Workflow Controller RBAC
  • The service.yaml tcp-http workflow-controller patch is no longer needed
  • Updated workflow-controller deployment to point to v3.2.6
  • Updated workflow-controller-configmap for emissary and default executor image
  • Updated ml-pipeline-ui environment variables for ARGO_ARCHIVE_LOGS
  • Updated images in ml-pipeline and ml-pipeline-persistenceagent deployments

Kubeflow V2 Manifests

The following is the kustomize that was used in Kubeflow V2:

• https://github.com/StatCan/aaw-kubeflow-manifests/tree/master/kustomize/application/pipeline

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
patchesStrategicMerge:
- deployment.yaml
- service.yaml
resources:
- ../../../.cache/manifests/manifests-1.2-branch/pipeline/installs/multi-user

AAW Dev / Prod Live Manifests

At the moment there is no difference in state then what is overridden above.

Note: While most everything 95% would have been automated, stored as config and is using what is referenced above. I believe a few things could have been done as manual adjustments that we should make sure we are keeping. Largely any manual yaml adjustments would have been documented in high level GitHub issues or tracked in the YAML repository under the AAW group.

Kubeflow V3 Manifests

The following is the P.R. that will be merged into the main branch for Kubeflow V3:

• feat(component): Add apps component pipeline #142

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# For a detailed summary of this Kustomization please see: https://github.com/StatCan/daaas/issues/683

# https://github.com/kubeflow/manifests/blob/master/apps/pipeline/upstream/env/platform-agnostic-multi-user/kustomization.yaml
bases:
  - github.com/kubeflow/manifests/apps/pipeline/upstream/base/installs/multi-user?ref=v1.3.1
  - github.com/kubeflow/manifests/apps/pipeline/upstream/base/metadata/base?ref=v1.3.1
  - github.com/kubeflow/manifests/apps/pipeline/upstream/base/metadata/options/istio?ref=v1.3.1

  # https://github.com/kubeflow/manifests/blob/master/apps/pipeline/upstream/third-party/argo/installs/cluster/kustomization.yaml
  - github.com/argoproj/argo-workflows/manifests/base/crds?ref=v3.2.6
  - github.com/argoproj/argo-workflows/manifests/cluster-install/workflow-controller-rbac?ref=v3.2.6
  - github.com/kubeflow/manifests/apps/pipeline/upstream/third-party/argo/base?ref=v1.3.1

  - github.com/kubeflow/manifests/apps/pipeline/upstream/third-party/mysql/base?ref=v1.3.1
  - github.com/kubeflow/manifests/apps/pipeline/upstream/third-party/mysql/options/istio?ref=v1.3.1
  - github.com/kubeflow/manifests/apps/pipeline/upstream/third-party/minio/base?ref=v1.3.1
  - github.com/kubeflow/manifests/apps/pipeline/upstream/third-party/minio/options/istio?ref=v1.3.1
  - github.com/kubeflow/manifests/apps/pipeline/upstream/third-party/metacontroller/base?ref=v1.3.1

# Identifier for application manager to apply ownerReference.
# The ownerReference ensures the resources get garbage collected
# when application is deleted.
commonLabels:
  application-crd-id: kubeflow-pipelines

# !!! If you want to customize the namespace,
# please also update base/cache-deployer/cluster-scoped/cache-deployer-clusterrolebinding.yaml
namespace: kubeflow

patchesStrategicMerge:
- ml-pipeline-ui.yaml
- workflow-controller-configmap-patch.yaml
- workflow-controller-deployment-patch.yaml

patches:
- path: ml-pipeline-config.json
  target:
    kind: Deployment
    name: ml-pipeline
    version: v1

- path: ml-pipeline-persistenceagent-config.json
  target:
    kind: Deployment
    name: ml-pipeline-persistenceagent
    version: v1

patchesJson6902:
- target:
    group: rbac.authorization.k8s.io
    version: v1
    kind: ClusterRoleBinding
    name: argo-binding
  path: workflow-controller-clusterrolebinding-patch.json

Testing

Usually a good idea to make sure all of the overrides are working is to run the following command and verify all of the yaml output for the component is what you expect and all of the overrides are taken into account.

• Installed Argo Workflow CRDS v3.2.6 and additional Workflow Controller RBAC successfully
• Updated workflow-controller deployment to point to v3.2.6 successfully
• Updated workflow-controller-configmap for emissary and default executor image successfully
• Updated ml-pipeline-ui environment variables for ARGO_ARCHIVE_LOGS successfully
• Updated images in ml-pipeline and ml-pipeline-persistenceagent deployments successfully

task stack:aaw:preview

Note: The command above will render all of the manifests into manifests top level folder with the name aaw.yaml. A trick to keep the yaml output small is under stacks/aaw/kustomization.yaml to only have the component you wish to test referenced.