[Epic] Kubernetes Operator Core Functionality Enhancements (PKOv2)

[rquitales]

Overview

Usage of the Pulumi Kubernetes Operator (PKO) has high friction due to a number of challenges for users. There are numerous performance enhancements that could be adopted by PKO, metrics and logging can sometimes be lacking as well as multi-tenancy concerns. We believe that targeted investments in this project can improve the user experience for users materially. By strengthening our core functionality, we can confidently embark on more ambitious innovations within PKO in the future.

Effort: 5 eng weeks

Key KPIs

1. Numerous top-voted enhancement requests can be closed, which pertain to scalability and isolation concerns.
2. Significantly improved stack reconciliation performance, esp. in use cases involving numerous stacks, long-running or complex stacks, slow-changing dependencies and/or source code, and period reconciliations.
3. multi-tenancy support within a Kubernetes cluster; stacks are significantly more isolated across namespaces, using a dedicated execution environment, no cross-namespace secret exposure, and non-root execution.

Key Stakeholders

Engineers: @EronWright

Approvers: @mjeffryes

Key Deliverables

1. An RPC server providing an "automation" API for stack operations.
2. A Kubernetes operator providing low-level automation primitives - a Workspace to provision an execution environment for a given program, an Update to drive one deployment operation against a given workspace, etc.
3. A revised Stack operator that use the lower-level primitives to reconcile its stacks. The Stack API may see some changes to accommodate the architecture change.
4. A new "v2 beta" release of the operator, with supporting materials (blog post, migration guide).

References 📔

• Project View
• PR/FAQ
• Design Doc
• UX Designs
• Decision Log

[cleverguy25]

Adding item #606

[cleverguy25]

Adding item #607

[cleverguy25]

Adding item #606

[cleverguy25]

Adding item #612

[cleverguy25]

Adding item #613

[cleverguy25]

Adding item #573

[cleverguy25]

Adding item #576

[cleverguy25]

Adding item #607

[cleverguy25]

Adding item #608

[cleverguy25]

Adding item #609

[cleverguy25]

Adding item #610

[cleverguy25]

Adding item #611

[cleverguy25]

Adding item #732

[cleverguy25]

Adding item #733

[cleverguy25]

Adding item #582

[cleverguy25]

Adding item #425

[cleverguy25]

Adding item #396

[cleverguy25]

Adding item #393

[cleverguy25]

Adding item #735

[cleverguy25]

Adding item #736

[cleverguy25]

Adding item #737

[cleverguy25]

Adding item #738

[cleverguy25]

Adding item #739

[cleverguy25]

Adding item #740

[EronWright]

This is done and shipped as PKOv2 (v2.0.0-beta.1) and accompanying blog post. We closed approximately a dozen outstanding issues. We reached out to known users of PKO, and were contacted by others, to gather some feedback in preparation for a "GA" release to come soon.

Here's a tracking issue for potential follow-ups:
#734