feat: allow chart deploy overrides ALPHA

adr/0023-chart-override.md

@@ -0,0 +1,65 @@
+# 23. Simplifying Helm Chart Value Overrides
+
+Date: 2024-03-25
+
+## Status
+
+Accepted
+
+
+## Context
+
+The process of deploying applications with Helm charts in Kubernetes environments often necessitates the customization of chart values to align with specific operational or environmental requirements. The current method for customizing these values—either through manual edits or `###ZARF_VAR_XYZ###`. A more streamlined approach would greatly enhance the deployment experience by offering both flexibility and reliability.
+
+## Decision
+
+To address this issue, we propose the introduction of a feature designed to simplify the process of overriding chart values at the time of deployment. This feature would allow users to easily specify overrides for any chart values directly via command-line arguments, eliminating the need to alter the chart's default values file or manage multiple command-line arguments for each override.
+
+Key aspects of the proposed implementation include:
+- Use existing `--set`  flags to specify overrides for chart values.
+- The ability to list all overrides in a structured and easily understandable format within `zarf.yaml`.
+- Ensuring that during deployment, these specified overrides take precedence over the chart's default values, thus facilitating customized deployments without necessitating permanent modifications to the chart.
+
+## Consequences
+
+Adopting this feature would lead to several key improvements:
+- **Streamlined Configuration Process**: Allowing helm values overrides in the zarf package schema simplifies the user experience by reducing the reliance on extensive custom `###ZARF_VAR_XYZ###` templating and aligning more closely with standard Helm practices
+
+Ultimately, this feature is aimed at enhancing the deployment workflow by offering a straightforward and efficient means of customizing Helm chart deployments via command-line inputs.
+
+## Example Configuration
+
+Below is an example of how the `zarf.yaml` configuration file might be structured to utilize the new override feature for Helm chart values:
+
+```yaml
+kind: ZarfPackageConfig
+metadata:
+  name: helm-charts
+  description: Example showcasing multiple ways to deploy helm charts
+  version: 0.0.1
+
+components:
+  - name: demo-helm-charts
+    required: true
+    charts:
+      - name: podinfo-local
+        version: 6.4.0
+        namespace: podinfo-from-local-chart
+        localPath: chart
+        valuesFiles:
+          - values.yaml
+        variables:
+          - name: REPLICA_COUNT
+            description: "Override the number of pod replicas"
+            path: replicaCount
+```
+This configuration allows for the specification of default values and descriptions for variables that can be overridden at deployment time. The variables section under each chart specifies the variables that can be overridden, along with a path that indicates where in the values file the variable is located.
+
+### Command Line Example
+
+To override the `REPLICA_COUNT` variable at deployment time, the following command can be used:
+
+```bash
+zarf package deploy zarf-package-helm-charts-arm64-0.0.1.tar.zst --set REPLICA_COUNT=5
+```
+This command demonstrates how users can easily customize their Helm chart deployments by specifying overrides for chart values directly via command-line arguments, in line with the proposed feature.

docs/3-create-a-zarf-package/4-zarf-schema.md

@@ -1185,6 +1185,95 @@ Must be one of:
 </blockquote>
 </details>
 
+<details open>
+<summary>
+<strong> <a name="components_items_charts_items_variables"></a>variables</strong>
+</summary>
+&nbsp;
+<blockquote>
+
+  ## components > charts > variables
+
+**Description:** [alpha] List of variables to set in the Helm chart
+
+|          |         |
+| -------- | ------- |
+| **Type** | `array` |
+
+![Min Items: N/A](https://img.shields.io/badge/Min%20Items%3A%20N/A-gold)
+![Max Items: N/A](https://img.shields.io/badge/Max%20Items%3A%20N/A-gold)
+![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
+![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
+
+ ### <a name="autogenerated_heading_13"></a>ZarfChartVariable  
+
+|                           |                                                                                                          |
+| ------------------------- | -------------------------------------------------------------------------------------------------------- |
+| **Type**                  | `object`                                                                                                 |
+| **Additional properties** | [![Not allowed](https://img.shields.io/badge/Not%20allowed-red)](# "Additional Properties not allowed.") |
+| **Defined in**            | #/definitions/ZarfChartVariable                                                                          |
+
+<details>
+<summary>
+<strong> <a name="components_items_charts_items_variables_items_name"></a>name *</strong>
+</summary>
+&nbsp;
+<blockquote>
+
+![Required](https://img.shields.io/badge/Required-red)
+
+**Description:** The name of the variable
+
+|          |          |
+| -------- | -------- |
+| **Type** | `string` |
+
+| Restrictions                      |                                                                               |
+| --------------------------------- | ----------------------------------------------------------------------------- |
+| **Must match regular expression** | ```^[A-Z0-9_]+$``` [Test](https://regex101.com/?regex=%5E%5BA-Z0-9_%5D%2B%24) |
+
+</blockquote>
+</details>
+
+<details>
+<summary>
+<strong> <a name="components_items_charts_items_variables_items_description"></a>description *</strong>
+</summary>
+&nbsp;
+<blockquote>
+
+![Required](https://img.shields.io/badge/Required-red)
+
+**Description:** A brief description of what the variable controls
+
+|          |          |
+| -------- | -------- |
+| **Type** | `string` |
+
+</blockquote>
+</details>
+
+<details>
+<summary>
+<strong> <a name="components_items_charts_items_variables_items_path"></a>path *</strong>
+</summary>
+&nbsp;
+<blockquote>
+
+![Required](https://img.shields.io/badge/Required-red)
+
+**Description:** The path within the Helm chart values where this variable applies
+
+|          |          |
+| -------- | -------- |
+| **Type** | `string` |
+
+</blockquote>
+</details>
+
+</blockquote>
+</details>
+
 </blockquote>
 </details>
 
@@ -1208,7 +1297,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_13"></a>ZarfDataInjection  
+ ### <a name="autogenerated_heading_14"></a>ZarfDataInjection  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -1371,7 +1460,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_14"></a>ZarfFile  
+ ### <a name="autogenerated_heading_15"></a>ZarfFile  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -1465,7 +1554,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_15"></a>symlinks items  
+ ### <a name="autogenerated_heading_16"></a>symlinks items  
 
 |          |          |
 | -------- | -------- |
@@ -1511,7 +1600,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_16"></a>images items  
+ ### <a name="autogenerated_heading_17"></a>images items  
 
 |          |          |
 | -------- | -------- |
@@ -1538,7 +1627,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_17"></a>repos items  
+ ### <a name="autogenerated_heading_18"></a>repos items  
 
 |          |          |
 | -------- | -------- |
@@ -1633,7 +1722,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_18"></a>valuesFiles items  
+ ### <a name="autogenerated_heading_19"></a>valuesFiles items  
 
 |          |          |
 | -------- | -------- |
@@ -1676,7 +1765,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_19"></a>fluxPatchFiles items  
+ ### <a name="autogenerated_heading_20"></a>fluxPatchFiles items  
 
 |          |          |
 | -------- | -------- |
@@ -1824,7 +1913,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_20"></a>env items  
+ ### <a name="autogenerated_heading_21"></a>env items  
 
 |          |          |
 | -------- | -------- |
@@ -1939,7 +2028,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_21"></a>ZarfComponentAction  
+ ### <a name="autogenerated_heading_22"></a>ZarfComponentAction  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -2029,7 +2118,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_22"></a>env items  
+ ### <a name="autogenerated_heading_23"></a>env items  
 
 |          |          |
 | -------- | -------- |
@@ -2094,7 +2183,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_23"></a>ZarfComponentActionSetVariable  
+ ### <a name="autogenerated_heading_24"></a>ZarfComponentActionSetVariable  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -2448,7 +2537,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_24"></a>ZarfComponentAction  
+ ### <a name="autogenerated_heading_25"></a>ZarfComponentAction  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -2479,7 +2568,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_25"></a>ZarfComponentAction  
+ ### <a name="autogenerated_heading_26"></a>ZarfComponentAction  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -2510,7 +2599,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_26"></a>ZarfComponentAction  
+ ### <a name="autogenerated_heading_27"></a>ZarfComponentAction  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -2590,7 +2679,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_27"></a>ZarfPackageConstant  
+ ### <a name="autogenerated_heading_28"></a>ZarfPackageConstant  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |
@@ -2709,7 +2798,7 @@ Must be one of:
 ![Item unicity: False](https://img.shields.io/badge/Item%20unicity%3A%20False-gold)
 ![Additional items: N/A](https://img.shields.io/badge/Additional%20items%3A%20N/A-gold)
 
- ### <a name="autogenerated_heading_28"></a>ZarfPackageVariable  
+ ### <a name="autogenerated_heading_29"></a>ZarfPackageVariable  
 
 |                           |                                                                                                          |
 | ------------------------- | -------------------------------------------------------------------------------------------------------- |

examples/helm-charts/zarf.yaml

@@ -16,6 +16,12 @@ components:
         localPath: chart
         valuesFiles:
           - values.yaml
+        # Variables are used to override the default values in the chart
+        # This can be overridden by the user at deployment time with the `--set` flag
+        variables:
+          - name: REPLICA_COUNT
+            description: "Override the number of pod replicas"
+            path: replicaCount
 
       - name: podinfo-oci
         version: 6.4.0
@@ -77,3 +83,4 @@ components:
                 name: cool-release-name-podinfo
                 namespace: podinfo-from-repo
                 condition: available
+

src/pkg/packager/deploy.go

@@ -547,24 +547,61 @@ func (p *Packager) pushReposToRepository(reposPath string, repos []string) error
 	return nil
 }
 
+// generateValuesOverrides creates a map containing overrides for chart values based on the given inputs.
+// It prioritizes the overrides in this order:
+// 1. Variables defined in the ZarfComponent, including both Set Variables and Chart Variables.
+// 2. Deployment options specified through DeployOpts.
+// This function is useful for customizing the deployment of a Helm chart by programmatically setting chart values.
+//
+// Returns:
+// - A map containing the final set of value overrides for the chart, where keys are the variable names.
+func generateValuesOverrides(chartVariables []types.ZarfChartVariable,
+	setVariableMap map[string]*types.ZarfSetVariable,
+	deployOpts types.ZarfDeployOptions,
+	componentName, chartName string) (map[string]any, error) {
+
+	valuesOverrides := make(map[string]any)
+	chartOverrides := make(map[string]any)
+
+	for _, variable := range chartVariables {
+		if setVar, ok := setVariableMap[variable.Name]; ok && setVar != nil {
+			// Use the variable's path as a key to ensure unique entries for variables with the same name but different paths.
+			if err := helpers.MergePathAndValueIntoMap(chartOverrides, variable.Path, setVar.Value); err != nil {
+				return nil, fmt.Errorf("unable to merge path and value into map: %w", err)
+			}
+		}
+	}
+
+	// Apply any direct overrides specified in the deployment options for this component and chart
+	if componentOverrides, ok := deployOpts.ValuesOverridesMap[componentName]; ok {
+		if chartSpecificOverrides, ok := componentOverrides[chartName]; ok {
+			valuesOverrides = chartSpecificOverrides
+		}
+	}
+
+	// Merge chartOverrides into valuesOverrides to ensure all overrides are applied.
+	// This corrects the logic to ensure that chartOverrides and valuesOverrides are merged correctly.
+	return helpers.MergeMapRecursive(valuesOverrides, chartOverrides), nil
+}
+
 // Install all Helm charts and raw k8s manifests into the k8s cluster.
 func (p *Packager) installChartAndManifests(componentPaths *layout.ComponentPaths, component types.ZarfComponent) (installedCharts []types.InstalledChart, err error) {
 	for _, chart := range component.Charts {
-
 		// zarf magic for the value file
 		for idx := range chart.ValuesFiles {
 			chartValueName := helm.StandardValuesName(componentPaths.Values, chart, idx)
 			if err := p.valueTemplate.Apply(component, chartValueName, false); err != nil {
 				return installedCharts, err
 			}
 		}
-
-		// TODO (@WSTARR): Currently this logic is library-only and is untested while it is in an experimental state - it may eventually get added as shorthand in Zarf Variables though
-		var valuesOverrides map[string]any
-		if componentChartValuesOverrides, ok := p.cfg.DeployOpts.ValuesOverridesMap[component.Name]; ok {
-			if chartValuesOverrides, ok := componentChartValuesOverrides[chart.Name]; ok {
-				valuesOverrides = chartValuesOverrides
-			}
+		// this is to get the overrides for the chart from the commandline and
+		// the chart variables set in the ZarfComponent and as well as DeployOpts set from the Library user.
+		// The order of precedence is as follows:
+		// 1. Set Variables and Chart Variables from the ZarfComponent
+		// 2. DeployOpts
+		valuesOverrides, err := generateValuesOverrides(chart.Variables, p.cfg.SetVariableMap, p.cfg.DeployOpts, component.Name, chart.Name)
+		if err != nil {
+			return installedCharts, err
 		}
 
 		helmCfg := helm.New(