Add `ValuesFiles` to HelmChart spec #305

arbourd · 2021-03-08T23:31:41Z

Allows the use of more than values file to be declared and used in a HelmRelease, HelmChart.

Closes #291

controllers/helmchart_controller.go

arbourd · 2021-03-08T23:55:54Z

I can recreate this test failure locally at least. I'll need to invest some time on getting that suite up and running properly so I can see what is causing the, I assume, CrashLoopBackOff.

controllers/helmchart_controller.go

hiddeco

Thank you for submitting this @arbourd 🥇

Did a quick first round of review with some minor comments.

api/v1beta1/helmchart_types.go

controllers/helmchart_controller.go

api/v1beta1/helmchart_types.go

controllers/helmchart_controller.go

api/v1beta1/helmchart_types.go

relu · 2021-03-10T10:59:30Z

Nice work so far, @arbourd! I like the idea of having the ability to use multiple override values files combined.

api/v1beta1/helmchart_types.go

controllers/helmchart_controller_test.go

controllers/helmchart_controller.go

controllers/helmchart_controller_test.go

arbourd · 2021-04-04T19:43:05Z

@relu and @hiddeco: I will need your help understanding these tests please 🙏

relu · 2021-04-06T16:16:18Z

I'm looking into this and will get back to you if I can figure it out.

controllers/helmchart_controller_test.go

arbourd · 2021-04-07T04:05:25Z

Because the test suite is not working this PR is now blocked by #332

arbourd · 2021-04-11T05:02:56Z

controllers/helmchart_controller.go

+			if v == "values.yaml" {
+				valuesMap = transform.MergeMaps(valuesMap, helmChart.Values)
+				continue
+			}


Default values are not available from the filesystem, only via helmChart.Values

Please note this @relu

arbourd · 2021-04-11T15:27:54Z

Okay so finally this passes all tests, for both Git repositories and Helm repositories.

As I was working on it last night, I had a realization that way we handle default values is problematic. I think we should always merge the default values. For most Helm users, helm install -f values-prod.yaml naturally implies that it will be merged with the default values. Flux does this differently, noting that it won't. This behaviour also diverges from other CloudNative CD platforms like ArgoCD, where valuesFiles represents a list of overrides.

I think that if someone is expecting a fully-declarative (no merging) values override they should implement the best practise of duplicating it exactly (as Bitnami used to do for example).

@hiddeco, @relu -- thoughts?

controllers/helmchart_controller.go

hiddeco · 2021-04-12T11:11:57Z

As I was working on it last night, I had a realization that way we handle default values is problematic. I think we should always merge the default values. For most Helm users, helm install -f values-prod.yaml naturally implies that it will be merged with the default values. Flux does this differently, noting that it won't.

This behavior was chosen because we also have a ValuesFrom on the HelmRelease that behaves more like the -f flag from Helm, but I can (now) see how this from a user's perspective doesn't make much sense.

No strong opinion on (not) changing this, so would welcome @relu's thoughts.

arbourd · 2021-04-15T13:10:15Z

@relu, any thoughts on this?

Also, @relu and @hiddeco are there any outstanding issues other than that discussion that are left on this PR? Would love to see this in the next release if possible. 😍

relu · 2021-04-15T14:33:43Z

The way I see it is that ValuesFrom on the HelmRelease resource refers to overrides (like you would use different values in different environments), while ValuesFiles on HelmChart would be used to package the chart with some defaults that may or may not be overridden in the HelmRelease using it. Considering this, I would think ideally we should leave it up to the user to decide what goes in or not into the re-packaged chart, essentially enabling the user to fully replace the default values.yaml.

@arbourd you mentioned that Argo and other tools are doing it differently, I don't think that's the case. Argo also has the ability to supply multiple values files at the level of the release, while we support that as well since ValuesFrom on the HelmRelease` resource accepts a list of override files.

arbourd · 2021-04-15T14:58:29Z

We use ArgoCD in production right now and this what the production application CRD looks like:

spec:
  destination:
    namespace: default
  source:
    path: ./path-to-chart
    repoURL: [email protected]:org/repo.git
    targetRevision: HEAD
    helm:
      valueFiles:
        - values-production.yaml

The default values are implicitly included in the helm template command that ArgoCD calls.

The trade off for this additional flexibility (being able to exclude the default values) is a difference in expectations from Helm itself, which may lead to confusion from Helm users new to Flux.

That being said, I also don't have a strong opinion on this. I brought it up on this PR because if we were to change it, this would be the time (already changing the spec).

hiddeco

This looks OK to me, thank you very much @arbourd 🥇

Signed-off-by: Dylan Arbour <[email protected]>

arbourd · 2021-04-19T13:22:53Z

Let me know if you need anything else to make it more than OK 😁

relu

All good from my side as well 🙌

hiddeco · 2021-04-19T13:34:00Z

There are two additional things @arbourd:

The field must be added to the template in the helm-controller API so that it can be configured by users there.
It should probably be documented on the website and in the migration guide, this documentation lives (for now) in fluxcd/flux2.

arbourd · 2021-04-19T13:42:49Z

Do we need to prerelease of the source-controller for that @hiddeco?

hiddeco · 2021-04-19T13:54:41Z

@arbourd you can create a PR that works with an API dependency from main, then change it to a released version once it exists.

arbourd commented Mar 8, 2021

View reviewed changes

controllers/helmchart_controller.go Outdated Show resolved Hide resolved

arbourd commented Mar 9, 2021

View reviewed changes

controllers/helmchart_controller.go Outdated Show resolved Hide resolved

hiddeco reviewed Mar 10, 2021

View reviewed changes

relu requested changes Mar 10, 2021

View reviewed changes

controllers/helmchart_controller.go Outdated Show resolved Hide resolved

api/v1beta1/helmchart_types.go Outdated Show resolved Hide resolved

hiddeco added area/helm Helm related issues and pull requests enhancement New feature or request labels Mar 10, 2021

hiddeco reviewed Mar 13, 2021

View reviewed changes

api/v1beta1/helmchart_types.go Outdated Show resolved Hide resolved

arbourd requested review from relu and hiddeco March 16, 2021 23:31

relu reviewed Mar 17, 2021

View reviewed changes

controllers/helmchart_controller_test.go Outdated Show resolved Hide resolved

arbourd mentioned this pull request Mar 29, 2021

Multiple valuesFiles in HelmChart source #291

Closed

arbourd commented Apr 3, 2021

View reviewed changes

controllers/helmchart_controller.go Show resolved Hide resolved

arbourd commented Apr 4, 2021

View reviewed changes

controllers/helmchart_controller_test.go Outdated Show resolved Hide resolved

relu requested changes Apr 6, 2021

View reviewed changes

controllers/helmchart_controller_test.go Outdated Show resolved Hide resolved

relu mentioned this pull request Apr 6, 2021

Fix HelmChart values tests #332

Merged

arbourd requested a review from relu April 9, 2021 16:56

arbourd marked this pull request as ready for review April 10, 2021 03:14

arbourd commented Apr 11, 2021

View reviewed changes

hiddeco reviewed Apr 12, 2021

View reviewed changes

controllers/helmchart_controller.go Outdated Show resolved Hide resolved

hiddeco approved these changes Apr 19, 2021

View reviewed changes

Add ValuesFiles to HelmChart spec

4a834e1

Signed-off-by: Dylan Arbour <[email protected]>

relu approved these changes Apr 19, 2021

View reviewed changes

hiddeco merged commit f56c96f into fluxcd:main Apr 19, 2021

arbourd deleted the values-files branch April 19, 2021 13:34

This was referenced Apr 19, 2021

Add ValuesFiles to HelmChart spec fluxcd/helm-controller#252

Merged

Add ValuesFiles documentation fluxcd/flux2#1296

Merged

hiddeco mentioned this pull request Apr 21, 2021

Release v0.12.0 #347

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `ValuesFiles` to HelmChart spec #305

Add `ValuesFiles` to HelmChart spec #305

arbourd commented Mar 8, 2021 •

edited

Loading

arbourd commented Mar 8, 2021

hiddeco left a comment

relu commented Mar 10, 2021

arbourd commented Apr 4, 2021 •

edited

Loading

relu commented Apr 6, 2021

arbourd commented Apr 7, 2021

arbourd Apr 11, 2021

arbourd Apr 15, 2021

arbourd commented Apr 11, 2021 •

edited

Loading

hiddeco commented Apr 12, 2021

arbourd commented Apr 15, 2021

relu commented Apr 15, 2021

arbourd commented Apr 15, 2021 •

edited

Loading

hiddeco left a comment

arbourd commented Apr 19, 2021

relu left a comment

hiddeco commented Apr 19, 2021

arbourd commented Apr 19, 2021

hiddeco commented Apr 19, 2021

Add ValuesFiles to HelmChart spec #305

Add ValuesFiles to HelmChart spec #305

Conversation

arbourd commented Mar 8, 2021 • edited Loading

arbourd commented Mar 8, 2021

hiddeco left a comment

Choose a reason for hiding this comment

relu commented Mar 10, 2021

arbourd commented Apr 4, 2021 • edited Loading

relu commented Apr 6, 2021

arbourd commented Apr 7, 2021

arbourd Apr 11, 2021

Choose a reason for hiding this comment

arbourd Apr 15, 2021

Choose a reason for hiding this comment

arbourd commented Apr 11, 2021 • edited Loading

hiddeco commented Apr 12, 2021

arbourd commented Apr 15, 2021

relu commented Apr 15, 2021

arbourd commented Apr 15, 2021 • edited Loading

hiddeco left a comment

Choose a reason for hiding this comment

arbourd commented Apr 19, 2021

relu left a comment

Choose a reason for hiding this comment

hiddeco commented Apr 19, 2021

arbourd commented Apr 19, 2021

hiddeco commented Apr 19, 2021

Add `ValuesFiles` to HelmChart spec #305

Add `ValuesFiles` to HelmChart spec #305

arbourd commented Mar 8, 2021 •

edited

Loading

arbourd commented Apr 4, 2021 •

edited

Loading

arbourd commented Apr 11, 2021 •

edited

Loading

arbourd commented Apr 15, 2021 •

edited

Loading