Kubectl does not support secret creation from .env files with multiline values

[plusiv]

What happened?

kubectl currently does not support the creation of Kubernetes secrets from .env files containing multiline values. This limitation poses a challenge for users who need to store multiline environment variables as secrets, such as certificates or private keys.

Assuming the following .env file:

SECRET_ONE_LINE=Value one line

PASSPORT_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAx+4sI6wK3b7q2C4RjN02pH/sy9vwZ9Xbb1hjQmQY/V2aG5QQ
... (rest of the private key) ...
-----END RSA PRIVATE KEY-----"

PASSPORT_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAx+4sI6wK3b7q2C4RjN02
... (rest of the public key) ...
-----END PUBLIC KEY-----"

And running kubectl create secret generic test-secret --from-env-file=.env the following output is given by kubectl:

error: "MIIEpAIBAAKCAQEAx+4sI6wK3b7q2C4RjN02pH/sy9vwZ9Xbb1hjQmQY/V2aG5QQ" is not a valid key name: a valid environment variable name must consist of alphabetic characters, digits, '_', '-', or '.', and must not start with a digit (e.g. 'my.env-name',  or 'MY_ENV.NAME',  or 'MyEnvName1', regex used for validation is '[-._a-zA-Z][-._a-zA-Z0-9]*')

What did you expect to happen?

The secret should be created successfully with the multiline value stored.

How can we reproduce it (as minimally and precisely as possible)?

1. Create a .env file with multiline values:

SECRET=Value
MULTILINE_SECRET="line1
line2
line3"

2. Create a Generic Secret:

kubectl create secret generic test-secret --from-env-file=.env

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.9+rke2r1

Cloud provider

OS version

# On MacOs:
$ uname -a
Darwin 200.225.2.32-clientes-izzi.mx 23.4.0 Darwin Kernel Version 23.4.0: Wed Feb 21 21:44:06 PST 2024; root:xnu-10063.101.15~2/RELEASE_ARM64_T8103 arm64

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

[k8s-ci-robot]

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[plusiv]

/sig cli

[mauri870]

Currently there is no support for parsing multi-line entries. We parse the .env file line by line.

I think you can get this to work by simply replacing the actual line breaks in the file with a \n, it should work when encased in quotation marks.

E.g:

PRIVATE_KEY="my\nprivate\nkey"

I'd consider this as a feature request, the dotenv JS library (probably the de-facto implementation) already supports entries with line breaks since v15.0.0 (Jan 2022).

[mauri870]

/sig support

[k8s-ci-robot]

@mauri870: The label(s) sig/support cannot be applied, because the repository doesn't have them.

In response to this:

/sig support

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[mauri870]

Oops, mispelling

/kind support

[mauri870]

Looking at the code, it seems that supporting multiline values would require a rewrite of the rather simple parser we have in cmdutil.AddFromEnvFile. I wonder if this is a good time to migrate to a library such as godotenv instead of implementing this ourselves.

[mauri870]

Noticed that we also rely on a rather unique behavior that is not supported on the majority of dotenv libraries. If we have only the name but no value and no equal sign, the variable name assumes the value from the environment (os.Getenv("B") in the example bellow:

A=a
B
C="c"

Since this is not a dotenv standard we cannot simply migrate to a library without introducing breaking changes.

[plusiv]

Hi @mauri870 !

Thanks for your reply and intention to help, I was reviewing the code behind, what about just checking prefixes and suffixes of each value (in this case "")?. That's the approach of most dotenv libraries.

[mauri870]

Yes, it looks rather simple. Given that you can use \n instead of actual line breaks in the values makes supporting this optional. Our current parser assumes each entry is a single line, whence why we need to rewrite it so it knows how to consume quoted values that span multiple lines. I'll wait on the team members to decide if this is worth supporting.

[HirazawaUi]

I think this use case is useful, but I'd like to hear from the kubectl maintainers.
/remove-kind support
/transfer kubectl

[plusiv]

@mauri870 @HirazawaUi Hi guys!

The PR kubernetes/kubernetes#125430 could help!

[mauri870]

/assign plusiv

[sftim]

/remove-kind bug
/kind feature

Early .env implementations didn't support multiline values with line breaks, so this feels like a feature request.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[plusiv]

/remove-lifecycle stale

[mpuckett159]

/triage accepted
I think we should just implement what is being done in the motdotla/dotenv repo so we can fall more in line with what is expected to be the standard by others. Unfortunately because of the behavior described here we can not simply import an existing library over top of the existing code, but I think we should be able to relatively easily just implement the behavior from the JS repo.

We could also implement the mentioned godotenv library as a separate flag. If someone would like to take that on, we could implement that via the KEP process with a scheduled deprecation of the old behavior.

[mpuckett159]

Personally I am very pro doing things the way that people will expect them behave and would love to see the KEP process for migrating from the old behavior to the standard behavior done.

[plusiv]

Hi @mpuckett159 ,

As mentioned in the comment below, I created a PR that solves this issue without using the dotenv package. The proposed solution aims to resolve the problem in the least intrusive way possible, without modifying any critical code.

@mauri870 @HirazawaUi Hi guys!

The PR kubernetes/kubernetes#125430 could help!