Do not alter secret key upper-/lowercase

[qwerty287]

Don't lowercase secret names and don't uppercase env vars.

Examples:

We have a secret SECRET_UPPER (it will show up exactly like this in UI).

• from_secret: secret_upper

  Matching with from_secret is case-insensitive, this injects SECRET_UPPER.

• secrets: [secret_upper]

  Matching these secrets is case-insensitive. The env var however looks exactly like the string here. In this example, SECRET_UPPER is injected as secret_upper env.

• secrets:
    - source: secret_upper
      target: some_different_name

  Here SECRET_UPPER is injected as some_different_name, it is not uppercased.

Closes #2368
Closes #3290
Closes #3096

[woodpecker-bot]

Deployment of preview was successful: https://woodpecker-ci-woodpecker-pr-3375.surge.sh

[codecov]

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (65d88be) 36.67% compared to head (9093ae7) 36.60%.
Report is 18 commits behind head on main.

Files	Patch %	Lines
pipeline/frontend/yaml/compiler/convert.go	0.00%	2 Missing ⚠️
server/api/repo_secret.go	0.00%	1 Missing ⚠️
server/pipeline/stepbuilder/stepBuilder.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3375      +/-   ##
==========================================
- Coverage   36.67%   36.60%   -0.07%     
==========================================
  Files         225      225              
  Lines       14788    14822      +34     
==========================================
+ Hits         5423     5426       +3     
- Misses       8972     9003      +31     
  Partials      393      393              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[6543]

If some consider it a gug or not ... it was intended as feature initialy ... to have keys upercase everywhere ...

[6543]

To make it non breaking we just have to add a db migration final adjusting the values in DB and in upcomming versions including this patchset the user then can decide.

Please have a look at the docs again or are you sure you catched all places mentioning secrets?

[xoxys]

If some consider it a gug or not ... it was intended as feature initialy ... to have keys upercase everywhere ...

What was the intention behind this feature? Why is it useful to have all keys uppercase? If it is useful to have them uniformed for internal use, we could just add another property to the scheme to store them unified while user-facing stuff is using the notation from the user input.

[6543]

no it just imitated what settings do for plugins. and try to guide the user so they do get it more often right. but if we completely get rid of the automatism, I'm fine with it too. just mind the migration ;)

[qwerty287]

Using a migration won't make this non-breaking.

If you currently use:

secrets: [ some_secret ]

the env var is SOME_SECRET.

With this change, it will be some_secret instead because it uses exactly the value you set in the yaml, no matter whether the secret is called some_secret or SOME_SECRET.

I don't really see a way to make this non-breaking. Maybe we can hold it back for some time and do a major release at some point?

[6543]

Uh right :/

[lafriks]

I'm not sure I like this change :/

[xoxys]

Well manipulating user inputs without a good reason is also a bad idea and given the amount of issues regarding this a special bad thing in this particular case.

[qwerty287]

@6543 To make this non-breaking, I added a fallback to inject both the name used by the user and the uppercased name for now. We can remove the uppercase one in next major.

[6543]

If we encourage #3406 we can let it stay all uppercase or lowecase.

As you still kan have any key:

Environment:
aNYkEy:
from_secret: anykey

[6543]

E.g. we should make it case insensitive to users :)

[qwerty287]

I don't think that's a good idea.

Using the secrets directive still changes the user's input which makes it harder to do debugging etc. Yes, you can easily work around this by using the from_secret syntax, but I don't think changing the user input without having a good reason (and we don't have a good reason for it) is good.

[xoxys]

It's not only a debugging issue. There are situations where env vars capitalization matters. One example is Terraform/OpenTofu, see https://opentofu.org/docs/language/values/variables/#environment-variables

For example the env var TF_VAR_image_id=ami-abc123 is case-sensitive and there is currently no way to set such an env var from a secret in Woodpecker.

[lafriks]

Why not set both uppercase and unchanged case env vars?

[xoxys]

Why not set both uppercase and unchanged case env vars?

#3375 (comment)

[lafriks]

Why not set both uppercase and unchanged case env vars?

#3375 (comment)

sorry, missed that comment. Only thing is that I don't see why we would need to deprecate it, personally I do prefer env variables upper case but not specifically like writing them in upper case in pipeline yaml 😅

[qwerty287]

I'd deprecate it because it's just useless duplication.
Why don't you like writing uppercase in pipeline yamls? You would have to do it anyways in the commands.

[lafriks]

just personal preference, not deal breaker 😉

[qwerty287]

I'll merge this now, if we decide to keep both we can still remove the todo/deprecation comment in a new PR.