Reject pipeline uploads containing redacted vars

[keithduncan]

Work in progress with @eleanorakh and @pzeballos to catch and prevent pipeline uploads from including interpolated redacted vars.

Fixes #1469

[keithduncan]

@eleanorakh I’ve taken another look at this and had some ideas about error messages before we merge, let me know what you think 🙇 😃

[keithduncan]

What do you think about tweaking this error to say something like:

"Couldn’t scan the %q pipeline for redacted variables. This parsed pipeline could not be serialized, ensure the pipeline YAML is valid, or disable secret redaction for this upload by passing --redacted-vars=''. (%s)", src, err

[eleanorakh]

Yea, I like that. It's hitting the three points of what makes a good error message. I'll update! :)

[keithduncan]

I did wonder about whether GetValuesToRedact could return tuples of (env_key, env_value) so that this error could say "Whoops your pipeline interpolated the value of the $env_key" and point folks closer to the answer. What do you think?

[eleanorakh]

Yea, a bit more specificity never hurts!
How about this?

Couldn't upload %q pipeline. Refusing to upload pipeline containing interpolated values of the $env_key. Ensure your pipeline does not include secret values or interpolated secret values

[keithduncan]

Yeah I think that would be great 😄

[keithduncan]

Here’s how this looks when your pipeline interpolates one of the environment variables matched by the --redacted-vars flag:

keithduncan@Keiths-Mac-mini agent % FROG_API_TOKEN=secretcats ./buildkite-agent pipeline upload .buildkite/pipeline.yml --agent-access-token foo
2021-12-07 12:59:04 INFO   Reading pipeline config from ".buildkite/pipeline.yml"
2021-12-07 12:59:04 FATAL  Couldn’t upload the "pipeline.yml" pipeline. Refusing to upload pipeline containing the value interpolated from the "FROG_API_TOKEN" environment variable. Ensure your pipeline does not include secret values or interpolated secret values.