Adding Environment Resource

[IaroslavTitov]

Summary

• Adding Environment resource
• Adding dependency on ESC client
• Fixes 255

Testing

Tested in dev stack using below Pulumi Program (Dotnet SDK), confirmed that things like imports, opening environment work as expected.

    String yaml = """
    imports:
      - dev-stacks
    values:
      aws:
        secrets:
          fn::open::aws-secrets:
            region: us-west-2
            login: ${aws.creds}
            get:
              allSecrets:
                secretId: iaro-dev-stack/pulumi-service
      secrets:
        fn::fromJSON: ${aws.secrets.allSecrets}
      pulumiConfig:
        gitHubOAuthID: ${secrets.gitHubOAuthID}
        gitHubOAuthSecret: ${secrets.gitHubOAuthSecret}
    """;

    var environ = new Environment(
        "Iaro's environment",
        new EnvironmentArgs {
            Organization = "IaroslavTitov",
            Name = "IaroEnv",
            Yaml = new StringAsset(yaml)
        }
    );

[pgavlin]

Should make absolutely certain that assets work with output properties. Not 100% sure about that.

[IaroslavTitov]

Confirmed it works with output using Random and below snippet of code:

var random = new RandomString("rand", new RandomStringArgs
    {
        Length = 5,
        Special = false,
    });

    Output<AssetOrArchive> asset = random.Result.Apply(res => {
        String yaml = """
            imports:
              - dev-stacks
            values:
              aws:
                secrets:
                  fn::open::aws-secrets:
                    region: us-west-2
                    login: ${aws.creds}
                    get:
                      allSecrets:
                        secretId: iaro-dev-stack/pulumi-service
              secrets:
                fn::fromJSON: ${aws.secrets.allSecrets}
              pulumiConfig:
                gitHubOAuthID: ${secrets.gitHubOAuthID}
                gitHubOAuthSecret: ${secrets.gitHubOAuthSecret}
                randomString: 
        """ + res;
        AssetOrArchive asset = new StringAsset(yaml);
        return asset;
        }
    );

    var environ = new Pulumi.PulumiService.Environment(
        "Iaro's environment",
        new EnvironmentArgs {
            Organization = "IaroslavTitov",
            Name = "IaroEnv",
            Yaml = asset
        }
    );

When I go to see the environment in console it has a random blurb zbhrh and the same in Random's resource properties.

[pgavlin]

Ah, I should clarify--I meant that I'm not sure that the output property Environment.Yaml will work with assets. I'd try passing the Yaml output of one environment to the Yaml input of another.

[IaroslavTitov]

Oooh, gotcha, didn't understand properly
Confirmed that this works, creating 2 environments with identical yaml:

var environ = new Pulumi.PulumiService.Environment(
        "Iaro's environment",
        new EnvironmentArgs {
            Organization = "IaroslavTitov",
            Name = "IaroEnv",
            Yaml = asset
        }
    );

    var environ2 = new Pulumi.PulumiService.Environment(
        "Iaro's environment 2",
        new EnvironmentArgs {
            Organization = "IaroslavTitov",
            Name = "IaroEnv2",
            Yaml = environ.Yaml.Apply(yaml => yaml)
        }
    );

[komalali]

FWIW @IaroslavTitov you shouldn't have to do this:

Yaml = environ.Yaml.Apply(yaml => yaml)

It can just be:

Yaml = environ.Yaml

[IaroslavTitov]

Oh, makes sense, got stuck in the Apply mindset

[pgavlin]

High-level question:

If my environment definition contains secrets, are those stored in plaintext unless the entire environment definition is marked secret? For example, in the following program, will foo's value be stored in plaintext?

const env = new Environment("env", { yaml: `{ values: { foo: { fn::secret: hunter2 } } }` });

[IaroslavTitov]

Tried this, shows up as plaintext in the Resources tab. (In Environments, shows up as ciphertext)

Unfortunately, I don't think we can avoid storing it as plaintext, in order for import to work, this is the same issue as with Deployment Settings. We can add some frontend logic to *** them out maybe? Still obv wouldn't actually hide the secrets, just visually.

[pgavlin]

So users can manually mark the environment def as secret in their Pulumi program:

const env = new Environment("env", { yaml: pulumi.secret(`{ values: { foo: { fn::secret: hunter2 } } }`) });

That will encrypt the YAML in the statefile. We might also be able to do this automatically as part of Check, but I'm not certain. Check might be able to wrap the input YAML in a Secret prior to returning the inputs to the engine. I think that's worth a look.

Unfortunately, I don't think we can avoid storing it as plaintext, in order for import to work, this is the same issue as with Deployment Settings

Yeah we need to chat more about our approach to diffing with secrets. I think that we have several options here, but all of them will involve tradeoffs.

[IaroslavTitov]

Check might be able to wrap the input YAML in a Secret prior

Ooh, I did not think of that, I think that shouldn't be hard, I'll look into it and update the PR.

we need to chat more about our approach to diffing with secrets

Yep, that's the purpose of that doc, to get a conversation going

[IaroslavTitov]

Updated to force yaml to always be a secret property.

I then went down the rabbit hole trying to solve noisy diff when user has extra spaces in the yaml, which ESC then removes and causes diff. Sadly, found no way of eliminating this issue with secrets, as Diff receives only hash of a secret Asset, not full text and this can only be compared exactly, and not partially. It is a rare chance anyone will hit that issue though (I totally accidentally stumbled into it), so I think it's fine.

[komalali]

@IaroslavTitov can you resolve the conflicts please?

[IaroslavTitov]

@IaroslavTitov can you resolve the conflicts please?

Done!