file resource type displays Sensitive information in clear text in output when Deferred

[bobnegri]

Describe the Bug

We are passing a Sensitive object from vault_lookup to the file resource type using stdlib::deferrable_epp.
Running puppet agent --test results in the updated secret being rendered to the screen in plain text.

Expected Behavior

The file resource type should not render Sensitive variable values in clear text in file difference output. The value rendered to output should be changed from Redacted to Redacted.

Steps to Reproduce

Steps to reproduce the behavior:

$my_vars = {
  'url' => $external_url,
  'app_id' => $external_app_id,
  'app_token => Deferred('vault_lookup::lookup', [
      $path_to_app_id_token, {
          'vault_addr' => $vault_url,
          'cert_role'  => $servertype,
          'field'      => 'app_id_token_pw',
      },
  ]),
}

file { '/opt/my_app/my_script.sh':
  ensure  => file,
  owner   => 'root',
  group   => 'root',
  mode    => '0750',
  content => stdlib::deferrable_epp("${module_name}/opt/my_app/my_script.sh.epp", $my_vars),
}

1. In HashiCorp vault, update the token value.
2. On the Linux server, execute: sudo puppet agent --test
3. The output should show the new token value and the old token value in clear text.

Environment

• Version puppet-agent 8.5.1, puppetlabs-stdlib 9.6.0, and puppet-vault_lookup 1.1.0
• Platform Ubuntu 22.04

Additional Context

I'm pretty sure the secret was not displayed when it changed under puppet 7.24.

[joshcooper]

I'm not convinced this is a puppet bug. The deferrable_epp function should preserve the sensitiveness of its inputs and so would be an issue with the stdlib module. If you pass a Sensitive value to the file resource then it works as expected, correct?

[bobnegri]

I'll do some experiments at work on Thursday and see where things break down for me.

[bobnegri]

Rather than using:
content => stdlib::deferrable_epp("${module_name}/opt/my_app/my_script.sh.epp", $my_vars),
I tried using:
content => Deferred('inline_epp', [find_template("${module_name}/opt/my_app/my_script.sh.epp").file, $my_vars]),

The first run creates a new file. No secrets show.
I change the secret in the file.
The next puppet agent run shows the change in the output in clear text.

I should point out that the .epp file is setup:

TOKEN='<%= $app_token %>

A co-worker reported having the variable in his .epp file declares as Sensitive but having to change to Variat[Senstive, String] for puppet agent 8.

[joshcooper]

The first run creates a new file. No secrets show.

When puppet creates a file (or any other ensurable resource), it just logs that the resource was created, but not what each of the desired properties are.

When puppet modifies an existing file (or other ensurable resources), it logs the properties that changed.

I should point out that the .epp file is setup:

Can you include your epp file?

[bobnegri]

The client's rules make it very hard to get permission to post code, which is why everything is rather generic.

The .epp file:

#!/usr/bin/ruby

require 'rubygems'
require 'net/https'
require 'uri'
require 'pp'
require 'rest_client'
require 'json'
require 'net/ldap'

GITLAB_URL='https://<%= $fqdn %>'
GITLAB_TOKEN='<%= $gitlab_mgmt_token %>'

Whenever the token changes under puppet-agent 8.5.1, the difference contains the secret in clear text.

[bobnegri]

In puppet.conf I added:
preprocess_deferred = true

This appears to have fixed the issue by making puppet agent 8.5.1 behave more like puppet agent 7.27/7.29.

I assume there is a reason the default was set to 'false' for puppet agent 8 (despite what the Puppet 8 documentation says it is.)

[joshcooper]

Thanks @bobnegri yes sounds like an issue with how deferred resources are preprocessed.

I assume there is a reason the default was set to 'false' for puppet agent 8 (despite what the Puppet 8 documentation says it is.)

Yes that's correct. We're working on fixing our docs configuration, see #9312

I verified that using a non-deferred Sensitive value works as expected. First the content is redacted when creating the file:

❯ tree modules/sensitive_test 
modules/sensitive_test
├── manifests
│   └── init.pp
└── templates
    └── my_script.sh.epp

❯ cat modules/sensitive_test/manifests/init.pp 
$my_vars = {
  'app_token' => Sensitive.new('opensesame'),
}

file { '/tmp/my_script.sh':
  ensure  => file,
  content => epp("sensitive_test/my_script.sh.epp", $my_vars),
}

❯ cat modules/sensitive_test/templates/my_script.sh.epp 
#!/usr/bin/ruby

GITLAB_TOKEN='<%= $app_token %>'

❯ rm /tmp/my_script.sh 

❯ bundle exec puppet apply --modulepath ./modules modules/sensitive_test/manifests/init.pp --show_diff
Notice: Compiled catalog for localhost in environment production in 0.01 seconds
Notice: /Stage[main]/Main/File[/tmp/my_script.sh]/ensure: changed [redacted] to [redacted]
Notice: Applied catalog in 0.03 seconds

And if I modify the file locally, then puppet redacts the password:

❯ sed -i -e 's/opensesame/empty/' /tmp/my_script.sh

❯ bundle exec puppet apply --modulepath ./modules modules/sensitive_test/manifests/init.pp --show_diff
Notice: Compiled catalog for localhost in environment production in 0.01 seconds
Notice: /Stage[main]/Main/File[/tmp/my_script.sh]/content: [diff redacted]
Notice: /Stage[main]/Main/File[/tmp/my_script.sh]/content: changed [redacted] to [redacted]
Notice: Applied catalog in 0.04 seconds

We'll take a look

[github-actions]

Migrated issue to PUP-12050