Lint vault files

[ThomasSteinbach]

I'd like to lint whole Ansible playbooks including vault files. An idea would be to provide ansible-lint the vault-pass parameter. Currently linting a vault file leads to a (correct) error:

ansible.errors.AnsibleError: ERROR! Decryption failed

[willthames]

Sure, implementations welcome

[willthames]

The approach in willthames/ansible-inventory-grapher#19 could likely be used here too.

[gersilex]

The problem still persists in ansible-lint 3.4.7. It does not seem to respect the settings from ansible.cfg, where the path to the vault password file is saved:

 ansible-lint roles/foo.bar/vars/main.vault.yml
Decryption failed on roles/foo.bar/vars/main.vault.yml

[thuandt]

Latest version and still return "Decryption failed"

[willthames]

Well, it would do, no one has actually implemented this yet. I've pointed to how one might go about this - and will favourably review pull requests implementing such a change.

[willthames]

I've removed the v3.2 milestone, it will get in the next version after someone has implemented it.

[tdmalone]

I know this hasn't been implemented, but just adding that the current error appears to be:

A vault password must be specified to decrypt data

I know ansible-lint can't take a password directly, so I've tried both ANSIBLE_VAULT_PASSWORD_FILE and ANSIBLE_VAULT_PASSWORD approaches, to no avail.

This means I'm choosing at the moment between linting, or committing secrets :(

I haven't really used Python before so I'm a bit lost on how to implement this, but taking the example in https://github.com/willthames/ansible-inventory-grapher/pull/19/files would it be adding:

• the parser.add_option calls to main, and
• the usage of the CLI library somewhere around the bottom of utils.py?

Having said that... utils.py also has this:

    # ansible-lint doesn't need/want to know about encrypted secrets, but it needs
    # Ansible 2.3+ allows encrypted secrets within yaml files, so we pass a string
    # as the password to enable such yaml files to be opened and parsed successfully.
    DEFAULT_VAULT_PASSWORD = 'x'

the DEFAULT_VAULT_PASSWORD is referenced elsewhere in the file twice.
Given this... is this actually already implemented and meant to be working?

[willthames]

@tdmalone what version are you running - 3.4.23 was released a couple of days ago which may or may not have included a fix (it definitely had a vault fix, just not sure if it's this vault fix)

This issue could use a minimal test case.

[tdmalone]

Thanks for the response! Yes I'm on 3.4.23.

I should be able to put a test case together. I just realised however my specific problem is with an inline string rather than a file - does this need a separate issue? (Only just started using Ansible Vault, and trying to get my head around it!)

[tdmalone]

Actually, I apologise - turns out it wasn't the vault strings I added... those are working fine. It was vault files someone else had added to the repo that I didn't know about. I actually don't need to be linting those, and have made my linting script a bit more targeted.

So, having said all that, I'm still happy to try to put a test case together if it would help! Is it just complete vault files which the problem is with?

[ewhauser]

The latest version of DataLoader does not have a property called set_vault_password. So the fix for me was to do this:

  class ByteHolder(object):
    """Not sure what object VaultLib actually wants here, so this is a hack"""
    def __init__(self, str):
        super(ByteHolder, self).__init__()
        self.bytes = str.encode('UTF-8')

    def parse_yaml_from_file(filepath):
        dl = DataLoader()
        dl.set_vault_secrets([('default', ByteHolder(os.environ['ANSIBLE_VAULT_PASSWORD']))])
        #if hasattr(dl, 'set_vault_password'):
        #    dl.set_vault_password(DEFAULT_VAULT_PASSWORD)
        return dl.load_from_file(filepath)

Cleaning up the code a bit for better error handling/etc. in our fork, but wanted to share the fix.

[AlKapkone]

Guys, is lint for vault files working or not ?

[ansiblejunky]

I just hit this issue with linting vault files. I ran the following command:

ansible-lint group_vars/dev/vault.yml

and received the following message back:

A vault password must be specified to decrypt data

There is a bigger problem though. It seems it causes the custom lint rules to not load at all. I run this command and ansible-lint should still run my custom lint rules but it doesn't.

ansible-lint -v site.yml group_vars/dev/vault.yml

Here is the output:

Examining site.yml of type playbook
Examining roles/galaxy/role1/tasks/redhat.yml of type tasks
Examining roles/galaxy/role1/tasks/windows.yml of type tasks
Examining roles/galaxy/role1/tasks/debian.yml of type tasks
Examining roles/galaxy/role1/tasks/main.yml of type tasks
Examining roles/galaxy/role1/handlers/main.yml of type handlers
Examining roles/galaxy/role1/meta/main.yml of type meta
A vault password must be specified to decrypt data

From the verbose mode it seems it's running the tests, but doesn't use my custom lint rule (as defined in the .ansible-lint config file I have).
Here is it without the vault file - you see the custom lint rule at the bottom now.

> ansible-lint site.yml  -v
Examining site.yml of type playbook
Examining roles/galaxy/role1/tasks/redhat.yml of type tasks
Examining roles/galaxy/role1/tasks/windows.yml of type tasks
Examining roles/galaxy/role1/tasks/debian.yml of type tasks
Examining roles/galaxy/role1/tasks/main.yml of type tasks
Examining roles/galaxy/role1/handlers/main.yml of type handlers
Examining roles/galaxy/role1/meta/main.yml of type meta
site.yml:4: [ECUSTOM0001] Play defines variables within `vars` section that do not comply with variable naming standards

[papanito]

Same problem here. I have a ansible.cfg file in the project which actually specifies the password file

vault_password_file = ~/.ansvault

Linting fails

ansible-lint -v playbooks/*.yml
A vault password must be specified to decrypt data

my version

ansible-lint 4.2.0

[rgaduput]

Hi @ssbarnea @ansiblejunky ,
we ran into the same issue when linting playbook with vault file A vault password must be specified to decrypt data,
It would be really helpful if this functionality can be be added by merging the drafted PR.

[ssbarnea]

If Ansible is configured to be able decrypt the linter will work, we do have test at examples/playbooks/contains_secrets.yml which is passing.

I will consider this bug fixed. Please do not open a new bug if you cannot pass ansible-playbook --syntax-check example.yml -- if this command fails, is no longer a problem of the linter, but more of local configuration issue with Ansible itself.

[Silejonu]

@ssbarnea This issue is not fixed. The test at examples/playbooks/contains_secrets.yml does not cover the issue.

The issue is with vault files included inside roles.

Here is a quick way to reproduce the bug:

.
├── ansible.cfg
├── roles
│   └── decrypt_vault
│       ├── tasks
│       │   └── main.yml
│       └── vars
│           └── vault.yml
├── test-ansible-lint.yml
└── vault_password.txt

ansible.cfg

[defaults]
vault_password_file = ./vault_password.txt

test-ansible-lint.yml

- name: Test ansible-lint reading vault password file
  hosts: localhost
  tasks:
    - name: Include test role
      ansible.builtin.include_role:
        name: decrypt_vault

vault_password.txt

password

roles/decrypt_vault/tasks/main.yml

- name: Test ansible-lint reading vault password file
  hosts: localhost
  tasks:
    - name: Include test role
      ansible.builtin.include_role:
        name: decrypt_vault
(venv) [test@testcentos test]$ cat roles/decrypt_vault/tasks/main.yml
- name: Decrypt vault
  ansible.builtin.include_vars:
    file: vault.yml

- name: Print secret
  ansible.builtin.debug:
    msg: "Decrypted secret is: {{ test_secret }}"

roles/decrypt_vault/vars/vault.yml

$ANSIBLE_VAULT;1.1;AES256
63666236666330386437643238353462363063353566643531316333323961303932623064643533
3562386332366631666630316538613337646462376237370a643436613261346233316330323861
66633266303838383433306530336464383737343036626665333532656337393832636662373766
3337643831376332620a396537653433636232376462643437343462313732656432653632653534
36393963653061633763363563383331333138373834333632376234653930376634

---

ansible-playbook --syntax-check executes correctly:

ansible-playbook --syntax-check test-ansible-lint.yml 
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost
does not match 'all'

playbook: test-ansible-lint.yml

However, as you can see, ansible-lint does not:

WARNING  Ignored exception from JinjaRule.matchyaml while processing roles/decrypt_vault/vars/vault.yml (vars): A vault password must be specified to decrypt data in file /home/test/test/roles/decrypt_vault/vars/vault.yml
WARNING  Ignored exception from VariableNamingRule.matchyaml while processing roles/decrypt_vault/vars/vault.yml (vars): A vault password must be specified to decrypt data in file /home/test/test/roles/decrypt_vault/vars/vault.yml

Passed: 0 failure(s), 0 warning(s) on 6 files. Last profile that met the validation criteria was 'production'.

However, if I move vault.yml anywhere else, suddenly ansible-lint does not complain. It complains only when the vault file is located inside roles/decrypt_vault/vars. Note that adding the full path to vault.yml to include_vars does not help. The issue is not about the way the file is included, it is about where it is located.

---

@ssbarnea please let me know if I should open a new bug, or if you're re-opening this one.