Trusting a Custom Certificate Authority not works for WinRM connection

[derhoeppi]

Please confirm the following

• I agree to follow this project's code of conduct.
• I have checked the current issues for duplicates.
• I understand that AWX is open source software provided for free and that I am not entitled to status updates or other assurances.

Summary

We configured AWX to trust custom CA's (described in https://github.com/ansible/awx-operator#trusting-a-custom-certificate-authority). After AWX is up and running, we use an adhoc command (win_ping) to test connectivity for our Microsoft systems. Winrm is configured to use credssp. The result of win_ping is "credssp: HTTPSConnectionPool(host='windows-system.my.domain', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)')))

AWX version

19.2.2

Installation method

kubernetes

Modifications

no

Ansible version

No response

Operating system

No response

Web browser

Edge

Steps to reproduce

1. Create a Windows system with enabled winrm and credssp. The certificate is generated by an custom ca.
2. Create a kubernetes secret with your custom ca.
3. Create an AWX deployment to use custom certificate authority like https://github.com/ansible/awx-operator#trusting-a-custom-certificate-authority.
4. After AWX deployment - use an adhoc command to test the winrm connection without the option "ansible_winrm_server_cert_validation = ignore"

Expected results

The module output shows success.

Actual results

The job output shows
"credssp: HTTPSConnectionPool(host='windows-system.my.domain', port=5986): Max retries exceeded with url: /wsman (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)')))

Additional information

I checked that my bundle-ca.crt is loaded to the task and web container. The file exists under /etc/pki/ca-trust/source/anchors/ and i can see my ca if i call trust list. I'm also able to use openssl (connected to the container) to test the connection to my Windows systems. The result is:
SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES256-GCM-SHA384 Session-ID: XXX Session-ID-ctx: Master-Key: XXX PSK identity: None PSK identity hint: None SRP username: None Start Time: 1629097650 Timeout : 7200 (sec) Verify return code: 0 (ok) Extended master secret: yes

So the connection can verify the trust of the presented certificate.

[derhoeppi]

Now i checked the ee Container and there i can run python3 -c "import certifi; print(certifi.where())" This returns the path of a pem file that is used by python. I don't find my certificates in it.
Is that the problem and how can i solve it?

[derhoeppi]

Today i tried to set an exported environment variable. On several pages i found hints that "REQUESTS_CA_BUNDLE" as env variable should point to my ca cert bundle. I can see it in awx-ee but certifi does not point to it.

[wenottingham]

-> #10787