Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[2.2.12] Fail to run Extension in FIPS mode #760

Closed
yuxisun1217 opened this issue Jun 13, 2017 · 7 comments
Closed

[2.2.12] Fail to run Extension in FIPS mode #760

yuxisun1217 opened this issue Jun 13, 2017 · 7 comments
Labels
feature request

Comments

@yuxisun1217
Copy link

yuxisun1217 commented Jun 13, 2017 •
edited
Loading

Hi,

In FIPS mode, the Extension doesn't work sell.
It also impact the VM provisioning if authenticate with ssh key in FIPS mode.

Packages:
RHEL-7.4
WALA-2.2.12
openssl-1.0.2k-8.el7.x86_64

Steps to Reproduce:

  1. Prepare a VM in Azure. Enable FIPS:
    1). yum install dracut-fips
    2). mv -v /boot/initramfs-$(uname -r).img{,.bak}
    dracut
    3). grubby --update-kernel=$(grubby --default-kernel) --args=fips=1
    uuid=$(findmnt -no uuid /boot)
    [[ -n $uuid ]] && grubby --update-kernel=$(grubby --default-kernel) --args=boot=UUID=${uuid}
    4). reboot
  2. Run "reset remote access" to install an Extension into the VM. There's no error logs in waagent.log
  3. Set "OS.EnableFIPS=y" in /etc/waagent.conf. Restart waagent service
  4. Check if the extension works. Check /var/log/waagent.log

Actual results:
The extension doesn't work. The waagent -run-exthandler process keeps restarting.

The error logs in waagent.log: (Seems the same as #668 )

2017/06/09 18:41:23.406056 WARNING Server preferred version:2015-04-05
2017/06/09 18:41:28.146195 ERROR Command: '/usr/bin/openssl cms -decrypt -in /var/lib/waagent/Certificates.p7m -inkey /var/lib/waagent/TransportPrivate.pem -recip /var/lib/waagent/TransportCert.pem | /usr/bin/openssl pkcs12 -nodes -password pass: -out /var/lib/waagent/Certificates.pem'
2017/06/09 18:41:28.184821 ERROR Return code: 1
2017/06/09 18:41:28.195972 ERROR Result: MAC verified OK
Error outputting keys and certificates
140308542494624:error:060740A0:digital envelope routines:EVP_PBE_CipherInit:unknown cipher:evp_pbe.c:181:
140308542494624:error:23077073:PKCS12 routines:PKCS12_pbe_crypt:pkcs12 algor cipherinit error:p12_decr.c:87:
140308542494624:error:2306A075:PKCS12 routines:PKCS12_item_decrypt_d2i:pkcs12 pbe crypt error:p12_decr.c:139:

2017/06/09 18:41:28.306785 ERROR Failed to run 'run-exthandlers': Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/agent.py", line 147, in main
    agent.run_exthandlers()
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/agent.py", line 117, in run_exthandlers
    update_handler.run()
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/ga/update.py", line 236, in run
    get_monitor_handler().run()
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/ga/monitor.py", line 96, in run
    self.init_sysinfo()
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/ga/monitor.py", line 121, in init_sysinfo
    protocol = self.protocol_util.get_protocol()
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/common/protocol/util.py", line 261, in get_protocol
    self.protocol = self._detect_protocol(protocols)
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/common/protocol/util.py", line 183, in _detect_protocol
    return self._detect_wire_protocol()
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/common/protocol/util.py", line 152, in _detect_wire_protocol
    protocol.detect()
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/common/protocol/wire.py", line 99, in detect
    self.client.update_goal_state(forced=True)
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/common/protocol/wire.py", line 729, in update_goal_state
    self.update_certs(goal_state)
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/common/protocol/wire.py", line 688, in update_certs
    self.certs = Certificates(self, xml_text)
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/common/protocol/wire.py", line 1147, in __init__
    self.parse(xml_text)
  File "/usr/lib/python2.7/site-packages/azurelinuxagent/common/protocol/wire.py", line 1218, in parse
    thumbprint = thumbprints[pubkey]
KeyError: u'-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAocW4DnlCqiI8MrQAj8ec\nZACpCKUwPCPg3vDYGLdwqvKs9H9bMxy1cXzgGFnPgfG/azfyzB3kbDlW+I9DMLq9\nw2ntdRdDn2esLlToWymQcQjs0FesvJhppgJSe0hOlUCBBgmWqFC1Lfom+SGDnxeR\nkc6z42ExX4VPRvNKeU7yZwoOqpTZmy2FXNxVe3db0nB87ZRRy15gXjHICFPMG4HV\nsPI/xDttaqTLlzmmGVh36oxE8WVCNiTarTOTNfA4udNmk07Xw2Y3lrms28jr2AKj\ngxpI+IUraN8reLUVNmkumeNwEl0ttdv6ngltkGCoNh+3lKVpnugahB+GCQ5hamCe\nGQIDAQAB\n-----END PUBLIC KEY-----\n'

I run the command manually and also get error messages. My steps:

  1. export OPENSSL_FIPS=1
  2. Run command:
#/usr/bin/openssl cms -decrypt -in /var/lib/waagent/Certificates.p7m -inkey /var/lib/waagent/TransportPrivate.pem -recip /var/lib/waagent/TransportCert.pem | /usr/bin/openssl pkcs12 -nodes -password pass: -out /var/lib/waagent/Certificates.pem
MAC verified OK
Error outputting keys and certificates
139851566958496:error:060740A0:digital envelope routines:EVP_PBE_CipherInit:unknown cipher:evp_pbe.c:181:
139851566958496:error:23077073:PKCS12 routines:PKCS12_pbe_crypt:pkcs12 algor cipherinit error:p12_decr.c:87:
139851566958496:error:2306A075:PKCS12 routines:PKCS12_item_decrypt_d2i:pkcs12 pbe crypt error:p12_decr.c:139:

I'm not sure if the openssl pkcs12 is supported in FIPS mode...
@brendandixon brendandixon added this to the 2.2.15 milestone Jun 20, 2017
@sriramsa
Copy link

Seen in Centos 6.7 too. Agent version WALiuxAgent-2.2.13 .

I found some solutions recommending using -descert option when dealing with PKCS12.
https://community.rsa.com/docs/DOC-51951

[  OK  ]Starting puppetmaster: 2017/06/22 06:59:00.412282 ERROR Command: '/usr/bin/openssl cms -decrypt -in /var/lib/waagent/Certificates.p7m -inkey /var/lib/waagent/TransportPrivate.pem -recip /var/lib/waagent/TransportCert.pem | /usr/bin/openssl pkcs12 -nodes -password pass: -out /var/lib/waagent/Certificates.pem'
2017/06/22 06:59:00.412560 ERROR Return code: 1
2017/06/22 06:59:00.412835 ERROR Result: MAC verified OK
Error outputting keys and certificates
139760389863240:error:060740A0:digital envelope routines:EVP_PBE_CipherInit:unknown cipher:evp_pbe.c:186:
139760389863240:error:23077073:PKCS12 routines:PKCS12_pbe_crypt:pkcs12 algor cipherinit error:p12_decr.c:83:
139760389863240:error:2306A075:PKCS12 routines:PKCS12_item_decrypt_d2i:pkcs12 pbe crypt error:p12_decr.c:130:2017/06/22 06:59:00.585337 ERROR Failed to run 'run-exthandlers': Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/agent.py", line 147, in main
    agent.run_exthandlers()
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/agent.py", line 117, in run_exthandlers
    update_handler.run()
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/ga/update.py", line 236, in run
    get_monitor_handler().run()
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/ga/monitor.py", line 96, in run
    self.init_sysinfo()
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/ga/monitor.py", line 121, in init_sysinfo
    protocol = self.protocol_util.get_protocol()
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/common/protocol/util.py", line 256, in get_protocol
    self.protocol = self._detect_protocol(protocols)
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/common/protocol/util.py", line 178, in _detect_protocol
    return self._detect_wire_protocol()
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/common/protocol/util.py", line 152, in _detect_wire_protocol
    protocol.detect()
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/common/protocol/wire.py", line 99, in detect
    self.client.update_goal_state(forced=True)
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/common/protocol/wire.py", line 729, in update_goal_state
    self.update_certs(goal_state)
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/common/protocol/wire.py", line 688, in update_certs
    self.certs = Certificates(self, xml_text)
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/common/protocol/wire.py", line 1147, in __init__
    self.parse(xml_text)
  File "/usr/lib/python2.6/site-packages/azurelinuxagent/common/protocol/wire.py", line 1218, in parse
    thumbprint = thumbprints[pubkey]
KeyError: u'-----BEGIN PUBLIC 
2017/06/22 06:59:01.000 INFO Event: ame=WALinuxAgent, op=sage=Agent WALiuxAgent-2.2.13 launched with command 'python -u /usr/sbin/waagent -run-exthandlers' is successfully running

@brendandixon brendandixon removed this from the 2.2.15 milestone Jul 14, 2017
@hglkrijger
Copy link
Member

I investigated this, and the certificates we receive are not FIPS compliant, so from the agent perspective there is nothing to be done here.

@sbohlen
Copy link

sbohlen commented Jan 11, 2018 •
edited
Loading

@hglkrijger Are you certain that its only the cert that's the issue here? These issues from RHEL Bugzilla seem to imply its also the encryption being applied:

https://bugzilla.redhat.com/show_bug.cgi?id=1460671
https://bugzilla.redhat.com/show_bug.cgi?id=1461243

Even with a different cert, am I misunderstanding that the same algo would still be used, seemingly resulting in a (still) invalid encryption for FIPS mode? Note this isn't my area of expertise, so it well be that I'm wrong here....

@yuxisun1217
Copy link
Author

Hi @hglkrijger ,
Do you mean that WALA cannot support FIPS mode? Thanks!

@jasonzio
Copy link
Member

@yuxisun1217 that's pretty much correct.

@yuxisun1217
Copy link
Author

@jasonzio OK. Thank you so much :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@sbohlen @brendandixon @hglkrijger @jasonzio @sriramsa @yuxisun1217 and others