Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RuntimeError: You must install paramiko package (pip install paramiko) to use the paramiko backend [Molecule verify, Ansible 2.8, Testinfra 3.x, AWS EC2 driver] #2070

Closed
jonashackt opened this issue May 28, 2019 · 4 comments · Fixed by #2076
Closed

RuntimeError: You must install paramiko package (pip install paramiko) to use the paramiko backend [Molecule verify, Ansible 2.8, Testinfra 3.x, AWS EC2 driver] #2070

jonashackt opened this issue May 28, 2019 · 4 comments · Fixed by #2076
Labels
bug critical

Comments

@jonashackt
Copy link
Contributor

jonashackt commented May 28, 2019 •
edited
Loading

Issue Type

  • Bug report

Molecule and Ansible details

Prerequisites (install testinfra first to fix #1727)

$ pip install testinfra molecule docker
Installing collected packages: testinfra, molecule, docker, ansible
Successfully installed ansible-2.8.0 docker-4.0.1 molecule-2.20.1 testinfra-3.0.4

$ ansible --version && molecule --version
ansible 2.8.0
  config file = None
  configured module search path = ['/Users/jonashecht/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.3 (default, Mar 27 2019, 09:23:15) [Clang 10.0.1 (clang-1001.0.46.3)]
molecule, version 2.20.1

Molecule installation method (one of):

  • pip

Ansible installation method (one of):

  • pip

Detail any linters or test runners used:

Desired Behavior

molecule verify --scenario-name aws-ec2-ubuntu should verify a AWS EC2 instance (like it does with Ansible 2.7.x and Testinfra 2.x)

Actual Behaviour

--> Test matrix
    
└── aws-ec2-ubuntu
    └── verify
    
--> Scenario: 'aws-ec2-ubuntu'
--> Action: 'verify'
--> Executing Testinfra tests found in /home/circleci/molecule-ansible-docker-vagrant/docker/molecule/aws-ec2-ubuntu/../tests//...
    ============================= test session starts ==============================
    platform linux -- Python 3.6.1, pytest-4.5.0, py-1.8.0, pluggy-0.11.0 -- /usr/local/bin/python
    rootdir: /home/circleci/molecule-ansible-docker-vagrant/docker/molecule
    plugins: testinfra-3.0.4
collected 3 items                                                              
    
    ../tests/test_docker.py::test_is_docker_installed[ansible:/aws-ec2-ubuntu] FAILED [ 33%]
    ../tests/test_docker.py::test_vagrant_user_is_part_of_group_docker[ansible:/aws-ec2-ubuntu] FAILED [ 66%]
    ../tests/test_docker.py::test_run_hello_world_container_successfully[ansible:/aws-ec2-ubuntu] FAILED [100%]
    
    =================================== FAILURES ===================================
    ______________ test_is_docker_installed[ansible://aws-ec2-ubuntu] ______________
    
    self = <testinfra.utils.ansible_runner.AnsibleRunner object at 0x7f30d7171ba8>
    host = 'aws-ec2-ubuntu'
    kwargs = {'ssh_config': None, 'ssh_identity_file': None}
    
        def get_host(self, host, **kwargs):
            try:
    >           return self._host_cache[host]
    E           KeyError: 'aws-ec2-ubuntu'
    
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:166: KeyError
    
    During handling of the above exception, another exception occurred:
    
        from __future__ import unicode_literals
        from __future__ import absolute_import
        
        import os
        
        try:
    >       import paramiko
    E       ModuleNotFoundError: No module named 'paramiko'
    
    /usr/local/lib/python3.6/site-packages/testinfra/backend/paramiko.py:20: ModuleNotFoundError
    
    During handling of the above exception, another exception occurred:
    
    host = <testinfra.host.Host object at 0x7f30d746e780>
    
        def test_is_docker_installed(host):
    >       package_docker = host.package('docker-ce')
    
    ../tests/test_docker.py:10:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:107: in __getattr__
        obj = module_class.get_module(self)
    /usr/local/lib/python3.6/site-packages/testinfra/modules/base.py:22: in get_module
        klass = cls.get_module_class(_host)
    /usr/local/lib/python3.6/site-packages/testinfra/modules/package.py:69: in get_module_class
        if host.system_info.type == 'windows':
    /usr/local/lib/python3.6/site-packages/testinfra/modules/systeminfo.py:143: in type
        return self.sysinfo["type"]
    /usr/local/lib/python3.6/site-packages/testinfra/utils/__init__.py:44: in __get__
        value = obj.__dict__[self.func.__name__] = self.func(obj)
    /usr/local/lib/python3.6/site-packages/testinfra/modules/systeminfo.py:33: in sysinfo
        uname = self.run_expect([0, 1], 'uname -s')
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:71: in run
        return self.backend.run(command, *args, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/ansible.py:46: in run
        ssh_identity_file=self.ssh_identity_file)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:173: in run
        return self.get_host(host, **kwargs).run(command)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:169: in get_host
        self.ansible_config, self.inventory, host, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:97: in get_ansible_host
        return testinfra.get_host(spec, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:132: in get_host
        backend = testinfra.backend.get_backend(hostspec, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/__init__.py:73: in get_backend
        klass = get_backend_class(kw["connection"])
    /usr/local/lib/python3.6/site-packages/testinfra/backend/__init__.py:41: in get_backend_class
        return getattr(importlib.import_module(module), name)
    /usr/local/lib/python3.6/importlib/__init__.py:126: in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    <frozen importlib._bootstrap>:978: in _gcd_import
        ???
    <frozen importlib._bootstrap>:961: in _find_and_load
        ???
    <frozen importlib._bootstrap>:950: in _find_and_load_unlocked
        ???
    <frozen importlib._bootstrap>:646: in _load_unlocked
        ???
    <frozen importlib._bootstrap>:616: in _load_backward_compatible
        ???
    /usr/local/lib/python3.6/site-packages/_pytest/assertion/rewrite.py:301: in load_module
        exec(co, mod.__dict__)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
        from __future__ import unicode_literals
        from __future__ import absolute_import
        
        import os
        
        try:
            import paramiko
        except ImportError:
            raise RuntimeError((
    >           "You must install paramiko package (pip install paramiko) "
                "to use the paramiko backend"))
    E       RuntimeError: You must install paramiko package (pip install paramiko) to use the paramiko backend
    
    /usr/local/lib/python3.6/site-packages/testinfra/backend/paramiko.py:23: RuntimeError
    _____ test_vagrant_user_is_part_of_group_docker[ansible://aws-ec2-ubuntu] ______
    
    self = <testinfra.utils.ansible_runner.AnsibleRunner object at 0x7f30d7171ba8>
    host = 'aws-ec2-ubuntu'
    kwargs = {'ssh_config': None, 'ssh_identity_file': None}
    
        def get_host(self, host, **kwargs):
            try:
    >           return self._host_cache[host]
    E           KeyError: 'aws-ec2-ubuntu'
    
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:166: KeyError
    
    During handling of the above exception, another exception occurred:
    
        from __future__ import unicode_literals
        from __future__ import absolute_import
        
        import os
        
        try:
    >       import paramiko
    E       ModuleNotFoundError: No module named 'paramiko'
    
    /usr/local/lib/python3.6/site-packages/testinfra/backend/paramiko.py:20: ModuleNotFoundError
    
    During handling of the above exception, another exception occurred:
    
    host = <testinfra.host.Host object at 0x7f30d746e780>
    
        def test_vagrant_user_is_part_of_group_docker(host):
    >       user_vagrant = host.user('vagrant')
    
    �[1m../tests/test_docker.py:16:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:107: in __getattr__
        obj = module_class.get_module(self)
    /usr/local/lib/python3.6/site-packages/testinfra/modules/base.py:22: in get_module
        klass = cls.get_module_class(_host)
    /usr/local/lib/python3.6/site-packages/testinfra/modules/user.py:110: in get_module_class
        if host.system_info.type.endswith("bsd"):
    /usr/local/lib/python3.6/site-packages/testinfra/modules/systeminfo.py:143: in type
        return self.sysinfo["type"]
    /usr/local/lib/python3.6/site-packages/testinfra/utils/__init__.py:44: in __get__
        value = obj.__dict__[self.func.__name__] = self.func(obj)
    /usr/local/lib/python3.6/site-packages/testinfra/modules/systeminfo.py:33: in sysinfo
        uname = self.run_expect([0, 1], 'uname -s')
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:71: in run
        return self.backend.run(command, *args, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/ansible.py:46: in run
        ssh_identity_file=self.ssh_identity_file)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:173: in run
        return self.get_host(host, **kwargs).run(command)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:169: in get_host
        self.ansible_config, self.inventory, host, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:97: in get_ansible_host
        return testinfra.get_host(spec, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:132: in get_host
        backend = testinfra.backend.get_backend(hostspec, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/__init__.py:73: in get_backend
        klass = get_backend_class(kw["connection"])
    /usr/local/lib/python3.6/site-packages/testinfra/backend/__init__.py:41: in get_backend_class
        return getattr(importlib.import_module(module), name)
    /usr/local/lib/python3.6/importlib/__init__.py:126: in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    <frozen importlib._bootstrap>:978: in _gcd_import
        ???
    <frozen importlib._bootstrap>:961: in _find_and_load
        ???
    <frozen importlib._bootstrap>:950: in _find_and_load_unlocked
        ???
    <frozen importlib._bootstrap>:646: in _load_unlocked
        ???
    <frozen importlib._bootstrap>:616: in _load_backward_compatible
        ???
    /usr/local/lib/python3.6/site-packages/_pytest/assertion/rewrite.py:301: in load_module
        exec(co, mod.__dict__)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
        from __future__ import unicode_literals
        from __future__ import absolute_import
        
        import os
        
        try:
            import paramiko
        except ImportError:
            raise RuntimeError((
    >           "You must install paramiko package (pip install paramiko) "
                "to use the paramiko backend"))
    E       RuntimeError: You must install paramiko package (pip install paramiko) to use the paramiko backend
    
    /usr/local/lib/python3.6/site-packages/testinfra/backend/paramiko.py:23: RuntimeError
    ____ test_run_hello_world_container_successfully[ansible://aws-ec2-ubuntu] _____
    
    self = <testinfra.utils.ansible_runner.AnsibleRunner object at 0x7f30d7171ba8>
    host = 'aws-ec2-ubuntu'
    kwargs = {'ssh_config': None, 'ssh_identity_file': None}
    
        def get_host(self, host, **kwargs):
            try:
    >           return self._host_cache[host]
    E           KeyError: 'aws-ec2-ubuntu'
    
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:166: KeyError
    
    During handling of the above exception, another exception occurred:
    
        from __future__ import unicode_literals
        from __future__ import absolute_import
        
        import os
        
        try:
    >       import paramiko
    E       ModuleNotFoundError: No module named 'paramiko'
    
    /usr/local/lib/python3.6/site-packages/testinfra/backend/paramiko.py:20: ModuleNotFoundError
    
    During handling of the above exception, another exception occurred:
    
    host = <testinfra.host.Host object at 0x7f30d746e780>
    
        def test_run_hello_world_container_successfully(host):
    >       hello_world_ran = host.run("sudo docker run hello-world")
    
    ../tests/test_docker.py:22:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:71: in run
        return self.backend.run(command, *args, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/ansible.py:46: in run
        ssh_identity_file=self.ssh_identity_file)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:173: in run
        return self.get_host(host, **kwargs).run(command)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:169: in get_host
        self.ansible_config, self.inventory, host, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:97: in get_ansible_host
        return testinfra.get_host(spec, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:132: in get_host
        backend = testinfra.backend.get_backend(hostspec, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/__init__.py:73: in get_backend
        klass = get_backend_class(kw["connection"])
    /usr/local/lib/python3.6/site-packages/testinfra/backend/__init__.py:41: in get_backend_class
        return getattr(importlib.import_module(module), name)
    /usr/local/lib/python3.6/importlib/__init__.py:126: in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    <frozen importlib._bootstrap>:978: in _gcd_import
        ???
    <frozen importlib._bootstrap>:961: in _find_and_load
        ???
    �[1m<frozen importlib._bootstrap>:950: in _find_and_load_unlocked
        ???
    <frozen importlib._bootstrap>:646: in _load_unlocked
        ???
    <frozen importlib._bootstrap>:616: in _load_backward_compatible
        ???
    /usr/local/lib/python3.6/site-packages/_pytest/assertion/rewrite.py:301: in load_module
        exec(co, mod.__dict__)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
        from __future__ import unicode_literals
        from __future__ import absolute_import
        
        import os
        
        try:
            import paramiko
        except ImportError:
            raise RuntimeError((
    >           "You must install paramiko package (pip install paramiko) "
                "to use the paramiko backend"))
    E       RuntimeError: You must install paramiko package (pip install paramiko) to use the paramiko backend
    
    /usr/local/lib/python3.6/site-packages/testinfra/backend/paramiko.py:23: RuntimeError
    =========================== 3 failed in 1.16 seconds ===========================
Exited with code 1

See build log https://circleci.com/gh/jonashackt/molecule-ansible-docker-vagrant/26

Seems that paramiko is now optional from Ansible 2.8.x on (see https://github.com/ansible/ansible/blob/stable-2.8/changelogs/CHANGELOG-v2.8.rst and look for paramiko).

Installing paramiko manually before the Molecule execution, the molecule verify step failes with paramiko.ssh_exception.AuthenticationException: Authentication failed. (see build log https://circleci.com/gh/jonashackt/molecule-ansible-docker-vagrant/27) or details here:

    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    self = <paramiko.auth_handler.AuthHandler object at 0x7f3c489add30>
    event = <threading.Event object at 0x7f3c489adcc0>
    
        def wait_for_response(self, event):
            max_ts = None
            if self.transport.auth_timeout is not None:
                max_ts = time.time() + self.transport.auth_timeout
            while True:
                event.wait(0.1)
                if not self.transport.is_active():
                    e = self.transport.get_exception()
                    if (e is None) or issubclass(e.__class__, EOFError):
                        e = AuthenticationException("Authentication failed.")
                    raise e
                if event.is_set():
                    break
                if max_ts is not None and max_ts <= time.time():
                    raise AuthenticationException("Authentication timeout.")
        
            if not self.is_authenticated():
                e = self.transport.get_exception()
                if e is None:
                    e = AuthenticationException("Authentication failed.")
                # this is horrible.  Python Exception isn't yet descended from
                # object, so type(e) won't work. :(
                if issubclass(e.__class__, PartialAuthentication):
                    return e.allowed_types
    >           raise e
    E           paramiko.ssh_exception.AuthenticationException: Authentication failed.
    
    /usr/local/lib/python3.6/site-packages/paramiko/auth_handler.py:250: AuthenticationException
    
    During handling of the above exception, another exception occurred:
    
    host = <testinfra.host.Host object at 0x7f3c4c68cb00>
    
        def test_run_hello_world_container_successfully(host):
    >       hello_world_ran = host.run("sudo docker run hello-world")
    
    ../tests/test_docker.py:22:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:71: in run
        return self.backend.run(command, *args, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/ansible.py:46: in run
        ssh_identity_file=self.ssh_identity_file)
    /usr/local/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:173: in run
        return self.get_host(host, **kwargs).run(command)
    /usr/local/lib/python3.6/site-packages/testinfra/host.py:71: in run
        return self.backend.run(command, *args, **kwargs)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/paramiko.py:117: in run
        if not self.client.get_transport().is_active():
    /usr/local/lib/python3.6/site-packages/testinfra/utils/__init__.py:44: in __get__
        value = obj.__dict__[self.func.__name__] = self.func(obj)
    /usr/local/lib/python3.6/site-packages/testinfra/backend/paramiko.py:98: in client
        client.connect(**cfg)
    /usr/local/lib/python3.6/site-packages/paramiko/client.py:437: in connect
        passphrase,
    /usr/local/lib/python3.6/site-packages/paramiko/client.py:749: in _auth
        raise saved_exception
    /usr/local/lib/python3.6/site-packages/paramiko/client.py:725: in _auth
        self._transport.auth_publickey(username, key)
    /usr/local/lib/python3.6/site-packages/paramiko/transport.py:1507: in auth_publickey
        return self.auth_handler.wait_for_response(my_event)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    self = <paramiko.auth_handler.AuthHandler object at 0x7f3c48a3fc88>
    event = <threading.Event object at 0x7f3c48a3fba8>
    
        def wait_for_response(self, event):
            max_ts = None
            if self.transport.auth_timeout is not None:
                max_ts = time.time() + self.transport.auth_timeout
            while True:
                event.wait(0.1)
                if not self.transport.is_active():
                    e = self.transport.get_exception()
                    if (e is None) or issubclass(e.__class__, EOFError):
                        e = AuthenticationException("Authentication failed.")
                    raise e
                if event.is_set():
                    break
                if max_ts is not None and max_ts <= time.time():
                    raise AuthenticationException("Authentication timeout.")
        
            if not self.is_authenticated():
                e = self.transport.get_exception()
                if e is None:
                    e = AuthenticationException("Authentication failed.")
                # this is horrible.  Python Exception isn't yet descended from
                # object, so type(e) won't work. :(
                if issubclass(e.__class__, PartialAuthentication):
                    return e.allowed_types
    >           raise e
    E           paramiko.ssh_exception.AuthenticationException: Authentication failed.
    
    /usr/local/lib/python3.6/site-packages/paramiko/auth_handler.py:250: AuthenticationException

Full documentation here: https://github.com/jonashackt/molecule-ansible-docker-vagrant/issues/3
@jonashackt jonashackt added the bug label May 28, 2019
@jonashackt jonashackt changed the title RuntimeError: You must install paramiko package (pip install paramiko) to use the paramiko backend [Molecule verify, Ansible 2.8, AWS EC2 driver] RuntimeError: You must install paramiko package (pip install paramiko) to use the paramiko backend [Molecule verify, Ansible 2.8, Testinfra 3.x, AWS EC2 driver] May 28, 2019
@decentral1se
Copy link
Contributor

Righhhhhht, thanks for this bug report! I was also seeing this in previous CI build logs and had no idea where to start looking. This makes sense and I'll try to dig into fixing this ASAP.

@decentral1se
Copy link
Contributor

Perhaps #2069 is related ...

@decentral1se decentral1se mentioned this issue May 28, 2019
Closed
4 tasks
@decentral1se decentral1se mentioned this issue May 28, 2019
Merged
decentral1se added a commit to decentral1se/molecule that referenced this issue May 28, 2019
@decentral1se
Add paramiko explicitly
d21ac3a 
Closes ansible#2070.

See https://github.com/ansible/ansible/blob/stable-2.8/changelogs/CHANGELOG-v2.8.rst.
See https://www.paramiko.org/installing.html.
@decentral1se decentral1se mentioned this issue May 28, 2019
Merged
@decentral1se
Copy link
Contributor

OK, I've given this a stab at #2076.

ssbarnea pushed a commit that referenced this issue May 29, 2019
@decentral1se @ssbarnea
Add paramiko explicitly (#2076)
95305db 
Closes #2070.

See https://github.com/ansible/ansible/blob/stable-2.8/changelogs/CHANGELOG-v2.8.rst.
See https://www.paramiko.org/installing.html.
@jonashackt jonashackt mentioned this issue Jun 13, 2019
Closed
@jonashackt
Copy link
Contributor Author

jonashackt commented Jun 13, 2019 •
edited
Loading

Same issue with Google GCE driver (jonashackt/molecule-ansible-google-cloud#2, build log: https://travis-ci.org/jonashackt/molecule-ansible-google-cloud/builds/545286129), but I guess #2076 will fix that too :) Waiting for 2.22 final release 🤙

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug critical
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add paramiko dependency (made optional in Ansible 2.8+) decentral1se/molecule
2 participants
@decentral1se @jonashackt