Version 3.1.10 generates failure with "ModuleNotFoundError: No module named 'openapi_spec_validator'"

[w0rmr1d3r]

bug

Read the issue template.

Looked for my issue or a similar one and haven't found anything.
It happens on version 3.1.10, which was released yesterday. The newest issue doesn't seem related either.

On version 3.1.10, tests fails, traceback is:

==================================== ERRORS ====================================
[16](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:17)
_________________ ERROR collecting tests/test_boto3_client.py __________________
[17](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:18)
ImportError while importing test module '/home/runner/work/cfripper/cfripper/tests/test_boto3_client.py'.
[18](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:19)
Hint: make sure your test modules/packages have valid Python names.
[19](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:20)
Traceback:
[20](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:21)
/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/importlib/__init__.py:126: in import_module
[21](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:22)
    return _bootstrap._gcd_import(name[level:], package, level)
[22](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:23)
tests/test_boto3_client.py:305: in <module>
[23](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:24)
    def test_export_values(boto3_client: Boto3Client):
[24](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:25)
/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/moto/__init__.py:30: in f
[25](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:26)
    module = importlib.import_module(module_name, "moto")
[26](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:27)
/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/importlib/__init__.py:126: in import_module
[27](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:28)
    return _bootstrap._gcd_import(name[level:], package, level)
[28](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:29)
/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/moto/cloudformation/__init__.py:1: in <module>
[29](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:30)
    from .models import cloudformation_backends
[30](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:31)
/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/moto/cloudformation/models.py:18: in <module>
[31](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:32)
    from .parsing import ResourceMap, OutputMap
[32](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:33)
/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/moto/cloudformation/parsing.py:17: in <module>
[33](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:34)
    from moto.apigateway import models  # noqa  # pylint: disable=all
[34](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:35)
/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/moto/apigateway/__init__.py:1: in <module>
[35](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:36)
    from .models import apigateway_backends
[36](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:37)
/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/moto/apigateway/models.py:9: in <module>
[37](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:38)
    from openapi_spec_validator import validate_spec
[38](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:39)
E   ModuleNotFoundError: No module named 'openapi_spec_validator'
[39](https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true#step:6:40)
---- generated xml file: /home/runner/work/cfripper/cfripper/build/test.xml ----

ref. -> https://github.com/Skyscanner/cfripper/runs/6573693883?check_suite_focus=true

We were using 3.1.6 and it worked fine. Tested the same on 3.1.9 and works fine.
When using 3.1.10 the coverage of the project breaks.
This is the PR where the issue has been found:
Skyscanner/cfripper#227

Happy to provide more information.

thank you :)

[bblommers]

Hi @w0rmr1d3r, thanks for raising this! Looks like our dependencies our indeed wrong - we should install openapi_spec_validator as part of the cloudformation-module.

As a workaround, you could install Moto like this:
moto[apigateway,cloudformation,s3]
to ensure it pulls in the correct dependencies (or indeed revert to 3.1.9 until it's fixed).

[w0rmr1d3r]

Hello @bblommers ,
thank you for such a fast response 😄

I see there's already a PR open to fix this issue, perfect!
We will use 3.1.9 for now and update in the future to newer versions.

thank you as well for the other workaround said !

[sodul]

@bblommers FYI we are getting the very same stack trace with 3.1.16, we are pinning ourselves to 3.1.9 for now.

  File "/Users/stephane/.pyenv/versions/3.10.5/lib/python3.10/site-packages/moto/apigateway/models.py", line 9, in <module>
    from openapi_spec_validator import validate_spec
ModuleNotFoundError: No module named 'openapi_spec_validator'

[bblommers]

@sodul How are you installing Moto, and at which point are getting the error?

[sodul]

@bblommers pip install moto[s3,sns,sqs]==3.1.9, although through a requirements.txt file. We get the stack trace while running our unittests.

The line that breaks is:

from moto import mock_cloudformation

So ...it looks like we just need to install cloudformation explicitly. Updating our requirements to moto[cloudformation,s3,sns,sqs]==3.1.16 worked.

Sorry for the false alarm.