-
-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Parametrize with fixtures #1660
Changes from all commits
dc55551
84f0dce
c6a711c
4f8b8c8
6938592
526c564
5860c60
acfdd85
eb98a8e
1b6bc4d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,148 @@ | ||
========================= | ||
Parametrize with fixtures | ||
========================= | ||
|
||
Problem | ||
------- | ||
|
||
As a user I have functional tests that I would like to run against various | ||
scenarios. | ||
|
||
In this particular example we want to generate a new project based on a | ||
cookiecutter template. We want to test default values but also data that | ||
emulates user input. | ||
|
||
- use default values | ||
|
||
- emulate user input | ||
|
||
- specify 'author' | ||
|
||
- specify 'project_slug' | ||
|
||
- specify 'author' and 'project_slug' | ||
|
||
This is how a functional test could look like: | ||
|
||
.. code-block:: python | ||
|
||
import pytest | ||
|
||
@pytest.fixture | ||
def default_context(): | ||
return {'extra_context': {}} | ||
|
||
|
||
@pytest.fixture(params=[ | ||
{'author': 'alice'}, | ||
{'project_slug': 'helloworld'}, | ||
{'author': 'bob', 'project_slug': 'foobar'}, | ||
]) | ||
def extra_context(request): | ||
return {'extra_context': request.param} | ||
|
||
|
||
@pytest.fixture(params=['default', 'extra']) | ||
def context(request): | ||
if request.param == 'default': | ||
return request.getfuncargvalue('default_context') | ||
else: | ||
return request.getfuncargvalue('extra_context') | ||
|
||
|
||
def test_generate_project(cookies, context): | ||
"""Call the cookiecutter API to generate a new project from a | ||
template. | ||
""" | ||
result = cookies.bake(extra_context=context) | ||
|
||
assert result.exit_code == 0 | ||
assert result.exception is None | ||
assert result.project.isdir() | ||
|
||
|
||
Issues | ||
------ | ||
|
||
* By using ``request.getfuncargvalue()`` we rely on actual fixture function | ||
execution to know what fixtures are involved, due to it's dynamic nature | ||
* More importantly, ``request.getfuncargvalue()`` cannot be combined with | ||
parametrized fixtures, such as ``extra_context`` | ||
* This is very inconvenient if you wish to extend an existing test suite by | ||
certain parameters for fixtures that are already used by tests | ||
|
||
pytest version 3.0 reports an error if you try to run above code:: | ||
|
||
Failed: The requested fixture has no parameter defined for the current | ||
test. | ||
|
||
Requested fixture 'extra_context' | ||
|
||
|
||
Proposed solution | ||
----------------- | ||
|
||
A new function that can be used in modules can be used to dynamically define | ||
fixtures from existing ones. | ||
|
||
.. code-block:: python | ||
|
||
pytest.define_combined_fixture( | ||
name='context', | ||
fixtures=['default_context', 'extra_context'], | ||
) | ||
|
||
The new fixture ``context`` inherits the scope from the used fixtures and yield | ||
the following values. | ||
|
||
- ``{}`` | ||
|
||
- ``{'author': 'alice'}`` | ||
|
||
- ``{'project_slug': 'helloworld'}`` | ||
|
||
- ``{'author': 'bob', 'project_slug': 'foobar'}`` | ||
|
||
Alternative approach | ||
-------------------- | ||
|
||
A new helper function named ``fixture_request`` tells pytest to yield all | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @hackebrot after discussing in person with @hpk42, he convinced me the first approach beats this one. 😁 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @nicoddemus can you please comment here, what made you change your mind? 😄 (I am aware it's been a while already 😁 ) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will try to remember then, but as you said it has been sometime already. 😁 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right, let me help you remember. Next beer is on me. 🍻 |
||
parameters of a fixture. | ||
|
||
.. code-block:: python | ||
|
||
@pytest.fixture(params=[ | ||
pytest.fixture_request('default_context'), | ||
pytest.fixture_request('extra_context'), | ||
]) | ||
def context(request): | ||
"""Returns all values for ``default_context``, one-by-one before it | ||
does the same for ``extra_context``. | ||
|
||
request.param: | ||
- {} | ||
- {'author': 'alice'} | ||
- {'project_slug': 'helloworld'} | ||
- {'author': 'bob', 'project_slug': 'foobar'} | ||
""" | ||
return request.param | ||
|
||
The same helper can be used in combination with ``pytest.mark.parametrize``. | ||
|
||
.. code-block:: python | ||
|
||
|
||
@pytest.mark.parametrize( | ||
'context, expected_response_code', | ||
[ | ||
(pytest.fixture_request('default_context'), 0), | ||
(pytest.fixture_request('extra_context'), 0), | ||
], | ||
) | ||
def test_generate_project(cookies, context, exit_code): | ||
"""Call the cookiecutter API to generate a new project from a | ||
template. | ||
""" | ||
result = cookies.bake(extra_context=context) | ||
|
||
assert result.exit_code == exit_code |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If possible something like this would be more pythonic:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will raise a
flake8
error as it an unused variable.More importantly @hpk42 and myself feel like the keyword arguments make this helper more verbose and comprehensible.