Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Parametrize with fixtures #1660

Merged
merged 10 commits into from
Jun 25, 2016
3 changes: 3 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,9 @@
Thanks `@Vogtinator`_ for reporting. Thanks to `@RedBeardCode`_ and
`@tomviner`_ for PR.

* Add proposal to docs for a new feature that enables users to combine multiple
fixtures into one. Thanks to `@hpk42`_ and `@hackebrot`_.

*

.. _#1580: https://github.com/pytest-dev/pytest/pull/1580
Expand Down
148 changes: 148 additions & 0 deletions doc/en/proposals/parametrize_with_fixtures.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
=========================
Parametrize with fixtures
=========================

Problem
-------

As a user I have functional tests that I would like to run against various
scenarios.

In this particular example we want to generate a new project based on a
cookiecutter template. We want to test default values but also data that
emulates user input.

- use default values

- emulate user input

- specify 'author'

- specify 'project_slug'

- specify 'author' and 'project_slug'

This is how a functional test could look like:

.. code-block:: python

import pytest

@pytest.fixture
def default_context():
return {'extra_context': {}}


@pytest.fixture(params=[
{'author': 'alice'},
{'project_slug': 'helloworld'},
{'author': 'bob', 'project_slug': 'foobar'},
])
def extra_context(request):
return {'extra_context': request.param}


@pytest.fixture(params=['default', 'extra'])
def context(request):
if request.param == 'default':
return request.getfuncargvalue('default_context')
else:
return request.getfuncargvalue('extra_context')


def test_generate_project(cookies, context):
"""Call the cookiecutter API to generate a new project from a
template.
"""
result = cookies.bake(extra_context=context)

assert result.exit_code == 0
assert result.exception is None
assert result.project.isdir()


Issues
------

* By using ``request.getfuncargvalue()`` we rely on actual fixture function
execution to know what fixtures are involved, due to it's dynamic nature
* More importantly, ``request.getfuncargvalue()`` cannot be combined with
parametrized fixtures, such as ``extra_context``
* This is very inconvenient if you wish to extend an existing test suite by
certain parameters for fixtures that are already used by tests

pytest version 3.0 reports an error if you try to run above code::

Failed: The requested fixture has no parameter defined for the current
test.

Requested fixture 'extra_context'


Proposed solution
-----------------

A new function that can be used in modules can be used to dynamically define
fixtures from existing ones.

.. code-block:: python

pytest.define_combined_fixture(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible something like this would be more pythonic:

context = combined_fixture('default_context', 'extra_context')

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will raise a flake8 error as it an unused variable.

More importantly @hpk42 and myself feel like the keyword arguments make this helper more verbose and comprehensible.

name='context',
fixtures=['default_context', 'extra_context'],
)

The new fixture ``context`` inherits the scope from the used fixtures and yield
the following values.

- ``{}``

- ``{'author': 'alice'}``

- ``{'project_slug': 'helloworld'}``

- ``{'author': 'bob', 'project_slug': 'foobar'}``

Alternative approach
--------------------

A new helper function named ``fixture_request`` tells pytest to yield all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hackebrot after discussing in person with @hpk42, he convinced me the first approach beats this one. 😁

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicoddemus can you please comment here, what made you change your mind? 😄 (I am aware it's been a while already 😁 )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to remember then, but as you said it has been sometime already. 😁

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, let me help you remember. Next beer is on me. 🍻

parameters of a fixture.

.. code-block:: python

@pytest.fixture(params=[
pytest.fixture_request('default_context'),
pytest.fixture_request('extra_context'),
])
def context(request):
"""Returns all values for ``default_context``, one-by-one before it
does the same for ``extra_context``.

request.param:
- {}
- {'author': 'alice'}
- {'project_slug': 'helloworld'}
- {'author': 'bob', 'project_slug': 'foobar'}
"""
return request.param

The same helper can be used in combination with ``pytest.mark.parametrize``.

.. code-block:: python


@pytest.mark.parametrize(
'context, expected_response_code',
[
(pytest.fixture_request('default_context'), 0),
(pytest.fixture_request('extra_context'), 0),
],
)
def test_generate_project(cookies, context, exit_code):
"""Call the cookiecutter API to generate a new project from a
template.
"""
result = cookies.bake(extra_context=context)

assert result.exit_code == exit_code