All ordering tests are failing (git develop)

[ArchangeGabriel]

This is the result of the test suite on our (Arch Linux) building system:

============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /build/python-pytest-ordering/src/pytest-ordering-0.6
collected 16 items

tests/test_misc.py ..                                                    [ 12%]
tests/test_ordering.py .FFFFFFFFFFFFF                                    [100%]

=================================== FAILURES ===================================
_______________________________ test_first_mark ________________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a4cc820>

    def test_first_mark(item_names_for):
        tests_content = """
        import pytest
    
        def test_1(): pass
    
        @pytest.mark.first
        def test_2(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_1']
E       AssertionError: assert ['test_1', 'test_2'] == ['test_2', 'test_1']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:43: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_first_mark0
collected 0 items

============================ no tests ran in 0.00s =============================
________________________________ test_last_mark ________________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a48fca0>

    def test_last_mark(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.last
        def test_1(): pass
    
        def test_2(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_1']
E       AssertionError: assert ['test_1', 'test_2'] == ['test_2', 'test_1']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:56: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_last_mark0
collected 0 items

============================ no tests ran in 0.00s =============================
____________________________ test_first_last_marks _____________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a439280>

    def test_first_last_marks(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.last
        def test_1(): pass
    
        @pytest.mark.first
        def test_2(): pass
    
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_3', 'test_1']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_2', 'test_3', 'test_1']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:72: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_first_last_marks0
collected 0 items

============================ no tests ran in 0.00s =============================
_______________________________ test_order_marks _______________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a4394c0>

    def test_order_marks(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=-1)
        def test_1(): pass
    
        @pytest.mark.run(order=-2)
        def test_2(): pass
    
        @pytest.mark.run(order=1)
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_2', 'test_1']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_3', 'test_2', 'test_1']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:89: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_order_marks0
collected 0 items

============================ no tests ran in 0.00s =============================
_________________________ test_non_contiguous_positive _________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a48ff70>

    def test_non_contiguous_positive(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=10)
        def test_1(): pass
    
        @pytest.mark.run(order=20)
        def test_2(): pass
    
        @pytest.mark.run(order=5)
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_1', 'test_2']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_3', 'test_1', 'test_2']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:106: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_non_contiguous_positive0
collected 0 items

============================ no tests ran in 0.00s =============================
_________________________ test_non_contiguous_negative _________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a439e50>

    def test_non_contiguous_negative(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=-10)
        def test_1(): pass
    
        @pytest.mark.run(order=-20)
        def test_2(): pass
    
        @pytest.mark.run(order=-5)
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_1', 'test_3']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_2', 'test_1', 'test_3']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:123: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_non_contiguous_negative0
collected 0 items

============================ no tests ran in 0.00s =============================
_________________________ test_non_contiguous_inc_zero _________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a34b0d0>

    def test_non_contiguous_inc_zero(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=10)
        def test_1(): pass
    
        @pytest.mark.run(order=20)
        def test_2(): pass
    
        @pytest.mark.run(order=5)
        def test_3(): pass
    
        @pytest.mark.run(order=-10)
        def test_4(): pass
    
        @pytest.mark.run(order=-20)
        def test_5(): pass
    
        @pytest.mark.run(order=-5)
        def test_6(): pass
    
        @pytest.mark.run(order=0)
        def test_7(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_7', 'test_3', 'test_1', 'test_2', 'test_5', 'test_4', 'test_6']
E       AssertionError: assert ['test_1', 't...'test_6', ...] == ['test_7', 't...'test_4', ...]
E         At index 0 diff: 'test_1' != 'test_7'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:152: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_non_contiguous_inc_zero0
collected 0 items

============================ no tests ran in 0.00s =============================
_________________________ test_non_contiguous_inc_none _________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a439ee0>

    def test_non_contiguous_inc_none(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=5)
        def test_1(): pass
    
        @pytest.mark.run(order=0)
        def test_2(): pass
    
        @pytest.mark.run(order=1)
        def test_3(): pass
    
        @pytest.mark.run(order=-1)
        def test_4(): pass
    
        @pytest.mark.run(order=-5)
        def test_5(): pass
    
        def test_6(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_3', 'test_1', 'test_6', 'test_5', 'test_4']
E       AssertionError: assert ['test_1', 't..._5', 'test_6'] == ['test_2', 't..._5', 'test_4']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:177: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_non_contiguous_inc_none0
collected 0 items

============================ no tests ran in 0.00s =============================
____________________________ test_first_mark_class _____________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a2c9940>

    def test_first_mark_class(item_names_for):
        tests_content = """
        import pytest
    
        def test_1(): pass
    
    
        @pytest.mark.first
        class TestSuite(object):
    
            def test_3(self): pass
    
            def test_2(self): pass
    
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_2', 'test_1']
E       AssertionError: assert ['test_1', 'test_3', 'test_2'] == ['test_3', 'test_2', 'test_1']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:196: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_first_mark_class0
collected 0 items

============================ no tests ran in 0.00s =============================
_____________________________ test_last_mark_class _____________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a34bd30>

    def test_last_mark_class(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.last
        class TestSuite(object):
    
            def test_1(self): pass
    
            def test_2(self): pass
    
    
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_1', 'test_2']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_3', 'test_1', 'test_2']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:214: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_last_mark_class0
collected 0 items

============================ no tests ran in 0.00s =============================
__________________________ test_first_last_mark_class __________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a2b8ee0>

    def test_first_last_mark_class(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.last
        class TestLast(object):
    
            def test_1(self): pass
    
            def test_2(self): pass
    
    
        def test_3(): pass
    
    
        @pytest.mark.first
        class TestFirst(object):
    
            def test_4(self): pass
    
            def test_5(self): pass
    
        """
    
>       assert item_names_for(tests_content) == ['test_4', 'test_5', 'test_3', 'test_1', 'test_2']
E       AssertionError: assert ['test_1', 't..._4', 'test_5'] == ['test_4', 't..._1', 'test_2']
E         At index 0 diff: 'test_1' != 'test_4'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:241: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_first_last_mark_class0
collected 0 items

============================ no tests ran in 0.00s =============================
____________________________ test_order_mark_class _____________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a439ee0>

    def test_order_mark_class(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=-1)
        class TestLast(object):
    
            def test_1(self): pass
    
            def test_2(self): pass
    
    
        @pytest.mark.run(order=0)
        def test_3(): pass
    
    
        @pytest.mark.run(order=-2)
        class TestFirst(object):
    
            def test_4(self): pass
    
            def test_5(self): pass
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_4', 'test_5', 'test_1', 'test_2']
E       AssertionError: assert ['test_1', 't..._4', 'test_5'] == ['test_3', 't..._1', 'test_2']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:268: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_order_mark_class0
collected 0 items

============================ no tests ran in 0.00s =============================
___________________________ test_markers_registered ____________________________

capsys = <_pytest.capture.CaptureFixture object at 0x7f108a164c10>

    def test_markers_registered(capsys):
        pytest.main(['--markers'])
        out, err = capsys.readouterr()
>       assert '@pytest.mark.run' in out
E       AssertionError: assert '@pytest.mark.run' in '@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/latest/w...: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.\n\n'

tests/test_ordering.py:274: AssertionError
=============================== warnings summary ===============================
tests/test_ordering.py::test_first_mark
tests/test_ordering.py::test_first_last_marks
tests/test_ordering.py::test_first_mark_class
tests/test_ordering.py::test_first_last_mark_class
  /usr/lib/python3.8/site-packages/_pytest/mark/structures.py:323: PytestUnknownMarkWarning: Unknown pytest.mark.first - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

tests/test_ordering.py::test_last_mark
tests/test_ordering.py::test_first_last_marks
tests/test_ordering.py::test_last_mark_class
tests/test_ordering.py::test_first_last_mark_class
  /usr/lib/python3.8/site-packages/_pytest/mark/structures.py:323: PytestUnknownMarkWarning: Unknown pytest.mark.last - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

tests/test_ordering.py::test_order_marks
tests/test_ordering.py::test_non_contiguous_positive
tests/test_ordering.py::test_non_contiguous_negative
tests/test_ordering.py::test_non_contiguous_inc_zero
tests/test_ordering.py::test_non_contiguous_inc_none
tests/test_ordering.py::test_order_mark_class
  /usr/lib/python3.8/site-packages/_pytest/mark/structures.py:323: PytestUnknownMarkWarning: Unknown pytest.mark.run - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

-- Docs: https://docs.pytest.org/en/latest/warnings.html
================== 13 failed, 3 passed, 14 warnings in 1.30s ===================

Seems that none of the tests were ordered…

[terrynguyen255]

Anyone have any ideas?

[ArchangeGabriel]

Yes:

pytest-ordering is no longer maintained, please use https://pypi.org/project/pytest-order/