Merge pull request #4 from KStocky/TestCaseWithSetup

TEST_CASE_FIXTURE: A new fixture macro for allowing persistent fixtures throughout a TEST_CASE

docs/cmake-integration.md

@@ -8,6 +8,7 @@
 [`CATCH_CONFIG_*` customization options in CMake](#catch_config_-customization-options-in-cmake)<br>
 [Installing Catch2 from git repository](#installing-catch2-from-git-repository)<br>
 [Installing Catch2 from vcpkg](#installing-catch2-from-vcpkg)<br>
+[Installing Catch2 from Bazel](#installing-catch2-from-bazel)<br>
 
 Because we use CMake to build Catch2, we also provide a couple of
 integration points for our users.

docs/other-macros.md

@@ -93,30 +93,6 @@ TEST_CASE("STATIC_CHECK showcase", "[traits]") {
 
 ## Test case related macros
 
-* `METHOD_AS_TEST_CASE`
-
-`METHOD_AS_TEST_CASE( member-function-pointer, description )` lets you
-register a member function of a class as a Catch2 test case. The class
-will be separately instantiated for each method registered in this way.
-
-```cpp
-class TestClass {
-    std::string s;
-
-public:
-    TestClass()
-        :s( "hello" )
-    {}
-
-    void testCase() {
-        REQUIRE( s == "hello" );
-    }
-};
-
-
-METHOD_AS_TEST_CASE( TestClass::testCase, "Use class's method as a test case", "[class]" )
-```
-
 * `REGISTER_TEST_CASE`
 
 `REGISTER_TEST_CASE( function, description )` let's you register

docs/test-fixtures.md

@@ -1,9 +1,26 @@
 <a id="top"></a>
 # Test fixtures
 
-## Defining test fixtures
+**Contents**<br>
+[Non-Templated test fixtures](#non-templated-test-fixtures)<br>
+[Templated test fixtures](#templated-test-fixtures)<br>
+[Signature-based parameterised test fixtures](#signature-based-parametrised-test-fixtures)<br>
+[Template fixtures with types specified in template type lists](#template-fixtures-with-types-specified-in-template-type-lists)<br>
 
-Although Catch allows you to group tests together as [sections within a test case](test-cases-and-sections.md), it can still be convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure:
+## Non-Templated test fixtures
+
+Although Catch allows you to group tests together as [sections within a test case](test-cases-and-sections.md), it can still be convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too with 3 different macros for non-templated test fixtures. They are: 
+
+| Macro    | Description |
+|----------|-------------|
+|1. `TEST_CASE_METHOD(className, ...)`| Creates a uniquely named class which inherits from the class specified by `className`. The test function will be a member of this derived class. An instance of the derived class will be created for every partial run of the test case. |
+|2. `METHOD_AS_TEST_CASE(member-function, ...)`| Uses `member-function` as the test function. An instance of the class will be created for each partial run of the test case. |
+|3. `TEST_CASE_FIXTURE(className, ...)`| Creates a uniquely named class which inherits from the class specified by `className`. The test function will be a member of this derived class. An instance of the derived class will be created at the start of the test run. That instance will be destroyed once the entire test case has ended. |
+
+### 1. `TEST_CASE_METHOD`
+
+
+You define a `TEST_CASE_METHOD` test fixture as a simple structure:
 
 ```c++
 class UniqueTestsFixture {
@@ -30,8 +47,59 @@ class UniqueTestsFixture {
  }
 ```
 
-The two test cases here will create uniquely-named derived classes of UniqueTestsFixture and thus can access the `getID()` protected method and `conn` member variables. This ensures that both the test cases are able to create a DBConnection using the same method (DRY principle) and that any ID's created are unique such that the order that tests are executed does not matter.
+The two test cases here will create uniquely-named derived classes of UniqueTestsFixture and thus can access the `getID()` protected method and `conn` member variables. This ensures that both the test cases are able to create a DBConnection using the same method (DRY principle) and that any ID's created are unique such that the order that tests are executed does not matter. 
+
+### 2. `METHOD_AS_TEST_CASE`
+
+`METHOD_AS_TEST_CASE` lets you register a member function of a class as a Catch2 test case. The class will be separately instantiated for each method registered in this way.
+
+```cpp
+class TestClass {
+    std::string s;
+
+public:
+    TestClass()
+        :s( "hello" )
+    {}
+
+    void testCase() {
+        REQUIRE( s == "hello" );
+    }
+};
+
+
+METHOD_AS_TEST_CASE( TestClass::testCase, "Use class's method as a test case", "[class]" )
+```
+
+This type of fixture is similar to [TEST_CASE_METHOD](#1-test_case_method) except in this case it will directly use the provided class to create an object rather than a derived class.
+
+### 3. `TEST_CASE_FIXTURE`
+
+> [Introduced](link-to-issue-or-PR) in Catch2 X.Y.Z
+
+`TEST_CASE_FIXTURE` behaves in the same way as [TEST_CASE_METHOD](#1-test_case_method) except that there will only be one instance created throughout the entire run of a test case. To demonstrate this have a look at the following example:
+
+```cpp
+struct MyFixture{
+    int MyInt = 0;
+};
+
+TEST_CASE_FIXTURE(MyFixture, "Tests with MyFixture") { 
+
+    const int val = MyInt++;
+
+    SECTION("First partial run") {
+        REQUIRE(val == 0);
+    }
+
+    SECTION("Second partial run") {
+        REQUIRE(val == 1);
+    }
+}
+```
+This test case will be executed twice as there are two leaf sections. On the first run `val` will be `0` and on the second run `val` will be `1`. This is useful if you would like to share some expensive setup code with all runs of your test case which can't be done at static initialization time.
 
+## Templated test fixtures
 
 Catch2 also provides `TEMPLATE_TEST_CASE_METHOD` and
 `TEMPLATE_PRODUCT_TEST_CASE_METHOD` that can be used together
@@ -93,7 +161,7 @@ _While there is an upper limit on the number of types you can specify
 in single `TEMPLATE_TEST_CASE_METHOD` or `TEMPLATE_PRODUCT_TEST_CASE_METHOD`,
 the limit is very high and should not be encountered in practice._
 
-## Signature-based parametrised test fixtures
+## Signature-based parameterised test fixtures
 
 > [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0.
 

examples/CMakeLists.txt

@@ -25,6 +25,7 @@ target_compile_definitions(231-Cfg_OutputStreams PUBLIC CATCH_CONFIG_NOSTDOUT)
 
 # These examples use the standard separate compilation
 set( SOURCES_IDIOMATIC_EXAMPLES
+    Fixture.cpp
     030-Asn-Require-Check.cpp
     100-Fix-Section.cpp
     110-Fix-ClassFixture.cpp

examples/Fixture.cpp

@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: BSL-1.0
+
+// Fixture.cpp
+
+// Catch has three ways to express fixtures:
+// - Sections
+// - Traditional class-based fixtures that are created and destroyed on every partial run
+// - Traditional class-based fixtures that are created at the start of a test case and destroyed at the end of a test case (this file)
+
+// main() provided by linkage to Catch2WithMain
+
+#include <catch2/catch_test_macros.hpp>
+
+struct MyFixture{
+    int MyInt = 0;
+};
+
+TEST_CASE_FIXTURE(MyFixture, "Tests with MyFixture") {
+
+    const int val = MyInt++;
+
+    SECTION("First partial run") {
+        REQUIRE(val == 0);
+    }
+
+    SECTION("Second partial run") {
+        REQUIRE(val == 1);
+    }
+}

src/catch2/catch_test_case_info.hpp

@@ -112,6 +112,14 @@ namespace Catch {
         TestCaseHandle(TestCaseInfo* info, ITestInvoker* invoker) :
             m_info(info), m_invoker(invoker) {}
 
+        void testCaseStarting() const { 
+            m_invoker->testCaseStarting();
+        }
+
+        void testCaseEnding() const { 
+            m_invoker->testCaseEnding(); 
+        }
+
         void invoke() const {
             m_invoker->invoke();
         }

src/catch2/catch_test_macros.hpp

@@ -43,6 +43,7 @@
   #define CATCH_TEST_CASE( ... ) INTERNAL_CATCH_TESTCASE( __VA_ARGS__ )
   #define CATCH_TEST_CASE_METHOD( className, ... ) INTERNAL_CATCH_TEST_CASE_METHOD( className, __VA_ARGS__ )
   #define CATCH_METHOD_AS_TEST_CASE( method, ... ) INTERNAL_CATCH_METHOD_AS_TEST_CASE( method, __VA_ARGS__ )
+  #define CATCH_TEST_CASE_FIXTURE( className, ... ) INTERNAL_CATCH_TEST_CASE_FIXTURE( className, __VA_ARGS__ )
   #define CATCH_REGISTER_TEST_CASE( Function, ... ) INTERNAL_CATCH_REGISTER_TESTCASE( Function, __VA_ARGS__ )
   #define CATCH_SECTION( ... ) INTERNAL_CATCH_SECTION( __VA_ARGS__ )
   #define CATCH_DYNAMIC_SECTION( ... ) INTERNAL_CATCH_DYNAMIC_SECTION( __VA_ARGS__ )
@@ -97,6 +98,7 @@
   #define CATCH_TEST_CASE( ... ) INTERNAL_CATCH_TESTCASE_NO_REGISTRATION(INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ))
   #define CATCH_TEST_CASE_METHOD( className, ... ) INTERNAL_CATCH_TESTCASE_NO_REGISTRATION(INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ))
   #define CATCH_METHOD_AS_TEST_CASE( method, ... )
+  #define CATCH_TEST_CASE_FIXTURE( className, ... )
   #define CATCH_REGISTER_TEST_CASE( Function, ... ) (void)(0)
   #define CATCH_SECTION( ... )
   #define CATCH_DYNAMIC_SECTION( ... )
@@ -142,6 +144,7 @@
   #define TEST_CASE( ... ) INTERNAL_CATCH_TESTCASE( __VA_ARGS__ )
   #define TEST_CASE_METHOD( className, ... ) INTERNAL_CATCH_TEST_CASE_METHOD( className, __VA_ARGS__ )
   #define METHOD_AS_TEST_CASE( method, ... ) INTERNAL_CATCH_METHOD_AS_TEST_CASE( method, __VA_ARGS__ )
+  #define TEST_CASE_FIXTURE( className, ... ) INTERNAL_CATCH_TEST_CASE_FIXTURE( className, __VA_ARGS__ )
   #define REGISTER_TEST_CASE( Function, ... ) INTERNAL_CATCH_REGISTER_TESTCASE( Function, __VA_ARGS__ )
   #define SECTION( ... ) INTERNAL_CATCH_SECTION( __VA_ARGS__ )
   #define DYNAMIC_SECTION( ... ) INTERNAL_CATCH_DYNAMIC_SECTION( __VA_ARGS__ )
@@ -195,6 +198,7 @@
   #define TEST_CASE( ... )  INTERNAL_CATCH_TESTCASE_NO_REGISTRATION(INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ), __VA_ARGS__)
   #define TEST_CASE_METHOD( className, ... ) INTERNAL_CATCH_TESTCASE_NO_REGISTRATION(INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ))
   #define METHOD_AS_TEST_CASE( method, ... )
+  #define TEST_CASE_FIXTURE( className, ... )
   #define REGISTER_TEST_CASE( Function, ... ) (void)(0)
   #define SECTION( ... )
   #define DYNAMIC_SECTION( ... )

src/catch2/interfaces/catch_interfaces_test_invoker.hpp

@@ -12,6 +12,8 @@ namespace Catch {
 
     class ITestInvoker {
     public:
+        virtual void testCaseStarting();
+        virtual void testCaseEnding();
         virtual void invoke() const = 0;
         virtual ~ITestInvoker(); // = default
     };

src/catch2/internal/catch_run_context.cpp

@@ -185,6 +185,7 @@ namespace Catch {
 
         auto const& testInfo = testCase.getTestCaseInfo();
         m_reporter->testCaseStarting(testInfo);
+        testCase.testCaseStarting();
         m_activeTestCase = &testCase;
 
 
@@ -254,6 +255,7 @@ namespace Catch {
             deltaTotals.testCases.failed++;
         }
         m_totals.testCases += deltaTotals.testCases;
+        testCase.testCaseEnding();
         m_reporter->testCaseEnded(TestCaseStats(testInfo,
                                   deltaTotals,
                                   CATCH_MOVE(redirectedCout),

src/catch2/internal/catch_test_registry.cpp

@@ -16,6 +16,8 @@
 #include <iterator>
 
 namespace Catch {
+    void ITestInvoker::testCaseStarting() {}
+    void ITestInvoker::testCaseEnding() {}
     ITestInvoker::~ITestInvoker() = default;
 
     namespace {

src/catch2/internal/catch_test_registry.hpp

@@ -47,6 +47,33 @@ Detail::unique_ptr<ITestInvoker> makeTestInvoker( void (C::*testAsMethod)() ) {
     return Detail::make_unique<TestInvokerAsMethod<C>>( testAsMethod );
 }
 
+template <typename C>
+class TestInvokerFixture : public ITestInvoker {
+    void ( C::*m_testAsMethod )();
+    Detail::unique_ptr<C> m_fixture = nullptr;
+
+public:
+    TestInvokerFixture( void ( C::*testAsMethod )() ) noexcept : m_testAsMethod( testAsMethod ) {}
+
+    void testCaseStarting() override { 
+        m_fixture = Detail::make_unique<C>(); 
+    }
+
+    void testCaseEnding() override { 
+        m_fixture.reset(); 
+    }
+
+    void invoke() const override {
+        auto* f = const_cast<C*>( m_fixture.get() );
+        ( f->*m_testAsMethod )();
+    }
+};
+
+template<typename C>
+Detail::unique_ptr<ITestInvoker> makeTestInvokerFixture( void ( C::*testAsMethod )() ) {
+    return Detail::make_unique<TestInvokerFixture<C>>( testAsMethod );
+}
+
 struct NameAndTags {
     constexpr NameAndTags( StringRef name_ = StringRef(),
                            StringRef tags_ = StringRef() ) noexcept:
@@ -143,6 +170,26 @@ static int catchInternalSectionHint = 0;
     #define INTERNAL_CATCH_TEST_CASE_METHOD( ClassName, ... ) \
         INTERNAL_CATCH_TEST_CASE_METHOD2( INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ), ClassName, __VA_ARGS__ )
 
+    ///////////////////////////////////////////////////////////////////////////////
+    #define INTERNAL_CATCH_TEST_CASE_FIXTURE2( TestName, ClassName, ... )      \
+        CATCH_INTERNAL_START_WARNINGS_SUPPRESSION                             \
+        CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS                              \
+        CATCH_INTERNAL_SUPPRESS_UNUSED_VARIABLE_WARNINGS                      \
+        namespace {                                                           \
+            struct TestName : INTERNAL_CATCH_REMOVE_PARENS( ClassName ) {     \
+                void test();                                                  \
+            };                                                                \
+            const Catch::AutoReg INTERNAL_CATCH_UNIQUE_NAME( autoRegistrar )( \
+                Catch::makeTestInvokerFixture( &TestName::test ),                    \
+                CATCH_INTERNAL_LINEINFO,                                      \
+                #ClassName##_catch_sr,                                        \
+                Catch::NameAndTags{ __VA_ARGS__ } ); /* NOLINT */             \
+        }                                                                     \
+        CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION                              \
+        void TestName::test()
+    #define INTERNAL_CATCH_TEST_CASE_FIXTURE( ClassName, ... )    \
+        INTERNAL_CATCH_TEST_CASE_FIXTURE2( INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ), ClassName, __VA_ARGS__ )
+
 
     ///////////////////////////////////////////////////////////////////////////////
     #define INTERNAL_CATCH_METHOD_AS_TEST_CASE( QualifiedMethod, ... ) \

tests/SelfTest/Baselines/automake.sw.approved.txt

@@ -66,6 +66,8 @@ Nor would this
 :test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1
 :test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3
 :test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6
+:test-result: FAIL A TEST_CASE_FIXTURE based test run that fails
+:test-result: PASS A TEST_CASE_FIXTURE based test run that succeeds
 :test-result: FAIL A TEST_CASE_METHOD based test run that fails
 :test-result: PASS A TEST_CASE_METHOD based test run that succeeds
 :test-result: PASS A Template product test case - Foo<float>

tests/SelfTest/Baselines/automake.sw.multi.approved.txt

@@ -64,6 +64,8 @@
 :test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1
 :test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3
 :test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6
+:test-result: FAIL A TEST_CASE_FIXTURE based test run that fails
+:test-result: PASS A TEST_CASE_FIXTURE based test run that succeeds
 :test-result: FAIL A TEST_CASE_METHOD based test run that fails
 :test-result: PASS A TEST_CASE_METHOD based test run that succeeds
 :test-result: PASS A Template product test case - Foo<float>

tests/SelfTest/Baselines/compact.sw.approved.txt

@@ -239,6 +239,10 @@ Class.tests.cpp:<line number>: failed: Nttp_Fixture<V>::value == 0 for: 6 == 0
 Class.tests.cpp:<line number>: passed: Nttp_Fixture<V>::value > 0 for: 1 > 0
 Class.tests.cpp:<line number>: passed: Nttp_Fixture<V>::value > 0 for: 3 > 0
 Class.tests.cpp:<line number>: passed: Nttp_Fixture<V>::value > 0 for: 6 > 0
+Class.tests.cpp:<line number>: passed: m_a++ == 0 for: 0 == 0
+Class.tests.cpp:<line number>: failed: m_a == 0 for: 1 == 0
+Class.tests.cpp:<line number>: passed: m_a++ == 0 for: 0 == 0
+Class.tests.cpp:<line number>: passed: m_a == 1 for: 1 == 1
 Class.tests.cpp:<line number>: failed: m_a == 2 for: 1 == 2
 Class.tests.cpp:<line number>: passed: m_a == 1 for: 1 == 1
 Misc.tests.cpp:<line number>: passed: x.size() == 0 for: 0 == 0
@@ -2840,7 +2844,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
 InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
 Misc.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: passed:
-test cases:  416 |  311 passed |  85 failed | 6 skipped | 14 failed as expected
-assertions: 2255 | 2074 passed | 146 failed | 35 failed as expected
+test cases:  418 |  312 passed |  86 failed | 6 skipped | 14 failed as expected
+assertions: 2259 | 2077 passed | 147 failed | 35 failed as expected
 
 

tests/SelfTest/Baselines/compact.sw.multi.approved.txt

@@ -237,6 +237,10 @@ Class.tests.cpp:<line number>: failed: Nttp_Fixture<V>::value == 0 for: 6 == 0
 Class.tests.cpp:<line number>: passed: Nttp_Fixture<V>::value > 0 for: 1 > 0
 Class.tests.cpp:<line number>: passed: Nttp_Fixture<V>::value > 0 for: 3 > 0
 Class.tests.cpp:<line number>: passed: Nttp_Fixture<V>::value > 0 for: 6 > 0
+Class.tests.cpp:<line number>: passed: m_a++ == 0 for: 0 == 0
+Class.tests.cpp:<line number>: failed: m_a == 0 for: 1 == 0
+Class.tests.cpp:<line number>: passed: m_a++ == 0 for: 0 == 0
+Class.tests.cpp:<line number>: passed: m_a == 1 for: 1 == 1
 Class.tests.cpp:<line number>: failed: m_a == 2 for: 1 == 2
 Class.tests.cpp:<line number>: passed: m_a == 1 for: 1 == 1
 Misc.tests.cpp:<line number>: passed: x.size() == 0 for: 0 == 0
@@ -2829,7 +2833,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
 InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
 Misc.tests.cpp:<line number>: passed:
 Misc.tests.cpp:<line number>: passed:
-test cases:  416 |  311 passed |  85 failed | 6 skipped | 14 failed as expected
-assertions: 2255 | 2074 passed | 146 failed | 35 failed as expected
+test cases:  418 |  312 passed |  86 failed | 6 skipped | 14 failed as expected
+assertions: 2259 | 2077 passed | 147 failed | 35 failed as expected