Skip to content

Commit

Permalink
Document method ordering support in the User Guide
Browse files Browse the repository at this point in the history
Issue: #13
  • Loading branch information
sbrannen committed Nov 9, 2018
1 parent 68b1e6f commit 1856ba2
Show file tree
Hide file tree
Showing 3 changed files with 82 additions and 1 deletion.
6 changes: 6 additions & 0 deletions documentation/src/docs/asciidoc/link-attributes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ endif::[]
:AfterAllCallback: {javadoc-root}/org/junit/jupiter/api/extension/AfterAllCallback.html[AfterAllCallback]
:AfterEachCallback: {javadoc-root}/org/junit/jupiter/api/extension/AfterEachCallback.html[AfterEachCallback]
:AfterTestExecutionCallback: {javadoc-root}/org/junit/jupiter/api/extension/AfterTestExecutionCallback.html[AfterTestExecutionCallback]
:Alphanumeric: {javadoc-root}/org/junit/jupiter/api/MethodOrderer.Alphanumeric.html[Alphanumeric]
:ArgumentsAccessor: {javadoc-root}/org/junit/jupiter/params/aggregator/ArgumentsAccessor.html[ArgumentsAccessor]
:ArgumentsAggregator: {javadoc-root}/org/junit/jupiter/params/aggregator/ArgumentsAggregator.html[ArgumentsAggregator]
:Assertions: {javadoc-root}/org/junit/jupiter/api/Assertions.html[org.junit.jupiter.api.Assertions]
Expand All @@ -62,10 +63,14 @@ endif::[]
:ExtendWith: {javadoc-root}/org/junit/jupiter/api/extension/ExtendWith.html[@ExtendWith]
:ExtensionContext: {javadoc-root}/org/junit/jupiter/api/extension/ExtensionContext.html[ExtensionContext]
:ExtensionContext_Store: {javadoc-root}/org/junit/jupiter/api/extension/ExtensionContext.Store.html[Store]
:MethodOrderer: {javadoc-root}/org/junit/jupiter/api/MethodOrderer.html[MethodOrderer]
:MethodSource: {javadoc-root}/org/junit/jupiter/params/provider/MethodSource.html[@MethodSource]
:Order: {javadoc-root}/org/junit/jupiter/api/Order.html[@Order]
:OrderAnnotation: {javadoc-root}/org/junit/jupiter/api/MethodOrderer.OrderAnnotation.html[OrderAnnotation]
:ParameterContext: {javadoc-root}/org/junit/jupiter/api/extension/ParameterContext.html[ParameterContext]
:ParameterizedTest: {javadoc-root}/org/junit/jupiter/params/ParameterizedTest.html[@ParameterizedTest]
:ParameterResolver: {javadoc-root}/org/junit/jupiter/api/extension/ParameterResolver.html[ParameterResolver]
:Random: {javadoc-root}/org/junit/jupiter/api/MethodOrderer.Random.html[Random]
:RegisterExtension: {javadoc-root}/org/junit/jupiter/api/extension/RegisterExtension.html[@RegisterExtension]
:RepetitionInfo: {javadoc-root}/org/junit/jupiter/api/RepetitionInfo.html[RepetitionInfo]
:ResourceLock: {javadoc-root}/org/junit/jupiter/api/parallel/ResourceLock.html[@ResourceLock]
Expand All @@ -74,6 +79,7 @@ endif::[]
:TestInfo: {javadoc-root}/org/junit/jupiter/api/TestInfo.html[TestInfo]
:TestInstanceFactory: {javadoc-root}/org/junit/jupiter/api/extension/TestInstanceFactory.html[TestInstanceFactory]
:TestInstancePostProcessor: {javadoc-root}/org/junit/jupiter/api/extension/TestInstancePostProcessor.html[TestInstancePostProcessor]
:TestMethodOrder: {javadoc-root}/org/junit/jupiter/api/TestMethodOrder.html[@TestMethodOrder]
:TestReporter: {javadoc-root}/org/junit/jupiter/api/TestReporter.html[TestReporter]
:TestTemplate: {javadoc-root}/org/junit/jupiter/api/TestTemplate.html[@TestTemplate]
:TestTemplateInvocationContext: {javadoc-root}/org/junit/jupiter/api/extension/TestTemplateInvocationContext.html[TestTemplateInvocationContext]
Expand Down
40 changes: 39 additions & 1 deletion documentation/src/docs/asciidoc/user-guide/writing-tests.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@ module.
| `@ParameterizedTest` | Denotes that a method is a <<writing-tests-parameterized-tests, parameterized test>>. Such methods are _inherited_ unless they are _overridden_.
| `@RepeatedTest` | Denotes that a method is a test template for a <<writing-tests-repeated-tests, repeated test>>. Such methods are _inherited_ unless they are _overridden_.
| `@TestFactory` | Denotes that a method is a test factory for <<writing-tests-dynamic-tests, dynamic tests>>. Such methods are _inherited_ unless they are _overridden_.
| `@TestInstance` | Used to configure the <<writing-tests-test-instance-lifecycle, test instance lifecycle>> for the annotated test class. Such annotations are _inherited_.
| `@TestTemplate` | Denotes that a method is a <<writing-tests-test-templates, template for test cases>> designed to be invoked multiple times depending on the number of invocation contexts returned by the registered <<extensions-test-templates, providers>>. Such methods are _inherited_ unless they are _overridden_.
| `@TestMethodOrder` | Used to configure the <<writing-tests-test-execution-order, test method execution order>> for the annotated test class; similar to JUnit 4's `@FixMethodOrder`. Such annotations are _inherited_.
| `@TestInstance` | Used to configure the <<writing-tests-test-instance-lifecycle, test instance lifecycle>> for the annotated test class. Such annotations are _inherited_.
| `@DisplayName` | Declares a custom display name for the test class or test method. Such annotations are not _inherited_.
| `@DisplayNameGeneration` | Declares a custom display name generator for the test class. Such annotations are _inherited_.
| `@BeforeEach` | Denotes that the annotated method should be executed _before_ *each* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, or `@TestFactory` method in the current class; analogous to JUnit 4's `@Before`. Such methods are _inherited_ unless they are _overridden_.
Expand Down Expand Up @@ -363,6 +364,43 @@ characters have been removed.
include::{testDir}/example/TaggingDemo.java[tags=user_guide]
----

[[writing-tests-test-execution-order]]
=== Test Execution Order

By default, test methods will be ordered using a default algorithm that is deterministic
but intentionally nonobvious. This ensures that subsequent runs of a test suite execute
test methods in the same order, thereby allowing for repeatable builds.

NOTE: In this context, a _test method_ is any instance method that is directly or
meta-annotated with `@Test`, `@RepeatedTest`, `@ParameterizedTest`, `@TestFactory`, or
`@TestTemplate`.

Although true _unit tests_ typically should not rely on the order in which they are
executed, there are times when it is necessary to enforce a specific test method
execution order -- for example, when writing _integration tests_ or _functional tests_
where the sequence of the tests is important, especially in conjunction
`@TestInstance(Lifecycle.PER_CLASS)`.

To control the order in which test methods are executed, annotate your test class or test
interface with `{TestMethodOrder}` and specify the desired `{MethodOrderer}`
implementation. You can implement your own custom `MethodOrderer` or use one of the
following built-in `MethodOrderer` implementations.

* `{Alphanumeric}`: sorts test methods _alphanumerically_ based on their names and formal
parameter lists.
* `{OrderAnnotation}`: sorts test methods _numerically_ based on values specified via the
`{Order}` annotation.
* `{Random}`: orders test methods _pseudo-randomly_ and supports configuration of a
custom _seed_.

The following example demonstrates how to guarantee that test methods are executed in the
order specified via the `@Order` annotation.

[source,java,indent=0]
----
include::{testDir}/example/OrderedTestsDemo.java[tags=user_guide]
----

[[writing-tests-test-instance-lifecycle]]
=== Test Instance Lifecycle

Expand Down
37 changes: 37 additions & 0 deletions documentation/src/test/java/example/OrderedTestsDemo.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
* Copyright 2015-2018 the original author or authors.
*
* All rights reserved. This program and the accompanying materials are
* made available under the terms of the Eclipse Public License v2.0 which
* accompanies this distribution and is available at
*
* http://www.eclipse.org/legal/epl-v20.html
*/

package example;

// tag::user_guide[]
import org.junit.jupiter.api.MethodOrderer.OrderAnnotation;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

@TestMethodOrder(OrderAnnotation.class)
class OrderedTestsDemo {

@Test
@Order(1)
void nullValues() {
}

@Test
@Order(2)
void emptyValues() {
}

@Test
@Order(3)
void validValues() {
}
}
// end::user_guide[]

0 comments on commit 1856ba2

Please sign in to comment.