Order listeners documentation

src/main/asciidoc/docs/testng_listeners.adoc

@@ -214,3 +214,148 @@ Finishing
 
 This mechanism allows you to apply the same set of listeners to an entire organization just by adding a jar file to the classpath, instead of asking every single developer to remember to specify these listeners in their `testng.xml` file.
 
+==== Ordering listeners in TestNG
+
+TestNG now allows you to control the order in which listeners are executed.
+
+This is particularly useful when you have multiple listeners that are altering the test result states and so you would like to ensure that they do so in some deterministic order.
+
+TIP: This feature is ONLY available from TestNG version `7.10.0`
+
+Following is how we can get this done.
+
+* To start with, you would need to build an implementation of the interface `org.testng.ListenerComparator`
+* Now this implementation needs to be plugged into TestNG via the configuration parameter `-listenercomparator`
+
+CAUTION: TestNG orders only user's listeners and which are not part of the exclusion list that can be specified via the JVM argument `-Dtestng.preferential.listeners.package` (Multiple fully qualified class names can be specified as comma separated values). If nothing is specified, TestNG by default excludes all the IntelliJ IDEA listeners under the package `com.intellij.rt.*`.
+
+Let's see an example.
+
+* Let's create a custom annotation that will help us define the order in which *our* listeners should be invoked.
+
+[source, java]
+
+----
+@Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
+@Target({ TYPE})
+
+public @interface RunOrder {
+  int value();
+}
+----
+
+* Now let's create an implementation of `org.testng.ListenerComparator` which honours this custom annotation that we just now created.
+
+[source, java]
+
+----
+import org.testng.ITestNGListener;
+import org.testng.ListenerComparator;
+import java.util.Optional;
+
+public class AnnotationBackedListenerComparator implements ListenerComparator {
+
+  @Override
+  public int compare(ITestNGListener l1, ITestNGListener l2) {
+    int first = getRunOrder(l1);
+    int second = getRunOrder(l2);
+    return Integer.compare(first, second);
+  }
+
+  private static int getRunOrder(ITestNGListener listener) {
+    RunOrder runOrder = listener.getClass().getAnnotation(RunOrder.class);
+    return Optional.ofNullable(runOrder)
+        .map(RunOrder::value)
+        .orElse(Integer.MAX_VALUE); //If annotation was not found then return a max value so that
+    //the listener can be plugged in to the end.
+  }
+}
+----
+
+* Lets say we have the below listeners as samples.
+
+[source, java]
+
+----
+import org.testng.IExecutionListener;
+
+public class ExecutionListenerHolder {
+
+  public static final String PREFIX = ExecutionListenerHolder.class.getName() + "$";
+
+  public abstract static class KungFuWarrior implements IExecutionListener {
+    @Override
+    public void onExecutionStart() {
+      System.err.println(getClass().getSimpleName() + ".onExecutionStart");
+    }
+
+    @Override
+    public void onExecutionFinish() {
+      System.err.println(getClass().getSimpleName() + ".onExecutionFinish");
+    }
+  }
+
+  @RunOrder(1)
+  public static class MasterOogway extends KungFuWarrior { }
+
+  @RunOrder(2)
+  public static class MasterShifu extends KungFuWarrior { }
+
+  public static class DragonWarrior extends KungFuWarrior { }
+}
+----
+
+* A sample code snippet that uses the TestNG APIs to run the test could look like below:
+
+[source, java]
+
+----
+TestNG testng = create(NormalSampleTestCase.class);
+testng.addListener(new ExecutionListenerHolder.DragonWarrior());
+testng.addListener(new ExecutionListenerHolder.MasterShifu());
+testng.addListener(new ExecutionListenerHolder.MasterOogway());
+testng.setListenerComparator(new AnnotationBackedListenerComparator());
+testng.run();
+----
+
+Here's a variant of the same above sample, which invokes the `main()` method:
+
+[source, java]
+
+----
+String prefix = ExecutionListenerHolder.PREFIX;
+String[] args = new String[] {
+    "-listener",
+    prefix + "DragonWarrior,"+
+    prefix + "MasterShifu,"+
+    prefix + "MasterOogway",
+    "-testclass",
+    NormalSampleTestCase.class.getName(),
+    "-listenercomparator",
+    AnnotationBackedListenerComparator.class.getName()
+};
+TestNG.main(args);
+----
+
+Here's how the trimmed version of execution output would look like:
+
+[source, bash]
+
+----
+MasterOogway.onExecutionStart
+MasterShifu.onExecutionStart
+DragonWarrior.onExecutionStart
+
+===============================================
+Command line suite
+Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
+===============================================
+
+DragonWarrior.onExecutionFinish
+MasterShifu.onExecutionFinish
+MasterOogway.onExecutionFinish
+----
+
+NOTE: As seen from the above output, we wanted to have the listener `MasterOogway` be invoked first, followed by the listener `MasterShifu` and finally `DragonWarrior` (because this class does not have any `@RunOrder` annotation and hence it should be ONLY added at the end)
+
+TIP: Also it should be noted that the teardown methods get executed in a symmetrical order. So if `onExecutionStart()` of the listener `DragonWarrior` got executed as the last listener, then its corresponding `onExecutionFinish()` would be called first.