How to Run a Grinder Build on Jenkins

How to run a Grinder test job at AdoptOpenJDK

Using AdoptOpenJDK Grinder job to test your new CS or rerun a test. These instructions also apply to Grinder jobs at the openj9 Jenkins server (and internal servers).

• Click Build with Parameters, or go to the Grinder job in your internal Jenkins instance and set your own parameters:
  • ADOPTOPENJDK_REPO : optional: your fork of openjdk-tests
  • ADOPTOPENJDK_BRANCH : optional, your branch off of your fork of openjdk-tests
  • PLATFORM : required, platform on which to run the test (used to determine machines labels to find machines on which to run)
  • JDK_VERSION : (required, JDK_VERSION that matches your SDK. e.g., 8, 9, 10, 11, 12)
  • JDK_IMPL : different JVM implementations (hotspot, openj9, sap, ibm)
  • BUILD_LIST : optional, reduces the amount of directories that will be compiled. e.g., openjdk, system, perf, external, functional. Any sub test directory you wish to compile
  • TARGET : required, which test target you want to run. Top-level targets such as sanity.openjdk, extended.system are inappropriate for Grinders as they contain so many sub-targets and have a long execution time. Test targets are defined in playlist.xml files (example jdk_math or jdk_custom). Refer to testCaseName in the playlist.xml files for the values to use for the TARGET parameter.
  • CUSTOM_TARGET : if TARGET=jdk_custom|hotspot_custom|langtools_custom, you can set this to be the specific test class name. For example test/jdk/java/math/BigInteger/BigIntegerTest.java
  • SDK_RESOURCE : relates to which build you wish to test (nightly, releases, customized)
  • CUSTOMIZED_SDK_URL : Link to download SDK. This is needed when SDK_RESOURCE=customized
    Optional, link to download native test libs. (Links can be separated by a space)
  • EXTRA_OPTIONS : set this to append additional JVM options to the test run
    In general, JVM options can be directly used. Please try to use JVM options as it is. For example, -Xaot:{jdk/incubator/vector/*Vector*Mask.lambda*anyTrue*}(traceFull,traceRelocatableDataCG,traceRelocatableDataDetailsCG,log=log.trc) can be directly used for openjdk tests. However, for system tests, the special characters may need to be escaped in EXTRA_OPTIONS and JVM_OPTIONS.
    Example 1:
    '-Xjit:count=0,\{java/lang/reflect/Method.getParameterAnnotations*\}\(traceFull,log=getParameterAnnotations.log\)'
    Example 2:
    -Xjit:\"{java/util/HashMap.*}(optlevel=noopt)\"
  • JVM_OPTIONS : set this to replace the JVM options of the test run
  • ITERATIONS : number of times to repeat the execution of the test run (on one machine)
  • PARALLEL: several modes of parallelization supported [None|Dynamic|Subdir|NodesByIterations], where:
    • None -> "run tests serially"
    • Dynamic -> "when running multiple targets, try to retrieve test execution times and dynamically calculate how to divide tests across NUM_MACHINES"
    • Subdir -> "for nested test directories like external, divide and run parallel by sub-directories"
    • NodesByIterations -> "when running a single target, run ITERATIONS of a test across NUM_MACHINES"
  • LABEL : if left blank, any machine matching the PLATFORM labels will be selected
  • OPENJ9_REPO : optional, your fork of the eclipse/openj9 repo
  • OPENJ9_BRANCH : optional, your branch off your fork of eclipse/openj9 repo
  • TEST_FLAG: optional. Only set to JITAAS for testing JITAAS SDK
  • ARCHIVE_TEST_RESULTS: optional. If checked, the test outputs will be archived regardless of the test result

Common scenario: Rerun an openjdk regression test that was failing

• You can find "Rerun in Grinder" links in 2 locations on the job that fails.

  • near the top of the job page, before the test run is complete, the "Rerun in Grinder" link of the entire job is posted. Clicking on that link opens a Grinder with pre-populated fields, but it sets the TARGET as the entire target used by the job (example, TARGET=sanity.openjdk).
  • at the bottom of the console output of the test run. Those links will show the test targets that failed. (example, TARGET=jdk_math which is one of the several targets that are contained in the sanity.openjdk set).
  

• To rerun a single test case in the openjdk group (for example the test case java/math/BigInteger/LargeValueExceptions.java, group jdk_math for example), click on either rerun link described above (so that the most of the other parameters used in the failing tests are pre-populated and then edit any parameters you want to be able to run a particular variation of the previous run. In this case, you can set TARGET=jdk_custom and CUSTOM_TARGET=java/math/BigInteger/LargeValueExceptions.java to rerun just that particular test class.

• In the case you want to rerun on a particular machine that the test failed on, because the test seems to only fail on particular machines, you could change LABEL=hostname for our example, LABEL=test-godaddy-centos7-x64-2

• As a general rule, if you want a finer granularity, as in, if only want to run a specific test, you can set TARGET=jdk_custom and CUSTOM_TARGET=jdk/test/java/math/BigInteger/LargeValueExceptions.java to rerun that particular test class

• If you want to run a directory of tests, TARGET=jdk_custom and CUSTOM_TARGET=jdk/test/java/math/BigInteger to run the test cases within that directory

Running system_custom in a Grinder:

• For running STF based system tests using system_custom target in a Grinder, please ensure the following is set:

  • BUILD_LIST=system
  • TARGET=system_custom
  • CUSTOM_TARGET=-test=<stf_test_class_name> - Optionally, if test requires arguments, CUSTOM_TARGET=-test=<stf_test_class_name> -test-args="x=a,y=b" (e.g. CUSTOM_TARGET=-test=MathLoadTest -test-args="workload=math,timeLimit=5m")

• Note : <stf_test_class_name> should be the name of the STF class to run, not a Playlist target name. For example, if you want to re-run ClassLoadingTest_5m, using system_custom, you can simply copy the last portion of the command line from the playlist (e.g. -test=ClassloadingLoadTest -test-args="timeLimit=5m") for CUSTOM_TARGET.