Running tests

The Testing HOWTO

There is more information about how to test in the Testing HOWTO. Below is a short description of what to do.

Building

First make sure ERL_TOP and PATH are configured

# Bourne Shell
export ERL_TOP=$PWD
export PATH=$ERL_TOP/bin:$PATH
# C Shell
setenv ERL_TOP $PWD
setenv PATH $ERL_TOP/bin:$PATH

Then build the tests like this

## The Erlang makefile system does not always catch changes made in files, so if the below steps do not work.
## Try again with a fresh clone or after doing `git clean -xfdq`
./otp_build all -a
export PATH=$PWD/release/*/bin:$PATH
./otp_build tests

The release is installed for tests that require an installed release to execute successfully (e.g., the reltool application tests). If you encounter an error due to problematic entries in ERL_LIBS, unset ERL_LIBS and re-run the failed ./otp_build call:

# Bourne Shell
unset ERL_LIBS
# C Shell
unsetenv ERL_LIBS

When working on a patch, you can use make to recompile everything without having to reconfigure. Just make sure to release the changes before re-running the tests. This is done by issuing make release in ERL_TOP. You can also use make inside the directory of the application you are working on, as long as ERL_TOP is defined.

Running

To run the test first do:

cd release/tests/test_server

and then start Erlang:

$ERL_TOP/release/*/bin/erl

Install the ts framework

ts:install().

To run all test suites do

ts:run().

Note that running all tests will require several hours, so you may want to run the test cases for a single application

ts:run(Application, [batch]).

or even part of the test suite for an application, for example

ts:run(emulator, bs, [batch]).

to run all test suite modules starting with bs (i.e. all modules that test the bit syntax).

To run a specific test case in a module, the full name of the module and test case must be spelled out:

ts:run(emulator, bs_bincomp_SUITE, byte_aligned, [batch]).

Run ts:help() to show some help.

There will currently be 15 or so failed test cases in the kernel application.

As of R14B02 it is also possibly to start all tests but the erl_interface tests by invoking Common Test directly from the released applications test directory, i.e.

cd release/tests/test_server
$ERL_TOP/tests_install/bin/ct_run -suite ../compiler_test/andor_SUITE -case t_orelse

Running from the command line still requires you to do the ts:install() step above. For more information about ct_run see http://www.erlang.org/doc/man/ct_run.html

Running with the `cover` tool

To run all tests with the `cover` tool, do:

ts:run([all_tests, cover]).

To run a single application using `cover`, do:

ts:run(Application, [batch, cover]).

Examining the results

Open the file release/tests/test_server/index.html in a web browser. Or open release/tests/test_server/last_test.html when a test suite is running to examine the results so far for the currently executing test suite (in and later R14B02 you want to open the release/tests/test_server/all_runs.html file to get to the currently running test)