e2e

E2E tests for Restate

DEPRECATED: Please add new tests to https://github.com/restatedev/sdk-test-suite

Modules

• services contains a collection of services for e2e testing:
  • java-services contains the Java SDK services
• test-utils contains utilities to develop e2e tests
• tests contains the test code
• contracts contains the different protobuf definitions, used by services and tests

Run tests

To run tests, just execute:

gradle build

This will populate your local image registry with the various service containers, required for testing, and then execute the tests.

In the default configuration, testcontainers will try to pull the Restate container frequently during tests. To prevent this behavior, e.g. when working offline, set the environment variable E2E_IMAGE_PULL_POLICY=cached to use the image available in the local container registry.

Tests

Source code of test runners is located in the Tests project, in particular:

• dev.restate.e2e contains common tests to all SDKs, testing the various SDK features
• dev.restate.e2e.runtime contains tests for some specific runtime behavior
• dev.restate.e2e.java contains tests for Java SDK specific features
• dev.restate.e2e.node contains tests for Node SDK specific features

Test configurations

Currently, we run tests in the following configurations:

• gradle :tests:test: Default runtime configuration
• gradle :tests:testAlwaysSuspending: Runtime setup to always suspend after replay, to mimic the behavior of RequestResponse stream type
• gradle :tests:testSingleThreadSinglePartition: Runtime setup with a single thread and single partition
• gradle :tests:testPersistedTimers: Runtime setup with timers in memory = 1, to trigger timer queue spilling to disk
• gradle :tests:testLazyState: Runtime setup disabling eager state when invoking the service endpoint

Developing tests

Parallel test execution

The test setup uses Gradle's maxParallelForks to distribute the test classes among different JVMs, providing parallelism per class.

Within the test classes, it is possible to tag individual test methods to be executed in parallel. Usually a test method is considered safe to execute in parallel when:

• Doesn't invoke directly or transitively singleton services
• When invoking directly or transitively keyed services, it uses a key with a random component, e.g. like a UUID
• Doesn't use additional stateful containers
• The containing test class uses RestateDeployerExtension

To tag a test method to be executed in parallel, use the annotation @Execution(ExecutionMode.CONCURRENT).

Debugging

VerificationTest seed

VerificationTest is using a random seed to generate the execution tree, logged at the beginning of the test. You can fix the seed to use setting the environment variable E2E_VERIFICATION_SEED.

Test report and container logs

For each deployment of RestateDeployer, the stdout and stderr of the containers and the docker inspect info are written in tests/build/test-results/[test-configuration]/container-logs/[test-name].

How to test Java SDK changes

In order to test local changes to the sdk-java, you need to check it out under ../sdk-java. When building the e2e project you have to set the environment variable SDK_JAVA_LOCAL_BUILD=true to include sdk-java as a composite build and substitute the dev.restate.sdk:sdk-java dependency with it. The build will fail if Gradle cannot find the sdk-java project.

How to test Restate runtime changes

You can manually build a docker image in the restate project using just docker. Then set the environment variable RESTATE_CONTAINER_IMAGE with the tag of the newly created image (printed at the end of the docker build log).

Retain runtime state after test

To retain the runtime RocksDB and Meta state, set the environment variable E2E_MOUNT_STATE_DIRECTORY=true to mount the state directory in the same directory of the container logs.

Running the services

For running the services see services/README.md.