Run tests from other directories than src/main/scala

[tomqaz]

It is possible to keep tests for example in src/test/scala directory?

[ktoso]

What tests?
Benchmarks are not tests.

Technically you could, but I'd rather not suggest that (just override the settings)

[mheimann]

Which setting would one need to override for using a custom source directory that contains the benchmarks (such as src/benchmarks)? I've tried sourceDirectory in Jmh := ... and sourceDirectory in (Jmh, compile) := ... but none of that worked.

I guess moving the benchmark code into its own module/project is a decent solution as well, but that would require making it a multi-module sbt project for some people :-).

[jeffrey-aguilera]

The gradle plugin for JMH (https://github.com/melix/jmh-gradle-plugin) uses src/jmh in the same module.

[ktoso]

Hey there,
I agree, it's a good idea - would be breaking for many people though hm...
Reason for keeping them in a different project is to avoid compiling them if you don't work on on them.

Would you like to contrib this fix? I'm a bit overloaded right now?

[magro]

We're just integrating a previously external jmh project into our multimodule project, so that it's kept in sync with code changes and lower the barrier to run benchmarks after changes. Because our benchmarks depend on the test artifact of another module (which provides e.g. scalacheck generators) and Intellij does not support "compile->test" dependencies (it just ignores the dependency) we're also "forced" to integrate jmh into the test configuration.

Here's a "hack" that seems to do the trick:

In our multimodule build.sbt we have this dependency:

lazy val microbench = project.dependsOn(otherModule % "compile;compile->test").enablePlugins(JmhPlugin)

In microbench/build.sbt:

sourceDirectory in Jmh := (sourceDirectory in Test).value

// JmhPlugin.generateJmhSourcesAndResources uses (classDirectory in Compile).value
classDirectory in Compile := (classDirectory in Test).value

libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.1" // must be in Compile scope, otherwise JmhBytecodeGenerator fails with CNFE org/scalacheck/Gen

// Running a clean 'jmh:run' resulted in 'Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to find the resource: /META-INF/BenchmarkList'
// which could be fixed with a prior 'jmh:compile'. Here we rewire tasks to that a clean 'jmh:run' is possible...
Keys.compile in Jmh <<= (Keys.compile in Jmh) dependsOn (Keys.compile in Test)
run in Jmh <<= (run in Jmh) dependsOn (Keys.compile in Jmh)

Hopefully this helps others until we have a better solution.

[alexandrnikitin]

It is possible to keep tests for example in src/test/scala directory?

To answer the OP's (@tomqaz) question, the following code in build.sbt worked for me:

sourceDirectory in Jmh := (sourceDirectory in Test).value
classDirectory in Jmh := (classDirectory in Test).value
dependencyClasspath in Jmh := (dependencyClasspath in Test).value