Share reporting code between SBT, Gradle and Maven

[maiflai]

Hi,

I think a CLI for generating reports would help me to avoid duplicating scoverage.ScoverageSbtPlugin#loadCoverage and scoverage.ScoverageSbtPlugin#writeReports in the gradle plugin.

The gradle plugin copies this functionality at the moment, but does not support multi-module aggregation and it would be nice if I could re-use the existing code by moving it from the SBT plugin into the parent module.

@gslowikowski - does the Maven plugin have its own duplicate of these functions, and if so, would a CLI help?

Thanks,
Stu.

[gslowikowski]

I tried to implement aggregated reports, but this just functionality does not work in scala-scoverage-plugin. Anyway if/when it will work I will introduce aggregate parameter for Maven reporting goals. This differs from how it's implemented in SBT plugin. I don't know, how you will implement it in Gradle, I see you don't henerage Scoverage xml report at all now.

[sksamuel]

@maiflai @gslowikowski I can pick this up this weekend. Question - do we want a cli rather than a library?

[gslowikowski]

I don't.

The goal is to decouple Maven/Gradle/SBT plugin from Scalac plugin to support different Scalac plugin versions from one Maven/Gradle/SBT plugin version.
In Maven I can change Scalac plugin/runtime versions used by overriding plugin's dependencies (Maven way of doing such things), like this:

            <plugin>
                <groupId>org.scoverage</groupId>
                <artifactId>scoverage-maven-plugin</artifactId>
                <version>1.0.2</version>
                <dependencies>
                    <dependency>
                        <groupId>org.scoverage</groupId>
                        <artifactId>scalac-scoverage-plugin_2.10</artifactId>
                        <version>1.0.3-SNAPSHOT</version>
                    </dependency>
                </dependencies>
            </plugin>

[gslowikowski]

Additionally, I have one nonstandard requirement - html report by default must be created inside target/site directory (default location is target/site/scoverage). This does not fit to the API proposed by @maiflai.

On the other hand xml reports (scoverage and cobertura) should be somewhere inside target, I don't any requirement here because I don't know what external tools will consume them and so what requirements they have. Maybe some users would like to place them inside site directory to deploy with the site, some others - not. Generally all three report locations should be parametrized independently if I would use common code.

[maiflai]

That's not my goal - my goal is to remove the dependency on Scala, and reduce the quantity of code contained within the gradle plugin to a minimum.

Specifically I would like to remove knowledge of the internals of the plugin:

       File coverageFile = Serializer.coverageFile(dataDir);
        File[] array = IOUtils.findMeasurementFiles(dataDir);
        // TODO: patch scoverage core to use a consistent collection type?
        Seq<File> measurementFiles = scala.collection.JavaConversions.asScalaBuffer(Arrays.asList(array));

        Coverage coverage = Serializer.deserialize(coverageFile);

        Set<Object> measurements = IOUtils.invoked(measurementFiles);
        coverage.apply(measurements);

Note that this snippet is only appropriate for single-module projects, multi-module is then a different approach.

[gslowikowski]

This goal is OK, but I see two problems with it:

1. I prefer Scala dependency over forking, we can have simpler API without the need to fork. When implementing the solution with forking you will immediately have to think about the way (parameters) to pass JVM arguments (memory options, remote debugging configuration, etc.).
2. I haven't seen multimodule project with working aggregated coverage yet. SBT 1.0.2 implementation does not work. Recently we discussed paths problem and I it works, but I still don't like the paths containing {modulename}/src/main/scala prefixes. I would prefer having this fixed in all plugins first.

[maiflai]

Are you suggesting that the simple generation of a report will require explicit memory options and remote debugging?

I'm asking for a trivial wrapper over a scala API.

I don't agree with your assessment of the path problem. I think there are more interesting problems relating to multi-module than the physical location of the files in an HTML report: scoverage/gradle-scoverage#33 (comment)

[gslowikowski]

How would you debug report generation in forked process?

[gslowikowski]

From my experience, every trick or workaround will sooner or later lead to more serious problems. That's why I always want to fix things asap.

[gslowikowski]

Memory problems - #89

[maiflai]

Why would you debug report generation from inside Maven?

If I really really want to debug it, then I will launch the CLI in a debug session from my scalac-scoverage-plugin project. It will give me a more focused debug session.

Re: Memory issues, I think if you re-read the stacktrace you will see

at scoverage.ScoverageInstrumentationComponent$$anon$1.run(plugin.scala:96)

This indicates to me that he has run out of memory during the instrumentation phase, not the reporting phase.

[gslowikowski]

As to debugging, you are right. I must rethink it.

#89 - you are right too, ignore my last comment.

[sksamuel]

The memory issue is fixed in #89

[sksamuel]

I think it makes most sense for the scalac plugin to have some reporting code, so you just need to go:

Reporter.doXMLReport(locationOfData, outputFolder) or whatever.
Reporter.doHTMLReport(locationOfData, outputFolder) or whatever.

Then you don't need to know how the scalac plugin handles the instrumentation files. Plus you can put XML / HTML reports in separate locations, wherever you want / whatever makes sense in your plugins. The "locationOfData" would just be some /target or /build folder where the scoverage temporary files go (currently in sbt its target/scoverage-data)

Is this acceptable?

[maiflai]

Sounds reasonable for the moment, and would support a trivial CLI over the top of it.

On a related note - do you think it would be possible to configure the data directory at runtime rather than compilation? I think that the current arrangement makes the separation (and reporting) of 'unit' coverage and 'integration' coverage difficult.

[sksamuel]

I don't think so, because when you run the tests, the instrumented code
needs to know where to write the instrumentation data to, so the full path
has to be baked in at that point. But you can configure it just as easily
right ?

On 24 January 2015 at 10:41, maiflai [email protected] wrote:

Sounds reasonable for the moment, and would support a trivial CLI over the
top of it.

On a related note - do you think it would be possible to configure the
data directory at runtime rather than compilation? I think that the current
arrangement makes the separation (and reporting) of 'unit' coverage and
'integration' coverage difficult.

—
Reply to this email directly or view it on GitHub
#81 (comment)
.

[maiflai]

But then both my unit tests and my integration tests write to the same directory, unless I recompile my source code for each set of tests.

If you consider a multi-module project, then each module writes to its own data directory, and I lose the ability to separate the direct project coverage from coverage generated by dependent projects. I would like to retain this separation, since I value unit test coverage more highly than integration tests.

The CoverageAggregator seems to aggregate the scoverage xml files, which represent a snapshot of the measurements; this means that the exact timing of the report task must be carefully selected if the contents of the directory change during the build.

For my use-case I would need to:

• clean
• compile, producing a statements file into each data directory
• run unit tests, producing measurement files into the data directories
• run reports, producing subproject reports
• purge only the measurement files from data directories
• run integration tests, producing new measurement files
• run reports, producing subproject reports

But then the CoverageAggregator searches the project for inputs, so I don't think I can produce separate reports for unit and integration tests. The aggregrated report will include both sets of coverage.

Of course, it might be suggested that multi-module projects are the real problem, and that you can solve this with a microservice approach, but I think that there can still be value in dividing a microservice into its components.

[sksamuel]

You would have to put the integration tests into another submodule and do
it that way. Then you can have two sets of reports. You can then compile
separately for your unit tests and int tests. When maven/gradle/sbt invoke
scalac, they pass it a parameter for the data directory. This data
directory is then encoded into Invoker.invoke(id, dataDir) calls. After the
compilation step, the tests are run. As part of the tests the compiled code
executes these invoker statements. After the tests, we can then investigate
the files to see which statements were executed and which were not.

This is why it cannot be done at runtime, and why it must be the same for
any given compilation cycle - unless you can think of a way that I've
missed.

On 24 January 2015 at 11:56, maiflai [email protected] wrote:

But then both my unit tests and my integration tests write to the same
directory, unless I recompile my source code for each set of tests.

If you consider a multi-module project, then each module writes to its own
data directory, and I lose the ability to separate the direct project
coverage from coverage generated by dependent projects. I would like to
retain this separation, since I value unit test coverage more highly than
integration tests.

The CoverageAggregator seems to aggregate the scoverage xml files, which
represent a snapshot of the measurements; this means that the exact timing
of the report task must be carefully selected if the contents of the
directory change during the build.

For my use-case I would need to:

• clean
• compile, producing a statements file into each data directory
• run unit tests, producing measurement files into the data directories
• run reports, producing subproject reports
• purge only the measurement files from data directories
• run integration tests, producing new measurement files
• run reports, producing subproject reports

But then the CoverageAggregator searches the project for inputs, so I
don't think I can produce separate reports for unit and integration tests.
The aggregrated report will include both sets of coverage.

Of course, it might be suggested that multi-module projects are the real
problem, and that you can solve this with a microservice approach, but I
think that there can still be value in dividing a microservice into its
components.

—
Reply to this email directly or view it on GitHub
#81 (comment)
.

[maiflai]

But I'm not looking for coverage of my test code, I'm looking for coverage of my source code? And the source code should be compiled once only.

Why can't the Invoker look for a system property to determine the data directory root?

[sksamuel]

But I'm not looking for coverage of my test code, I'm looking for
coverage of my source code?

I wrote scoverage, so I'm aware of this ;) If the source code is compiled
once, and since it uses (currently) hardcoded path, your int tests and your
unit tests will write to the same folder and the only way (currently) if
you want two sets of output data is to compile twice.

Why can't the Invoker look for a system property to determine the data
directory root?

A property might not be propagated to forked processes on all platforms? It
could use an env var but can that be reliably set by the build processes?
Sounds like it would solve your problem if it works and would be easy
enough to add in. Need to answer both these questions.

On 24 January 2015 at 12:07, maiflai [email protected] wrote:

But I'm not looking for coverage of my test code, I'm looking for coverage
of my source code? And the source code should be compiled once only.

Why can't the Invoker look for a system property to determine the data
directory root?

—
Reply to this email directly or view it on GitHub
#81 (comment)
.

[maiflai]

:-)

I think that Maven, SBT and Gradle all support setting system properties when running JVM-based unit tests.

[maiflai]

And you've just reminded me how hard that was with SBT :-(

In the end it encouraged me to use typesafe/config rather than sys.props.

[maiflai]

(and pass in environment variables)

[sksamuel]

We need to ensure that the system property can be set for forked processes
too, as SBT will fork its tests (usually).

Setting env vars in Java is a proper hack, and I've got unit tests that
don't run on windows in the scalac-scoverage-plugin because of that.

On 24 January 2015 at 12:18, maiflai [email protected] wrote:

(and pass in environment variables)

—
Reply to this email directly or view it on GitHub
#81 (comment)
.

[maiflai]

Sadly I agree it's a little more complicated.

It's not guaranteed to fork and therefore you can't be sure that the property will be passed through. javaOptions only takes effect when forked.

fork in Test := true
javaOptions in test += "-Dscoverage.dataDir=..."

I think a data directory override capability would be useful, but we would then need a way to locate the measured statements files at report time, and also a way to identify the exact statement. Currently it assumes one statement file per data directory. I don't think this is compatible with running with multiple instrumented jars.

Now that I think about it, I imagine that both Maven and Gradle only support one instrumented project at the moment anyway, since the test classpaths will be composed of:

• instrumented project classes
• plain project classes
• other dependency project classes
• other dependency jars

Configuring these classpaths to include the instrumented classes of other projects would require a bit of thought.

[sksamuel]

The statement file won't change between compilations anyway, so having one
is fine.

What I think you want is this:

compile your classes with instrumentation
run your unit tests with instrumentation results going to folder1
run your integration tests with instrumentation results going to folder2
generate report1 from folder1
generate report2 from folder2

That can be done with a system property, but whether its propagated or not
is your question.

On 24 January 2015 at 13:38, maiflai [email protected] wrote:

Sadly I agree it's a little more complicated.

It's not guaranteed to fork and therefore you can't be sure that the
property will be passed through. javaOptions only takes effect when
forked.

fork in Test := true
javaOptions in test += "-Dscoverage.dataDir=..."

I think a data directory override capability would be useful, but we would
then need a way to locate the measured statements files at report time, and
also a way to identify the exact statement. Currently it assumes one
statement file per data directory. I don't think this is compatible with
running with multiple instrumented jars.

Now that I think about it, I imagine that both Maven and Gradle only
support one instrumented project at the moment anyway, since the test
classpaths will be composed of:

• instrumented project classes
• plain project classes
• other dependency project classes
• other dependency jars

Configuring these classpaths to include the instrumented classes of other
projects would require a bit of thought.

—
Reply to this email directly or view it on GitHub
#81 (comment)
.

[maiflai]

Yep - and that means that the report generation is no longer a function of the data directory and the output directory, but it now also requires the statement file (since we have separated the statement file from the measurement directory).

(and assuming that we do not want to measure more than one project concurrently, which seems fair for a first step)

[sksamuel]

The statement file can be set by a property too though - or in fact just
kept constant, as it doesn't matter where that is.

On 24 January 2015 at 14:08, maiflai [email protected] wrote:

Yep - and that means that the report generation is no longer a function of
the data directory and the output directory, but it now also requires the
statement file (since we have separated the statement file from the
measurement directory).

(and assuming that we do not want to measure more than one project
concurrently, which seems fair for a first step)

—
Reply to this email directly or view it on GitHub
#81 (comment)
.

[gslowikowski]

Hi, I read your discussion, experimented with https://github.com/scoverage/gradle-scoverage-sample/tree/master/separate-tests (thanks for this project) ported to Maven and have some thoughts and ideas.

1. In Maven plugin when compiling a module in multi-module build, there are scoverage-enhanced classes locations of dependent modules in the classpath. Class files are located in target/scoverage-classes instead of target/classes and classpath contains target/scoverage-classes locations of dependent modules. This is important because one testing module tests all dependent code modules at once. It's different from what @maiflai wrote in Share reporting code between SBT, Gradle and Maven #81 (comment). How does it exactly look like in Grdle and SBT?
2. In Maven default test runner (Surefire) system properties work with and without forking. I never used evn vars. I didn't test it with other testing plugins: for Scalatest and for Specs2.
3. Instead of system properties or env vars scalac plugin could create a resource (a file in classes directory) containing data directory location. This resource filename would be constant. It would be created during compilation and would contain the data directory passed to compiler. This would be backward compatible because would need no additional configuration. Optionally this file could be overwritten before report generation for tests in another module. In Maven (I don't know if in SBT and Gradle it's similar) project classpath contains module classes directory before dependent modules classes directories. This resource file doesn't need to be physically overwritten before invoking tests in another module (this would require additional task or configuration, so I don't like this solution). You just need to add a resource with the same name in that module and it will override the one from the module under test.
   When on test module depends on more than one modules containing classes, coverage for all of them should be collected IMO. In this case one resource defining data directory in test module would override resources from all dependent modules automatically.
   WDYT?
4. I'm against running multiple compilations.
5. Small problem - when there are no tests in one of the modules in multimodule project you could render useless (0% coverage) report or not render it. This is parametrizable in some Maven reports I know. What should be default behaviour?
6. gradle reportScoverage does not work on https://github.com/scoverage/gradle-scoverage-sample/tree/master/separate-tests - again problem with path relativization.

[gslowikowski]

Ad. 3. Forget about resource. This was wrong idea. I'm in favor of system property. It allows defining many test executions with different scoverage data locations for different reports in one module (if someone would want/need it).

[maiflai]

re: 1

I confirm that Gradle tests instrumented main classes for the subproject under test, and un-instrumented classes for any dependent projects (even though instrumented classes may exist). I imagine that SBT works in the same way, because of the way that dependencies are specified in these tools.

It sounds as though this is different to the Maven plugin, in that I can see https://github.com/scoverage/scoverage-maven-plugin/blob/master/src/main/java/org/scoverage/plugin/SCoveragePreCompileMojo.java#L240 and I think this would place all the instrumented classes on the classpath, regardless of which module they were in.

However, I wonder if this is actually working as we might expect; I agree that all the instrumented classes are presented, and new measurements are taken. I suspect though that the coverage will be crystallised at the time that the submodule is reported, which means that code covered by a downstream module will have no impact on the coverage statistics, because those measurements were taken after scoverage.xml was created.

I'll try to produce a test project tomorrow, and look forward to proving this hypothesis is wrong.

[maiflai]

I've confirmed to myself that the reports are written module-by-module; i.e. code only covered by a dependent module is not included in the report. This means that Maven behaves in the same way as Gradle, despite the inclusion of instrumented classes throughout the reactor projects.

@gslowikowski - do you agree?

It seems we have two separate use-cases (i.e. 'direct' coverage and 'indirect' coverage).

[gslowikowski]

I've compared Maven and Gradle working with your simple multimodule project https://github.com/scoverage/gradle-scoverage-sample/tree/master/separate-tests. They work internally completely differently (to achieve the same goal). I will check SBT later.

Executed commangs:

• "gradle testScoverage"
• "mvn scoverage:report" (I have no "testScoverage" equivalent, only "report" and "check" tasks)

My findings are:

1. Gradle executed many more tasks than Maven.
   Gradle log:

:a:compileJava UP-TO-DATE
:a:compileScala
:a:processResources UP-TO-DATE
:a:classes
:a:compileScoverageJava UP-TO-DATE
:a:compileScoverageScala
:a:compileTestJava UP-TO-DATE
:a:compileTestScala UP-TO-DATE
:a:processTestResources UP-TO-DATE
:a:testClasses UP-TO-DATE
:a:testScoverage SKIPPED
:a:jar
:a:processScoverageResources UP-TO-DATE
:a:scoverageClasses
:a-tests:compileJava UP-TO-DATE
:a-tests:compileScala UP-TO-DATE
:a-tests:processResources UP-TO-DATE
:a-tests:classes UP-TO-DATE
:a-tests:compileScoverageJava UP-TO-DATE
:a-tests:compileScoverageScala UP-TO-DATE
:a-tests:compileTestJava UP-TO-DATE
:a-tests:compileTestScala
[ant:scalac] Element 'G:\scm.gslowikowski\github.git\gslowikowski\scoverage-tests\trunk\separate-tests.1\a-tests\build\classes\main' does not exist.
[ant:scalac] Element 'G:\scm.gslowikowski\github.git\gslowikowski\scoverage-tests\trunk\separate-tests.1\a-tests\build\resources\main' does not exist.
:a-tests:processTestResources UP-TO-DATE
:a-tests:testClasses
:a-tests:testScoverage

Maven log is more verbose, but the tasks executed for every module are:

[INFO] --- scoverage-maven-plugin:1.0.5-SNAPSHOT:pre-compile (default-cli) @ a --- changes 'outputDirectory' and some other project properties used by resources, compiler and tester plugins
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ a --- copies resources to 'scoverage-classes'
[INFO] --- sbt-compiler-maven-plugin:1.0.0-beta5:compile (default-sbt-compile) @ a --- compiles to 'scoverage-classes'
[INFO] --- scoverage-maven-plugin:1.0.5-SNAPSHOT:post-compile (default-cli) @ a ---
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ a ---
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ a ---
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ a --- 'scoverage-classes' in classpath
[INFO] <<< scoverage-maven-plugin:1.0.5-SNAPSHOT:report (default-cli) < [scoverage]test @ a <<<

Differences:

• Gradle compiles module twice, without and with scoverage instrumentation, writing classes to build/classes and build/scoverage, Maven tweaks default compilation configuration so there is only one compile task execution, with scoverage instrumentation, classes are written to target/scoverage-classes; IMPORTANT - there are no 'clean' classes (in target/classes directory)
• I don't know what are Gradle classes and testClasses goals for, there are no such tasks in Maven
• Gradle executes jar task, produced a/build/libs/a.jar file is added to scoverage classpath in a-tests module; in Maven non-instrumented classes and jar file are not produced during scoverage life cycle, only scoverage instrumented classes are produced so they must be used in a-tests module's compilation classpath

Of course in Maven non-instrumented classes can be available (in target/classes directory) but only in compile task was executed before (for example in mvn clean compile scoverage:report invocation they are present, but in mvn clean scoverage:report - not). Because it's not guaranteed they will be available I cannot use them in dependent modules classpaths. That's why only scoverage-instrumented classes are used.

I learned Cobertura Maven plugin implementation before writing Scoverage plugin, but Cobertura is different. It does not tweak compilation. After compilation non-instrumented compiled classes are copied from target/classes to target/generated-classes/cobertura and instrumented there. You always have regular and instrumented classes without additional compilation overhead. This is not possible with Scoverage.

I don't like this difference between Gradle and Maven, but don't know what to do with it yet.

As I wrote, I will also test SBT. All I know now is that it does not compile instrumented classes to different location, but to target/scala_2.11/classes (or target/scala_2.10/classes depending on Scala version used by project) overwriting non-instrumented classes if they were there. This differs from both Gradle and Maven implementations.

[maiflai]

re: Gradle compiling the main sources twice - I think this supported excluding files from coverage before the feature was available in the scalac plugin. It can probably be removed now.

I think Gradle classes and testClasses are broadly the equivalent of the Maven 'compile' and 'testCompile' lifecycle phases. Gradle does not differentiate between a phase and an execution, they are all tasks.

[gslowikowski]

"I think this supported excluding files from coverage before the feature was available in the scalac plugin." - I don't understand, can you clarify?

If you remove regular compilation, non-instrumented classes will not be available as dependencies in other modules (like in Maven now). What about jar task?

[maiflai]

Excluding source files from instrumented compilation allows the class loader to fall back to the plain uninstrumented class (both output directories are included on the test class path).

This requires that you compile everything at least once, likely twice.

A better solution to the problem 'I do not want to instrument everything' is now available as a property that is passed to the Scoverage compiler plugin on the command line.

There is no 'instrumented' jar file produced at the moment, but it is a one-liner to add to your gradle file if you want it.

The existing jar task includes plain class files, and the other modules use those too.

[gslowikowski]

Hi, I tried the solution with system property for data directory and it works! I have not enough time tu publish everything now. Required changes are:

1. In Invoker.scala class (scala-scoverage-runtime' module) changeinvoked` method fragment from:

      val writer = files.getOrElseUpdate(dataDir, new FileWriter(measurementFile(dataDir), true))
      writer.append(id.toString + '\n').flush()

      ids.put((dataDir, id), ())

to

      val resolvedDataDir = Option(System.getProperty("scoverage.dataDir")).getOrElse(dataDir)
      new File(resolvedDataDir).mkdirs();//TODO-check result
      val writer = files.getOrElseUpdate(resolvedDataDir/*dataDir*/, new FileWriter(measurementFile(resolvedDataDir/*dataDir*/), true))
      writer.append(id.toString + '\n').flush()

      ids.put((resolvedDataDir/*dataDir*/, id), ())

1. In SBT/Gradle/Maven reporting code split dataDirectory into two directories: dataDirectory (containing scoverage.coverage.xml file) and measurementsDirectory (containing scoverage.measurements.* files) - load coverage data from one directory and apply measurements from another. By default measurementsDirectory parameter should point to dataDirectory.
2. Very old problem - it would be good to add support for multiple: source, data directories to be able to test more than one module at once (usual case for integration tests).
3. It would be even better to add support for multiple measurements directories. This would be another, better IMO, implementation of aggregated reports. It would allow to deprecate current aggregation tasks. Instead of building aggregated report from scoverage.xml files created in previous run, it would be build based on data and measurements files. Aggregated reports would differ from non-aggregated only by configuration, no additional task and, what's more important, no second run required.

WDYT? If you agree, it would be good to apply this change, build and deploy 1.0.5-SNAPSHOT version of scalac plugin to Sonatype snapshot repository. This would allow implementing necessary changes in SBT/Gradle and Maven plugins and testing them before next release.

[maiflai]

I'm not sure that is sufficient.

I think that the current relationship is that a single compilation run produces a single measured statements file. The compiled classes then write to a single measurements directory.

Statements are identified by an integer, generated from a sequence that is reset at compile time. You cannot mix classes from separate compilations and have them write to a single directory, you will get identity clashes. This must not be allowed.

If you allow a user to specify where a single location where measurements should be written, then you have to have a unique identifier for each statement. If you have more than one compilation step (i.e. a multi-module project) then you need something more unique than an integer sequence.

I think that with the current system it will be difficult to explicitly state the location where each compilation run should write its measurements at invocation time.

[sksamuel]

@maiflai is right in his analysis. The compilation run uses an integer sequence for uniquely identifying the statements. It is reset each time, as the code has probably changed anyway.

What is the issue that seperate compilations would solve?

[maiflai]

I think that in a multi-module project each module is compiled separately. It might be a gradle peculiarity that a new process is forked for each compilation?

The original problem was 'how do I see what code was covered in my integration tests' (which are testing code from multiple instrumented modules).

[maiflai]

(I'll double check the forking behaviour with the ant compiler, but I think the zinc compiler is initialised separately for each project)

[maiflai]

But isn't the behaviour of the CoverageAggregator designed to work around this?

[gslowikowski]

Scalac compiler plugin needs additional, string type, optional parameter uniquely identifying every module (in Maven for example "groupId:artifactId"). In runtime - additional version of Invoker.invoked method with this parameter.

Generally speaking, if we want to get the coverage of integration tests of multimodule project, these tests must operate on scoverage-instrumented classes from all modules. Nobody writes integration tests for single module.

[maiflai]

I think that doing so exposes the internal workings of the plugin downstream; I prefer that the plugin should generate a globally unique identifier.

re: integration tests - some might consider that tests accessing external systems are 'integration' tests. I thought this was normally done with the failsafe plugin in maven, as part of a single module's lifecycle.

[gslowikowski]

You are right, scalac plugin will expose the internal workings to Gradle/SBT/Maven plugin, but not to user project. That's not bad, but of course it would be better to not expose it at all. This was just the first idea.

Integration test can test integration between modules in multi-module system/application too. From coverage point of view it's not important by what plugin they are run.

For example, I have multi-module Play! Framework web application with Selenium integration tests. I would be happy to see their code coverage of the whole application, not only one module per report.