GitHub Action
jacoco-badge-generator
The jacoco-badge-generator GitHub Action parses a jacoco.csv
from a JaCoCo coverage report,
computes coverage percentages from JaCoCo's Instructions and Branches counters, and
generates badges for one or both of these (configurable with action inputs) to provide an easy
to read visual summary of the code coverage of your test cases. The action supports
both the basic case of a single jacoco.csv
, as well as multi-module projects in which
case the action can produce coverage badges from the combination of the JaCoCo reports
from all modules, provided that the individual reports are independent.
The jacoco-badge-generator can also optionally be used as part of a pull-request check. Specifically, you can configure it to fail the workflow run if coverage decreased relative to prior run, and/or if coverage is below a target threshold. See the Inputs section for details of how to configure it for this purpose.
The developers of the jacoco-badge-generator GitHub Action are not affiliated with the developers of JaCoCo, although we are a fan and user of their excellent test coverage tool.
The documentation is organized into the following sections:
- The Coverage Metrics: Explains the JaCoCo metrics that are supported by the badge generator, such as what they measure, and why they were chosen for inclusion for the jacoco-badge-generator GitHub Action.
- Badge Style and Content: Provides examples of the appearance of the badges that are generated, including a description of the color scheme used, and the formatting of the percentages.
- Inputs: Detailed descriptions of the action inputs.
- Outputs: Detailed descriptions of the action inputs.
- Example Workflows: Example GitHub workflows demonstrating usage of the jacoco-badge-generator action.
- Multi-Module Example Workflows: Example GitHub workflows demonstrating usage of the jacoco-badge-generator action with multi-module projects.
- Examples in Other Projects: Links to a few repositories that are actively using the action, as well as direct links to the relevant workflow files.
The jacoco-badge-generator GitHub Action currently supports generating badges for the two primary coverage metrics generated by JaCoCo: Instructions (C0 Coverage), and Branches (C1 Coverage). Here is a summary of what these compute and why they were chosen for inclusion by this badge generator.
The default behavior of the badge generator is to generate only the Instructions Coverage
badge, which is labeled on the badge simply as "coverage". JaCoCo
measures C0 Coverage
from the Java bytecode instructions in the compiled .class
files. One of the advantages
to counting the bytecode instructions executed or missed, rather than lines of source code,
is that it is independent of coding style and formatting. As a simple example, consider
the sequence of Java statements to swap the values in two
variables: int temp = a; a = b; b = temp;
. A line counter will count this as 1 line if
written on a single line, or 3 lines if each statement is written on its own line. However,
JaCoCo's instructions counter treats these two cases as equivalent since they compile
to the same bytecode. Consider a more complex example of calling a method while passing a simple
value, such as foo(5)
versus passing the result of a calculation to the method, such as
foo(2.0 + bar/11.0)
. Line counting considers both of these as 1 line; while the second case
will factor in more heavily into JaCoCo's instruction counting than will the first case. For
these reasons, although JaCoCo also provides line coverage data, we do not currently support
generating a badge from JaCoCo's line counter data. JaCoCo's use of bytecode instructions
in its definition of C0 Coverage is a more meaningful measure of coverage than is counting
lines of code.
The badge generator also optionally supports generating a badge for Branches Coverage
(or C1 Coverage), with the generated badge labeled as "branches". See
the inputs section for a description of the action
inputs. JaCoCo
measures C1 Coverage or Branches Coverage
from the Java bytecode in the compiled .class
files, so the result may be a bit
different than what you might expect from branch coverage. At first, you may even mistakenly
guess that it is counting conditions (C2 coverage) rather than branches, but it is counting
branches (in bytecode rather than in source code). Consider this example to illustrate
the difference: if (a && b) foo(); else bar();
. If we count branches in source code,
there are 2 branches, which would require a minimum of two tests for full coverage, one where
both a
and b
are true
, and a second test where at least one of them is false
.
If we instead count conditions, there are 4 conditions (a==true
, a==false
, b==true
,
b==false
), which can be covered with as few as two tests (e.g., a==true
and b==false
as test one, and b==true
and a==false
as test two), without actually covering both
branches. JaCoCo's branches counter is neither of these. JaCoCo's branches counter
counts branches in the Java bytecode. What does that mean for this example? Imagine instead
that the if statement above was written as a pair of nested if
statements: if (a) { if (b) { foo(); } else { bar(); } } else { bar(); }
. There are
a total of 4 branches in this case (and is essentially what JaCoCo would count as branches).
To cover all 4 branches would require a minimum of 3 test cases: one where a
and b
are both
true
, one in which a
is true
and b
is false
, and a third where a
is false
and b
's
value doesn't matter. In this way, JaCoCo's branches counter leads to a stronger form
of C1 Coverage than is usually implied by branches coverage.
The badges that are generated are inspired by the style of the badges of Shields.io, however, the badges are entirely generated within the jacoco-badge-generator GitHub Action, with no external calls.
Here are a few samples of what the badges look like if you use the default colors:
- We use bright green for 100% coverage.
- We use green for coverage from 90% up through 99.9%.
- We use yellow green for coverage from 80% up through 89.9%.
- We use yellow for coverage from 70% up through 79.9%.
- We use orange for coverage from 60% up through 69.9%.
- We use red for coverage from 0% up through 59.9%.
- A sample of a branch coverage badge.
The jacoco-badge-generator action provides two inputs that can be used
to customize the colors used for the badges. The colors
input enables you to
pass a list of colors to the action. The intervals
input enables you to pass
a list of percentages used to determine color choice. If you like the default
colors, but want to start the colors at different percentages, then you can use
the intervals
input to accomplish that. These two inputs can be used either
individually or in combination depending upon what you want to do. See the
Inputs section for more details.
The coverage displayed in the badge is the result of truncating to one decimal place. If that decimal place is 0, then it is displayed as an integer. The rationale for truncating to one decimal place, rather than rounding is to avoid displaying a just failing coverage as passing. For example, if the user of the action considers 80% to be a passing level, then we wish to avoid the case of 79.9999% being rounded to 80% (it will instead be truncated to 79.9%).
If you use the action's default badges directory and default badge filenames, then
you can add the coverage badge to your repository's readme with the following
markdown: ![Coverage](.github/badges/jacoco.svg)
, and likewise for the
branch coverage badge: ![Branches](.github/badges/branches.svg)
.
See the Inputs section for how to change the directory and filenames of
the badges. You can of course also link these to the JaCoCo coverage report if you host it
online, or perhaps to the workflow that generated it, such as with (just replace
USERNAME and REPOSITORY with yours):
[![Coverage](.github/badges/jacoco.svg)](https://github.com/USERNAME/REPOSITORY/actions/workflows/build.yml)
The above assumes that the relevant workflow is build.yml
(replace as needed). This will
link the badge to the runs of that specific workflow.
All inputs include default values, and are thus optional provided the defaults are relevant to your use-case.
This input is the full path, relative to the root of the repository, to
the jacoco.csv
file, including filename. It defaults
to target/site/jacoco/jacoco.csv
, which is the default location and filename
assuming you are using the JaCoCo Maven plugin and don't change the default
output location. Note that if you are using Gradle to run your build, you will
definitely need to use this input, because the default location and name of the
jacoco csv report is different than it is in Maven. If you use Gradle's default
output directories, then you will need to set this input with something
like: jacoco-csv-file: build/reports/jacoco/test/jacocoTestReport.csv
.
If you have a multi-module project, you can pass the paths (including filenames)
to all of the jacoco.csv
files for all of the sub-projects. Separate these by spaces,
and in particular see the Multi-Module Example Workflows
for an example of how to do this. Multi-module support is limited to cases where
each module has its own test coverage report, and where those reports don't overlap.
The action assumes that all reports passed via this input are independent of each other. If you are using matrix testing, such that each group of tests produces a report, and where the groups overlap in what they are testing (e.g., one group tests a portion of a class or method, and another group tests another portion, etc), then the coverage computed by this action will not be correct. The csv reports don't contain enough information to properly merge such overlapping reports. If this applies to your use-case, then you will need to have JaCoCo produce a single JaCoCo report first (for example, see jacoco:report-aggregate).
This input is the directory for storing badges, relative to the root of the
repository. The default is .github/badges
. The action will create the badges
directory if it doesn't already exist, although the action itself does not commit.
This input controls whether or not to generate the coverage badge (Instructions
Coverage), and defaults to true
.
This input is the filename for the coverage badge (Instructions or C0
Coverage). The default filename
is jacoco.svg
. The file format is an svg
. The badge file will be
created within the badges-directory
directory. The action doesn't commit the badge file. You will
need to have additional steps in your workflow to do that.
This input controls whether or not to generate the branches coverage badge, and defaults
to false
. This defaults to false
to avoid surprising users who upgrade from earlier
versions with a badge they didn't know would be generated.
This input is the filename for the branches coverage badge (C1
Coverage). The default filename
is branches.svg
. The file format is an svg
. The badge file will be
created within the badges-directory
directory. The action doesn't commit the badge file. You will
need to have additional steps in your workflow to do that.
This input can be used to change the colors used for the badges.
It defaults to colors: '#4c1 #97ca00 #a4a61d #dfb317 #fe7d37 #e05d44'
,
which are the hex color codes for the colors described previously in
section Default Color Scheme.
Because #
has special meaning to YAML (it is used for comments), you
must either put quotes around the input value as shown in this example, or
you can escape each #
. The list of colors that you pass here can either
be space separated (as shown) or comma separated. The colors in this list
can be specified either with hex (as in the example above), or with any
named colors that are recognized by SVG, or some combination of the two.
Here is an example with named
colors: colors: green yellow orange red purple blue
. Notice that you don't need
quotes around the input if none of the colors are specified by hex.
Although the default uses six colors and six coverage intervals, you can have
as many or as few as you want. For example, if you want to use green
regardless
of percentage, you can set colors like this: colors: green
. If you pass more
colors than there are intervals, then the extra colors will be ignored. If you
pass an empty list of colors, then the action will simply use the default colors.
The action does not do any validation of the colors that you pass.
This input enables specifying the coverage intervals for the
different colors. It is a simple list of percentages. The default
is intervals: 100 90 80 70 60 0
, which corresponds to what is
described in the section Default Color Scheme
earlier. The action assumes that the percentages in this list are in
decreasing order. You can space separate or
comma separate the percentages. For example, intervals: 100 90 80 70 60 0
is equivalent to intervals: 100, 90, 80, 70, 60, 0
. A mix of spaces and commas
will also work.
If you specify too many intervals, the extras will simply be ignored. If there
are C colors altogether, then only the first (C-1) percentages specified in
this input are used, with the last color designated for coverages that are below
that last cutoff. For example, if you use the default set of 6 colors, then
intervals: 100 90 80 70 60
is equivalent to intervals: 100 90 80 70 60 0
.
Although these examples have integer percentages, the action
supports floating-point values. For example, you can specify something
like intervals: 99.5 90 80 70 60
.
If you only want to use the first three default colors (bright green, green,
and yellow green), then you don't necessarily need to change the value
of the colors
input. You can keep the default colors, and
then you can use something like intervals: 80 60
, which will assign
80 and above to bright green, 60 and above to green, and less than 60 to yellow
green.
If you like some of the default colors, but want to skip over some of them,
then you can either use a combination of the colors
input and intervals
input to accomplish this, or you can leave colors
at the default and
exploit the action's assumption of decreasing percentages in the intervals
input to skip the ones you don't like. For example, if
you want to use bright green for 80 and above,
yellow for 60 and above, and red for less than 60, you might do something like the
following: intervals: 80 80 80 60 60 0
. The 0 at the end is optional.
This input controls what happens if one or more jacoco.csv
files do not exist.
This input accepts one of three possible values: fail
, quiet
, or badges
.
The behavior of these is defined as follows:
- The default is
on-missing-report: fail
, in which case the action will return a non-zero exit code (causing the workflow run to fail) if one or more files listed in thejacoco-csv-file
input do not exist, or if an empty list of files is passed to the action. We recommend that you use this default since missing coverage report files in most cases probably means that there is either a bug in your workflow (e.g., typo in path to jacoco.csv) or that something went wrong in an earlier step (e.g., unit tests failed, halting generation of the coverage report). - You can use
on-missing-report: quiet
if you would rather the workflow itself not fail, in which case the action will instead quietly exit without producing badges if any JaCoCo reports are missing. - Although not recommended, a third option,
on-missing-report: badges
, will cause the action to produce badges from the report files that do exist, simply ignoring missing report files, provided that at least one such report file exists. We do not recommend this option since such a case is likely due to an error in your workflow, and any badges produced are likely computed with missing data.
Regardless of value passed to this input, the action will log warnings for
any files listed in the jacoco-csv-file
input that do not exist, for your
inspection in the workflow run.
This input enables directing the action to fail the workflow run if
the computed coverage is less than a minimum. The default is 0, effectively
disabling the option. You can specify it as either a floating point value
in the interval 0.0 to 1.0, or as a percent (with or without the percent sign).
For example, all of the following are equivalent: fail-if-coverage-less-than: 0.6
,
fail-if-coverage-less-than: 60
, or fail-if-coverage-less-than: "60%"
.
Note that in the last case, you need the quotes due to the percent sign.
Values greater than 1 are assumed percents.
This input enables directing the action to fail the workflow run if
the computed branches coverage is less than a minimum. The default is 0, effectively
disabling the option. You can specify it as either a floating point value
in the interval 0.0 to 1.0, or as a percent (with or without the percent sign).
For example, all of the following are equivalent: fail-if-branches-less-than: 0.6
,
fail-if-branches-less-than: 60
, or fail-if-branches-less-than: "60%"
.
Note that in the last case, you need the quotes due to the percent sign.
Values greater than 1 are assumed percents.
This input enables directing the action to fail the workflow run if
the computed coverage is less than it was on the previous run as recorded in the
existing coverage badge, if one exists. The default is false
.
Use fail-on-coverage-decrease: true
to enable. The generate-coverage-badge
input must also be true
(the default), as the action will otherwise assume
that there is no existing badge from which to get the prior coverage.
This input enables directing the action to fail the workflow run if
the computed branches coverage is less than it was on the previous run as recorded in the
existing branches badge, if one exists. The default is false
.
Use fail-on-branches-decrease: true
to enable. The generate-branches-badge
input must also be true
, as the action will otherwise assume
that there is no existing badge from which to get the prior branches coverage.
The action also outputs the actual computed coverage percentages as double-precision floating-point numbers. So you can add a step to your workflow to access these if desired (these action outputs are values in the interval from 0.0 to 1.0).
This output is the actual computed coverage percentage in the interval from 0.0 to 1.0. This is coverage computed from the instructions coverage data in the JaCoCo csv report.
This output is the actual computed branches coverage percentage in the interval from 0.0 to 1.0. This is the percentage of branches covered, computed from the branches data in the JaCoCo csv report.
The example workflows assume that you are using Maven to build and test
a Java project, and that you have the jacoco-maven-plugin
configured in your pom.xml
in the test phase with something
along the lines of the following:
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.7</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>generate-code-coverage-report</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Note that the jacoco-badge-generator action has been tested with
the jacoco.csv
files generated by jacoco-maven-plugin
versions
0.8.6 and 0.8.7, and has not been tested with earlier versions
of JaCoCo.
If you use gradle as your build tool, then you can configure JaCoCo
in build.gradle.kts
with:
plugins {
jacoco
}
tasks.jacocoTestReport {
reports {
csv.isEnabled = true
}
}
Or the equivalent in build.gradle
:
plugins {
id 'jacoco'
}
jacocoTestReport {
reports {
csv.enabled true
}
}
If you use Maven as your build tool, then you will have steps in your workflow along the lines of the following (which assumes that Maven is configured to run JaCoCo during the test phase:
- name: Build with Maven
run: mvn -B test
- name: Generate JaCoCo Badge
uses: cicirello/jacoco-badge-generator@v2
with:
generate-branches-badge: true
The equivalent for Gradle is:
- name: Run Tests
run: ./gradlew test
- name: Run Test Coverage
run: ./gradlew jacocoTestReport
- name: Generate JaCoCo Badge
uses: cicirello/jacoco-badge-generator@v2
with:
generate-branches-badge: true
jacoco-csv-file: build/reports/jacoco/test/jacocoTestReport.csv
You can also use a specific release with:
- name: Generate JaCoCo Badge
uses: cicirello/[email protected]
with:
generate-branches-badge: true
This sample workflow runs on pushes to the main
branch. It first sets up Java, and runs the tests with Maven. If you
have JaCoCo configured to run during the test phase, this will also
produce the JaCoCo reports. The jacoco-badge-generator action is then
run to parse the jacoco.csv
, compute the coverage percentage, and
generate the badge. The coverage percentage is then logged in the
workflow so you can inspect later if necessary. The next step of the workflow
checks if any changes were made to the badge, and if so, does a commit and a push (note
that there are also GitHub Actions that you can use for that step). And
finally, the JaCoCo coverage reports are uploaded as a workflow
artifact using the actions/upload-artifact
GitHub Action, so you can inspect them if necessary.
name: build
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up the Java JDK
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
- name: Build with Maven
run: mvn -B test
- name: Generate JaCoCo Badge
id: jacoco
uses: cicirello/jacoco-badge-generator@v2
- name: Log coverage percentage
run: |
echo "coverage = ${{ steps.jacoco.outputs.coverage }}"
echo "branch coverage = ${{ steps.jacoco.outputs.branches }}"
- name: Commit the badge (if it changed)
run: |
if [[ `git status --porcelain` ]]; then
git config --global user.name 'YOUR NAME HERE'
git config --global user.email '[email protected]'
git add -A
git commit -m "Autogenerated JaCoCo coverage badge"
git push
fi
- name: Upload JaCoCo coverage report
uses: actions/upload-artifact@v2
with:
name: jacoco-report
path: target/site/jacoco/
This example workflow is just like the above example, however, it generates both badges (instructions coverage percentage and branches coverage percentage). This example also uses the EndBug/add-and-commit action to commit and push the badge, whereas the previous example did this step with shell commands.
name: build
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up the Java JDK
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
- name: Build with Maven
run: mvn -B test
- name: Generate JaCoCo Badge
id: jacoco
uses: cicirello/jacoco-badge-generator@v2
with:
generate-branches-badge: true
- name: Log coverage percentage
run: |
echo "coverage = ${{ steps.jacoco.outputs.coverage }}"
echo "branch coverage = ${{ steps.jacoco.outputs.branches }}"
- name: Commit and push the badge (if it changed)
uses: EndBug/add-and-commit@v7
with:
default_author: github_actions
message: 'commit badge'
add: '*.svg'
- name: Upload JaCoCo coverage report
uses: actions/upload-artifact@v2
with:
name: jacoco-report
path: target/site/jacoco/
This example workflow generates both badges (instructions coverage percentage
and branches coverage percentage) for a multi-module project. The badges that are generated
are computed over all modules. To do so, simply pass the paths to all of the JaCoCo reports
that you want to include via the jacoco-csv-file
input. The >
is just Yaml's way of writing
a string across multiple lines. You can also just list all on a single space-separated line,
but your workflow file will be easier for you to read if you put them one per line.
In this example, there are three subprojects: module1
, module2
, and module3
.
name: build
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up the Java JDK
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
- name: Build with Maven
run: mvn -B test
- name: Generate JaCoCo Badge
id: jacoco
uses: cicirello/jacoco-badge-generator@v2
with:
generate-branches-badge: true
jacoco-csv-file: >
module1/target/site/jacoco/jacoco.csv
module2/target/site/jacoco/jacoco.csv
module3/target/site/jacoco/jacoco.csv
- name: Log coverage percentage
run: |
echo "coverage = ${{ steps.jacoco.outputs.coverage }}"
echo "branch coverage = ${{ steps.jacoco.outputs.branches }}"
- name: Commit the badge (if it changed)
run: |
if [[ `git status --porcelain` ]]; then
git config --global user.name 'YOUR NAME HERE'
git config --global user.email '[email protected]'
git add -A
git commit -m "Autogenerated JaCoCo coverage badge"
git push
fi
If you would prefer to generate separate coverage badges for each of the
modules of a multi-module project, then just include multiple steps of the
jacoco-badge-generator
in your workflow, such as in this example. Be sure to use
the inputs to specify names for the badge files, otherwise with the defaults
the subsequent steps will overwrite the previous. This example assumes that there
are two modules.
name: build
on:
push:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up the Java JDK
uses: actions/setup-java@v2
with:
java-version: '11'
distribution: 'adopt'
- name: Build with Maven
run: mvn -B test
- name: Generate JaCoCo Badges for Module 1
id: jacocoMod1
uses: cicirello/jacoco-badge-generator@v2
with:
generate-branches-badge: true
jacoco-csv-file: module1/target/site/jacoco/jacoco.csv
coverage-badge-filename: jacoco1.svg
branches-badge-filename: branches1.svg
- name: Generate JaCoCo Badges for Module 2
id: jacocoMod2
uses: cicirello/jacoco-badge-generator@v2
with:
generate-branches-badge: true
jacoco-csv-file: module2/target/site/jacoco/jacoco.csv
coverage-badge-filename: jacoco2.svg
branches-badge-filename: branches2.svg
- name: Commit the badge (if it changed)
run: |
if [[ `git status --porcelain` ]]; then
git config --global user.name 'YOUR NAME HERE'
git config --global user.email '[email protected]'
git add -A
git commit -m "Autogenerated JaCoCo coverage badge"
git push
fi
If you would like to see examples where the action is actively used, here
are a few repositories that are actively using the jacoco-badge-generator
action.
The table provides a link to repositories using the action, and direct links to the
relevant workflow as well as the relevant build configuration (e.g., Maven pom.xml
or Gradle build.gradle.kts
) so you can see how JaCoCo is
configured. Note that in all of the Maven examples, JaCoCo is configured within a
Maven profile within the pom.xml
, which is then activated via a command line
option when mvn
is run by the workflow on all push/pull request events. Configuration
can instead be done in the <build>
section if you'd rather not use a profile.
Repository | Workflow | Build Configuration |
---|---|---|
Chips-n-Salsa | build.yml | pom.xml |
JavaPermutationTools | build.yml | pom.xml |
ZigguratGaussian | build.yml | pom.xml |
XpathQS | build.yml | build.gradle.kts |
This GitHub action is released under the MIT License.