Skip to content
Steve Gilham edited this page May 4, 2021 · 52 revisions

How do I get my F# inline functions to show whether they are partly or fully covered?

Unfortunately, the only way is to do the coverage runs with the inline declaration conditionally commented out. The inlined code is just added to the call site, with no debug reference to the original source location. Nor is there any indication in the IL that a function was declared inline, so it can't be optionally suppressed in the way that e.g. automatic properties are.

If I do dotnet test my.sln how do I get separate coverage files for each test project?

As of release v7.1.795, if you include the literal $(ProjectName) in the report file path, then that will be explicitly replaced, in the circumstances that MSBuild hasn't done so already, so, for example:

/p:AltCoverReport=/_temp/$(ProjectName).coverage.xml

will drop a set of distinctly named files into the temporary folder.

This also works for the tokens $(SolutionDir) and $([System.Guid]::NewGuid())

How do I tell which test covered which line, like with OpenCover's coverbytest option?

The --callContext command line option (or equivalently named parameter for other means of invocation) does this, and in a more general fashion.

To track all unit tests, use the attribute name in square brackets option, giving your unit test framework's attribute, like [Test], [Fact], [TestMethod], or whatever.
To track a subset of tests, either mark them with a custom attribute, or give an explicit list of test method names (either by multiple --callContext options for the command line, a | separated list for MSBuild or dotnet test, or as an appropriately typed collection of names for everything else)
For integration test cases, where the focus is what happens per API call, track the API methods either by name or a common attribute
For integration test cases, where test steps can be marked against wall clock time, e.g. finding what is covered by a user operation that causes a sequence of API calls, use the timer interval option to match visits with test steps.

  • Method and attribute name matches are exact (apart from the optional Attribute part of an attribute type name), not wild-card or regex based.
  • Branch visits are also tracked

I only have .net core 3 or above on my machine -- how do I get things to work?

Upgrade to v6.5 or later. From v5.3, the global tools should work anyway.

The command-line tool AltCover.dll can also be run using the roll forwards option as in dotnet --roll-forward Major AltCover.dll ..., as should the old-style dotnet-cli tool as in dotnet --roll-forward Major altcover ... If required, the blunt instrument is to set the environment variable DOTNET_ROLL_FORWARD to have value Major.

If I use the Quick Start, I get the coverage including the unit test code or a third-party test runner. How do I exclude the unit test related code like this?

Use the -e or -assemblyExcludeFilter command line argument (the /p:AltCoverAssemblyExcludeFilter MSBuild parameter). to exclude unit test assemblies (or other assemblies in the test stack which depend on the ones you're interested in covering e.g. test adapters). This stops any visit instrumentation being written, but will still perform any necessary rewriting to link correctly against the instrumented assemblies, and to apply --callContext tracking.

My system under test drags in a large number of other assemblies, so I'm having to use a whole bunch of filter exclusions. Is there a way to just opt-in an assembly to instrument?

Yes; from release 6.2.714 filter expressions with a leading ? are treated as excluding non-matching, so "exclude all strings that don't have substring 'xyzzy'" can be written as "?xyzzy". With earlier releases, more complicated and fragile expressions using negative lookahead were required like "^((?!xyzzy).)*$", which had problems scaling.

My system under test invokes some platform specific (possibly native code) libraries that appear in x86 or x64 subdirectories, but these aren't carried over into the instrumented code, so running with the instrumented test assembly, the test set fails for not finding them. How do I fix this?

The --inplace option is probably what you need here; then you can point the unit tests at the instrumented assembly in $(TargetDir). Alternatively, you need a manual copy step between instrumenting and testing.

What about using AltCover on C++/CLI assemblies?

The Mono.Cecil library used to instrument the assembles does not read the non-IL code in mixed-mode assemblies, and does not write assemblies without the ILOnly core flag set. As the C++/CLI compiler does not reliably set that flag even using the now deprecated /clr:pure compiler option, that effectively excludes that language from support.

The global tool visualizer doesn't seem to work. What gives?

tl;dr version -- Upgrade to the latest version (7.1.780 or later).

For the previous implementation, having GTK+3 installed on your machine and in your PATH was a pre-req. For Windows, this requires jumping through a few hoops, see e.g. https://github.com/GtkSharp/GtkSharp/wiki/Installing-Gtk-on-Windows

Also, GTK+ needs the GSETTINGS_SCHEMA_DIR environment variable set to pick up standard assets.

If that variable isn't set by your install, then you can make it a part of the process-level environment, by command-line argument --schemadir=path-of-schema-directory -- that value will be saved into the application configuration for later uses. For GTK+ installed as above on a vanilla 64-bit system, the appropriate path is C:\msys64\mingw64\share\glib-2.0\schemas

I do "Instrument now, test later" style coverage gathering, and I get zero CRAP scores.

Yes, coverage percentages, and any quantities derived from those percentages, are not computed in the end-of-process handler. Use a "runner" mode to perform those calculations in their own process, or use the Write-OpenCoverDerivedState cmdlet (or the underlying Toolkit API AltCover.OpenCover.PostProcess) which extracts just that step of the "runner" operations.