Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cabal new-test doesn't work with library-coverage for components with other-modules #5213

Closed
RyanGlScott opened this issue Mar 14, 2018 · 23 comments · Fixed by #7493
Closed

Comments

@RyanGlScott
Copy link
Member

$ cabal --version
cabal-install version 2.0.0.0
compiled using version 2.0.1.0 of the Cabal library

I'll work on preparing a minimal example in a bit. In the meantime, you can trigger this issue with scotty-0.11.0:

$ cabal get scotty-0.11.0
Unpacking to scotty-0.11.0/
$ cd scotty-0.11.0/
$ cat > cabal.project << EOF
> packages: .
> package scotty
>   library-coverage: true
> EOF
$ cabal new-test
Resolving dependencies...
In order, the following will be built (use -v for more details):
 - scotty-0.11.0 (lib) (first run)
 - scotty-0.11.0 (test:spec) (first run)
Configuring library for scotty-0.11.0..
Preprocessing library for scotty-0.11.0..
Building library for scotty-0.11.0..
<build output elided>
Configuring test suite 'spec' for scotty-0.11.0..
Preprocessing test suite 'spec' for scotty-0.11.0..
Building test suite 'spec' for scotty-0.11.0..
<build output elided>
Running 1 test suites...
Test suite spec: RUNNING...
<test output elided>
Test suite spec: PASS
Test suite logged to:
/home/ryanglscott/Documents/Hacking/Haskell/scotty-0.11.0/dist-newstyle/build/x86_64-linux/ghc-8.2.2/scotty-0.11.0/c/spec/test/scotty-0.11.0-spec.log
hpc: can not find scotty-0.11.0-inplace/Web.Scotty.Util in ./.hpc, /home/ryanglscott/Documents/Hacking/Haskell/scotty-0.11.0/dist-newstyle/build/x86_64-linux/ghc-8.2.2/scotty-0.11.0/c/spec/hpc/vanilla/mix/scotty-0.11.0, /home/ryanglscott/Documents/Hacking/Haskell/scotty-0.11.0/dist-newstyle/build/x86_64-linux/ghc-8.2.2/scotty-0.11.0/c/spec/hpc/vanilla/mix/spec
CallStack (from HasCallStack):
  error, called at libraries/hpc/Trace/Hpc/Mix.hs:122:15 in hpc-0.6.0.3:Trace.Hpc.Mix
cabal: Tests failed for test:spec from scotty-0.11.0.

This error doesn't happen for another project, wai-middleware-static-0.8.1. Unlike scotty, wai-middleware-static has no other-modules, so I suspect that is the culprit here.

@RyanGlScott
Copy link
Member Author

A minimal reproducer can be found at https://github.com/RyanGlScott/cabal-gh5213:

$ git clone https://github.com/RyanGlScott/cabal-gh5213
Cloning into 'cabal-gh5213'...
remote: Counting objects: 16, done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 16 (delta 1), reused 12 (delta 0), pack-reused 0
Unpacking objects: 100% (16/16), done.
$ cd cabal-gh5213/
$ cabal new-test
Resolving dependencies...
In order, the following will be built (use -v for more details):
 - cabal-gh5213-0.1 (lib) (first run)
 - cabal-gh5213-0.1 (test:tests) (first run)
Configuring library for cabal-gh5213-0.1..
Preprocessing library for cabal-gh5213-0.1..
Building library for cabal-gh5213-0.1..
[1 of 2] Compiling CabalGH5213Other ( src/CabalGH5213Other.hs, /home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/build/CabalGH5213Other.o )
[2 of 2] Compiling CabalGH5213Exposed ( src/CabalGH5213Exposed.hs, /home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/build/CabalGH5213Exposed.o )
Configuring test suite 'tests' for cabal-gh5213-0.1..
Preprocessing test suite 'tests' for cabal-gh5213-0.1..
Building test suite 'tests' for cabal-gh5213-0.1..
[1 of 1] Compiling Main             ( tests/Main.hs, /home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/build/tests/tests-tmp/Main.o )
Linking /home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/build/tests/tests ...
Running 1 test suites...
Test suite tests: RUNNING...
42
Test suite tests: PASS
Test suite logged to:
/home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/test/cabal-gh5213-0.1-tests.log
hpc: can not find cabal-gh5213-0.1-inplace/CabalGH5213Other in ./.hpc, /home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/hpc/vanilla/mix/cabal-gh5213-0.1, /home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/hpc/vanilla/mix/tests
CallStack (from HasCallStack):
  error, called at libraries/hpc/Trace/Hpc/Mix.hs:122:15 in hpc-0.6.0.3:Trace.Hpc.Mix
cabal: Tests failed for test:tests from cabal-gh5213-0.1.

@RyanGlScott
Copy link
Member Author

-v3 revealed that this is how hpc is being invoked:

/opt/ghc/8.2.2/bin/hpc markup /home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/hpc/vanilla/tix/tests/tests.tix '--destdir=/home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/hpc/vanilla/html/tests' '--hpcdir=/home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/hpc/vanilla/mix/tests' '--hpcdir=/home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/hpc/vanilla/mix/cabal-gh5213-0.1' '--exclude=Main'

It's worth noting that CabalGH5213Other.mix is being created:

$ find dist-newstyle/ -wholename "*cabal-gh5213-0.1-inplace/CabalGH5213Other*"
dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/hpc/dyn/mix/cabal-gh5213-0.1/cabal-gh5213-0.1-inplace/CabalGH5213Other.mix
dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/hpc/vanilla/mix/cabal-gh5213-0.1/cabal-gh5213-0.1-inplace/CabalGH5213Other.mix

I'm not sure why hpc can't find it, though, especially since it's passed '--hpcdir=/home/ryanglscott/Documents/Hacking/Haskell/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/c/tests/hpc/vanilla/mix/cabal-gh5213-0.1'.

@hvr
Copy link
Member

hvr commented Mar 14, 2018

I quick strace reveals, that hpc is looking at e.g.

open("./.hpc/cabal-gh5213-0.1-inplace/CabalGH5213Other.mix", O_RDONLY|O_NOCTTY|O_NONBLOCK) = -1 ENOENT (No such file or directory)
open("./dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/t/tests/hpc/vanilla/mix/cabal-gh5213-0.1/cabal-gh5213-0.1-inplace/CabalGH5213Other.mix", O_RDONLY|O_NOCTTY|O_NONBLOCK) = -1 ENOENT (No such file or directory)
open("./dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/t/tests/hpc/vanilla/mix/tests/cabal-gh5213-0.1-inplace/CabalGH5213Other.mix", O_RDONLY|O_NOCTTY|O_NONBLOCK) = -1 ENOENT (No such file or directory)

whereas it should have been looking at

./dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/hpc/vanilla/mix/cabal-gh5213-0.1/cabal-gh5213-0.1-inplace/CabalGH5213Other.mix

for reference, the flags in my case were:

'--destdir=/tmp/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/t/tests/hpc/vanilla/html/tests'

'--hpcdir=dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/t/tests/hpc/vanilla/mix/tests'

'--hpcdir=dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/t/tests/hpc/vanilla/mix/cabal-gh5213-0.1'

So it appears that cabal is not passing the right --hpcdir to hpc...

...or rather, that the .mix file isn't created in the right folder; the folder passed as --hpcdir is the proper one.

We're indeed missing the --hpcdir pointing to the library component's .mix files!

In other words, the

--hpcdir=dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/t/tests/hpc/vanilla/mix/cabal-gh5213-0.1

erroneously pointing into the test-components build-dir, should instead have been

--hpcdir=dist-newstyle/build/x86_64-linux/ghc-8.2.2/cabal-gh5213-0.1/hpc/vanilla/mix/cabal-gh5213-0.1

pointing to the library component's build-dir.

The more I investigate, the more issues/tech-debt I notice... :-/

@hvr
Copy link
Member

hvr commented Sep 5, 2018

I forgot to tackle this in time for 2.4, but I'll try to get to this within the next 1-2 weeks; but it's probably not going to be ready in time for the cabal-install 2.4.0.0 rls; more like 2.4.1.0

@hvr hvr added the meta: hvr label Sep 5, 2018
@vu3rdd
Copy link

vu3rdd commented Sep 5, 2018

Thanks @hvr.

@erikd
Copy link
Member

erikd commented Feb 5, 2019

Just hit this myself with cabal 2.4.1.0. Is that expected?

@23Skidoo
Copy link
Member

23Skidoo commented Feb 5, 2019

"Expected" in the sense that it's a known issue, yes.

@cartazio
Copy link
Contributor

cartazio commented May 3, 2019

#4798 is related right?

@cartazio
Copy link
Contributor

cartazio commented May 4, 2019

a related problem folks with fancy code setups hit is when they try to do coverage and have local/not hackage deps, and naively cabal tries do enable coverage on the local package. the work around for now is to set --disable-coverage for that local dep in the cabal.project file. But i think thats an issue that might be related. but probably merits its own ticket

@cartazio
Copy link
Contributor

quoting @davean

5:53 PM <davean> ocharles: you can specificly disable coverage on the other parts of the project and it will work
5:54 PM <davean> package reflex-time
5:54 PM <davean>   coverage: False
5:54 PM <davean>   library-coverage: False
5:54 PM <davean> as an example
5:55 PM <davean> It should, of course, work to have coverage for all of them
5:55 PM <davean> it just doesn't

a near term improvment would be ANY sort of warning saying "this is a known issue, heres what you need to do "

@cartazio
Copy link
Contributor

cc @ocharles

@Mikolaj
Copy link
Member

Mikolaj commented Jun 30, 2021

This quite likely got fixed in #7250 and/or #7358, which are both merged. Could somebody please check, e.g., using a dev releast that can be obtained as in #7458 (comment)?

@RyanGlScott
Copy link
Member Author

I downloaded a cabal artifact from commit f0d0594 per the instructions in #7458 (comment):

$ ~/Software/cabal --version
cabal-install version 3.5
compiled using version 3.5.0.0 of the Cabal library

However, I am still able to reproduce the bug when using the minimal example from #5213 (comment):

$ ~/Software/cabal new-test
Resolving dependencies...
Build profile: -w ghc-9.0.1 -O1
In order, the following will be built (use -v for more details):
 - cabal-gh5213-0.1 (lib) (first run)
 - cabal-gh5213-0.1 (test:tests) (first run)
Configuring library for cabal-gh5213-0.1..
Preprocessing library for cabal-gh5213-0.1..
Building library for cabal-gh5213-0.1..
[1 of 2] Compiling CabalGH5213Other ( src/CabalGH5213Other.hs, /home/ryanglscott/Software/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/build/CabalGH5213Other.o, /home/ryanglscott/Software/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/build/CabalGH5213Other.dyn_o )
[2 of 2] Compiling CabalGH5213Exposed ( src/CabalGH5213Exposed.hs, /home/ryanglscott/Software/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/build/CabalGH5213Exposed.o, /home/ryanglscott/Software/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/build/CabalGH5213Exposed.dyn_o )
Configuring test suite 'tests' for cabal-gh5213-0.1..
Warning: The package has an extraneous version range for a dependency on an
internal library: cabal-gh5213 >=0 && ==0.1. This version range includes the
current package but isn't needed as the current package's library will always
be used.
Preprocessing test suite 'tests' for cabal-gh5213-0.1..
Building test suite 'tests' for cabal-gh5213-0.1..
[1 of 1] Compiling Main             ( tests/Main.hs, /home/ryanglscott/Software/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/t/tests/build/tests/tests-tmp/Main.o )
Linking /home/ryanglscott/Software/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/t/tests/build/tests/tests ...
Running 1 test suites...
Test suite tests: RUNNING...
Test suite tests: PASS
Test suite logged to:
/home/ryanglscott/Software/cabal-gh5213/dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/t/tests/test/cabal-gh5213-0.1-tests.log
hpc: can not find cabal-gh5213-0.1-inplace/CabalGH5213Exposed in ./.hpc, ./dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/t/tests/hpc/vanilla/mix/cabal-gh5213-0.1, ./dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1/t/tests/hpc/vanilla/mix/tests
CallStack (from HasCallStack):
  error, called at libraries/hpc/Trace/Hpc/Mix.hs:122:15 in hpc-0.6.1.0:Trace.Hpc.Mix
cabal: Tests failed for test:tests from cabal-gh5213-0.1.

@Mikolaj
Copy link
Member

Mikolaj commented Jul 1, 2021

@RyanGlScott: thank you very much for verifying. A similar issue turned out to be fixed by the PRs but this one still holds. Good to know and since it's already marked as high priority, just a shout out to whomever feels heroic enough to tackle it. I dare say, with the recent changes, it may be easier to diagnose and code-dive now.

@Mikolaj
Copy link
Member

Mikolaj commented Jul 1, 2021

BTW, once the crash (probably caused by wrong paths given to hpc, as @hvr discovered) is fixed, are other modules supposed to be presented in the hpc markup report, or only exposed modules? I'm guessing the latter is now the case:

markup hpc hpcVer verbosity
(tixFilePath distPref way testName') mixDirs
htmlDir_
(exposedModules library)

but I think I'd be interested in dead or not tested enough code in my hidden library modules as well. OTOH, perhaps people are only interested in whether their tests hit all API entry points and don't care about particular code paths?

Edit: I think exposedModules library is wrong also because it only considers the main library of the package, unlike concatMap (exposedModules) $ PD.allLibraries pkg_descr down below, which has been correctly fixed recently. Paths have not been fixed in either case AFAICT, but this is only about the multi-lib case, so not a cause of this ticket's crash.

Edit2: the line from which the wrong hpc paths come is

distPref = fromFlag $ testDistPref flags

It should probably be

distPref = fromFlag (buildDistPref flags)

instead, but that's only a guess.

Edit3: but it doesn't typecheck.

Edit4: and it's unused in the code path from the repro. One of the places using markupTest is used and that's where we should change things.

Edit5: the place is src/Distribution/Simple/Test/ExeV10.hs.

Edit6: this typechecks fine in ExeV10.runTest:

    distPrefBuild = fromFlag $ configDistPref $ LBI.configFlags lbi

where lbi is

 -> LBI.LocalBuildInfo      -- ^information from the configure step

but the path is actually the same as the old one, probably because that is a configure step of the test command, not of the library building that was performed as part of it [edit: or in a previous invocation of cabal]. I have no idea where to take the library build phase dist directory from. The last chance of an easy solution is clbi :: LBI.ComponentLocalBuildInfo that is available there. I wonder if the main library component is inside it, with its dist directory location somehow recoverable.

Edit7: oh noes, this is a single ComponentLocalBuildInfo and it most probably comes from the test component.

Edit8: LBI.buildDir lbi is wrong as well, it just adds /build to the already wrong path with the unneeded /t/test. I'm grasping at straws at this moment.

@Mikolaj
Copy link
Member

Mikolaj commented Jul 3, 2021

I'm stuck at this point. I can't find the distPref directory used when building the main library component readily available anywhere. It seems we need to pass it explicitly to the test component somehow, e.g,. extend the TestComponentLocalBuildInfo variant of ComponentLocalBuildInfo with an extra field. However, I don't know what, if any, data structure keeps the stuff from previous components so that we can store the path there when building the main library and then put it into TestComponentLocalBuildInfo when starting the build of the test component.

A crude way would be to just remove /t/test from the end of the distPref` directory of the test component, but I bet these can be affected by config files, commandline arguments, environment variables (which is one more reason to try to eliminate those and stick to .caba/config configuration), possibly separately for each component. Or may be in the future.

Hints welcome.

@gbaz
Copy link
Collaborator

gbaz commented Jul 3, 2021

Ok so this is the offending bit:

distPref = fromFlag $ testDistPref flags

And we need to calculate and pass a second distPref as well, for the lib component.

The correct dir is accidentally (in a sense) the dataDir of the LocalBuildInfo. A hack would be to use that. A more correct thing would be to put a (potentially list) of dirs in the localBuildInfo for hpcdirs, calculated just as the dataDir is, but not risking being overridden by some flag. In either case, its the LocalBuildInfo and not the ComponentLocalBuildInfo that I think should be modified/used.

@Mikolaj
Copy link
Member

Mikolaj commented Jul 15, 2021

Given that dataDir doesn't work, I've gone the way of passing the correct directory as an extra field of LocalBuildInfo, but I'm stuck again. Namely, the incorrect directory (the one with spurious t/nameOfTestComponent/) is created in one go here

https://github.com/haskell/cabal/blob/master/cabal-install/src/Distribution/Client/DistDirLayout.hs#L193-L205

and nowhere else is, e.g., distParamCompilerId used. So the correct directory is not created anywhere in cabal at all. I'm guessing GHC creates it independently and this is how hpc knows where to store the .mix files (BTW, if that's true, that is quite fragile). I could create it independently as well, but I'd rather avoid that and creating it once and passing additionally from cabal-install to Cabal seems like a lot of extra code and polluted APIs. I hope I'm missing something.

Edit: to spell it out, the think I hope I'm missing is the correct directory (such as dist-newstyle/build/x86_64-linux/ghc-9.0.1/cabal-gh5213-0.1) being created anywhere at all in the cabal repo's codebase when invoked as cabal new-test (assuming the library component is built and up to date, so the correct directory won't be created as part of rebuilding the library).

@gbaz
Copy link
Collaborator

gbaz commented Jul 15, 2021

I think its fine for now to rely on ghc's behavior here. Arguably we should upstream flags to ghc so that we can control where mix files end up (it seems hardwired for now) but I'm fine hardcoding in something that matches ghc's hardcoding for now. cc @bgamari

@Mikolaj
Copy link
Member

Mikolaj commented Jul 21, 2021

Note to self (I'm not stuck): I think I've fixed the place in the code that we suspected of being buggy, but it still fails and in the same way. I think I was right the correct directory is not created in cabal code, but I think I was wrong it's created (hardwired) in GHC --- in fact, it seems we create the director via calls to Hpc.mixDir in Cabal/src/Distribution/Simple/GHC.hs. but we create a wrong one, similarly as in the place we originally indicated as erroneous. So I guess I need to hack the correct directory (by dropping the last two components of path, such as t/test/) in both places (actually, it seems in 3 or 4 places total). I think I will create a separate hacky function and apply it whenever we can tell the HPC processing is for tests, not a library.

alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 5, 2023
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 12, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 12, 2023
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 12, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
andreabedini pushed a commit to alt-romes/cabal that referenced this issue Dec 18, 2023
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
andreabedini pushed a commit to alt-romes/cabal that referenced this issue Dec 18, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 18, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
erikd pushed a commit to erikd/cabal that referenced this issue Apr 22, 2024
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
erikd pushed a commit to erikd/cabal that referenced this issue Apr 22, 2024
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants