Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Program coverage crashes in cabal new-test #4798

Closed
jiaochangyang opened this issue Sep 27, 2017 · 26 comments · Fixed by #7358 or #9464
Closed

Program coverage crashes in cabal new-test #4798

jiaochangyang opened this issue Sep 27, 2017 · 26 comments · Fixed by #7358 or #9464

Comments

@jiaochangyang
Copy link
Collaborator

jiaochangyang commented Sep 27, 2017

Preprocessing test suite 'testsuite' for simple-server-0.1.0.0..
Building test suite 'testsuite' for simple-server-0.1.0.0..
Running 1 test suites...
Test suite testsuite: RUNNING...
Tests
  core unit tests
    testing storeInsert empty:                                     OK
    testing storeInsert when already in small:                     OK
    testing storeInsert when already in commit:                    OK
    testing storeInsert when data is in both:                      OK
    testing isSecondaryIndexed when in store:                      OK
    testing isSecondaryIndexed when not in store:                  OK
    testing secondaryLookup when not in secondary:                 OK
    testing secondaryLookup when in small:                         OK
    testing secondaryLookup when in commit:                        OK
    testing secondaryInsert when not in secondary:                 OK
    testing read (ReadAll) when not in store:                      OK
    testing read (lookup) when in small store:                     OK
    testing read (lookup) when in commit store:                    OK
    testing write successfully in small store:                     OK
    testing write unsuccessfully (duplicate) in small store:       OK
    testing write unsuccessfully (duplicate) in commit store:      OK
    testing write failure in store (somehow):                      OK
    testing write failure in secondary (already in store somehow): OK
    testing write failure in store (already in secondary somehow): OK

All 19 tests passed (0.00s)
Test suite testsuite: PASS
Test suite logged to:
/Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/test/simple-server-0.1.0.0-testsuite.log
hpc: can not find simple-server-0.1.0.0-inplace/Data.API.Read in ./.hpc, /Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/simple-server-0.1.0.0, /Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/testsuite
CallStack (from HasCallStack):
  error, called at libraries/hpc/Trace/Hpc/Mix.hs:122:15 in hpc-0.6.0.3:Trace.Hpc.Mix
cabal: Tests failed for test:testsuite from simple-server-0.1.0.0.

@cartazio was looking through this with me and can reproduce the same issue

we are using ghc 8.2.1 and cabal new-build / new-test

cabal --version
cabal-install version 2.0.0.0
compiled using version 2.0.0.2 of the Cabal library

a verbose (-v3) output would be as follows

te testsuite: PASS
Test suite logged to:
/Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/test/simple-server-0.1.0.0-testsuite.log
Environment: [("Apple_PubSub_Socket_Render","/private/tmp/com.apple.launchd.Q3JKpyPRfN/Render"),("COLORFGBG","7;0"),("COLORTERM","truecolor"),("HASKELL_DIST_DIR","/Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite"),("HOME","/Users/cyj"),("ITERM_PROFILE","Default"),("ITERM_SESSION_ID","w0t2p0:F80CC278-087E-4137-B59D-639A0978E05D"),("LANG","en_US.UTF-8"),("LOGNAME","cyj"),("PATH","/Users/cyj/.ghc-installs/ghc-8.2.1-hq/bin:/Users/cyj/.cabal/bin:/Users/cyj/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/cyj/.cabal/bin:/Users/cyj/.cabal/bin"),("PWD","/Users/cyj/Documents/simple-server"),("SECURITYSESSIONID","186a6"),("SHELL","/bin/bash"),("SHLVL","1"),("SSH_AUTH_SOCK","/private/tmp/com.apple.launchd.AaBcIdWfTT/Listeners"),("TERM","xterm-256color"),("TERM_PROGRAM","iTerm.app"),("TERM_PROGRAM_VERSION","3.1.2"),("TERM_SESSION_ID","w0t2p0:F80CC278-087E-4137-B59D-639A0978E05D"),("TMPDIR","/var/folders/fp/_6rb74hd3j39xp_scrxxf4ww0000gn/T/"),("USER","cyj"),("XPC_FLAGS","0x0"),("XPC_SERVICE_NAME","0"),("__CF_USER_TEXT_ENCODING","0x1F5:0x0:0x0")]
/Users/cyj/.ghc-installs/ghc-8.2.1-hq/bin/hpc markup /Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/hpc/vanilla/tix/testsuite/testsuite.tix '--destdir=/Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/hpc/vanilla/html/testsuite' '--hpcdir=/Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/testsuite' '--hpcdir=/Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/simple-server-0.1.0.0' '--exclude=Main'
hpc: can not find simple-server-0.1.0.0-inplace/Data.API.Read in ./.hpc, /Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/simple-server-0.1.0.0, /Users/cyj/Documents/simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/testsuite
CallStack (from HasCallStack):
  error, called at libraries/hpc/Trace/Hpc/Mix.hs:122:15 in hpc-0.6.0.3:Trace.Hpc.Mix
/Users/cyj/.ghc-installs/ghc-8.2.1-hq/bin/hpc returned ExitFailure 1
packages definitely up to date: simple-server-0.1.0.0-inplace
packages previously probably up to date: simple-server-0.1.0.0-inplace, simple-server-0.1.0.0-inplace-simple-server-scotty, simple-server-0.1.0.0-inplace-simple-server-snap
packages now probably up to date: simple-server-0.1.0.0-inplace, simple-server-0.1.0.0-inplace-simple-server-scotty, simple-server-0.1.0.0-inplace-simple-server-snap
packages newly up to date:
packages out to date: simple-server-0.1.0.0-inplace-testsuite
packages invalid due to dep change: simple-server-0.1.0.0-inplace-testsuite
packages invalid due to build failure: simple-server-0.1.0.0-inplace-testsuite
CallStack (from HasCallStack):
  die', called at ./Distribution/Client/ProjectOrchestration.hs:782:8 in main:Distribution.Client.ProjectOrchestration
cabal: Tests failed for simple-server-0.1.0.0-inplace-testsuite.

we can make the simple application code available to any cabal maintainer privately (well, @cartazio will deal with that), but can't make it public at the moment, (its a pretty small project, but minimizing this failure would require more ideas about how cabal works than we do! :) )

@cartazio
Copy link
Contributor

pinging @ttuegel and @hvr and @bgamari and @23Skidoo :)

i want my program coverage magic!

it does seem to correctly generate mix and tix files ... i think ... but i never have tried coverage checking plus new build before :(

@23Skidoo
Copy link
Member

@ttuegel is the one most knowledgeable about these bits of Cabal.

@cartazio
Copy link
Contributor

cartazio commented Sep 28, 2017 via email

@cartazio
Copy link
Contributor

@23Skidoo @ttuegel lemme know if theres any way i can help
... should i use cabal with stack trace, or dig into the two source locations with those errors?

@EButlerIV
Copy link

EButlerIV commented Sep 30, 2017

@cartazio
I just replicated it myself, and I don't think it's project- or local settings-dependent. This is what happened when I ran it against a freshly downloaded package from hackage.

The test suite log doesn't contain any errors, but my build dir definitely doesn't contain a .hpc directory. It's likely that any coverage files generated are being stored somewhere else.

eugene@Eugenes-MacBook-Pro ~/D/P/blah> cabal get u2f
Downloading u2f-0.1.0.1...
Unpacking to u2f-0.1.0.1/
eugene@Eugenes-MacBook-Pro ~/D/P/blah> cd u2f-0.1.0.1/
eugene@Eugenes-MacBook-Pro ~/D/P/b/u2f-0.1.0.1> cabal new-test --enable-coverage
[snip]
Finished in 0.0223 seconds
5 examples, 0 failures
Test suite hspec-suite: PASS
Test suite logged to:
/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/test/u2f-0.1.0.1-hspec-suite.log
hpc: can not find u2f-0.1.0.1-inplace/U2F.Types in ./.hpc, /Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/vanilla/mix/u2f-0.1.0.1, /Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/vanilla/mix/hspec-suite
CallStack (from HasCallStack):
  error, called at libraries/hpc/Trace/Hpc/Mix.hs:122:15 in hpc-0.6.0.3:Trace.Hpc.Mix
cabal: Tests failed for test:hspec-suite from u2f-0.1.0.1.

E: Manually moving the files to the expected location results in successful generation of coverage reports. It looks like this is the issue.:

eugene@Eugenes-MacBook-Pro ~/D/P/b/u2f-0.1.0.1> mkdir .hpc
eugene@Eugenes-MacBook-Pro ~/D/P/b/u2f-0.1.0.1>
cp -r dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/hpc/vanilla/mix/u2f-0.1.0.1/u2f-0.1.0.1-inplace .hpc/

eugene@Eugenes-MacBook-Pro ~/D/P/b/u2f-0.1.0.1> ls -lh .hpc/
total 0
drwxr-xr-x  4 eugene  staff   136B Sep 30 15:45 u2f-0.1.0.1-inplace
eugene@Eugenes-MacBook-Pro ~/D/P/b/u2f-0.1.0.1> cabal new-test --enable-coverage
In order, the following will be built (use -v for more details):
 - u2f-0.1.0.1 (test:hspec-suite) (first run)
Preprocessing test suite 'hspec-suite' for u2f-0.1.0.1..
Building test suite 'hspec-suite' for u2f-0.1.0.1..
Running 1 test suites...
Test suite hspec-suite: RUNNING...

Request/Response Parsing
  should work for registration requests
Registration Flow
  should check that request and response challenges are equivalent
  should validate properly formed request and response
Signin Flow
  should check that request and response challenges are equivalent
  should validate properly formed request and response

Finished in 0.0056 seconds
5 examples, 0 failures
Test suite hspec-suite: PASS
Test suite logged to:
/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/test/u2f-0.1.0.1-hspec-suite.log
Writing: u2f-0.1.0.1-inplace/U2F.Types.hs.html
Writing: u2f-0.1.0.1-inplace/U2F.hs.html
Writing: hpc_index.html
Writing: hpc_index_fun.html
Writing: hpc_index_alt.html
Writing: hpc_index_exp.html
Test coverage report written to
/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/vanilla/html/hspec-suite/hpc_index.html
1 of 1 test suites (1 of 1 test cases) passed.
Writing: u2f-0.1.0.1-inplace/U2F.Types.hs.html
Writing: u2f-0.1.0.1-inplace/U2F.hs.html
Writing: hpc_index.html
Writing: hpc_index_fun.html
Writing: hpc_index_alt.html
Writing: hpc_index_exp.html
Package coverage report written to
/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/vanilla/html/u2f-0.1.0.1/hpc_index.html

@EButlerIV
Copy link

EButlerIV commented Sep 30, 2017

Ok, last update. It looks like the real issue is caused by bad arguments in this use of hpc markup. The last hpcdir argument appears to be incorrect.

This is what it uses

/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/prof/mix/u2f-0.1.0.1

This is what it should actually be

/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/hpc/prof/mix/

Incorrect invocation in full

/Users/eugene/.installed-ghc/ghc-8.0.2/bin/hpc markup /Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/prof/tix/hspec-suite/hspec-suite.tix --destdir=/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/prof/html/hspec-suite --hpcdir=/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/prof/mix/hspec-suite --hpcdir=/Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/hpc/prof/mix/u2f-0.1.0.1 --exclude=Main

E: Can confirm that normal cabal test does run and create coverage reports without issue when coverage is enabled.

@cartazio
Copy link
Contributor

so either
a) ghc isn't being told where to write the mix files correctly by cabal
b) cabal isn't telling hpc where to read the mix files correctly ?

@cartazio
Copy link
Contributor

cartazio commented Sep 30, 2017

it seems that cabal new-test didn't adjust how args/paths are passed to the the markup routines that invoke hpc... though @EButlerIV and i are really just poking around and guessing atm :)

that is: when we build ghc programs with coverage, /Users/eugene/Development/Personal/blah/u2f-0.1.0.1/dist-newstyle/build/x86_64-osx/ghc-8.0.2/u2f-0.1.0.1/c/hspec-suite/hpc/prof/mix/u2f-0.1.0.1 should be setup properly ... or something

@cartazio
Copy link
Contributor

@23Skidoo would fixing this be also a bug fix for 2.0 series cabal :)

@cartazio
Copy link
Contributor

a) possible culprit: program coverage flags not set correctly for new build in general?
shouldn't these two definitions of hpc dir

/Users/carter/Desktop/repoScratcher/CabalSept2017/Cabal/Cabal/Distribution/Simple/GHC.hs:
  543        -- TH always needs default libs, even when building for profiling
  544  
  545:   -- Determine if program coverage should be enabled and if so, what
  546    -- '-hpcdir' should be.
  547:   let isCoverageEnabled = libCoverage lbi
  548        -- TODO: Historically HPC files have been put into a directory which
  549        -- has the package name.  I'm going to avoid changing this for
  ...
  554        hpcdir way
  555          | forRepl = mempty  -- HPC is not supported in ghci
  556:         | isCoverageEnabled = toFlag $ Hpc.mixDir distPref way pkg_name
  557          | otherwise = mempty
  558  
  ...
 1121    -- modules?  FIX: what about exeName.hi-boot?
 1122  
 1123:   -- Determine if program coverage should be enabled and if so, what
 1124    -- '-hpcdir' should be.
 1125:   let isCoverageEnabled = exeCoverage lbi
 1126        distPref = fromFlag $ configDistPref $ configFlags lbi
 1127        hpcdir way
 1128          | gbuildIsRepl bm   = mempty  -- HPC is not supported in ghci
 1129:         | isCoverageEnabled = toFlag $ Hpc.mixDir distPref way (gbuildName bm)
 1130          | otherwise         = mempty
 1131  

is the results of distBuildDirectory not being passed correctly somewhere?

@23Skidoo
Copy link
Member

23Skidoo commented Oct 1, 2017

would fixing this be also a bug fix for 2.0 series cabal :)

It depends on how hard it will be to backport.

@ttuegel
Copy link
Member

ttuegel commented Oct 12, 2017

The root of this issue is that the wrong build directory is being passed to the test phase. The test phase is getting dist-newstyle/build/x86_64-linux/ghc-8.0.2/u2f-0.1.0.1/t/hspec-suite when it should be getting dist-newstyle/build/x86_64-linux/ghc-8.0.2/u2f-0.1.0.1. I have no idea why or where this happens because I still cannot wrap my head around the new-* phases.

@ttuegel
Copy link
Member

ttuegel commented Oct 12, 2017

One more thing: even if I can fix that, there is the underlying problem that each component now has a different build directory, which is incompatible with the existing API of hpcDir. In other words, I could not backport this hypothetical bug fix to Cabal 2.0.

@cartazio
Copy link
Contributor

cartazio commented Oct 12, 2017 via email

@ezyang
Copy link
Contributor

ezyang commented Oct 12, 2017

Oh, I guess the problem here makes sense: for a non-Custom setup, new-test will create separate build directories for the library and test (like, literally separate, with each having their own LocalBuildInfo). And I guess the coverage mechanism was never clued in about how to deal with this. So this is more of a feature request.

One way to work around the problem, then, is to turn off per-component build. I think there are some knobs for this, but you can at least easily check by changing the build-type to Custom.

@ttuegel
Copy link
Member

ttuegel commented Oct 12, 2017

In 2.0 they’re all in the same component afaict so it should be a simpler version of the same fix I think

@cartazio
If they were all in the same component, then they should have the same build directory and then this wouldn't be a problem. I haven't looked at 2.0 yet, but if what you say about per-component build is correct, then it must suffer from a different problem with similar symptoms.

@jiaochangyang
Copy link
Collaborator Author

attached is a full log
coverage-log.log

the bit at the end in particular is


All 19 tests passed (0.00s)
Test suite testsuite: PASS
Test suite logged to:
/Users/cyj/enterprise-simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/c/testsuite/test/enterprise-simple-server-0.1.0.0-testsuite.log
/Users/cyj/.ghc-installs/ghc-8.2.1-hq/bin/hpc markup /Users/cyj/enterprise-simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/c/testsuite/hpc/vanilla/tix/testsuite/testsuite.tix '--destdir=/Users/cyj/enterprise-simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/c/testsuite/hpc/vanilla/html/testsuite' '--hpcdir=/Users/cyj/enterprise-simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/testsuite' '--hpcdir=/Users/cyj/enterprise-simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/enterprise-simple-server-0.1.0.0' '--exclude=Main'
hpc: can not find enterprise-simple-server-0.1.0.0-inplace/Data.API.Read in ./.hpc, /Users/cyj/enterprise-simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/enterprise-simple-server-0.1.0.0, /Users/cyj/enterprise-simple-server/dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/c/testsuite/hpc/vanilla/mix/testsuite
CallStack (from HasCallStack):
  error, called at libraries/hpc/Trace/Hpc/Mix.hs:122:15 in hpc-0.6.0.3:Trace.Hpc.Mix
CallStack (from HasCallStack):
  die', called at ./Distribution/Client/ProjectOrchestration.hs:782:8 in main:Distribution.Client.ProjectOrchestration
cabal: Tests failed for enterprise-simple-server-0.1.0.0-inplace-testsuite.

@ttuegel
Copy link
Member

ttuegel commented Oct 17, 2017

@cj285 Could you please also post the output of find dist-newstyle -name hpc?

@jiaochangyang
Copy link
Collaborator Author

This is the result of find dist-newstyle -name hpc

~/enterprise-simple-server ((323160c7…)|…) $ find dist-newstyle -name hpc
dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/c/testsuite/hpc
dist-newstyle/build/x86_64-osx/ghc-8.2.1/enterprise-simple-server-0.1.0.0/hpc

@cartazio
Copy link
Contributor

cartazio commented Oct 24, 2017 via email

@ttuegel
Copy link
Member

ttuegel commented Oct 24, 2017

Let us know if there’s any further information we can provide or other ways we can help

I know what the problem is and I have a solution, but I am blocked on testing by not having a working Cabal installation at the moment due to other bugs.

@cartazio
Copy link
Contributor

cartazio commented Nov 18, 2017 via email

@ttuegel
Copy link
Member

ttuegel commented Nov 21, 2017

Not really... I "only" need to figure out how to get the build path information into the right places.

At the moment, per-component build is a problem: when we're running the test component, the library component isn't in scope, so we can't determine the library build directory, so we can't pass the correct HPC directories to the test runner.

The easy solution is to disable per-component build when coverage is requested. How badly would this impact you?

ttuegel added a commit to ttuegel/cabal that referenced this issue Nov 22, 2017
Creating the test program coverage report requires the test runner to know the
build directory for the library component. If per-component builds are enabled,
this is not possible because the library is not in-scope at the same time as the
test component. The only apparent solution that does not require rethinking
per-component builds entirely is to disable it when coverage is enabled.

Resolves: haskell#4798
@cartazio
Copy link
Contributor

cartazio commented Nov 22, 2017 via email

ttuegel added a commit to ttuegel/cabal that referenced this issue Nov 27, 2017
Creating the test program coverage report requires the test runner to know the
build directory for the library component. If per-component builds are enabled,
this is not possible because the library is not in-scope at the same time as the
test component. The only apparent solution that does not require rethinking
per-component builds entirely is to disable it when coverage is enabled.

Resolves: haskell#4798
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 5, 2023
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 7, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 8, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 11, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 12, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 12, 2023
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 12, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
andreabedini pushed a commit to alt-romes/cabal that referenced this issue Dec 18, 2023
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
andreabedini pushed a commit to alt-romes/cabal that referenced this issue Dec 18, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
alt-romes added a commit to alt-romes/cabal that referenced this issue Dec 18, 2023
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
erikd pushed a commit to erikd/cabal that referenced this issue Apr 22, 2024
This commits re-enables per-component builds when coverage checking is
enabled. This restriction was previously added in haskell#5004 to fix haskell#4798.

- haskell#4798 was subsequently fixed "again" with the fix for haskell#5213, in haskell#7493 by
fixing the paths of the testsuite `.mix` files to the same location as
that of the main library component.

Therefore the restriction to treat testsuites per-package
(legacy-fallback) is no longer needed.

We went further and fixed coverage for internal sublibraries, packages
with backpack (but without generating coverage information for
indefinite and instantiated units -- it is not clear what it would mean
for HPC to support this), and coverage for multi-package projects.

1. We allow hpc in per-component builds

2. To generate hpc files in the appropriate component directories in the
distribution tree, we remove the hack from haskell#7493 and instead determine
the `.mix` directories that are included in the call to `hpc markup` by
passing the list of components in the project from the cabal-install
invocation of test.
We also drop an unnecessary directory in the hpc file hierarchy.

3. To account for internal (non-backpack) libraries, we include the mix
   dirs and modules of all (non-indefinite and non-instantiations)
   libraries in the project

   Indefinite libraries and instantiations are ignored as it is not
   obvious what it means for HPC to support backpack, e.g. covering a
   library function that two different instantiations

4. We now only reject coverage if there are no libraries at all in the
   project, rather than if there are no libraries in the package.

This allows us to drop the coverage masking logic in
cabal.project.coverage while still having coverage of cabal-install
(i.e. cabal test --enable-coverage cabal-install now works without the
workaround)

Even though we allow multi-package project coverage, we still cover each
package independently -- the tix files resulting from all packages are
not combined for the time being.

Multi-package project coverage is fixed in Cabal, however, the
paths to the source files listed in the `.mix` files will be incorrect
because package sources will no longer be in the root of the project
tree, but rather under the subdir with the package. We add an error for
multi-package projects when coverage is enabled, and track lifting this
error in haskell#9493.

Includes tests for haskell#6440, haskell#6397, haskell#8609, and haskell#4798 (the test for haskell#5213 already exists)

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage)
, doesn't yet fix haskell#8609 (multi-package coverage report) and fixes in a new way the
previously fixed haskell#4798, haskell#5213.
erikd pushed a commit to erikd/cabal that referenced this issue Apr 22, 2024
This commit re-designs the mechanism by which we make the .mix files of
libraries available to produce the Haskell Program Coverage report after
running testsuites.

The idea, for the Cabal library, is:

* Cabal builds libraries with -fhpc, and store the hpc artifacts in
  build </> `extraCompilationArtifacts`
* At Cabal install time, `extraCompilationArtifacts` is copied into the
  package database
* At Cabal configure time, we both
    - receive as --coverage-for flags unit-ids of library components
      from the same package (ultimately, when haskell#9493 is resolved, we will
      receive unit ids of libraries in other packages in the same
      project too),
    - and, when configuring a whole package instead of just a testsuite
      component, we determine the unit-ids of libraries in the package
  these unit-ids are written into `configCoverageFor` in `ConfigFlags`
* At Cabal test time, for each library to cover (stored in
  `configCoverageFor`), we look in the package database for the hpc
  dirs, which we eventually pass along to the `hpc markup` call as
  `--hpcdir` flags

As for cabal-install:

* After a plan has been elaborated, we select the packages which can be
  covered and pass them to Cabal's ./Setup configure as
  --coverage-for=<unit-id> flags.
    - Notably, valid libraries are non-indefinite and
      non-instantiations, since HPC does not support backpack.
    - Furthermore, we only include libraries in the same package as the
      component being configured, despite possibly there being
      more library components in other packages of the same project.
      When haskell#9493 is resolved, we could lift this restriction and pass
      all libraries local to the package as --coverage-for. See
      `determineCoverageFor` and `shouldCoverPkg` in Distribution.Client.ProjectPlanning.

Detail:
    We no longer pass the path to the testsuite's mix dirs to `hpc
    markup` because we only ever include modules in libraries, which
    means they were previously unused.

Fixes haskell#6440 (internal libs coverage), haskell#6397 (backpack breaks coverage),
doesn't yet fix haskell#8609 (multi-package coverage report) which is tracked
in haskell#9493, and fixes in a new way the previously fixed haskell#4798, haskell#5213.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment