Post-multilibs checklist

[fgaz]

#5526 (multiple public libraries) is in, but there are still a few things to add and polish

Visibility enforcement

• Add a visibility field to sublibraries 🔥 🔥 🔥 ❗ ❗ ❗
  • add the field
  • update ghc-pkg
  • populate the field
  • enforce the field: Prevent dependency on private library #5848

cabal-install integration

• Make the solver multilibs-aware Make the cabal-install solver multilibs-aware #6039
  • Enforce visibility at the solver level Solver: Support dependencies on sub-libraries (issue #6039). #6047 Solver: Support dependencies on sub-libraries (issue #6039) (3rd iteration) #6836
  • Extend the compatibility to GHC<8.8 when using cabal-install Solver: Support dependencies on sub-libraries (issue #6039) (3rd iteration) #6836
  • Handle installed packages
• Update cabal init to take into account sublibraries when populating build-depends

Tests

• Add tests
  • positive (ghc>=8.8) (isssue Add positive test for multilibs. #6409)
  • negative (all ghcs) (done in Prevent dependency on private library #5848)

Documentation

• Update changelog 4d2ca52
  • Visibility field not listed under cabal spec changes. #6362 Also update the cabal spec changelog, which I forgot
• Document multiple public libraries #8364 Update the Cabal and cabal-install docs (can be adapted from my post)

Bugfixes

• cabal-install: cannot depend on public sublibrary when the package does not contain a main library too #6038 (cannot depend on public sublibrary when the package does not contain a main library too)
• Cabal: cannot depend on public sublibrary when the package does not contain a main library too #8095 same but with Setup.hs/Cabal and installed packages
• Wrong "cabal: Encountered missing or private dependencies" error on sublibraries that start with "z" #7375 (same as Cabal: cannot depend on public sublibrary when the package does not contain a main library too #8095??)
• Boundless sublibrary dependency fails to parse #5846 (boundless dependency fails to parse)
• Cabal 3: in build-depends, package name gets treated as internal library name ("legacy" syntax) even when using the pkg:lib qualified syntax #6083 (edge case with the old syntax)
• mylib:{ } ^>= 42 syntax is accepted, though docs say it is not (Cabal 3: in build-depends, package name gets treated as internal library name ("legacy" syntax) even when using the pkg:lib qualified syntax #6083 related, makes sense to fix at the same time?) - fixedin 9b3686f
• Parse error when trying to use multiple public libraries within mixins #6281 (parse failure with mixins)
• Reconfiguration avoidance not affected by library visibility change #6358 Reconfiguration avoidance not affected by library visibility change
• 🔥 Dependency on installed public sublibrary without --dependency flag #7270 🔥 Cabal's simple solver may not pick up sublibraries unless --dependency is used?
  • Deprecate lookupDependency in favor of lookupInternalDependency and maybe a new lookupPackageDependency that looks up all libraries of a package

Ecosystem changes

• Make hackage render sublibraries too
  • haddock
  • cabal haddock run as part of new-build ignores internal libraries #5949
  • hackage Accept multi-lib packages with public sublibraries hackage-server#1119
• Nix Handle internal libraries correctly NixOS/cabal2nix#539

[phadej]

I mark this as high priority as 3.0 cannot be released before Add a visibility field to sublibraries is done.

ping @23Skidoo @hvr

[phadej]

Should solver be able to flip flags based on visible field? I think it should, but I don't know if it will make implementation tricky.

I don't know a good motivation, maybe pkgA:foo is visible, only if pkgB:bar is available, and author of pkgA.

e.g. pkgA.cabal:

cabal-version: 3.0
name: pkgA
...

library
   -- available always
  build-depends: foo

flag foo
  description: Provide pkgA:foo
  manual: False
  default: True

library foo
  -- this is contrived example, but ...
  -- also it's somewhat fragile, as there isn't a complement constraint; but it should work
  if foo
    visible: True
    build-depends: pkgB:bar

 -- if not pkgB:bar, there's still some stuff to be used by pkgA

[phadej]

we will need MIN_VERSION_pkgB:foo kind of thing. No idea how the name should be generated though. Maybe double underscore: MIN_VERSION_pkgB__foo? as there cannot be package pkgB--foo?

[phadej]

Should visibility be recorded into InstalledInfo. Then ./Setup.hs will be able to fail if you try to "manually" use a internal library which was registered as non-visible.

Note: this change would need GHC support (as package database is ghc-pkg business), so this makes things tricky. (practically limiting multilibs for ghc-8.8+)

[fgaz]

re IPI: yeah, I came to the same conclusion but didn't know how to go at it :-/

Is it just a matter of updating the cabal submodule in the ghc repo?

[phadej]

@fgaz essentially yes.

[phadej]

I'm chaging the milestone to 3.2.

@fgaz are you confident you can do this by the mid December (only two weeks!), or should we be waiting this only for 3.4?

[phadej]

Moving to 3.4

[cartazio]

how can this be progressed? :)

[fgaz]

Hmm actually some of these may be done. Let me update it first

[fgaz]

@phadej thank you for all the work you've been doing on fixing those bugs, especially #6893!

[phadej]

@fgaz is anyone working on Handle installed packages and Make the solver multilibs-aware #6039?

If not, I'll stop waiting for them for Cabal-3.4?

---

Also if you could check what's missing for Also update the cabal spec changelog, which I forgot, I can do that as well if there are missing points.

[fgaz]

@phadej

is anyone working on Handle installed packages and Make the solver multilibs-aware

not me, maybe @grayjay?
The solver is multilibs-aware now btw (I just checked the box to make it clearer).
The remaining subtask is not super important, because with v2- there's almost always a source package from which visibility can be determined. The only counterexample I can think of is ghc-bundled packages, but those don't have sublibraries.

what's missing for Also update the cabal spec changelog

Well, there isn't any mention of the multilibs feature, despite it being implemented Cabal-side. Would it be ok to add it to the 3.0 changelog or is that frozen and the 3.4 one should be used?

[fgaz]

The only counterexample I can think of is ghc-bundled packages

Duh, there's also distros' packages :-/
Does anyone use those in combination with cabal-install v2- though? I can only think of nix

[phadej]

The remaining subtask is not super important, because with v2- there's almost always a source package from which visibility can be determined. The only counterexample I can think of is ghc-bundled packages, but those don't have sublibraries.

I don't agree. I have heard this argument before, and ended up fixing issues myself because someone run into them. I won't be spending any time arguing whether it's important or not. If it's not done and known issue, the feature is not ready.

The Nix(pkgs) folks how install packages into global package database will run into this issue.

[grayjay]

I haven't had time to look into handling installed packages.

[kokobd]

Is this still being worked on?

[fgaz]

@kokobd yes... when contributors have the time. Missing items are mostly nice-to-haves and edge cases though. The ongoing stuff is:

• In the cabal-install solver, consider installed packages with multiple public libraries (last part of Make the cabal-install solver multilibs-aware #6039)
• On the hackage side, Accept multi-lib packages with public sublibraries hackage-server#1119

Help is appreciated of course!

[ulysses4ever]

Another item on that list in the OP, perhaps:

• Cabal confused with sublibrary when same-named remote package exists #9122