Nix shell does not provide Alex, Happy to Cabal-Install

[ScottFreeCode]

Describe the bug

I have a Haskell package using callCabal2nix, with build-tool-depends on alex and happy. If I nix-shell straight into my default.nix and use the phase hooks, including the less commonly known compileBuildDriverPhase, I can build the project in a Nix shell environment. However, if I use a shell.nix that adds cabal-install and uses .env then I get an error from cabal build claiming it can't find/build happy (and the phase hooks aren't there).

Upstream issue: haskell/cabal#7197 (comment)

While my minimal example uses callCabal2nix, the same issue also affects IOG's haskell.nix (where .shellFor is equivalent to .env and is needed to nix-shell at all unless you want to specify individual components for nix-shell with -A tmp.components.exes.tmp): input-output-hk/haskell.nix#798

Steps To Reproduce

tmp.cabal:

cabal-version:      3.0
name:               tmp
version:            0.1.0.0

author:             Demo

executable tmp
    main-is:          Main.hs

    build-depends:    base ^>=4.15.1.0
    hs-source-dirs:   app
    default-language: Haskell2010

    build-tool-depends:
          alex:alex
        , happy:happy

default.nix:

{ pkgs ? import <nixpkgs> {} }:
pkgs.haskellPackages.callCabal2nix "tmp" ./. {}

shell.nix:

{ pkgs ? import <nixpkgs> {} }:
(pkgs.haskell.lib.addBuildTool
  (import ./. { inherit pkgs; })
  pkgs.cabal-install
).env

How to get the error:

1. nix-shell
2. cabal build

output

Warning: No remote package servers have been specified. Usually you would have
one specified in the config file.
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: tmp-0.1.0.0 (user goal)
[__1] unknown package: tmp:happy:exe.happy (dependency of tmp)
[__1] fail (backjumping, conflict set: tmp, tmp:happy:exe.happy)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: tmp, tmp:happy:exe.happy

Expected behavior

Cabal builds the project locally.

Screenshots

N/A

Additional context

If you do the following, you bypass the issue:

app/Main.hs:

main = print "Hello world"

1. nix-shell default.nix (i.e. don't use .env)
2. eval "$compileBuildDriverPhase"
3. eval "$configurePhase"
4. eval "$buildPhase"

Program is built to: dist/build/tmp/tmp

The way that nix-build does it (and which avoiding .env allows you to do by explicitly evaling the hooks) uses Setup.hs instead of the cabal executable. The default Setup.hs it uses appears to be:

import Distribution.Simple
main = defaultMain

It looks like it prefers a local Setup.hs, even though I don't have custom build type set, so that's… interesting and potentially an incorrectly handled edge case?

Notify maintainers

@cdepillabout

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.43, NixOS, 22.05 (Quokka), 22.05.481.40e2b1ae053`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.1`
 - channels(root): `"nixos-22.05"`
 - channels(scott): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[cdepillabout]

cc @NixOS/haskell

I was able to reproduce this locally as well.

It seems unfortunate that cabal-install doesn't know to take build-tools from Nix, although I guess it makes sense that it wouldn't know this.

---

I feel like I've seen that another work-around for this is to use v1-build. If I create the project layout as described above and run nix-shell, then cabal v1-build, cabal-install is able to build everything without explicitly needing to build alex and happy. (Although I guess nothing is explicitly calling them. I didn't test what exactly cabal v1-build would do if alex or happy actually needed to be called.)

Here's one more issue that looks related: input-output-hk/haskell.nix#839

[sternenseemann]

Duplicate of #130556

[ScottFreeCode]

Thanks for looking into this!

I'd be interested to know what the pros and cons are of using v1-build in a Nix shell sandbox (does it try to build its own happy/alex or just get less picky about using Nix's, does it try to use a global package DB or still use the nix-provided one, etc.).

We're up to a few different workarounds, as another mentioned in some of these issues is to make the project dependent on Nix and simply remove the tools from the .cabal file, instead putting them in default.nix explicitly so they're provided but Cabal doesn't think it needs to be picky about them. So we have that, use of v1-, and nix-shell'ing the main derivation directly and then calling the phases rather than using cabal at all.

Looking over the issues, I also see a couple different proposals around fixing things. One is to add it to the package-db, if that even works for executables (though it's not really "correct" for them even if it works). Another is theoretically adding a new package-db equivalent for executables and then teaching Nix to use that. And another is input-output-hk/haskell.nix#839 (comment) though I don't understand that explanation very well (.conf files?).

This could also be affected in the future if Setup.hs goes away, a la haskell/cabal#7906 – I also wonder how that would impact use of a tool like BNFC, the discussion seems centered on integration of other package managers like Rust's.

[sternenseemann]

v2 insists that executables it needs are in the package db (so it can do constraint checking), while v1 is happy if a correctly named executable is in in PATH. However, putting something in the package db also allows linking against the package’s library, meaning we can no longer separate build (»native«) and runtime dependencies which is a correctness issue in the case of cross compiling. Not that you do need to use the v1 /sandbox/ nor build anything yourself.

…

On 9. Jun 2022, at 03:12, Scott Santucci ***@***.***> wrote:  Thanks for looking into this! I'd be interested to know what the pros and cons are of using v1-build in a Nix shell sandbox (does it try to build its own happy/alex or just get less picky about using Nix's, does it try to use a global package DB or still use the nix-provided one, etc.). We're up to a few different workarounds, as another mentioned in some of these issues is to make the project dependent on Nix and simply remove the tools from the .cabal file, instead putting them in default.nix explicitly so they're provided but Cabal doesn't think it needs to be picky about them. So we have that, use of v1-, and nix-shell'ing the main derivation directly and then calling the phases rather than using cabal at all. Looking over the issues, I also see a couple different proposals around fixing things. One is to add it to the package-db, if that even works for executables (though it's not really "correct" for them even if it works). Another is theoretically adding a new package-db equivalent for executables and then teaching Nix to use that. And another is input-output-hk/haskell.nix#839 (comment) though I don't understand that explanation very well (.conf files?). This could also be affected in the future if Setup.hs goes away, a la haskell/cabal#7906 – I also wonder how that would impact use of a tool like BNFC, the discussion seems centered on integration of other package managers like Rust's. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are on a team that was mentioned.

[ScottFreeCode]

Mmm, and based on a little experiment, cabal v1-build with Nix enabled in my user-level Cabal config basically seems to do the same thing that Nix's phases do, handle everything locally in dist with (I think) Nix-provided packages, except that it is stricter about putting imported modules in the custom-setup setup-depends (I have to add process to import System.Process – good, I should figure out if I can get Nix builds to be this strict).

So using v1- is a much nicer workaround than my trick of calling Nix's phases.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-use-flake-inputs-instead-of-a-cabal-project/25110/2

[ParetoOptimalDev]

It seems unfortunate that cabal-install doesn't know to take build-tools from Nix, although I guess it makes sense that it wouldn't know this.

@cdepillabout There's an upstream issue for it and the maintainers are open to PR's adding support:

haskell/cabal#8434 (comment)

[ScottFreeCode]

I've collected GitHub issues relating to this problem here: https://gist.github.com/ScottFreeCode/ef9f254e2dd91544bba4a068852fc81f