compilation with stack fails because argument list to hsc2hs is too long #49206
Comments
|
Note that this seems to be a problem with either a newer version of If I use ghc-8.0.2 to compile $ git clone [email protected]:cdepillabout/haskell-gi.git
$ git checkout hsc2hs-error-ghc-8.0
$ stack build --verbose --cabal-verbose --ghc-options '-v3'
...
(everything succeeds)
... |
Can you check if this is just coincidence (e.g. the command lines involved being slightly shorter in that version)? |
|
@nh2 Thanks for the response.
I went back to try to figure this out and... I couldn't reproduce the error from above! Yesterday right after I posted this issue, I just happened to upgrade from 18.03 to 18.09. And now I can no longer reproduce the error.
I guess I'm fine with this, and I'm not interested in investigating further if the solution seems to be just to upgrade to a later release of NixOS. I'll go ahead and close this issue. Hopefully problem will now be google-able. |
|
Unfortunately I've run into this problem again. As far as I can tell, it only occurs when building with I'm going to reopen this since it doesn't seem like it is fixed just by upgrading to a newer version of nixpkgs. When I have more time I'm going to try to dig more into this to figure out what is going on. |
|
What is the package count? |
|
Can you verify that you have the latest version of Nixpkgs in your shell? For instance try this diff on your project: I was pretty sure we fixed this problem with Haskell by 18.09 (so it should only appear in nixpkgs-unstable from like April to July). I think stack uses your builtin channels unless you do something like the above. |
|
I got some free time this weekend so I've tried to come up with a way to reliably reproduce this on NixOS. Here's the method I've come up with. You need
This should fail with the following error when trying to build the The Here is the full list of unique flags. Nothing looks too strange. The only strange thing is that there were originally tons of repeats: https://gist.github.com/cdepillabout/0a7d79e5bf856bd8d14ca9ab271bc488 The last line ( |
|
I thought that the above (step 3) would only happen after installing many Haskell packages, but I was incorrect. The above still happens when trying to install It appears the the error from step 3 will happen if all you do is just try to build |
|
@cdepillabout Could you try this commit: This is definitely a generic-stack-builder thing because we aren't getting it in other places. |
|
@matthewbauer Thanks for taking a look at this. Unfortunately matthewbauer@83fef59 doesn't seem to help. I am still getting the same error as above. Here is a commit for termonad where I change the nixpkgs pin: It is on this branch: https://github.com/cdepillabout/termonad/tree/try-to-fix-hsc2hs-problem If you are interested in testing this out yourself, you could clone Termonad, checkout the above branch, and run I thought this problem might be happening because of the environment variables that are set by nix (and stack?), so here is the output https://gist.github.com/cdepillabout/74e7560440997973aa56ad3b7e26d8b1 There are a couple environment variables with relatively long values:
|
|
The I'm thinking that this is not a problem in the |
matthewbauer
added
0.kind: regression
|
I've tried to update Termonad to a more recent version of nixpkgs (master from around 2018-12-23), but it doesn't look like it has fixed this problem. Here's a commit that can be used to reproduce this problem: $ git clone [email protected]:cdepillabout/termonad.git
$ git checkout 5a4e05d1c4
$ stack build
...This should fail when building I had an idea of trying to work around this with something like |
|
I was able to use Here's the PR where I add the derivation using |
|
@cdepillabout can you try #53618 with your project. It looks to have fixed it by I am not sure if I am reproducing exactly what you describe. |
|
@matthewbauer Unfortunately this doesn't appear to fix it for me. In #53618 (comment) you said:
That's what I was thinking as well. @nh2 sent a PR to cabal that fixed #41340 for most things: However I'm wondering if somewhere else in Cabal also needs to be fixed. I'm wondering if the place where |
|
This is fixed for me on 19.03 with this commit matthewbauer/haskell-gi@ac1134b Feel free to reopen if this is still an issue. |
|
Unfortunately this doesn't seem to be fixed. I'm still seeing this error when trying to build Termonad with This should (hopefully) be reproducible: The above is trying to build Termonad at commit f4776be73, which changes to use the latest commit for the It looks like I don't have permissions to reopen this issue though. |
|
Is this on Linux or macOS? Have you been able to file this with stack? I think there is something weird going on with how they handle the --extra-lib-dirs and --extra-include-dirs flag. It looks like they are somehow duplicating them for each dependency so you get tons and tons of duplicates. Right now Nixpkgs should only be setting the flag once for each place. At the very least, the fact we can build haskell-gi on Cabal+Nix makes me think Stack is accumulating these variables weirdly. |
|
@matthewbauer The error from #49206 (comment) was on Linux. I just tried compiling on MacOS with $ stack build --nix
...
-- While building package haskell-gi-0.21.5 using:
/Users/illabout/.stack/setup-exe-cache/x86_64-osx-nix/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.3 --builddir=.stack-work/dist/x86_64-osx-nix/Cabal-2.4.0.1 build --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
Process exited with code: ExitFailure 1
Logs have been written to: /Users/illabout/git/termonad/.stack-work/logs/haskell-gi-0.21.5.log
Configuring haskell-gi-0.21.5...
Preprocessing library for haskell-gi-0.21.5..
hsc2hs: /nix/store/wdjf5qf597b5pcxfw031zxm5rk9w4p1y-clang-wrapper-5.0.2/bin/cc: rawSystem: runInteractiveProcess: exec: resource exhausted (Argument list too long)
I actually hadn't thought about |
|
Yeah, so each include directory is listed 38 times each. I believe what's happening is that stack passes the |
I believed I had fixed at least one instance of this in |
|
@nh2 I haven't looked into this issue deeply on the Cabal-side, but it appears to be occurring when cabal calls My guess is that haskell/cabal#5356 fixed the problem when compiling Haskell modules, but there is still a lingering problem when calling external programs (like |
|
This issue might be related to haskell/hsc2hs#22, as is posted by @gbaz. |
|
building with an hsc2hs from the branch of that PR indeed fixes this issue, but it then just opens the way for the next iteration of the problem, which is that the cc-wrapper itself runs into problems, as discussed in #41340 |
|
I am having the same problem. I am using linux Mint 19 and stack version 2.1.3. ghc version 8.6.5. -- While building package haskell-gi-0.21.5 using: |
|
@jdevoto I don't think the underlying problem has been solved here yet. However, as a workaround, I'm running You should be able to use this like the following: $ nix-shell ./stack-fhs-env.nix
$ stack build # this is within the nix-shellOr, just directly using Also, since you're not actually on NixOS, one possibility is just to install the GTK libraries with your system package manager, and not involve |
|
Thanks for the help |
|
Hello, I'm a bot and I thank you in the name of the community for opening this issue. To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human. The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it. If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them. Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel. |
stale
bot
added
the
2.status: stale
Issue description
I am seeing an error when using
stackto compile Haskell code on NixOS. It appears that the argument list passed tohsc2hsis extremely long, andhsc2hscallsgccwith that long argument list.gccfails to deal with an extremely long argument list.This seems to be another symptom of #41340.
Steps to reproduce
I've created a reproduction with the
haskell-gipackage. You can try it out with the following steps:I thought this might be fixed with haskell/cabal#5356, so I tried backporting it to Cabal-2.2 in ghc-8.4.3, but this did not appear to fix the problem. You can try out my fix like the following, although it recompiles GHC, so it takes a long time:
Just like #41340, I guess this is ultimately a Nix problem, but it would be nice if
Cabalcould be fixed to not send duplicate arguments tohsc2hs. It would also be nice ifhsc2hswould be fixed similarly.Should I open up an issue on the
Cabalorhsc2hsrepos?Pinging @nh2, @ElvishJerricco, @Ericson2314, @domenkozar, and @dmjio since they seemed to be active on issue #41340. Also pinging @23Skidoo, @IvanMalison, and @angerman since they were active on haskell/cabal#5356. Finally, pinging @garetxe since he is the author of the
haskell-gilibrary, which I am using to cause the error above (although I don't thinkhaskell-giis particularly doing anything bad here).Technical details
"x86_64-linux"Linux 4.14.51, NixOS, 18.03.132768.94d80eb7247 (Impala)yesnonix-env (Nix) 2.0.4"nixos-18.03""unstable-18.09pre145679.dae9cf6106d, nixos-18.03"/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgsThe text was updated successfully, but these errors were encountered: