Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix GHC loadArchive clang++ error on macOS #149942

Closed
wants to merge 1 commit into from
Closed

Fix GHC loadArchive clang++ error on macOS #149942

wants to merge 1 commit into from

Conversation

jkachmar
Copy link
Contributor

@jkachmar jkachmar commented Dec 9, 2021

Closes #149937.

Motivation for this change
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@jkachmar
Copy link
Contributor Author

jkachmar commented Dec 9, 2021

Note that I haven't tested this patch on GHC 8.8.4, 9.0.1, or 9.2.1.

I've only applied it to those expressions following IOHK's guidance (as they apply this patch in their repository to all versions of GHC following 8.6.4).


EDIT: If possible I would really appreciate it if we could back port this to earlier versions of nixpkgs (21.11 at a minimum, but 21.05 would also be wonderful).

I was bitten by it on x86_64-darwin before (but I wasn't sufficiently motivated to track down a solution at the time); today I ran into it again on aarch64-darwin now that I'm trying to use nixpkgs with one of the new M1 MacBook Pros.

@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Dec 10, 2021
GHC 8.6.4 introduced an issue where linking to `-libc++` will cause
GHC to resolve the library patch to `c++`, which in turn is a link to
clang++.

This results in the following error:

```
ghc: loadArchive: Neither an archive, nor a fat archive: `/nix/store/?-clang-wrapper-5.0.2/bin/clang++'

<no location info>: error:
    loadArchive "/nix/store/?-clang-wrapper-5.0.2/bin/clang++": failed
```

This commit applies the relevant patch to revert GHC's linking behavior
on darwin for the applicable versions of GHC (i.e. after 8.6.4).
@ofborg ofborg bot requested a review from guibou December 10, 2021 18:18
@ofborg ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Dec 10, 2021
@jkachmar
Copy link
Contributor Author

@bgamari made the following observation when we discussed this in another channel:

Oof, that c++ issue is rather frustrating since it's arguably a workaround for a linker bug which may break currently-working programs

If I understand correctly this patch will break users who specify, e.g., extra-libraries: z in their cabal file

I think a more correct way to work around this bug would be to incorporate a check to ensure that we aren't trying to link against an executable

@cdepillabout
Copy link
Member

If I understand correctly this patch will break users who specify, e.g., extra-libraries: z in their cabal file

@jkachmar Were you able to figure out if this is the case?

If so, it sounds pretty bad? My guess is that there are quite a few packages using extra-libraries (although I'm not actually that sure).

Also, most of the Haskell maintainers (other than sternenseemann) don't use darwin as far as I know, so we might have to ping @ NixOS/darwin-maintainers in order to get this reviewed.

@jkachmar
Copy link
Contributor Author

jkachmar commented Feb 9, 2022

I looked into this a bit more again tonight and I still don't really know what the best path forwards is here.

Some related issues:


@jwiegley I saw that you opened the GHC issue linked above; did you ever find a satisfactory workaround to the issue (or was it resolved for you in an updated GHC release)?

@mariatsji
Copy link

Thanks, this patch solved the problem I was having with cabal build in a nix-shell using GHC 8.10.7 on darwin:

ghc: loadArchive: Neither an archive, nor a fat archive: `/nix/store/cvz17hkb0wpbsyx58l4zcaamqs2bhlmz-clang-wrapper-11.1.0/bin/clang++'
ghc: panic! (the 'impossible' happened)
  (GHC version 8.10.7:
        loadArchive "/nix/store/cvz17hkb0wpbsyx58l4zcaamqs2bhlmz-clang-wrapper-11.1.0/bin/clang++": failed

hasura-bot pushed a commit to hasura/graphql-engine that referenced this pull request Sep 30, 2022
Improvements to the Nix configuration so that macOS is supported.

Microsoft SQL Server is still not supported (yet; I have something in mind there), but the rest works. You can still use Homebrew to install the SQL Server drivers.

I had to make the following changes:

* I updated nixpkgs, because it's been a while.

* I made `ODBCINSTINI` optional so that it's not loaded on macOS, as it depends on `msodbcsql17`, which is broken on macOS.

* I upgraded OpenSSL.

* I set `DYLD_LIBRARY_PATH` on macOS so GHC finds OpenSSL; otherwise, it uses the wrong version of `libcrypto`, and fails with a fun error:

  > WARNING: ghc is loading libcrypto in an unsafe way

* I patched GHC to fix compilation on macOS (copied from NixOS/nixpkgs#149942).

To test this out, you can run `nix develop` (or install [direnv][] and add `use flake` to _.envrc.local_), and then try building HGE in the shell provided.

Fair warning: GHC needs to be built, and takes _aaages_ the first time. If this becomes useful to others, we can set up a shared cache.

[direnv]: https://direnv.net/

PR-URL: hasura/graphql-engine-mono#5989
GitOrigin-RevId: 7130738d47709c37778b31c134061758ce23e959
@ghost
Copy link

ghost commented Dec 1, 2022

A bit surprised this is still not fixed. Can't build cabal-install on darwin because of that. :(

@sternenseemann sternenseemann marked this pull request as draft December 1, 2022 19:11
@sternenseemann
Copy link
Member

As stated above, the workaround is not satisfactory, as it may break legitimate use cases that are already working today.

Building cabal-install on darwin is unaffected by this issue:

@jkachmar
Copy link
Contributor Author

jkachmar commented Dec 2, 2022

i agree that this solution is unsatisfactory, however as far as i can tell "something" does need to be done to address this. i'm fine with closing this PR, but i hesitated to do so because it has more activity (and more context) than the discussion at #149937.

if anyone has any ideas for how we should fix this, i'd be more than willing to dedicate some time to exploring them and patching things properly.

zwilias added a commit to NoRedInk/servant-elm that referenced this pull request Feb 3, 2023
There's a transitive dependency somewhere that links against c++ which
causes all kinds of havoc. For now, easy way out: use
NixOS/nixpkgs#149942
@maksar
Copy link
Contributor

maksar commented Mar 18, 2023

iohk repo doesn't have a patch for ghc944. Structure of the files has changed in GHC. I used this patch and can confirm issue disappeared:

--- a/compiler/GHC/Linker/Loader.hs
+++ b/compiler/GHC/Linker/Loader.hs
@@ -1616,8 +1616,6 @@
        | verbatim = [lib]
        | otherwise = [ "lib" ++ lib ++ lib_tag <.> "a"
                      , lib <.> "a" -- native code has no lib_tag
-                     , "lib" ++ lib
-                     , lib
                      ]
      lib_tag = if is_hs && loading_profiled_hs_libs then "_p" else ""

@sternenseemann
Copy link
Member

@maksar This is equivalent to the patch proposed in this PR and as mentioned above, GHC upstream thinks that the patch breaks legitimate use cases.

@jkachmar jkachmar closed this by deleting the head repository Nov 25, 2023
@harryaskham
Copy link
Contributor

Experiencing the same issue with GHC 9.8.1 on nix-darwin, tried with gcc/clang from each of OSX and nixpkgs in the path. I'll see if patching still works following e.g. e9d090f

@maksar
Copy link
Contributor

maksar commented Jun 15, 2024

@harryaskham JFYI we are still using this patch on 9.8.2 and it helps on darwin and doesn't hurt on linux.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: darwin Running or building packages on Darwin 6.topic: haskell 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GHC unusable on darwin when linking against -libc++
6 participants