Document static compilation (was: Static compile support for AWS Linux (Redhat/CentOS))

[chadbrewbaker]

If you are compiling for AWS Lambda you have to statically link everything, i.e. stack ghc Foo.hs -threaded -static -optl-static

This requires static versions of libffi, glibc, libgmp, ... finally got it working but it was a pain. Could really use support for static compiles out of the box in stack.

[snoyberg]

Can you list the steps you initially took and the error message you received?

[snoyberg]

Also, the steps you used to resolve the problem would be very useful too.

[rvion]

some people including myself would love to have a simple --static argument to easilly produce an as-static-as-possible executable

[chadbrewbaker]

I used the compile flags above. Had to use straight GHC since something broke inside Stack on the Redhat 7 install. If you run ldd a.out you get a list of dynamic libs you have to bring in. For each I went to there project page, downloaded them and did the ./configure;make; sudo make install .

[snoyberg]

@chadbrewbaker As mentioned above, I'll need to see the error messages generated. And how did you get the stack binary in the first place? What changes are you asking be made (changes to the RPM, for example?). I'm still missing information on this.

[chadbrewbaker]

Stack does not install both static and dynamic versions of all pre-compiled
libraries. You need static linking when deploying to machines with none of
the dynamic libs installed. I will detail my latest Stack on AWS Linux
install issue in another ticket as that problem is orthogonal.

On Sat, Sep 26, 2015 at 11:25 PM, Michael Snoyman [email protected]
wrote:

@chadbrewbaker https://github.com/chadbrewbaker As mentioned above,
I'll need to see the error messages generated. And how did you get the
stack binary in the first place? What changes are you asking be made
(changes to the RPM, for example?). I'm still missing information on this.

—
Reply to this email directly or view it on GitHub
#1032 (comment)
.

[snoyberg]

Maybe I just don't have enough experience with Redhat, but this issue still isn't clear to me. Can you clarify what you mean by "pre-compiled libraries," for instance? Is this Haskell libraries, system libraries, etc?

[chadbrewbaker]

/usr/lib type libraries. To statically link you need libfoo.a, not libfoo.so or libfoo.dylib. If libfoo.a doesn't exist you either have to find a libfoo-static package, or download the libfoo source and compile it yourself as a static library.

[snoyberg]

If you'd like to send a PR to modify the rpms to include the relevant
system libraries, we can probably include that (assuming it doesn't
drastically increase the download size). Short of that, I'm not really sure
what changes stack can make for this.

On Thu, Oct 1, 2015, 2:37 PM Chad Brewbaker [email protected]
wrote:

/usr/lib type libraries. To statically link you need libfoo.a, not
libfoo.so or libfoo.dylib. If libfoo.a doesn't exist you either have to
find a libfoo-static package, or download the libfoo source and compile it
yourself as a static library.

—
Reply to this email directly or view it on GitHub
#1032 (comment)
.

[borsboom]

I'm not quite clear. Stack doesn't install any libraries in /usr/lib. The distro packages (RPMs/.Debs) do list minimal dependencies so that regular Stack functionality works, but I don't know that we should extend the dependencies to cover every possible use case. RPMs don't seem to support recommended vs. required dependencies the way Debian packages packages do either so we can't just add them to a recommended/suggested list in the case of CentOS/RHEL/Amazon. We've already more-or-less decided that "configuration management" (e.g. installing system libs) is out of scope for Stack, much as we'd all love for Stack to do everything.

At minimum, though, it's certainly worth documenting what is required to build static binaries.

[chadbrewbaker]

I don't disagree. I'll write a bash script this weekend that pulls down other needed packages and builds a static hello world for AWS Lambda then make a pull request to the docs.

[DemiMarie]

Problem: GMP is LGPL so distributing a statically linked binary without corresponding .o files (so that a modified GMP can be used) violates the GMP license. So integer-simple will need to be used, unless someone can find a good, permissively licensed alternative.

[chadbrewbaker]

No LGPL violation if you include the source.

[proger]

There is no LGPL violation if the software is not being distributed — I guess using AWS Lambda for private stuff / company use counts here.

Is it possible to build a custom ghc for use with stack btw? I'd like to be able to not use glibc with ghc (and the one that stack downloads uses it), but with something like ghc-musl instead.

[borsboom]

@proger: Yes it's possible to use a custom GHC with Stack, using the ghc-variant option.

[diogob]

👍 for having a --static flag :D

[sjakobi]

Maybe some bits from this r/haskell thread might be useful for fixing this issue.

[jml]

The instructions here are helpful for linux, but I struggle to get them to work on OS X.

I've asked a question on Stack Overflow about this; http://stackoverflow.com/questions/39805657/how-can-i-create-static-executables-on-os-x-with-stack

[jml]

I've also created a repo that minimally reproduces the problems you might see when trying to statically link an executable that requires a C library: https://github.com/jml/haskell-static-minimal-repro

[jml]

https://ghc.haskell.org/trac/ghc/ticket/10912 is this ticket filed against GHC.

[erebe]

Hello,
The crtbeginT hack is not at all required to compile a static binary.
The only thing needed (beside static versions of librairies) is to add the option ld-options: -static in your cabal file and compile with --ghc-options="-fPIC"

Please see this
https://stackoverflow.com/questions/41419102/haskell-stack-static-binary-relocation-r-x86-64-32-against-tmc-end-can-not/41427067#41427067 and https://www.reddit.com/r/haskell/comments/5lk33p/struggling_building_a_static_binary_for_aws/
for an in-depth why.

I would like to fight this miss-conception of the crtbeginT hack, as it is mentioned almost everywhere when this subject is talked about and too many post now refers to it.
For me the easiest option to do that would be to add an --static option to stack.

I tried to grok the code in order to propose a PR, but sadly I don't understand zip of the code.
If someone can guide me trough it so I can add the option.

[songdongsheng]

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.0.1

$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux bullseye/sid"
NAME="Debian GNU/Linux"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

$ ghc -O2  -threaded -static -optl-static hello.hs
[1 of 1] Compiling Main             ( hello.hs, hello.o )
Linking hello ...
/usr/bin/ld.gold: error: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(lowlevellock.o): multiple definition of '__lll_lock_wait_private'
/usr/bin/ld.gold: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc.a(libc-lowlevellock.o): previous definition here
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)

[chadbrewbaker]

Stack is very similar to https://rustup.rs . Would it make sense to have some synergy in trying to pin to the same LLVM and try to link in the same manner from the user install path instead of using what may be missing/old versions of libc or libpthread etc on the host? Throw in Clang too so Rust, Haskell, C/C++ all play nice together at the LLVM IR level.