Build fails on DESTDIR= build environments: No rule to make target '/usr/bin/nix'

[trofi]

Linux distributions usually install packages into sandboxed environments before moving packages into live system:

$ ./configure --prefix=/usr
$ make install DESTDIR=/sandbox
$ mv /sandbox/* /

nix fails at make install stage:

make: *** No rule to make target '/usr/bin/nix', needed by 'doc/manual/nix.json'.  Stop.
make: *** Waiting for unfinished jobs....

On NixOS it's easiest to reproduce by using inaccessible prefix:

$ nix develop -f ~/nixpkgs/master nixUnstable
$ ./bootstrap.sh
$ ./configure --prefix=/usr
$ make install DESTDIR=$PWD/__sandbox__
...
make: *** No rule to make target '/usr/bin/nix', needed by 'doc/manual/nix.json'.  Stop.

I sprinlked s@$(bindir)/nix@$(DESTDIR)$(bindir)/nix@g around with:

--- a/doc/manual/local.mk
+++ b/doc/manual/local.mk
@@ -20,7 +20,7 @@ dummy-env = env -i \
        NIX_STATE_DIR=/dummy \
        NIX_CONFIG='cores = 0'

-nix-eval = $(dummy-env) $(bindir)/nix eval --experimental-features nix-command -I nix/corepkgs=corepkgs --store dummy:// --impure --raw
+nix-eval = $(dummy-env) $(DESTDIR)$(bindir)/nix eval --experimental-features nix-command -I nix/corepkgs=corepkgs --store dummy:// --impure --raw

 $(d)/%.1: $(d)/src/command-ref/%.md
        @printf "Title: %s\n\n" "$$(basename $@ .1)" > $^.tmp
@@ -44,7 +44,7 @@ $(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/command-ref/new-cli
        $(trace-gen) cat doc/manual/src/SUMMARY.md.in | while IFS= read line; do if [[ $$line = @manpages@ ]]; then cat doc/manual/src/command-ref/new-cli/SUMMARY.md; else echo "$$line"; fi; done > [email protected]
        @mv [email protected] $@

But it's not enough as nix relies on it's own local libraries:

$ dev>LANG=C make install DESTDIR=$PWD/__sandbox__
  INST   /home/slyfox/dev/git/nix/__sandbox__/usr/libexec/nix/build-remote
  GEN    doc/manual/nix.json
/home/slyfox/dev/git/nix/__sandbox__/usr/bin/nix: error while loading shared libraries: libnixexpr.so: cannot open shared object file: No such file or directory
make: *** [doc/manual/local.mk:57: doc/manual/nix.json] Error 127

What would be the best way to fix it?

[dmacvicar]

Note that there is something weird about DESTDIR being required in the make stage (before make install). I stumbled into this bug when upgrading my openSUSE package to 2.5.1.

I first got the error:

[   19s] install -d "/usr/lib64/nix/"
[   19s] install: cannot change permissions of '/usr/lib64/nix/': No such file or directory
[   19s] make: *** [mk/lib.mk:117: /usr/lib64/nix/] Error 1

when doing:

%make_build profiledir="%{_sysconfdir}/profile.d"

Then I added DESTDIR to the make stage and ended here.

IMHO just building should not end trying to chmod files in the destination directory.

[thkoch2001]

Debian Maintainer of package nix here probably with the same problem while packaging version 2.5.1:

   dh_auto_build
        make -j4
make[1]: Entering directory '/<<PKGBUILDDIR>>'
rm -f Makefile.config && ./config.status --quiet --file=Makefile.config
bison -v -o src/libexpr/parser-tab.cc src/libexpr/parser.y -d
bison -v -o src/libexpr/parser-tab.cc src/libexpr/parser.y -d
flex --outfile src/libexpr/lexer-tab.cc --header-file=src/libexpr/lexer-tab.hh src/libexpr/lexer.l
flex --outfile src/libexpr/lexer-tab.cc --header-file=src/libexpr/lexer-tab.hh src/libexpr/lexer.l
src/libexpr/lexer.l:296: warning, -s option given but default rule can be matched
src/libexpr/lexer.l:296: warning, -s option given but default rule can be matched
cat src/nix-channel/unpack-channel.nix >> src/nix-channel/unpack-channel.nix.gen.hh.tmp
<SNIP>                                       
cat src/libexpr/fetchurl.nix >> src/libexpr/fetchurl.nix.gen.hh.tmp
g++ -o src/libutil/affinity.o -c src/libutil/affinity.cc <SNIP>
<SNIP>
g++ -o src/libutil/xml-writer.o -c src/libutil/xml-writer.cc <SNIP>            
install -d "/usr/lib/nix/"
install: cannot change permissions of ‘/usr/lib/nix/’: No such file or directory
make[1]: *** [mk/lib.mk:117: /usr/lib/nix/] Error 1

Or is this a different issue?

[trofi]

Proposed #6005 to restore basic DESTDIR= support. Should not degrade base case.

[edolstra]

I would actually prefer to get rid of DESTDIR, which is a bit of a 1990s hack to deal with the problem of installing into a prefix for which we don't have write access, that comes at the cost of infecting the entire build system with $(DESTDIR) references. Nowadays there are better sandboxing mechanisms, e.g. using a mount namespace.

It's also worth noting that for Nix, you really want to run make installcheck, which doesn't work with DESTDIR at all.

[trofi]

I would actually prefer to get rid of DESTDIR, which is a bit of a 1990s hack to deal with the problem of installing into a prefix for which we don't have write access, that comes at the cost of infecting the entire build system with $(DESTDIR) references. Nowadays there are better sandboxing mechanisms, e.g. using a mount namespace.

It's also worth noting that for Nix, you really want to run make installcheck, which doesn't work with DESTDIR at all.

From what I understand DESTDIR is The standard to package software in many linux and non-linux distributions. At least I'm not aware of a supported direct alternative for it.

Do you know of any FHS distribution that uses mount namespaces to handle software installs? I'm curious to look at the way they deal with all the portability complications and other corner cases. I don't think mount namespaces work as is for such a layout: one still needs to allow make install to write to (say) /usr/lib while disallowing write to any file outside /usr/lib. I'd expect such a setup to at least require overlayfs (and then sort out the collisions).

Sounds a lot less portable and more nuanced than adding $DESTDIR as a prefix when copying a few files as a final step.

In my opinion proliferation of DESTDIR (as opposed to having it in 1-2 places of `$(INSTALL) call) in case of nix's build system is the result of a few unusual design decisions:

1. nix runs tools from install $prefix. Other software normally runs binaries from inplace location they were just built (gcc, ghc, re2c) or don't run it at all. As a result of mixing $prefix in nix blurs make and make install phases (and as a bonus also complicates linking when $prefix already contains previous nix version).
2. nix maintains it's own GNU-make based build system and does not handle cases like DESTDIR gracefully. Most other build systems (automake, meson, cmake, Cabal, cargo, ) already maintain LD_LIBRARY_PATH or RPATH hacks to make inplace execution reasonable and installation to $DESTDIR prefix to just work for a typical config.

Requiring custom sandboxing mechanism just to make nix installed from source sounds like a very high bar to package nix.

Is it expected to be more complicated than an average daemon? Then perhaps nix manual should explicitly guide on how distribution packagers are expected to provide nix packages when building it from source (say, by providing an example for concrete distribution like Fedora or Debian).

[dmacvicar]

@edolstra I think getting rid of DESTDIR will make hard to package nix. Given that nix itself introduces daemons and systemd services, it is expected that nix itself is installed as a package.

https://build.opensuse.org/package/show/home:dmacvicar/nix

How are distributions expected to build nix?

[dmacvicar]

I managed to build 2.7.0 on openSUSE using @trofi 's patch from #6005.

[trofi]

d536c57 fixed all DESTDIR= problems for me.