chroot environments have no /usr/bin/env?

[peti]

This may be a stupid question, but I just realized that Nix chroot environments (as configured by NixOS) have no /usr/bin/env path. This comes as a surprise to me. Has this always been this way? How are scripts supposed to specify a portable shebang to, say bash, without having a reliable path to env?

[lucabrunox]

It's always been that way. Use patchShebangs.

On Sun, Feb 8, 2015 at 1:50 PM, Peter Simons [email protected]
wrote:

This may be a stupid question, but I just realized that Nix chroot
environments (as configured by NixOS) have no /usr/bin/env path. This
comes as a surprise to me. Has this always been this way? How are scripts
supposed to specify a portable shebang to, say bash, without having a
reliable path to env?

—
Reply to this email directly or view it on GitHub
#6227.

NixOS Linux http://nixos.org

[peti]

IMHO it's silly that a perfectly portable shell script needs patching to run on NixOS. Especially considering the fact that the NixOS system does provide /usr/bin/env. The fact that build environments don't have this path is counter-intuitive to me.

[lucabrunox]

/usr/bin/env in nixos is solely for the user daily convenience, not for
being used in builds. Considering we want to also remove /bin/sh, I think
using /usr/bin/env back in builds is a step backwards.

On Sun, Feb 8, 2015 at 1:55 PM, Peter Simons [email protected]
wrote:

IMHO it's silly that a perfectly portable shell script needs patching to
run on NixOS. Especially considering the fact that the NixOS system does
provide /usr/bin/env. The fact that build environments don't have this
path is counter-intuitive to me.

—
Reply to this email directly or view it on GitHub
#6227 (comment).

NixOS Linux http://nixos.org

[peti]

Providing /bin/sh and /usr/bin/env in NixOS, but only /bin/sh in build environments is inconsistent and is bound to confuse users. It certainly confuses me! A consistent state can be reached either by (a) adding /usr/bin/env to the build environment or by (b) removing /bin/sh from the build environment, but the current state of affairs is counter-intuitive and inconsistent.

[lucabrunox]

It's not inconsistent, it's counter-intuitive maybe. Tell people to enable chroot builds and that's it. For me this issue can be closed, unless you want better docs in the nix manual.

[wmertens]

@peti patchShebangs works fast and fine and removes all doubt about what a script runs while debugging builds.

[peti]

It's not inconsistent.

Do you have any rational reasoning for that position, or is this argument valid simply by means of stamping your foot on the ground real hard?

[lucabrunox]

@peti perhaps you have a reasoning of why it's inconsistent, or it's just because you were surprised by this behavior and maybe need some time thinking.

[peti]

@wmertens, I feel it's important that well-written portable scripts can run in a Nix build environment without the need for patching. The path /bin/sh makes this possible for Bourne shell scripts. But how do you run an interpreter other than sh? The universally accepted way to accomplish that is /usr/bin/env bash, for example, and this is considered a well-written portable script, IMHO. So it should run in a Nix build environment without patching!

Another problem is that people use the /usr/bin/env interpreter approach in scripts, too, that are generated programatically to be run later, and patchShebang cannnot fix those cases because the time the hook runs the script doesn't exist yet. In fact, I just have build failures in a Haskell library which did precisely that: it wrote a script and ran it (using the /usr/bin/env trampoline). The only way to make that program work is to patch the Haskell source code. That sucks, IMHO. I don't see what advantages the absence of /usr/bin/env provides us with -- but I see the disadvantages, namely the fact that I as a maintainer have to put extra work into making code work that otherwise functions just fine on NixOS.

[wmertens]

@peti I'm not 100% sure I follow, you want to run scripts from your homedir
in a build environment?

Could you post a simple step-by-step illustrating why not having env is a
problem?

On Sun Feb 08 2015 at 2:21:38 PM Peter Simons [email protected]
wrote:

@wmertens https://github.com/wmertens, I feel it's important that
well-written portable scripts can run in a Nix build environment without
the need for patching. The path /bin/sh makes this possible for Bourne
shell scripts. But how do you run an interpreter other than sh? The
universally accepted way to accomplish that is /usr/bin/env bash, for
example, and this is considered a well-written portable script, IMHO. So it
should run in a Nix build environment without patching!

Another problem is that people use the /usr/bin/env interpreter approach
in scripts, too, that are generated programatically to be run later, and
patchShebang cannnot fix those cases because the time the hook runs the
script doesn't exist yet. In fact, I just have build failures in a Haskell
library which did precisely that: it wrote a script and ran it (using the
/usr/bin/env trampoline). The only way to make that program work is to
patch the Haskell source code. That sucks, IMHO. I don't see what
advantages the absence of /usr/bin/env provides us with -- but I see the
disadvantages, namely the fact that I as a maintainer have to put extra
work into making code work that otherwise functions just fine on NixOS.

—
Reply to this email directly or view it on GitHub
#6227 (comment).

[peti]

let
  pkgs = import <nixpkgs> {};
in
pkgs.stdenv.mkDerivation {
  name = "foo-0";
  unpackPhase = ":";
  doCheck = true;
  checkPhase = ''
    # Run some programs that compute lots of stuff and output a test
    # script as a result.
    echo >$TMPDIR/run-tests  '#! /usr/bin/env bash'
    echo >>$TMPDIR/run-tests 'exit 0'
    chmod +x $TMPDIR/run-tests
    $TMPDIR/run-tests
  '';
  installPhase = "install -D /dev/null $out/ok:";
}

[peti]

utdemir/handsy#5 is a concrete example that illustrates this kind of failure. The code assumes that /usr/bin/env exists and tries to use it for portable program execution. patchShebang cannot fix that, because those assumptions are hidden somewhere in the Haskell code.

[wmertens]

Ok, so in these particular cases, ${stdenv.shell} should have been used
instead of /usr/bin/env bash in the creation of the script and it would
have referred to the correct bash from the start. There's also writeScript
which helps making the script (
https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/trivial-builders.nix#L14-L38
)

In fact you don't even need to give the shebang for bash scripts, if
there's no shebang and it's executable, bash will run it as a bash script.
There are many writeScript invocations in nixpkgs that depend on this.

The only thing I can think of where it would be nice to have env is when
running scripts from your homedir while debugging a build, and even you
could manually do "perl myscript" or whatever. I'm not convinced, I'm in
fact happy that the lack of env forces better dependency writing.

After all, when the build completes and there are still calls to env
somehow, they will not be counted in the hash-based dependency discovery.

So I'm still not convinced.

On Sun Feb 08 2015 at 2:49:31 PM Peter Simons [email protected]
wrote:

utdemir/handsy#5 utdemir/handsy#5 is a
concrete example that illustrates this kind of failure. The code assumes
that /usr/bin/env exists and tries to use if for portable program
execution. patchShebang cannot fix it, though, because those assumptions
are hidden somewhere in the Haskell code.

—
Reply to this email directly or view it on GitHub
#6227 (comment).

[bjornfor]

@wmertens: I think that was just an example. If the code is written in nix to begin with, I agree, use ${stdenv.shell}. But what if the code came from an upstream project?

patchShebangs doesn't automatically solve all problems. It cannot handle scripts that are generated and run in the middle of a build.

If an /usr/bin/env script is placed in $out, fixupPhase will rewrite the shebang. So I don't think having /usr/bin/env shebangs in builders changes the hash-based dependency in any way.

[wmertens]

So basically we're trying to fix a hypothetical upstream project that uses
env in scripts that are generated on the fly?

I'm not convinced... I'd rather fix that problem if and when it appears.

I think it would be much nicer to remove /bin/sh as well...

On Sun, Feb 8, 2015, 5:10 PM Bjørn Forsman [email protected] wrote:

@wmertens https://github.com/wmertens: I think that was just an
example. If the code is written in nix to begin with, I agree, use
${stdenv.shell}. But what if the code came from an upstream project?

patchShebangs doesn't automatically solve all problems. It cannot handle
scripts that are generated and run in the middle of a build.

If an /usr/bin/env script is placed in $out, fixupPhase will rewrite the
shebang. So I don't think having /usr/bin/env shebangs in builders changes
the hash-based dependency in any way.

—
Reply to this email directly or view it on GitHub
#6227 (comment).

[peti]

@wmertens, what makes you say that this is issue is hypothetical?

[wmertens]

Well point me to an actual package with this problem then?

On Sun Feb 08 2015 at 6:39:10 PM Peter Simons [email protected]
wrote:

@wmertens https://github.com/wmertens, what makes you say that this is
issue is hypothetical?

—
Reply to this email directly or view it on GitHub
#6227 (comment).

[peti]

@wmertens, #6227 (comment) maybe?

[copumpkin]

@peti would you be more okay with this if we were getting rid of /bin/sh too? i.e., is it just the inconsistency that bugs you, or is it also the inconvenience of a deeper patching of the builds?

I ask because I too am striving to get rid of /bin/sh in the new pure darwin stdenv (with sandboxed builds), and am also emulating nix-on-linux by not allowing access to /usr/bin/env.

[peti]

@copumpkin, I don't know how I'd prefer to remedy this issue, I haven't formed an opinion yet. So far, the options we are aware of are:

1. Add /usr/bin/env into the chroot environment.

   Pro: Some builds that currently fail or require patching would succeed without patching.
   Con: Builds that succeed unmodified only when /usr/bin/env is available might generate a broken derivation, so it would have been better to have them fail in the first place.

2. Remove /bin/sh from the chroot environment.

   Pro: I can't think of anything.
   Con: Some builds that currently succeed and generate working derivations would then require patching.

Are there any alternative suggestions? Further pros or cons that I missed?

[7c6f434c]

1. Remove /bin/sh from the chroot environment.

   Pro: I can't think of anything.

Pro: weird LD_LIBRARY_PATH-related failures on glibc updates go away.

These are chrooted Nix-on-NixOS build failures.

Con: Some builds that currently succeed and generate working derivations would then require patching.

Are there any alternative suggestions? Further pros or cons that I missed?

[peti]

@7c6f434c, I'm not aware of any LD_LIBRARY_PATH-related failures. When do these occur?

[7c6f434c]

@7c6f434c, I'm not aware of any LD_LIBRARY_PATH-related failures. When do these occur?

#1424

[wmertens]

@peti hmm I missed the fact that handsy was not in nixpkgs. So indeed it does happen, although the fix is generally simple.

Once #1424 is solved it's no longer a factor for or against this issue.

So now it's weighing the trouble of fixing builds that need sh or env vs the trouble of using packages that use /bin/sh or /usr/bin/env. Perhaps there should be a warning/error when a finished build still contains references to either after fixupPhase, and then allow both in the chroot?

[peti]

@7c6f434c, that issue is important, but is it related to the problem we're discussing here? The build failure in #1424 was caused by the fact that glibc depends on /bin/sh, and the fix for that problem is to remove that dependency from glibc. Whether /bin/sh is present in the chroot environment or not is not a factor. If glibc would find sh in $PATH -- like Eelco suggested in #1424 (comment) --, then those pypy builds would succeed just fine even if /bin/sh is present in the chroot environment. So while #1424 provides a strong incentive to take the dependency on /bin/sh out of glibc, IMHO, it does not provide a strong incentive to take /bin/sh out of chroot environments.

[peti]

@wmertens, based on what empirical data do you assert that "the fix is generally simple"?

I don't believe that generalization is valid, because the fix for handsy (utdemir/handsy@8bb651a), for example, certainly doesn't strike me as "simple".

EDIT: Also, please note that the problems we've had with handsy are completely unrelated to #1424. Even if we had a patched glibc that doesn't depend on /bin/sh, we'd stil have run into trouble with handsy.

[wmertens]

@peti Given that we're generally talking about scripts, just patching the
scripts to switch /bin/sh to the actual shell would work, as it would have
in this case.

On Mon Feb 09 2015 at 10:57:24 AM Peter Simons [email protected]
wrote:

@wmertens https://github.com/wmertens, based on what empirical data do
you assert that "the fix is generally simple"?

I don't believe that generalization is valid, because the fix for handsy (
utdemir/handsy@8bb651a
utdemir/handsy@8bb651a),
for example, certainly doesn't strike me as "easy".

—
Reply to this email directly or view it on GitHub
#6227 (comment).

[peti]

@wmertens, okay. This is probably not going anywhere. Let us agree to disagree.

[7c6f434c]

@7c6f434c, that issue is important, but is it related to the problem we're discussing here? The build failure in #1424 was caused by the fact that glibc depends on /bin/sh, and the fix for that problems is to make glibc, like, not depend on /bin/sh. Whether /bin/sh is present in the chroot environment or not is not a factor. If glibc would execute sh from $PATH instead of using a hard-coded path -- like Eelco suggested in #1424 (comment) --, then those pypy builds would succeed just fine even if /bin/sh is present in the chroot environment. So while #1424 provides a strong incentive to take the dependency on /bin/sh out of glibc, IMHO, it does not provide a strong incentive to take /bin/sh out of chroot environments.

Anything using /bin/sh while LD_LIBRARY_PATH is set would trigger that
issue on next glibc update.

[lucabrunox]

@peti don't know what that code does exactly, however it reads the contents of /bin/sh for doing a couple of tests, not even executing it. Why not replace that /bin/bin string with ${stdenv.shell}? Should be one line of code in the derivation. Not to say that your patch actually makes more sense in general.

[peti]

@7c6f434c, okay, I see your point. Any attempt to execute /bin/sh will break the build, so arguably it's better to have no /bin/sh in the first place. That's a good reason for removing both /bin/sh and /usr/bin/env from chroot environments.

[vcunat]

Having sh closure from the build machine seems a nontrivial impurity. Personally, I'd prefer to keep env instead in build-chroot-dirs (perhaps a statically-linked version to avoid the glibc problem) and use that to find (ba)sh on $PATH, typically provided by stdenv.

Of course, it's probably best to avoid any (/usr)/bin references when possible (stdenv.shell, patchShebangs).

[copumpkin]

As a data point, I've had actual failures in my pure-Darwin work as a result of it using the global shell

On Feb 9, 2015, at 06:08, Vladimír Čunát [email protected] wrote:

Having sh closure from the build machine seems a nontrivial impurity. Personally, I'd prefer to keep env instead in build-chroot-dirs (perhaps a statically-linked version to avoid the glibc problem) and use that to find (ba)sh on $PATH, typically provided by stdenv.

Of course, it's probably best to avoid any (/usr)/bin references when possible (stdenv.shell, patchShebangs).

—
Reply to this email directly or view it on GitHub.

[peti]

@copumpkin, "global shell" means "Darwin's /bin/sh" in your case, right?

[copumpkin]

Yeah, sorry

On Monday, February 9, 2015, Peter Simons [email protected] wrote:

@copumpkin https://github.com/copumpkin, "global shell" means "Darwin's
/bin/sh" in your case, right?

—
Reply to this email directly or view it on GitHub
#6227 (comment).

[peti]

Closing this issue because it seems unlikely that a consensus for action in any direction can be reached, so let's leave everything as is. It's working good enough anyway.

[vcunat]

A very related question: I want to patch some scripts to pick up the interpreter from $PATH (TeX scripts for #287). From this it seems I have to make them depend on ${coreutils}/bin/env; I'd like to avoid that. Do you know a way? I would just leave it with /usr/bin/env, but that wouldn't work for builds using TeX...

The reason is that I want to share about a gigabyte of paths among all platforms and stdenvs (by putting them into fixed-output derivations).

[wmertens]

Can't you simply let the shell pick up the path with a wrapper?

On Wed, Aug 26, 2015 at 2:36 PM Vladimír Čunát [email protected]
wrote:

A very related question: I want to patch some scripts to pick up the
interpreter from $PATH (TeX scripts for #287
#287). From this it seems I have
to make them depend on ${coreutils}/bin/env; I'd like to avoid that. Do
you know a way? I would just leave it with /usr/bin/env, but that
wouldn't work for builds using TeX...

The reason is that I want to share about a gigabyte of paths among all
platforms and stdenvs (by putting them into fixed-output derivations).

—
Reply to this email directly or view it on GitHub
#6227 (comment).

Wout.
(typed on mobile, excuse terseness)

[vcunat]

Hmm, /bin/sh instead, that might work.

[0xABAB]

@peti I am on your side with this one (not that anyone is counting).

[Atry]

I understand /bin/sh is harmful because there are different implementation of /bin/sh. But what's wrong with /usr/bin/env?

[fumieval]

I've been bitten by this issue, due to webpack CLI's /usr/bin/env node in which patchShebangs does not affect. I took a really long time to figure this out so I'd like to leave a comment at least. Here is my workaround:

    sed -i -e "s|#!/usr/bin/env node|#! ${pkgs.nodejs}/bin/node|" /build/frontend/node_modules/.bin/webpack

[bergkvist]

Workaround:

{ pkgs ? import <nixpkgs> {}
}:

pkgs.runCommand "example" {} ''
  # Ensure that /usr/bin/env exists in our build sandbox
  mkdir -m 0755 -p /usr/bin && ln -sfn "${pkgs.coreutils}/bin/env" /usr/bin/env

  # Perform whatever build steps you want here
  mkdir -p $out && cd $out
  ${./some-script-that-uses-usr-bin-env.sh}
''

This allows you to run scripts that use #!/usr/bin/env bash or similar shebangs without patching them.

This could also be turned into a hook:

{ pkgs ? import <nixpkgs> {}
}:
let
  usrBinEnvHook = pkgs.makeSetupHook { name = "usr-bin-env-hook"; } (pkgs.writeText "create-usr-bin-env.sh" ''
    mkdir -m 0755 -p /usr/bin
    ln -sfn "${pkgs.coreutils}/bin/env" /usr/bin/env
  '');
in
pkgs.runCommand "example" { buildInputs = [ usrBinEnvHook ]; } ''
  mkdir -p $out && cd $out
  ${./some-script-that-uses-usr-bin-env.sh}
''

So I guess a relevant question here is whether we would want this hook to be part of nixpkgs (for example pkgs.usrBinEnvHook).

[nbraud]

FWIW I'm running into this issue as well while packaging cppyy, with scripts generated during the build assuming that /usr/bin/env exists.

pkgs.runCommand "example" {} ''
  # Ensure that /usr/bin/env exists in our build sandbox
  mkdir -m 0755 -p /usr/bin && ln -sfn "${pkgs.coreutils}/bin/env" /usr/bin/env

This does not work, but the error did not cause the drv build to fail due to the use of &&

mkdir: cannot create directory '/usr': Permission denied

PS: Quoting my reply to the (non-functional?) usrBinEnv hook's PR:

For what it's worth, I ran into a fair few cases of things which expect /usr/bin/env to be present during build, which seems fair-enough given that POSIX requires it to be available IIRC.

patchShebangs is a workable solution if the scripts are present in the source tree, though it gets tedious when there are many scripts and it's not obvious whether they are run at build- or run-time; worse, errors won't be apparent until someone cross-compiles the drv and tries to use it.

However, it gets downright painful when the scripts are generated during the build, and even more error-prone: one must patch whatever is generating the scripts (or somehow run part of the build, then patchShebangs then resume the build) and it's even more difficult to ensure the scripts are only used during the build and not shipped in the output.

All in all, that seems to me like extra tedium for packagers, and a source of subtle cross-compilation bugs.