Bash not found when running nextflow project in NixOs

[apraga]

Describe the bug

Nextflow expects /bin/bash to exists when running projects. This is fixed by
ln -s /run/current-system/sw/bin/bash /bin/bash as mentionned in nextflow-io/nextflow#1598.

Should it be part of the nix package ?

Steps To Reproduce

Steps to reproduce the behavior:

1. Put the following in main.nf

 process BWA {
     output:
        stdout 
     """
     echo "test"
     """
 }

 workflow {
=    BWA | view
 }

2. nextflow run main.f

Expected behavior

A normal execution.

Screenshots

Additional context

Error message

Caused by:
  Cannot run program "/bin/bash" (in directory "/home/alex/code/bisonex/work/85/f38efc28549cd640297afdeb3f5704"): error=2, No such file or directory

Command executed:

  echo "lol"

Command exit status:
   -

Notify maintainers

@Etjean

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.16.3-microsoft-standard-WSL2, NixOS, 22.11 (Raccoon), 22.11pre401467.8ea014acc33`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.10.3`
 - channels(root): `"home-manager, nixos-22.11pre401467.8ea014acc33"`
 - channels(nixos): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[milahu]

your nf script is wrong. this works:

// main.nf
nextflow.enable.dsl=2
process sayHello {
  output:
    stdout
  """
  echo hello
  """
}
workflow {
  sayHello | view
}

nextflow run main.nf 
N E X T F L O W  ~  version 21.10.6
Launching `main.nf` [evil_hoover] - revision: 403516deb9
executor >  local (1)
[ff/1c3dfa] process > sayHello [100%] 1 of 1, failed: 1 ✘
Error executing process > 'sayHello'

Caused by:
  Cannot run program "/bin/bash" (in directory "/tmp/tmp.ufhuIdnYwB/work/ff/1c3dfa9da3c492cc2bad8470e7caaf"): error=2, No such file or directory

Command executed:

  echo hello

/bin/bash is called from
nextflow/modules/nextflow/src/main/resources/nextflow/executor/command-run.txt
and others ...
but fixing that is out-of-scope per nextflow-io/nextflow#1598 (comment)

we could rewrite syscalls from

[pid 1851702] execve("/bin/bash", ["/bin/bash", "-ue", ".command.run"], 0x7ffd6edc7208 /* 159 vars */ <unfinished ...>

to our version of bash

[IllustratedMan-code]

I went ahead and build a FHS User environment using this flake:

{
    inputs = {};
    outputs = {self, nixpkgs, ...}:
    let
        system = "x86_64-linux";
        pkgs = import nixpkgs{inherit system;};
    in
    {
        devShells.${system}.default = with pkgs; (pkgs.buildFHSUserEnv {
            name = "nextflow";
            targetPkgs = pkgs: [pkgs.openjdk];
        }).env;
    };

}

run nix develop and you can just run nextflow like normal.

[apraga]

Thanks for the tip. Is there any way to run nextflow on NixOs outside a shell though ?

[IllustratedMan-code]

You could actually package the nextflow executable in a FHS environment and specify the runScript parameter as "nextflow". I might hack something together tomorrow.

[IllustratedMan-code]

I'll probably get around to making a pull request to nixpkgs at some point, but in the meantime here is a flake that you can use.

https://github.com/IllustratedMan-code/nextflowFlake

[IllustratedMan-code]

It probably makes more sense to do some kind of substitution, but buildFHSUserEnv does seem to work.

[edmundmiller]

Just cross-posting this nextflow-io/nextflow#1598 (comment)

I've had zero issues after doing that for anyone who just wants it to run.

[IllustratedMan-code]

Well, looks like I've done this work for nothing. It was already packaged!: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/interpreters/nextflow/default.nix

@apraga you should close this issue.

[apraga]

@IllustratedMan-code Unless I'm mistaken, the current version does package bash as coreutils inside the $PATH but nextflow really wants /bin/bash as a hard-code path.

@emiller88 Your solution worked for me but I wanted to discuss a more flexible solution if possible.

[IllustratedMan-code]

@apraga you're right, I am able to reproduce the issue with the version in nixpkgs.

[IllustratedMan-code]

I'll try to clean up my derivation and submit to nixpkgs.

[IllustratedMan-code]

@apraga, just letting you know I made a pull request with the buildFHSUserEnv solution.

[apraga]

@IllustratedMan-code Thanks for your work ! I no longer have access to a NixOs environment for the moment, but it may help other people :)