MPICH with OpenPBS

[yashkurkure]

Hello!

I am trying to run MPI jobs using OpenPBS 23.06.06 on Ubuntu 20.04.

Note that my mpiexec works as a standalone without pbs.

The job script:

#!/bin/bash
#PBS -q workq
#PBS -l walltime=00:10:00
#PBS -l nodes=2:ppn=2
#PBS -o hello_world.out
#PBS -e hello_world.err

# Change to the working directory
cd $PBS_O_WORKDIR

# Make sure mpiexec is in PATH
source /etc/profile && which mpiexec

mpiexec ./hello_world

The output of the job:

Host key verification failed.
[[email protected]] ui_cmd_cb (/local/mpich-4.2.0rc2/src/pm/hydra/mpiexec/pmiserv_pmci.c:51): Launch proxy failed.
[[email protected]] HYDT_dmxu_poll_wait_for_event (/local/mpich-4.2.0rc2/src/pm/hydra/lib/tools/demux/demux_poll.c:76): callback returned error status
[[email protected]] HYD_pmci_wait_for_completion (/local/mpich-4.2.0rc2/src/pm/hydra/mpiexec/pmiserv_pmci.c:173): error waiting for event
[[email protected]] main (/local/mpich-4.2.0rc2/src/pm/hydra/mpiexec/mpiexec.c:260): process manager error waiting for completion

I came across these environment variables in the PBS BigBok:

For Intel MPI Hydra set:

I_MPI_HYDRA_BOOTSTRAP=rsh
I_MPI_HYDRA_BOOTSTRAP_EXEC=/opt/pbs/bin/pbs_tmrsh 

For MPICH:

Job submitters use this command by setting the appropriate environment variable to pbs_tmrsh.

For example, to inte-grate MPICH, set P4_RSHCOMMAND to pbs_tmrsh.

Thanks!

[yashkurkure]

On all nodes do:
echo " StrictHostKeyChecking no" >> /etc/ssh/ssh_config