use BatchJobs to submit Rmpi job

[ghost]

Hi Team, we are trying to leverate BatchJobs to submit mpi job to job scheduler. After a bit investigation/enhancement, we tested below approach on OpenLava job scheduler. we want to get some inputs from the team.

1. R script that submit a Rmpi job using BatchJobs:
   library(BatchJobs)
   conf = BatchJobs:::getBatchJobsConf()
   conf$cluster.functions = makeClusterFunctionsOpenLava("/home/clusteradmin/rmpi.tmpl")
   reg = makeRegistry(id = "RMPIJobsExample")
   f = function(x) {
   library("Rmpi")
   …
   mpi.spawn.Rslaves(nslaves=slaveno)
   …
   mpi.remote.exec(paste("I am",mpi.comm.rank(),"of",mpi.comm.size()))
   …
   }
   batchMap(reg, f, 1) specify 1 Rmpi job
   submitJobs(reg, np=2) specify number of processors to use
   showStatus(reg)
   waitForJobs(reg)
   showStatus(reg)
2. the job template: rmpi.tmpl
   /opt/openlava/bin/openmpi-mpirun -np 1 R CMD BATCH "<%= rscript %>"
3. we did a bit code change like below
   3.1 R/submitJobs.R
   added one more parameter "np" to submitJobs() method. and pass the parameter when call submitJob.

67c67

< job.delay = FALSE, progressbar = TRUE, np = 1L) {

job.delay = FALSE, progressbar = TRUE) {
218d217
< np = np,

3.2 R/clusterFunctionsOpenLava.R
add one more parameter to submitJob(). if it is more than 1, give args "-n np" to bsub when call runOSCommandLinux.

33c33

< submitJob = function(conf, reg, job.name, rscript, log.file, job.dir, resources, arrayjobs, np = 1L) {

submitJob = function(conf, reg, job.name, rscript, log.file, job.dir, resources, arrayjobs) {
36,42c36
< if (np > 1L) {
< args <- paste(c("-n ", np), collapse="")
< res = runOSCommandLinux("bsub", args = args, stdin = outfile, stop.on.exit.code = FALSE)
< } else {
< res = runOSCommandLinux("bsub", stdin = outfile, stop.on.exit.code = FALSE)
< }

<

res = runOSCommandLinux("bsub", stdin = outfile, stop.on.exit.code = FALSE)

clusteradmin@ip-172-31-24-74:~$

What we want to get your input are if this makes sense and if this change would break existing features?

thank you!
Feng Li
Teraproc