Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider moving R to GCC level, splitting MPI extensions into a separate bundle #15195

Closed
jfgrimm opened this issue Mar 28, 2022 · 4 comments
Closed

Comments

@jfgrimm
Copy link
Member

jfgrimm commented Mar 28, 2022

Unlike Python, Perl, etc., R sits at foss level due to requiring BLAS/LAPACK and MPI for various extensions. This pulls any easyconfigs depending on R all the way to foss, even if (and this often seems to be the case) they are not MPI-aware.
I would propose moving R itself to GCC (for flexiblas), and splitting any extensions that need to be at foss level into a separate bundle, similar to Python/SciPy-bundle

@branfosj
Copy link
Member

branfosj commented Mar 28, 2022

MPI is definitely needed for Rmpi, but I am not sure much else beyond that and there are not a lot of R packages that depend on Rmpi (as far as I know).

No package in R-4.1.2-foss-2021b.eb or R-bundle-Bioconductor-3.14-foss-2021b-R-4.1.2.eb requires Rmpi.

@jfgrimm
Copy link
Member Author

jfgrimm commented Apr 29, 2022

had a look at packages depending on MPI (directly or indirectly) in the current R 4.1.2 easyconfig:

R package dependency library linked
Rssa FFTW FFTW/3.3.10-gompi-2021b/lib/libfftw3.so.3
hdf5r HDF5 HDF5/1.12.1-gompi-2021b/lib/libhdf5_hl.so.200
Rmpi OpenMPI OpenMPI/4.1.1-GCC-11.2.0/lib/libmpi.so.40
sf netCDF netCDF/4.8.1-gompi-2021b/lib/libnetcdf.so.19
terra netCDF netCDF/4.8.1-gompi-2021b/lib/libnetcdf.so.19
fftw FFTW FFTW/3.3.10-gompi-2021b/lib/libfftw3.so.3
fftwtools FFTW FFTW/3.3.10-gompi-2021b/lib/libfftw3.so.3
imager FFTW FFTW/3.3.10-gompi-2021b/lib/libfftw3.so.3
rgdal netCDF netCDF/4.8.1-gompi-2021b/lib/libnetcdf.so.19
ncdf4 netCDF netCDF/4.8.1-gompi-2021b/lib/libnetcdf.so.19

@bartoldeman
Copy link
Contributor

At Compute Canada we compile R with the gcccoreflexiblas toolchain:
https://github.com/ComputeCanada/easybuild-computecanada-config/blob/main/toolchains/gcccoreflexiblas.py
but this is a bit tricky as BLIS and OpenBLAS are at the GCC level (they have to be since Fortran interfaces differ between GCC and Intel), and having BLIS at both GCCcore and Intel can be confusing with an HMNS.

We also have
https://github.com/ComputeCanada/easybuild-computecanada-config/blob/main/toolchains/gccflexiblas.py

But it may be ok to simply use flexiblas as a dependency instead, and FFTW-serial.
Would need to look up if R can use HDF5/netCDF's MPI interfaces or not, if not it could use the -serial versions of those too.

@Micket
Copy link
Contributor

Micket commented Mar 22, 2024

R now sits at gfbf level, which is as low as it can get in our toolchains, becuase R itself depends on BLAS/LAPACK (it cant ever be moved to compiler level).

@Micket Micket closed this as completed Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants