Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix logic w.r.t. enabling Python support in PETSc #2299

Merged
merged 3 commits into from
Jan 5, 2021

Conversation

boegel
Copy link
Member

@boegel boegel commented Jan 4, 2021

@boegel
Copy link
Member Author

boegel commented Jan 4, 2021

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS PETSc-3.12.4-intel-2019b-Python-3.7.4.eb
  • SUCCESS PETSc-3.11.1-foss-2019a-Python-3.7.2.eb
  • SUCCESS PETSc-3.9.3-foss-2018a.eb

Build succeeded for 3 out of 3 (3 easyconfigs in total)
node3104.skitty.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/112494f4f0edb8a896c2e3fd7b7d6612 for a full test report.

@boegel boegel requested a review from branfosj January 4, 2021 15:51
branfosj
branfosj previously approved these changes Jan 4, 2021
Copy link
Member

@branfosj branfosj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@branfosj branfosj dismissed their stale review January 4, 2021 17:12

found a potential problem

Copy link
Member

@branfosj branfosj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has not fully solved the issue due to a bug. My test has:

./configure --prefix=/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/PETSc/3.11.0-foss-2018b-downloaded-deps --with-hdf5=1 --with-hdf5-dir=$EBROOTHDF5 --with-debugging --with-cxx-dialect=C++11 --download-hypre=1  --download-ptscotch=1  --download-mumps=1  --download-superlu_dist=1  --download-parmetis=1  --download-triangle=1  --download-suitesparse=1  --download-prometheus=1  --download-metis=1  --download-ml=1  --download-superlu=1  --download-hypre-shared=0  --download-metis-shared=0  --download-ml-shared=0  --download-mumps-shared=0  --download-parmetis-shared=0  --download-prometheus-shared=0  --download-ptscotch-shared=0  --download-suitesparse-shared=0  --download-superlu-shared=0  --download-superlu_dist-shared=0  --with-cc="mpicc"  --with-cxx="mpicxx" --with-c++-support  --with-fc="mpifort"  --CFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fPIC"  --CXXFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fPIC -DOMPI_SKIP_MPICXX -DMPICH_SKIP_MPICXX"  --FFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fPIC"  --with-mpi=1  --with-build-step-np=8  --with-shared-libraries=1  --with-debugging=0  --with-pic=1  --with-x=0 --with-windows-graphics=0  --with-numpy=1  --with-mpi4py=1  --with-fftw=1  --with-fftw-include=/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/FFTW/3.3.8-gompi-2018b/include  --with-fftw-lib=[/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/FFTW/3.3.8-gompi-2018b/lib/libfftw3_mpi.a,libfftw3.a]  --with-scalapack=1  --with-scalapack-include=/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1/include  --with-scalapack-lib=[/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/ScaLAPACK/2.0.2-gompi-2018b-OpenBLAS-0.3.1/lib/libscalapack.a,libopenblas.a,libgfortran.a]  --with-blas-lapack-lib=[/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/OpenBLAS/0.3.1-GCC-7.3.0-2.30/lib/libopenblas.a,libgfortran.a]  --with-flex=1 --with-flex-dir=/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/flex/2.6.4-GCCcore-7.3.0  --with-bison=1 --with-bison-dir=/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/Bison/3.0.5-GCCcore-7.3.0  --with-python=1 --with-python-dir=/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/Python/2.7.15-foss-2018b  --with-hdf5=1 --with-hdf5-dir=/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/HDF5/1.10.2-foss-2018b  --with-valgrind=1 --with-valgrind-dir=/rds/bear-apps/devel/eb-sjb-up/EL7/EL7-cascadelake/software/Valgrind/3.14.0-foss-2018b

The important parts of that are: --with-flex=1, --with-bison=1, and --with-python=1. I.e. all the build deps are sent through to PETSc.

That is because the self.cfg.builddependencies()

depfilter = self.cfg.builddependencies() + sep_deps
returns more than just the names - depfilter is:

[{'full_mod_name': 'CMake/3.12.1-GCCcore-7.3.0', 'short_mod_name': 'CMake/3.12.1-GCCcore-7.3.0', 'name': 'CMake', 'version': '3.12.1', 'versionsuffix': '', 'toolchain': {'name': 'GCCcore', 'version': '7.3.0'}, 'toolchain_inherited': False, 'system': False, 'hidden': False, 'build_only': True, 'external_module': False, 'external_module_metadata': {}}, {'full_mod_name': 'flex/2.6.4-GCCcore-7.3.0', 'short_mod_name': 'flex/2.6.4-GCCcore-7.3.0', 'name': 'flex', 'version': '2.6.4', 'versionsuffix': '', 'toolchain': {'name': 'GCCcore', 'version': '7.3.0'}, 'toolchain_inherited': False, 'system': False, 'hidden': False, 'build_only': True, 'external_module': False, 'external_module_metadata': {}}, {'full_mod_name': 'Bison/3.0.5-GCCcore-7.3.0', 'short_mod_name': 'Bison/3.0.5-GCCcore-7.3.0', 'name': 'Bison', 'version': '3.0.5', 'versionsuffix': '', 'toolchain': {'name': 'GCCcore', 'version': '7.3.0'}, 'toolchain_inherited': False, 'system': False, 'hidden': False, 'build_only': True, 'external_module': False, 'external_module_metadata': {}}, {'full_mod_name': 'Python/2.7.15-foss-2018b', 'short_mod_name': 'Python/2.7.15-foss-2018b', 'name': 'Python', 'version': '2.7.15', 'versionsuffix': '', 'toolchain': {'name': 'foss', 'version': '2018b'}, 'toolchain_inherited': False, 'system': False, 'hidden': False, 'build_only': True, 'external_module': False, 'external_module_metadata': {}}, 'BLACS', 'BLAS', 'CMake', 'FFTW', 'LAPACK', 'numpy', 'mpi4py', 'papi', 'ScaLAPACK', 'SciPy-bundle', 'SuiteSparse']

@branfosj
Copy link
Member

branfosj commented Jan 4, 2021

The same logic is present in SLEPc and Trillinos easyblocks. I've PR-ed a fix in #2300 and #2301

@boegel
Copy link
Member Author

boegel commented Jan 4, 2021

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS PETSc-3.12.4-intel-2019b-Python-3.7.4.eb
  • SUCCESS PETSc-3.11.1-foss-2019a-Python-3.7.2.eb
  • SUCCESS PETSc-3.9.3-foss-2018a.eb

Build succeeded for 3 out of 3 (3 easyconfigs in total)
node3104.skitty.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/dc12236fe3ed51e573e7a99c584f656e for a full test report.

Copy link
Member

@branfosj branfosj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@branfosj
Copy link
Member

branfosj commented Jan 5, 2021

Test report by @branfosj

Overview of tested easyconfigs (in order)

  • SUCCESS PETSc-3.11.1-foss-2019a-Python-3.7.2.eb
  • SUCCESS PETSc-3.11.1-intel-2019a-Python-3.7.2.eb
  • SUCCESS PETSc-3.12.4-foss-2019b-Python-3.7.4.eb
  • SUCCESS PETSc-3.12.4-foss-2020a-Python-3.8.2.eb
  • SUCCESS PETSc-3.12.4-intel-2019b-Python-2.7.16.eb
  • SUCCESS PETSc-3.12.4-intel-2019b-Python-3.7.4.eb
  • SUCCESS PETSc-3.12.4-intel-2020a-Python-3.8.2.eb

Build succeeded for 7 out of 7 (7 easyconfigs in total)
bear-pg0211u03a.bear.cluster - Linux centos linux 7.8.2003, x86_64, Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (cascadelake), Python 3.6.8
See https://gist.github.com/960901b36b1419da2b44b166384a22ba for a full test report.

@branfosj
Copy link
Member

branfosj commented Jan 5, 2021

Going in, thanks @boegel!

@branfosj branfosj merged commit c4f3765 into easybuilders:develop Jan 5, 2021
@boegel boegel deleted the petsc_fixes branch January 5, 2021 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants