Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor/petsc openmpi fix #1064

Merged
merged 19 commits into from
Sep 1, 2023
Merged

Refactor/petsc openmpi fix #1064

merged 19 commits into from
Sep 1, 2023

Conversation

shuds13
Copy link
Member

@shuds13 shuds13 commented Aug 21, 2023

The catch-22 is that when running with mpi4py you must have PETSc import globally, and when running with openMPI (and using the executor) it must NOT import globally, making a one size fits all solution non-trivial.

The default imports are removed, and rc options are retained. The imports are also added to the functions. If already globally imported (by the rc option), these will not do anything.

Note: The reason OpenMPI must not import globally (when using executor), is that PETSc imports mpi4py, and if that is done globally, it creates a nested comms, which openMPI does not support.

Note2: As the imports are in the functions, an alternative to the rc line for global import is to just import the package in the user calling script.

Remove default imports.
Add imports to functions.
rc option still supported.
This is used to test when OpenMPI is used with the executor.
PETSc must not be imported globally, or nested MPI
@shuds13 shuds13 self-assigned this Aug 21, 2023
@shuds13
Copy link
Member Author

shuds13 commented Aug 24, 2023

Unit tests for openmpi are failing because MPI being imported - anywhere in unit test dir - does not matter which test.

  • It is imported in test_executor.py but as mpi4py.rc.initialize = False - it seems to be okay (this test file works by itself).
  • It is in test_manager_main - but I dont think needs to be.
  • It also seems to get imported in test_ensemble.py - though not explicitly in that file

@shuds13
Copy link
Member Author

shuds13 commented Aug 24, 2023

I dont like that automatic tools dictate lines that go together have been separated. I would prefer the "petsc" line was right under the gen_funcs line, as it originally was.

import libensemble.gen_funcs

# Import libEnsemble items for this test
from libensemble.libE import libE
from libensemble.sim_funcs.six_hump_camel import six_hump_camel as sim_f

libensemble.gen_funcs.rc.aposmm_optimizers = "petsc"

@shuds13 shuds13 mentioned this pull request Aug 31, 2023
34 tasks
@shuds13 shuds13 merged commit ddfddf4 into develop Sep 1, 2023
12 of 14 checks passed
@jmlarson1 jmlarson1 deleted the refactor/petsc_openmpi_fix branch September 21, 2023 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants