Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make observables run in parallel #4748

Merged
merged 1 commit into from
Jul 24, 2023

Conversation

reinaual
Copy link
Contributor

@reinaual reinaual commented Jul 6, 2023

Fixes #3154

Description of changes:

  • make observables run in parallel
    • only send the extracted data via MPI (the original implementation sent the entire particle data structure)
    • remove a major performance bottleneck in parallel simulations
  • remove 8 callbacks from the deprecated MpiCallbacks framework

@reinaual reinaual force-pushed the parallelize_observables branch 4 times, most recently from fc05c0b to b17b8f7 Compare July 7, 2023 14:27
Co-authored-by: Alexander Reinauer <[email protected]>
@jngrad
Copy link
Member

jngrad commented Jul 24, 2023

Here is an updated version of the plot #3154 (comment) which uses the new implementation of the observables, using a TimeSeries of the particle positions:

Benchmarks for a Lennard-Jones simulation with two types of writer: MPI-IO, observables accumulated with a time series. For simulations with 1 or 4 cores, observables are as fast as MPI-IO. For 2 cores, MPI-IO is faster. For 8 core or more, observables are faster.

The benchmark writes to an SSD, either raw unsorted particle positions for MPI-IO every 20 time steps, or sorted particle positions for observables written once at the very end (sampled every 20 time steps with an auto-accumulator).

Attachments: benchmark.py, plot.py

@jngrad jngrad added the automerge Merge with kodiak label Jul 24, 2023
@kodiakhq kodiakhq bot merged commit a770f49 into espressomd:python Jul 24, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Observables: Particle selection and filtering should be independent of observable
2 participants