Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Find the right boundary between runtimes and programs #13

Merged
merged 2 commits into from
Mar 30, 2020
Merged

Find the right boundary between runtimes and programs #13

merged 2 commits into from
Mar 30, 2020

Conversation

rlouf
Copy link
Owner

@rlouf rlouf commented Mar 28, 2020 •
edited
Loading

I am not sure what runtimes and programs' respective responsibilities are yet. This PR is an experiment to find the right API.

Requirements

  • The programs need not be aware of the models so they can be used with custom loglikelihoods. As such they can be moved to the inference module;
  • It should be simple to use meta-algorithms like HMC Swindles and Replica exchange with an existing program;
  • Runtimes are program-agnostic; they communicate via a unified API;
  • One should be able to choose the integrator used in HMC;
  • Warmup adapts to the parameters passed to the program;
  • Runtimes come with reasonable defaults (dynamicHMC or empiricalHMC), but can easily be used with any algorithm.

Slow initialisation

The initialisation is unacceptably slow. It is caused by repeatedly calling ravel_pytree for each chain. Luckily for us, ravel_pytree arranges the values of the dictionary in a deterministic way, sorting them by key. To fix the performance, we stack the arrays in the dictionary returned by sample_forward to obtained flattened position, and apply ravel_pytreeto one dictionary to get the unraveling function.

This PR follows issue #12.

@rlouf
move most of the initialization to the runtime
c88960d
@rlouf
separated programs and runtimes neatly
cd51114 
The boundary between programs (sampling algorithms) and runtimes
(executors) was not very clear. I remove any dependance on the model
from the program and responsibilities are now clear. I also improved the
performance of the creation of initial states.
@rlouf
Copy link
Owner Author

rlouf commented Mar 30, 2020

Good enough for now, merging.

@rlouf rlouf closed this Mar 30, 2020
@rlouf rlouf reopened this Mar 30, 2020
@rlouf rlouf merged commit 363da6d into master Mar 30, 2020
@rlouf rlouf deleted the runtime-program-boundary branch March 30, 2020 15:23
@rlouf rlouf mentioned this pull request Mar 30, 2020
Closed
rlouf pushed a commit that referenced this pull request Apr 3, 2020
@rlouf
Find the right boundary between runtimes and programs (#13)
9299a7c 
The boundary between programs (sampling algorithms) and runtimes
(executors) was not very clear. I remove any dependence on the model
from the program and responsibilities are now clear. Most of the initialization has been transferred to the runtime. I also improved the
performance of the creation of initial states.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@rlouf