Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated LocScaleReparam's documentation #1599

Merged
merged 2 commits into from
Jun 1, 2023

Conversation

Madhav-Kanda
Copy link
Contributor

Refer #1598

@Madhav-Kanda Madhav-Kanda changed the title Update reparam.py Updated LocScaleReparam's documentation in reparam.py Jun 1, 2023
@Madhav-Kanda Madhav-Kanda changed the title Updated LocScaleReparam's documentation in reparam.py Updated LocScaleReparam's documentation Jun 1, 2023
@@ -72,8 +72,7 @@ class LocScaleReparam(Reparam):
1. *Automatic Reparameterisation of Probabilistic Programs*,
Maria I. Gorinova, Dave Moore, Matthew D. Hoffman (2019)

:param float centered: optional centered parameter. If None (default) learn
a per-site per-element centering parameter in ``[0,1]``. If 0, fully
:param float centered: optional centered parameter. If None (default) considers centered parameter to be 0.5. If 0, fully
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you change this to something like:

    :param float centered: optional centered parameter. If None (default) learn
        a per-site per-element centering parameter in ``[0,1]`` initialized at value 0.5.
        To sample the parameter, consider using :class:`~numpyro.handlers.lift` handler with a
        prior like ``Uniform(0, 1)`` to cast the parameter to a latent variable. If 0, fully

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I would change this, but based on my understanding of the current implementation, it would not learn the center param even when we keep it None. It would simply keep the centered param as 0.5. To learn the correct parameter based on Maria Gorinova's paper, we will have to use Variational Inference. Please correct me if I need to correct something. Thanks for the quick response.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's why we say this is a parameter, not a latent variable. When using MCMC, you are drawing samples of latent variables. When you say "optimizing", it typically means that you are using inference methods with optimizers like SGD, Adam and gradient descent to optimize the corresponding parameters. When you say "sampling", it means that you are using MCMC to draw samples of latent "random variable".

Copy link
Member

@fehiepsi fehiepsi Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that in numpyro, MCMC does not sample "parameters", it only samples latent variables. If you think the word "parameters" is not clear enough, I think you can change it to something like

If None (default) create a ``numpyro.param`` site for a per-site per-element centering parameter ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, okay. Thank you!

@fehiepsi
Copy link
Member

fehiepsi commented Jun 1, 2023

Thanks @Madhav-Kanda!

@fehiepsi fehiepsi closed this Jun 1, 2023
@fehiepsi fehiepsi reopened this Jun 1, 2023
@fehiepsi fehiepsi merged commit 4e3c50e into pyro-ppl:master Jun 1, 2023
OlaRonning pushed a commit to aleatory-science/numpyro that referenced this pull request Jun 2, 2023
* Update reparam.py

Refer pyro-ppl#1598

* Making suggested changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants