You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sharing parameter values between a model and its guide is helpful in some applications, e.g. symmetric autoencoders. This can almost be accomplished by defining a param as usual in the guide and calling numypro.param("shared_site_name") in the model, but since SVI only replays sample values and not parameter values before tracing the model, the return value of this param() call defaults to None which breaks any downstream code that uses it.
Explicitly substitute initialized parameter values from the guide before tracing the model. Here's a strawman implementation which uses substitute() to avoid changing the semantics of replay(): master...msegado:numpyro:feat-shared-param-init.
Change the replay() handler to support replaying param values. This should probably be opt-in via an include_params kwarg or similar to avoid breaking existing user code, and would also require a change to the docs.
The first seems like the more natural approach to me, but I figured it would be good to ask before opening a PR 🙂 Thoughts?
The text was updated successfully, but these errors were encountered:
(Context: https://forum.pyro.ai/t/sharing-params-between-model-guide-i-e-tied-weights/5033/)
Sharing parameter values between a model and its guide is helpful in some applications, e.g. symmetric autoencoders. This can almost be accomplished by defining a param as usual in the guide and calling
numypro.param("shared_site_name")
in the model, but sinceSVI
only replays sample values and not parameter values before tracing the model, the return value of thisparam()
call defaults toNone
which breaks any downstream code that uses it.The desired behavior in SVI is to avoid initializing any model parameters already specified in the guide. I can see a couple of ways of accomplishing this:
substitute()
to avoid changing the semantics ofreplay()
: master...msegado:numpyro:feat-shared-param-init.replay()
handler to support replaying param values. This should probably be opt-in via aninclude_params
kwarg or similar to avoid breaking existing user code, and would also require a change to the docs.The first seems like the more natural approach to me, but I figured it would be good to ask before opening a PR 🙂 Thoughts?
The text was updated successfully, but these errors were encountered: