-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for linking distributions with embedded support #461
Comments
Regarding the first point: Maybe we could just |
So that's easy enough to do! We can just change the corresponding Lines 785 to 786 in 0d82a65
But this still leaves the problem of Because |
There might be a way if we move the |
I think it is ok if we stop adding new features to |
But the current |
I understand that we can default to |
Not when used with |
We can use a combination of
This is probably fine for now but can be improved later via tracing the model once to seed |
Not possible 😕 (unless we write some pretty significant pieces of code to reconstruct container values from a seuqence of |
Attempt at addressing #461. I think the approach here is somewhat correct, but it's currently very dirty because of the intricacies of `VarInfo` implementation. This can be cleaned up, but will take some effort. Until I've done this, I leave this as a draft PR. We also most certainly need to do some benchmarking before merging this as it could lead to some additional overhead. NOTE: This is based on #457 Co-authored-by: Hong Ge <[email protected]>
Fixed by #462 |
For certain distributions, the random variable represented by the
Distribution
has support which is lower-dimensional than the return-type indicates; that is, the returned realizations are embedded in a higher dimensional space.For example,
LKJ
is a distribution over correlation-matrices. Correlation matrices are required to be positive-definite (PD) and have 1 along the diagonal. PD means that we only have(n choose 2) + n
degrees of freedom, and 1 along the diagonal removes the additional factor ofn
, leaving us with only(n choose 2)
degrees of freedom. That is, as a vector space, the dimension of the correlation-matrices is actually just(n choose 2)
, notn × n
as might be indicated by the returnedMatrix{Float64}
fromrand(::LKJ)
!For
SimpleVarInfo
, this is trivial to support becauseSimpleVarInfo
only contains the realizations themselves, no information related to the distributions, etc. Therefore, with something like TuringLang/Bijectors.jl#246, things just workIn contrast, with
VarInfo
things are not so simple:With
VarInfo
there are multiple challenges:link!!
occurs in-place and expects the same shape as the original (untransformed) value.getindex(vi, vn, dist)
usesreconstruct(dist, val)
to reshape the underlying flattened representation inVarInfo
to whatdist
expects. This is done before passing it to the bijector/transformation, and so we if we're working with aVector
(because we're in transformed space), then callreconstruct(dist, val::Vector)
we get back aMatrix
aaaand the inverse transformation, which expects aVector
, fails. We could start looking into potentially adding the transformation used to thereconstruct
call, i.e. letting(dist, transform)
-pairs define thereconstruct
rather than justdist
, but then the problem is that inVarInfo
whether a variable is transformed or not is decided at runtime, which in turn causes type-instabilities (reconstruct
would then returnVector
in some cases andMatrix
in others, decided upon at runtime).So. We need a good way of doing this with
VarInfo
and I figured I'd make an issue so we can discuss this in more detail together.The text was updated successfully, but these errors were encountered: