Should Distributions have two transform kwargs?

[ricardoV94]

When users specify a transform (e.g. Ordered, LocScale(TBA)) on a distribution that has a default transform (e.g., Log), they usually end up missing/forgetting the latter and sampling becomes flaky.

The safe thing would be to chain the two transforms, but this is a bit verbose if done manually and most users are not even aware there is a default transform.

What if we convert the current transform kwarg to default_transform and make transform an optional additional transform that we automatically chain behind the scenes? Or call the new extra_transform which is more verbose (most users would want to fiddle with this one) but backwards compatible.

One other advantage is that we could warn users in prior predictive sampling as extra transforms almost always render the model different than forward sampling, similar to our Potentials warning

See:

#4721

#4213

#5277

[twiecki]

I think this makes sense to change the default. I'm a bit concerned about the case where I set transform=None (often for illustrative or debugging purposes). Would that be a special case?

[ricardoV94]

We can issue a warning, since the default is UNSET, and None would never have an effect.

So if a user sets None we can say Behavior changed... bla bla. Set default_transform to None, to completely disable distribution transformation.

[junpenglao]

So IIUC the second transform kwarg is to specify additional transformation, so we have chain([additional_transformation, default_transformation])? In the case where we want to rewrite transformation user will need to set default transformation to UNSET and addtional_transformation to user input?

[ricardoV94]

If the user wants to override the default_transformation they would need to either:

1. Set default_transform to something else and leave transform untouched
2. Set default_transform=None and transform to something else

We probably can raise an error in the latter case.

[junpenglao]

OK i see. We can make a distinction that:

1. Transformation that for unbounding RV (for HMC and VI). For example, with Uniform RV and default interval transformation, user can overwrite it to use sigmoid instead.
2. Other transformation: for creating ordered RV, unit vector etc

[ricardoV94]

Yeah that's the idea. Default transforms are almost always for unconstrained sampling and and the other would be for additional features (sorting, shifting and scaling, artificially bounding, etc...)

Naming is important here to avoid too much confusion. Transforms are already very opaque for many users

[ricardoV94]

This is also relevant for prior predictive. If we have the distinction we could easily switch to NUTS for doing prior predictive when model contains non-default transformations.

This makes transforms like ordered much easier to work with.

[twiecki]

Seems like we should make a decision here and it sounds like there are good arguments for and not very good arguments against.

[ricardoV94]

I think there were no disagreements, I'll open an issue