Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

from_pymc3(save_warmup=True) puts tuning samples into posterior if draws=0 #1210

Closed
michaelosthege opened this issue May 26, 2020 · 2 comments

Comments

@michaelosthege
Copy link
Contributor

Describe the bug
When a trace given to az.from_pymc3(trace, save_warmup=True) contains tuning samples, but no draws, the tuning samples end up in the posterior group and the warmup group remains empty.

To Reproduce

with pm.Model() as model:
    x = pm.Normal("x", mu=0, sigma=1)
    trace = pm.sample(
        tune=100,
        draws=0,
        discard_tuned_samples=False,
        chains=1,
    )
    idata = az.from_pymc3(trace, save_warmup=True)
# >> Sampling 1 chain for 100 tune and 0 draw iterations (100 + 0 draws total) took 0 seconds.

print(idata.warmup_posterior.dims)  # >> Frozen(SortedKeysDict({'chain': 1, 'draw': 0}))
print(idata.posterior.dims)         # >> Frozen(SortedKeysDict({'chain': 1, 'draw': 100}))

Expected behavior
Tuning samples should end up in the warmup group - even if there are no further draws.

Additional context
ArviZ master
PyMC3 master

@michaelosthege
Copy link
Contributor Author

The existing test can be parametrized to detect it:

Line 403 in test_data_pymc.py

    @pytest.mark.skipif(
        not hasattr(pm.backends.base.SamplerReport, "n_draws"),
        reason="requires pymc3 3.9 or higher",
    )
    @pytest.mark.parametrize("save_warmup", [False, True])
    @pytest.mark.parametrize("chains", [1, 2])
    @pytest.mark.parametrize("tune,draws", [(0, 200), (100, 200), (100, 0)])
    def test_save_warmup(self, save_warmup, chains, tune, draws):
        with pm.Model():
            pm.Uniform("u1")
            pm.Normal("n1")
            trace = pm.sample(
                tune=tune,
                draws=draws,
                chains=chains,
                cores=1,
                step=pm.Metropolis(),
                discard_tuned_samples=False,
            )
            assert isinstance(trace, pm.backends.base.MultiTrace)
            idata = from_pymc3(trace, save_warmup=save_warmup)
        prefix = "" if save_warmup else "~"
        test_dict = {
            "posterior": ["u1", "n1"],
            "sample_stats": ["~tune", "accept"],
            f"{prefix}warmup_posterior": ["u1", "n1"],
            f"{prefix}warmup_sample_stats": ["~tune"],
            "~warmup_log_likelihood": [],
            "~log_likelihood": [],
        }
        fails = check_multiple_attrs(test_dict, idata)
        assert not fails
        assert idata.posterior.dims["chain"] == chains
        assert idata.posterior.dims["draw"] == draws
        if save_warmup:
            assert idata.warmup_posterior.dims["chain"] == chains
            assert idata.warmup_posterior.dims["draw"] == tune

@michaelosthege
Copy link
Contributor Author

It's a slicing problem.. Doing a PR...

michaelosthege added a commit to michaelosthege/arviz that referenced this issue May 26, 2020
michaelosthege added a commit to michaelosthege/arviz that referenced this issue May 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant