Multitraces should iterate over samples from all chains

[AustinRochford]

Resolves #2457

[ferrine]

Does it affect this code somehow?

[junpenglao]

Does it affect this code somehow?

I think it would.

[AustinRochford]

@ferrine it's not obvious to me that it will affect that code, but the tests will let us know for sure. I certainly expect some to fail, and will also be adding a few new ones before merging.

[ferrine]

this line assumes old length https://github.com/pymc-devs/pymc3/blob/master/pymc3/variational/approximations.py#L311

[AustinRochford]

Thanks for the heads up, that one's a simple enough change, at least. I'll wait to see how many other places there are that depend on the old len. If there are many, I may need to take a different approach.

[AustinRochford]

Coverage decreased by ~40%

[aseyboldt]

Not sure what we should do here, all options I can see are a bit strange.

• We could keep the current behaviour (len and iter ignore all but the first chain), but add a method that returns an iterator over all samples
• Keep len as it is, but change iter to return all samples (the current PR). This kind of breaks common assumptions, usually len(list(obj)) should match len(obj) for all iterables.
• Change both len and iter to return all samples

I think the first one might be a good compromise between reasonable behaviour and backward compatibility.

[AustinRochford]

@aseyboldt I agree. I mostly want to see which tests this approach breaks. After trying the third option, which seems to break many parts of the code base, I'm inclined to go with the first, although it is a bit unsatisfying.

[twiecki]

Hm, yeah, first option seems the most realistic.

[ferrine]

I vote for the third

[aseyboldt]

Any updates on this? We still haven't decided which way we'd like to go, right?

[AustinRochford]

@aseyboldt this got lost in my vacation :|. I think we should go with the first option. Will try to get a new PR up in the next few days.

[junpenglao]

@AustinRochford any progress? Would be great to fix this before 3.2 ;-)

[AustinRochford]

Yes it would. Unfortunately no progress yet, but I'll see about putting something together tonight.