Add always-visible titles to the example gallery. #1484

rpgoldman · 2021-01-03T00:15:22Z

Previously the only titles were in tooltips, which made it near impossible to scan the page and find the example you wanted.

Note that this is a WIP if only because there's an Easter Egg in one of the titles that needs fixing!

ColCarroll

I found the easter egg!

Is there a reason to not just use ex_title as the title? We could even start throwing exceptions or something if it wasn't found. I don't feel strongly either way.

doc/sphinxext/gallery_generator.py

examples/matplotlib/mpl_plot_ess_evolution.py

ColCarroll · 2021-01-03T13:23:35Z

examples/matplotlib/mpl_plot_loo_pit_overlay.py

@@ -3,6 +3,7 @@
 ====================

 _thumb: .5, .7
+_example_title: Plot LOO overlayed on <i>some damn thing</i>.


watch your language!

I was hoping you knew what it was overlayed on...

...and whether it's "overlayed" or "overlaid"!

overlaid i believe

I would write something like Plot LOO-PIT KDE overlaid on uniform distributions KDEs, but I find it quite hard to summarize that more that what is said in the Examples section of its API docs without loosing so much meaning it can only be understood if you already know what it means

Can we assume someone using this would know what are CDF and inverse-CDF? Maybe Wikipedia's PIT article helps https://en.wikipedia.org/wiki/Probability_integral_transform ?

@avehtari Probably we can make that assumption, in which case we can give this a title that explains it sufficiently for such a person.

Plot LOO overlaid on uniform distributions to assess convergence.

Would that be good?

This does beg the question -- if a person does not know the purpose of this plot, where should they look? API docs? Examples? Or do we need a tutorial of some form?

I would say that ideally they would look at the API docs (each example already has a link to the relevant function) if necessary they can go to other resources from there, reference papers, it's page in the educational materials repo (which does not exist yet) and other relevant links that may be present.

I put in the above suggested title -- "Plot LOO overlaid on uniform distributions to assess convergence." Marking this as resolved.

It's missing a word: Plot LOO "what" overlaid on...
LOO can mean many things, and the confusion could that some would think it's LOO elpds. If you think that probability integral transformation is not well known, how about "LOO predictive ECDF" ?

It's not about convergence, but about model checking. So the full sentence could be
"Plot LOO predictive ECDF overlaid on ECDFs of uniform samples to assess predictive calibration"

rpgoldman · 2021-01-03T15:44:56Z

I found the easter egg!

Is there a reason to not just use ex_title as the title? We could even start throwing exceptions or something if it wasn't found. I don't feel strongly either way.

I put changes in ex_title for a couple of reasons:

It lets us put something different on the thumbnail for people who are searching for a particular plot.
It also lets us have longer titles, if we want, that wouldn't fit on the examples page.

codecov · 2021-01-03T17:38:12Z

Codecov Report

Merging #1484 (7dec9fd) into master (ae8a613) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1484   +/-   ##
=======================================
  Coverage   91.92%   91.92%           
=======================================
  Files         105      105           
  Lines       11252    11252           
=======================================
  Hits        10343    10343           
  Misses        909      909

Impacted Files	Coverage Δ
arviz/plots/khatplot.py	`94.87% <ø> (ø)`
arviz/plots/separationplot.py	`73.68% <ø> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ae8a613...7dec9fd. Read the comment docs.

rpgoldman · 2021-01-11T15:40:53Z

One thing that would be nice to fix before merging: separation plot.

the only description of the separation plot cited is from an article hidden behind a paywall.
the thumbnail is just a wall of blue.

rpgoldman · 2021-01-12T02:18:52Z

I just pushed a new version with a better-centered matplotlib separation plot.

rpgoldman · 2021-01-12T02:19:09Z

Calling this good enough and clearing the WIP flag.

rpgoldman

The only one I'm not sure about is the change to mpl_plot_ess_local -- the reason I added the phrase that you took out was that the term "local" was not clear. What makes an expected sample size local?

rpgoldman

Other than that one point, these all seem like clear improvements.

rpgoldman · 2021-01-13T18:40:00Z

These test failures aren't due to this MR.

rpgoldman · 2021-01-13T18:40:48Z

Last thing, if we have the energy, would be to apply these changes to the Bokeh figures, as well.

OriolAbril · 2021-01-13T19:22:15Z

What makes an expected sample size local?

The quantile ess is calculated with P(theta < theta_quantile) (see equation 16 in the reference) whereas the local ess is calculated with P(theta_quantile < theta < theta_quantile+delta) (eq 19). From the paper:

(...) This gives us a local efficiency measure which is more localized than efficiency measure for quantiles and can be used to build intuition about what types of posterior functionals can be computed as illustrated in the examples. While the expectation of a function that only depends on intermediate values can be usually estimated with relative ease, expectations of tail probabilities or other posterior functionals that depend critically on the tail of the distribution will be usually more difficult to estimate. In addition, small probability intervals can be used in practical equivalence testing

reference: https://arxiv.org/pdf/1903.08008.pdf

if we have the energy, would be to apply these changes to the Bokeh figures

I would wait for that, I want to try simplifying the example gallery using tabs like https://sphinx-panels.readthedocs.io/en/latest/#tabbed-content. After all, the title is the same, the api reference is the same, and it would allow having a single thumbnail and avoid generating bokeh thumbnails which is the main complication when generating the docs and requires weird dependencies.

rpgoldman · 2021-01-14T00:02:50Z

In that case, I think we are ready to merge.

Previously, it was hard to find things in the gallery generator, because the only information was in tooltips that were hidden by default.

Extend the docstring based on explanation from Aki Vehtari

Improve the reference to make it easier for readers to find.

Embedded Latex causes complaints about anomalous strings from pydocstyle and pylint.

rpgoldman requested a review from ColCarroll January 3, 2021 00:15

rpgoldman added the User Documentation Documentation outside of the codebase label Jan 3, 2021

ColCarroll reviewed Jan 3, 2021

View reviewed changes

rpgoldman force-pushed the doc-thumbnail-captions branch from 19773d2 to 18142db Compare January 3, 2021 17:20

rpgoldman marked this pull request as ready for review January 12, 2021 02:19

rpgoldman commented Jan 13, 2021

View reviewed changes

rpgoldman and others added 11 commits January 16, 2021 03:07

Add titles to gallery_generator.

022751d

Previously, it was hard to find things in the gallery generator, because the only information was in tooltips that were hidden by default.

Add titles to matplotlib examples.

86d6879

Add Khat plot description.

df3af34

Extend the docstring based on explanation from Aki Vehtari

Extend docstring for separation plot.

da9e441

Improve the reference to make it easier for readers to find.

Improvements suggested by @ColCarroll

17b6f4f

Fix title per @ColCarroll.

b503aca

Fix complaints about docstring.

2896d9d

Embedded Latex causes complaints about anomalous strings from pydocstyle and pylint.

Replace example title for LOO-PIT overlay plot.

1318f74

Fix thumbnail centering.

9ee7d21

minor tweaks to plot titles

31671ea

add to changelog

4f13f4d

OriolAbril force-pushed the doc-thumbnail-captions branch from 0ede2bd to 4f13f4d Compare January 16, 2021 01:09

Merge branch 'master' into doc-thumbnail-captions

7dec9fd

OriolAbril merged commit e9d2f2b into arviz-devs:master Jan 16, 2021

OriolAbril mentioned this pull request Jan 23, 2021

[docs] Tabs for example gallery! #1521

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add always-visible titles to the example gallery. #1484

Add always-visible titles to the example gallery. #1484

rpgoldman commented Jan 3, 2021

ColCarroll left a comment

ColCarroll Jan 3, 2021

rpgoldman Jan 3, 2021

rpgoldman Jan 3, 2021

canyon289 Jan 10, 2021

OriolAbril Jan 10, 2021

avehtari Jan 11, 2021

rpgoldman Jan 11, 2021

OriolAbril Jan 11, 2021

rpgoldman Jan 12, 2021

avehtari Jan 12, 2021

rpgoldman commented Jan 3, 2021

codecov bot commented Jan 3, 2021 •

edited

Loading

rpgoldman commented Jan 11, 2021

rpgoldman commented Jan 12, 2021

rpgoldman commented Jan 12, 2021

rpgoldman left a comment

rpgoldman left a comment

rpgoldman commented Jan 13, 2021

rpgoldman commented Jan 13, 2021

OriolAbril commented Jan 13, 2021

rpgoldman commented Jan 14, 2021

Add always-visible titles to the example gallery. #1484

Add always-visible titles to the example gallery. #1484

Conversation

rpgoldman commented Jan 3, 2021

ColCarroll left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rpgoldman commented Jan 3, 2021

codecov bot commented Jan 3, 2021 • edited Loading

Codecov Report

rpgoldman commented Jan 11, 2021

rpgoldman commented Jan 12, 2021

rpgoldman commented Jan 12, 2021

rpgoldman left a comment

Choose a reason for hiding this comment

rpgoldman left a comment

Choose a reason for hiding this comment

rpgoldman commented Jan 13, 2021

rpgoldman commented Jan 13, 2021

OriolAbril commented Jan 13, 2021

rpgoldman commented Jan 14, 2021

codecov bot commented Jan 3, 2021 •

edited

Loading