merge scipy19 docs #3557

keewis · 2019-11-20T23:25:50Z

This PR tries to merge the scipy19-docs branch into master while also making sure that the readthedocs build does not fail (see #3199 and #3369).

Passes black . && mypy . && flake8

There is also a preview page, but it currently fails because conda is taking too much memory (which the official builds do not). Does anyone know why that is?

keewis · 2019-11-20T23:48:01Z

something that could in the long run reduce the memory usage might be conda-metachannel (blog post), but I think the integration into conda is in its early stages?

keewis · 2019-11-21T00:58:46Z

it looks like it would be easier to do this by rebasing / cherry-picking onto master: there would not be as many conflicts as using merge would have (look at the list: 146 changed files). I'll cherry-pick and force push afterwards.

dcherian · 2019-11-21T01:16:08Z

👍

.pre-commit-config.yaml

rabernat · 2019-11-21T03:11:56Z

Sorry for creating this mess! 🤦‍♂️ Thank you for trying to clean it up.

* switching out examples to use nbsphinx * added jupyter_client to doc env * added ipykernel to doc env

* switching out examples to use nbsphinx * added jupyter_client to doc env * moved gallery to notebook

* switching out examples to use nbsphinx * added jupyter_client to doc env * allow non netcdf tutorial files

* change name of test env to xarray-tests (pydata#3110) * ROMS_ocean_model example added * Allow other tutorial filename extensions (pydata#3121) * switching out examples to use nbsphinx * added jupyter_client to doc env * allow non netcdf tutorial files * Changed load to xr.tutorial.open_dataset(), and added some extra documentation. * change name of test env to xarray-tests (pydata#3110) * ROMS_ocean_model example added * Changed load to xr.tutorial.open_dataset(), and added some extra documentation. * fixed colormap issues leftover from cmocean import * Added intro paragraph to ROMS example notebook, removed comments, and added citation in whats-new.

…ydata#3199) * Adds an example of ERA5 and GRIB data to the gallery * Add markdown narrative cells to GRIB example * Update load method to use xr.tutorial * Fix load method

keewis · 2019-11-21T14:33:07Z

okay, now the cherry-picking is done and the number of touched files is down to a more reasonable 14, though I dropped the non-PR commits. I'm not sure which of these was really necessary, but I guess that's something that we will notice once we try to get CI and readthedocs to work?

The original branch is still available for comparison on scipy19-docs

@dcherian (or anyone else who knows a bit more about RTD than me), can you help me fix the readthedocs builds? I can't test anything as long as the environment creation fails.

dcherian · 2019-11-21T15:44:14Z

Does anyhting on this page help? It's likely we have requested extra resources for xarray.

https://docs.readthedocs.io/en/stable/guides/build-using-too-many-resources.html#use-pip-when-possible

keewis · 2019-11-21T15:45:22Z

and now I ran into the shadowing issue we had in #3199. I think the culprit is cfgrib which uses extras_require to pull in xarray. Is there any way to tell conda not to use that? We could also try to pull it using pip, since that works (at least for me) and bypasses the download of the packaged xarray.

dcherian · 2019-11-21T15:46:25Z

maybe pip is the solution to all of these problems

keewis · 2019-11-21T15:55:26Z

probably, but the only issue I found is this: https://pypi.org/project/iris/, which means iris is only available on conda-forge?

dcherian · 2019-11-21T16:06:53Z

How about we pip install iris master just to see if we can get things to work?

keewis · 2019-11-21T16:08:53Z

I'm currently trying to move everything except python and iris into the pip section, let's see what kind of memory usage that gives us.

Edit: that almost worked, it failed in the execution state. I'll try to install iris from github.

keewis · 2019-11-21T17:09:07Z

iris from github would work, but cartopy has no wheels, so installing using pip fails. I'll stop investigating this for now and try to make CI pass.

dcherian · 2019-11-21T17:45:26Z

keewis · 2019-11-21T18:08:24Z

now that CI passes we only need to get the readthedocs build to work (and maybe re-apply #3464?).

keewis · 2019-11-21T18:16:28Z

oh, and we have some warnings regarding missing document labels. Is there a way to reference labels in notebooks?

Missing: section links in visualization_gallery.ipynb don't work yet, also the one in io.rst (it has a unicode char).

keewis · 2019-11-21T22:06:04Z

oh, and is there a reason why we merge into stable instead of master?

dcherian · 2019-11-21T22:10:48Z

I just switched it so we can tweet about binderable examples ;)

dcherian · 2019-11-21T22:13:30Z

Parsing rasterio geocoordinates? 🤷‍♂️

Is that the last thing to fix?

keewis · 2019-11-21T22:41:21Z

I would think so, but I think we can fix anything that pops up later in new PRs.

keewis · 2019-11-21T22:50:33Z

one thing that might be done in a future PR is enabling the HTML reprs?

max-sixty · 2019-11-21T22:53:09Z

Feel free to ignore since it's not a big deal relative to the benefits, but looks like this adds some recent changes on master to stable (e.g. https://github.com/pydata/xarray/pull/3557/files#diff-99289912975ea19afc02f51c240ca451R18)

keewis · 2019-11-21T22:54:16Z

yeah, should we merge to stable we should probably rebase first

keewis · 2019-11-21T22:55:39Z

@dcherian, I'd merge into master and then cherry-pick into stable?

max-sixty · 2019-11-21T22:56:31Z

Should we change the base of this PR to master? Then we can choose when we push to stable? (But let's not get stuck on this, fine to just go ahead if we can't resolve)

dcherian · 2019-11-21T23:00:10Z

OK I jsut got too excited.

keewis · 2019-11-21T23:03:35Z

ah, maybe we should move the whats-new.rst entries?

dcherian · 2019-11-21T23:05:31Z

Yes! there seems to be one new example entry

keewis · 2019-11-21T23:09:08Z

would that be documentation or enhancements?

dcherian · 2019-11-21T23:13:09Z

Documentation

keewis · 2019-11-21T23:13:18Z

while we have an entry for #3116 the one for #3199 is missing (also, @rabernat, do you want an entry for switching to nbsphinx?)

dcherian · 2019-11-21T23:14:27Z

Yes let's add appropriate entries for all those.

keewis · 2019-11-21T23:21:55Z

how about

- Switch doc examples to use nbsphinx and replace sphinx_gallery with notebook.
  (:pull:`3105`, :pull:`3106`, :pull:`3121`)
  By `Ryan Abernathey <https://github.com/rabernat>`
- Added example notebook demonstrating the visualization of ERA5 GRIB data.
  (:pull:`3199`).
  By `Zach Bruick <https://github.com/zbruick>` and `Stephan Siemen <https://github.com/StephanSiemen>`

dcherian · 2019-11-21T23:33:47Z

👍

keewis · 2019-11-21T23:45:15Z

done

keewis · 2019-11-21T23:54:15Z

I think we can merge after making sure RTD still builds?

dcherian · 2019-11-22T01:14:56Z

Great work as usual @keewis !

* upstream/master: Add drop to api.rst under pending deprecations (pydata#3561) replace duplicate method _from_vars_and_coord_names (pydata#3565) propagate indexes in to_dataset, from_dataset (pydata#3519) Switch examples to notebooks + scipy19 docs improvements (pydata#3557) fix whats-new.rst (pydata#3554)

* upstream/master: (22 commits) Resolve the version issues on RTD (pydata#3589) Add bottleneck & rasterio git tip to upstream-dev CI (pydata#3585) update whats-new.rst (pydata#3581) Examples for quantile (pydata#3576) add cftime intersphinx entries (pydata#3577) Add pyXpcm to Related Projects doc page (pydata#3578) Reimplement quantile with apply_ufunc (pydata#3559) add environment file for binderized examples (pydata#3568) Add drop to api.rst under pending deprecations (pydata#3561) replace duplicate method _from_vars_and_coord_names (pydata#3565) propagate indexes in to_dataset, from_dataset (pydata#3519) Switch examples to notebooks + scipy19 docs improvements (pydata#3557) fix whats-new.rst (pydata#3554) Tweaks to release instructions (pydata#3555) Clarify conda environments for new contributors (pydata#3551) Revert to dev version 0.14.1 whatsnew (pydata#3547) sparse option to reindex and unstack (pydata#3542) Silence sphinx warnings (pydata#3516) Numpy 1.18 support (pydata#3537) ...

* upstream/master: (35 commits) fix plotting with transposed nondim coords. (pydata#3441) make coarsen reductions consistent with reductions on other classes (pydata#3500) Resolve the version issues on RTD (pydata#3589) Add bottleneck & rasterio git tip to upstream-dev CI (pydata#3585) update whats-new.rst (pydata#3581) Examples for quantile (pydata#3576) add cftime intersphinx entries (pydata#3577) Add pyXpcm to Related Projects doc page (pydata#3578) Reimplement quantile with apply_ufunc (pydata#3559) add environment file for binderized examples (pydata#3568) Add drop to api.rst under pending deprecations (pydata#3561) replace duplicate method _from_vars_and_coord_names (pydata#3565) propagate indexes in to_dataset, from_dataset (pydata#3519) Switch examples to notebooks + scipy19 docs improvements (pydata#3557) fix whats-new.rst (pydata#3554) Tweaks to release instructions (pydata#3555) Clarify conda environments for new contributors (pydata#3551) Revert to dev version 0.14.1 whatsnew (pydata#3547) sparse option to reindex and unstack (pydata#3542) ...

keewis mentioned this pull request Nov 20, 2019

Fix documentation #3554

Merged

1 task

max-sixty reviewed Nov 21, 2019

View reviewed changes

.pre-commit-config.yaml Outdated Show resolved Hide resolved

rabernat and others added 5 commits November 21, 2019 15:11

Switch doc examples to use nbsphinx (pydata#3105)

e574663

* switching out examples to use nbsphinx * added jupyter_client to doc env * added ipykernel to doc env

Replace sphinx_gallery with notebook (pydata#3106)

789a3d2

* switching out examples to use nbsphinx * added jupyter_client to doc env * moved gallery to notebook

Allow other tutorial filename extensions (pydata#3121)

751ec0f

* switching out examples to use nbsphinx * added jupyter_client to doc env * allow non netcdf tutorial files

Add an example of ERA5 and GRIB data & visualization to the gallery (p…

3f85659

…ydata#3199) * Adds an example of ERA5 and GRIB data to the gallery * Add markdown narrative cells to GRIB example * Update load method to use xr.tutorial * Fix load method

keewis force-pushed the scipy19-docs branch from 4dc2e56 to 3f85659 Compare November 21, 2019 14:20

keewis added 2 commits November 21, 2019 15:27

require nbsphinx for the documentation builds

66d5428

add more nbsphinx dependencies

9511758

keewis added 3 commits November 21, 2019 18:11

install cfgrib using pip

64abead

add the eccodes library to the dependencies

423ba37

remove the dependency on sphinx-gallery

15fced0

add the ERA5 GRIB example to the list

36b5996

update the documentation links

d502ae2

Missing: section links in visualization_gallery.ipynb don't work yet, also the one in io.rst (it has a unicode char).

fix the reference to the rasterio geocoordinates docs

26f4d07

dcherian changed the base branch from stable to master November 21, 2019 22:59

update whats-new.rst

8222468

dcherian merged commit b0064b2 into pydata:master Nov 22, 2019

keewis mentioned this pull request Nov 24, 2019

version collisions on readthedocs #3567

Closed

keewis deleted the scipy19-docs branch November 25, 2019 11:29

merge scipy19 docs #3557

merge scipy19 docs #3557

Conversation

keewis commented Nov 20, 2019 • edited Loading

keewis commented Nov 20, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

rabernat commented Nov 21, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019 • edited Loading

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019

keewis commented Nov 21, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019

keewis commented Nov 21, 2019

max-sixty commented Nov 21, 2019

keewis commented Nov 21, 2019

keewis commented Nov 21, 2019

max-sixty commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019 • edited Loading

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 21, 2019

keewis commented Nov 21, 2019

keewis commented Nov 21, 2019

dcherian commented Nov 22, 2019

keewis commented Nov 20, 2019 •

edited

Loading

keewis commented Nov 21, 2019 •

edited

Loading

keewis commented Nov 21, 2019 •

edited

Loading