Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add functions to import the datasets #3

Merged
merged 20 commits into from
Nov 19, 2020
Merged

Add functions to import the datasets #3

merged 20 commits into from
Nov 19, 2020

Conversation

stefsmeets
Copy link
Contributor

@stefsmeets stefsmeets commented Nov 3, 2020 •
edited
Loading

In this PR I set up the basic setup.py, and add some functionality to load the timeseries data.

from esmvaltool_sample_data import load_timeseries_data
cubelist = load_timeseries_data()

Needed for ESMValGroup/ESMValCore#856

@stefsmeets stefsmeets self-assigned this Nov 3, 2020
@stefsmeets
Set cube helper logging level to ERROR to hide warnings
f3c9a30 
i.e. `tracking_id, history and creation_date attributes inconsistent`
@stefsmeets
Comment out problematic dataset
294833e
@stefsmeets
Copy link
Contributor Author

I have some issues with the dataset:
CMIP6.CMIP.FIO-QLNM.FIO-ESM-2-0.historical.r1i1p1f1.Amon.ta.gn.v20191204
which raises
ValueError: Cube 'air_temperature' must contain a single 1D y coordinate.

@stefsmeets stefsmeets mentioned this pull request Nov 5, 2020
Merged
27 tasks
@stefsmeets stefsmeets mentioned this pull request Nov 6, 2020
Closed
11 tasks
@stefsmeets stefsmeets marked this pull request as ready for review November 6, 2020 13:45
bouweandela
bouweandela reviewed Nov 9, 2020
View reviewed changes
setup.py Show resolved Hide resolved
bouweandela
bouweandela reviewed Nov 9, 2020
View reviewed changes
setup.py Show resolved Hide resolved
@stefsmeets
Copy link
Contributor Author

stefsmeets commented Nov 10, 2020 •
edited
Loading

Two more datasets give warnings:

esmvaltool_sample_data/data/timeseries/CMIP6/CMIP/NCAR/CESM2-WACCM-FV2/historical/r1i1p1f1/Amon/ta/gn/v20191120
/home/stef/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py:2290: UserWarning: Gracefully filling 'lat' dimension coordinate masked points
  warnings.warn(msg.format(str(cf_coord_var.cf_name)))
/home/stef/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py:2290: UserWarning: Gracefully filling 'lat' dimension coordinate masked points
  warnings.warn(msg.format(str(cf_coord_var.cf_name)))

esmvaltool_sample_data/data/timeseries/CMIP6/CMIP/NCAR/CESM2-FV2/historical/r1i1p1f1/Amon/ta/gn/v20191120
/home/stef/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py:2290: UserWarning: Gracefully filling 'lat' dimension coordinate masked points
  warnings.warn(msg.format(str(cf_coord_var.cf_name)))
/home/stef/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/iris/fileformats/_pyke_rules/compiled_krb/fc_rules_cf_fc.py:2290: UserWarning: Gracefully filling 'lat' dimension coordinate masked points
  warnings.warn(msg.format(str(cf_coord_var.cf_name)))

@bouweandela
Copy link
Member

Two more datasets give warnings:

If they don't work, just delete the files and add them to the ignore list in datasets.yml so they will not be downloaded again.

@stefsmeets
Copy link
Contributor Author

This is ready to be merged for all I'm concerned. @nielsdrost could you have a look?
I will address problems with the setup.py in a separate PR (metadata etc.).

@stefsmeets stefsmeets mentioned this pull request Nov 18, 2020
Closed
nielsdrost
nielsdrost approved these changes Nov 18, 2020
View reviewed changes
Copy link
Member

@nielsdrost nielsdrost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

I also tried it out. Package seems to work just fine :-)

esmvaltool_sample_data/__init__.py Show resolved Hide resolved
esmvaltool_sample_data/datasets.yml Show resolved Hide resolved
@stefsmeets stefsmeets merged commit b0b5b6d into master Nov 19, 2020
@stefsmeets stefsmeets deleted the make_package branch November 19, 2020 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bouweandela bouweandela bouweandela left review comments

@nielsdrost nielsdrost nielsdrost approved these changes

Assignees

@stefsmeets stefsmeets

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@stefsmeets @bouweandela @nielsdrost