Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote Datasets: Adjust attributes - remove "title", use default of "name" and "long_name", introduce "description" #3048

Merged
merged 68 commits into from
Apr 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
0b3b9f1
Remove title and make key and name consistent
yvonnefroehlich Feb 7, 2024
5342d89
Adjust load functions
yvonnefroehlich Feb 7, 2024
6f37e01
Adjust error message
yvonnefroehlich Feb 7, 2024
7d45695
Adjust documentation
yvonnefroehlich Feb 7, 2024
178c6f3
Make documentation and long_name consistent
yvonnefroehlich Feb 7, 2024
16926f3
Adjust tests
yvonnefroehlich Feb 7, 2024
b38635f
Adjust tutorial 'grid_equalization.py'
yvonnefroehlich Feb 7, 2024
57be08d
Adjust earth_age test
yvonnefroehlich Feb 7, 2024
28df448
Fix typo
yvonnefroehlich Feb 7, 2024
bfd447b
Adjust test 'load_remote_datasets'
yvonnefroehlich Feb 7, 2024
83b647f
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Feb 17, 2024
74d9bc8
Fix typo
yvonnefroehlich Mar 1, 2024
15f42b7
Separte Earth relief dataset in 'igpp' and 'gebco'
yvonnefroehlich Mar 4, 2024
3b999e7
Sort alphabetically
yvonnefroehlich Mar 4, 2024
582235c
Adjust tests
yvonnefroehlich Mar 4, 2024
3ca6f32
Fix programming language
yvonnefroehlich Mar 4, 2024
098b45a
Fix typo
yvonnefroehlich Mar 4, 2024
2393ec4
Fix tutorial
yvonnefroehlich Mar 4, 2024
d2b2c43
Shorten documentation
yvonnefroehlich Mar 9, 2024
02cc018
Use 'earth_mag' instead of 'earth_emag'
yvonnefroehlich Mar 9, 2024
d09db24
Use 'earth_mag' instead of 'earth_emag'
yvonnefroehlich Mar 9, 2024
36cf851
Adjust test
yvonnefroehlich Mar 9, 2024
24055f5
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Mar 9, 2024
fabae4c
TEST - Change keyword
yvonnefroehlich Mar 9, 2024
4d160fc
TEST - Change keyword back
yvonnefroehlich Mar 9, 2024
25b98bf
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Mar 10, 2024
39085c7
TEST - Remove 'grid_equalization' tutorial
yvonnefroehlich Mar 10, 2024
48608f2
Revert "TEST - Remove 'grid_equalization' tutorial"
yvonnefroehlich Mar 10, 2024
6aa8e74
Adjust keyword
yvonnefroehlich Mar 10, 2024
7c5f27a
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Mar 10, 2024
9b6b8ff
Adjust name for emag2 dataset
yvonnefroehlich Mar 11, 2024
7a59250
Adjust name for wdmam dataset
yvonnefroehlich Mar 11, 2024
ab00c93
Sort alphabeticaly
yvonnefroehlich Mar 11, 2024
4636eb0
Adjust tests
yvonnefroehlich Mar 11, 2024
4058691
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Mar 11, 2024
36a8efe
Fix tests, names | Use upper case letters related to shortcut
yvonnefroehlich Mar 11, 2024
1779461
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Mar 13, 2024
001c7b2
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Mar 17, 2024
5fb9748
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Mar 18, 2024
2856008
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Mar 19, 2024
7db4edf
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Apr 5, 2024
6af2390
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Apr 18, 2024
e4b27bb
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Apr 21, 2024
8a7e317
Use 'z' as 'name' and 'quanity_unit' as 'long_name' attributes
yvonnefroehlich Apr 21, 2024
a39f44a
[format-command] fixes
actions-bot Apr 21, 2024
d385716
Fix
yvonnefroehlich Apr 21, 2024
3d07d7a
Fix coding style
yvonnefroehlich Apr 21, 2024
9311971
Fix
yvonnefroehlich Apr 21, 2024
4add858
Fix input data type
yvonnefroehlich Apr 21, 2024
ae29f76
Fix
yvonnefroehlich Apr 21, 2024
9f2b747
Fix
yvonnefroehlich Apr 21, 2024
1df4324
Test testing 'description' attribut
yvonnefroehlich Apr 21, 2024
eec68ec
Move 'description' attribut to 'extra_attributes'
yvonnefroehlich Apr 21, 2024
e75a6d1
Revert changes
yvonnefroehlich Apr 21, 2024
d4541ad
Revert changes
yvonnefroehlich Apr 21, 2024
5f508db
Add 'description attribut
yvonnefroehlich Apr 21, 2024
59e042f
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Apr 22, 2024
26ff532
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Apr 23, 2024
2f95200
Use match-case statements
yvonnefroehlich Apr 23, 2024
04bf516
Remove setting the 'name' and 'long_name' attributes
yvonnefroehlich Apr 23, 2024
d7a8ea0
Remove setting the 'name' and 'long_name' attributes
yvonnefroehlich Apr 23, 2024
3c4cde7
Remove setting the 'name' and 'long_name' attributes
yvonnefroehlich Apr 23, 2024
7418d6d
Update tests
yvonnefroehlich Apr 23, 2024
1546ada
Update tests
yvonnefroehlich Apr 23, 2024
2772e2d
Update example of 'grd2xyz'
yvonnefroehlich Apr 23, 2024
4f9def4
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Apr 23, 2024
67f31cf
Merge branch 'main' into adjust-names-datasets
seisman Apr 24, 2024
32e3349
Merge branch 'main' into adjust-names-datasets
yvonnefroehlich Apr 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/tutorials/advanced/grid_equalization.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
grid = pygmt.datasets.load_earth_relief(
resolution="03s", region=[-119.825, -119.4, 37.6, 37.825]
)
grid_dist = pygmt.grd2xyz(grid=grid, output_type="pandas")["elevation"]
grid_dist = pygmt.grd2xyz(grid=grid, output_type="pandas")["z"]
seisman marked this conversation as resolved.
Show resolved Hide resolved


# %%
Expand Down
8 changes: 4 additions & 4 deletions pygmt/datasets/earth_free_air_anomaly.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
Function to download the IGPP Earth Free-Air Anomaly dataset from the GMT data server,
Function to download the IGPP Earth free-air anomaly dataset from the GMT data server,
and load as :class:`xarray.DataArray`.

The grids are available in various resolutions.
Expand All @@ -20,13 +20,13 @@ def load_earth_free_air_anomaly(
registration: Literal["gridline", "pixel", None] = None,
):
r"""
Load the IGPP Earth Free-Air Anomaly dataset in various resolutions.
Load the IGPP Earth free-air anomaly dataset in various resolutions.

.. figure:: https://www.generic-mapping-tools.org/remote-datasets/_images/GMT_earth_faa.jpg
:width: 80 %
:align: center

IGPP Earth Free-Air Anomaly dataset.
IGPP Earth free-air anomaly dataset.

The grids are downloaded to a user data directory
(usually ``~/.gmt/server/earth/earth_faa/``) the first time you invoke
Expand Down Expand Up @@ -102,7 +102,7 @@ def load_earth_free_air_anomaly(
... )
"""
grid = _load_remote_dataset(
dataset_name="earth_free_air_anomaly",
dataset_name="earth_faa",
dataset_prefix="earth_faa_",
resolution=resolution,
region=region,
Expand Down
6 changes: 3 additions & 3 deletions pygmt/datasets/earth_geoid.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
Function to download the EGM2008 Earth Geoid dataset from the GMT data server, and load
Function to download the EGM2008 Earth geoid dataset from the GMT data server, and load
as :class:`xarray.DataArray`.

The grids are available in various resolutions.
Expand All @@ -20,13 +20,13 @@ def load_earth_geoid(
registration: Literal["gridline", "pixel"] = "gridline",
):
r"""
Load the EGM2008 Earth Geoid dataset in various resolutions.
Load the EGM2008 Earth geoid dataset in various resolutions.

.. figure:: https://www.generic-mapping-tools.org/remote-datasets/_images/GMT_earth_geoid.jpg
:width: 80 %
:align: center

EGM2008 Earth Geoid dataset.
EGM2008 Earth geoid dataset.

The grids are downloaded to a user data directory
(usually ``~/.gmt/server/earth/earth_geoid/``) the first time you invoke
Expand Down
2 changes: 1 addition & 1 deletion pygmt/datasets/earth_magnetic_anomaly.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ def load_earth_magnetic_anomaly(
"Valid values are 'emag2', 'emag2_4km', and 'wdmam'."
)
dataset_prefix = magnetic_anomaly_sources[data_source]
dataset_name = "earth_wdmam" if data_source == "wdmam" else "earth_magnetic_anomaly"
dataset_name = "earth_wdmam" if data_source == "wdmam" else "earth_mag"
grid = _load_remote_dataset(
dataset_name=dataset_name,
dataset_prefix=dataset_prefix,
Expand Down
4 changes: 2 additions & 2 deletions pygmt/datasets/earth_mask.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ def load_earth_mask(
registration: Literal["gridline", "pixel"] = "gridline",
):
r"""
Load the GSHHG Earth Mask dataset in various resolutions.
Load the GSHHG Earth mask dataset in various resolutions.

.. figure:: https://www.generic-mapping-tools.org/remote-datasets/_images/GMT_earth_mask.jpg
:width: 80 %
:align: center

GSHHG Earth Mask dataset.
GSHHG Earth mask dataset.

The grids are downloaded to a user data directory
(usually ``~/.gmt/server/earth/earth_mask/``) the first time you invoke
Expand Down
9 changes: 7 additions & 2 deletions pygmt/datasets/earth_relief.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,9 +161,14 @@ def load_earth_relief(
)
else:
dataset_prefix = earth_relief_sources[data_source]

# Choose earth relief dataset
match data_source:
case "igpp" | "synbath":
dataset_name = "earth_igpp"
case "gebco" | "gebcosi":
dataset_name = "earth_gebco"
grid = _load_remote_dataset(
dataset_name="earth_relief",
dataset_name=dataset_name,
dataset_prefix=dataset_prefix,
resolution=resolution,
region=region,
Expand Down
6 changes: 3 additions & 3 deletions pygmt/datasets/earth_vertical_gravity_gradient.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
Function to download the IGPP Earth Vertical Gravity Gradient dataset from the GMT data
Function to download the IGPP Earth vertical gravity gradient dataset from the GMT data
server, and load as :class:`xarray.DataArray`.

The grids are available in various resolutions.
Expand All @@ -20,13 +20,13 @@ def load_earth_vertical_gravity_gradient(
registration: Literal["gridline", "pixel", None] = None,
):
r"""
Load the IGPP Earth Vertical Gravity Gradient dataset in various resolutions.
Load the IGPP Earth vertical gravity gradient dataset in various resolutions.

.. figure:: https://www.generic-mapping-tools.org/remote-datasets/_images/GMT_earth_vgg.jpg
:width: 80 %
:align: center

IGPP Earth Vertical Gravity Gradient dataset.
IGPP Earth vertical gravity gradient dataset.

The grids are downloaded to a user data directory
(usually ``~/.gmt/server/earth/earth_vgg/``) the first time you invoke
Expand Down
143 changes: 66 additions & 77 deletions pygmt/datasets/load_remote_dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,33 +41,25 @@ class GMTRemoteDataset(NamedTuple):

Attributes
----------
title : str
The title of the dataset, used in error messages.
name : str
The name assigned as an attribute to the DataArray.
long_name : str
The long name assigned as an attribute to the DataArray.
units : str
description : str
The name assigned as an attribute to the DataArray.
units : str, None
The units of the values in the DataArray.
resolutions : dict
Dictionary of available resolution as keys and Resolution objects as values.
extra_attributes : dict
A dictionary of extra or unique attributes of the dataset.
"""

title: str
name: str
long_name: str
description: str
units: str | None
resolutions: dict[str, Resolution]
extra_attributes: dict


datasets = {
"earth_age": GMTRemoteDataset(
title="seafloor age",
name="seafloor_age",
long_name="age of seafloor crust",
description="EarthByte Earth seafloor crustal age",
units="Myr",
extra_attributes={"horizontal_datum": "WGS84"},
resolutions={
Expand All @@ -84,10 +76,8 @@ class GMTRemoteDataset(NamedTuple):
"01m": Resolution("01m", registrations=["gridline"], tiled=True),
},
),
"earth_free_air_anomaly": GMTRemoteDataset(
title="free air anomaly",
name="free_air_anomaly",
long_name="IGPP Earth Free-Air Anomaly",
"earth_faa": GMTRemoteDataset(
description="IGPP Earth free-air anomaly",
units="mGal",
extra_attributes={"horizontal_datum": "WGS84"},
resolutions={
Expand All @@ -104,10 +94,30 @@ class GMTRemoteDataset(NamedTuple):
"01m": Resolution("01m", registrations=["pixel"], tiled=True),
},
),
"earth_gebco": GMTRemoteDataset(
description="GEBCO Earth relief",
units="meters",
extra_attributes={"vertical_datum": "EGM96", "horizontal_datum": "WGS84"},
resolutions={
"01d": Resolution("01d"),
"30m": Resolution("30m"),
"20m": Resolution("20m"),
"15m": Resolution("15m"),
"10m": Resolution("10m"),
"06m": Resolution("06m"),
"05m": Resolution("05m", tiled=True),
"04m": Resolution("04m", tiled=True),
"03m": Resolution("03m", tiled=True),
"02m": Resolution("02m", tiled=True),
"01m": Resolution("01m", tiled=True),
"30s": Resolution("30s", tiled=True),
"15s": Resolution("15s", registrations=["pixel"], tiled=True),
"03s": Resolution("03s", registrations=["gridline"], tiled=True),
"01s": Resolution("01s", registrations=["gridline"], tiled=True),
},
),
"earth_geoid": GMTRemoteDataset(
title="Earth geoid",
name="earth_geoid",
long_name="EGM2008 Earth Geoid",
description="EGM2008 Earth geoid",
units="m",
extra_attributes={"horizontal_datum": "WGS84"},
resolutions={
Expand All @@ -124,10 +134,30 @@ class GMTRemoteDataset(NamedTuple):
"01m": Resolution("01m", registrations=["gridline"], tiled=True),
},
),
"earth_magnetic_anomaly": GMTRemoteDataset(
title="Earth magnetic anomaly",
name="magnetic_anomaly",
long_name="Earth magnetic anomaly",
"earth_igpp": GMTRemoteDataset(
description="IGPP Earth relief",
units="meters",
extra_attributes={"vertical_datum": "EGM96", "horizontal_datum": "WGS84"},
resolutions={
"01d": Resolution("01d"),
"30m": Resolution("30m"),
"20m": Resolution("20m"),
"15m": Resolution("15m"),
"10m": Resolution("10m"),
"06m": Resolution("06m"),
"05m": Resolution("05m", tiled=True),
"04m": Resolution("04m", tiled=True),
"03m": Resolution("03m", tiled=True),
"02m": Resolution("02m", tiled=True),
"01m": Resolution("01m", tiled=True),
"30s": Resolution("30s", tiled=True),
"15s": Resolution("15s", registrations=["pixel"], tiled=True),
"03s": Resolution("03s", registrations=["gridline"], tiled=True),
"01s": Resolution("01s", registrations=["gridline"], tiled=True),
},
),
"earth_mag": GMTRemoteDataset(
description="EMAG2 Earth Magnetic Anomaly Model",
units="nT",
extra_attributes={"horizontal_datum": "WGS84"},
resolutions={
Expand All @@ -144,9 +174,7 @@ class GMTRemoteDataset(NamedTuple):
},
),
"earth_mask": GMTRemoteDataset(
title="Earth mask",
name="earth_mask",
long_name="Mask of land and water features",
description="GSHHG Earth mask",
units=None,
extra_attributes={"horizontal_datum": "WGS84"},
resolutions={
Expand All @@ -165,34 +193,8 @@ class GMTRemoteDataset(NamedTuple):
"15s": Resolution("15s"),
},
),
"earth_relief": GMTRemoteDataset(
title="Earth relief",
name="elevation",
long_name="Earth elevation relative to the geoid",
units="meters",
extra_attributes={"vertical_datum": "EGM96", "horizontal_datum": "WGS84"},
resolutions={
"01d": Resolution("01d"),
"30m": Resolution("30m"),
"20m": Resolution("20m"),
"15m": Resolution("15m"),
"10m": Resolution("10m"),
"06m": Resolution("06m"),
"05m": Resolution("05m", tiled=True),
"04m": Resolution("04m", tiled=True),
"03m": Resolution("03m", tiled=True),
"02m": Resolution("02m", tiled=True),
"01m": Resolution("01m", tiled=True),
"30s": Resolution("30s", tiled=True),
"15s": Resolution("15s", registrations=["pixel"], tiled=True),
"03s": Resolution("03s", registrations=["gridline"], tiled=True),
"01s": Resolution("01s", registrations=["gridline"], tiled=True),
},
),
"earth_vgg": GMTRemoteDataset(
title="Earth vertical gravity gradient",
name="earth_vgg",
long_name="IGPP Earth Vertical Gravity Gradient",
description="IGPP Earth vertical gravity gradient",
units="Eotvos",
extra_attributes={"horizontal_datum": "WGS84"},
resolutions={
Expand All @@ -210,9 +212,7 @@ class GMTRemoteDataset(NamedTuple):
},
),
"earth_wdmam": GMTRemoteDataset(
title="WDMAM magnetic anomaly",
name="wdmam",
long_name="World Digital Magnetic Anomaly Map",
description="WDMAM World Digital Magnetic Anomaly Map",
units="nT",
extra_attributes={"horizontal_datum": "WGS84"},
resolutions={
Expand All @@ -228,9 +228,7 @@ class GMTRemoteDataset(NamedTuple):
},
),
"mars_relief": GMTRemoteDataset(
title="Mars relief",
name="mars_relief",
long_name="NASA Mars (MOLA) relief",
description="NASA Mars (MOLA) relief",
units="meters",
extra_attributes={},
resolutions={
Expand All @@ -251,9 +249,7 @@ class GMTRemoteDataset(NamedTuple):
},
),
"moon_relief": GMTRemoteDataset(
title="Moon relief",
name="moon_relief",
long_name="USGS Moon (LOLA) relief",
description="USGS Moon (LOLA) relief",
units="meters",
extra_attributes={},
resolutions={
Expand All @@ -274,9 +270,7 @@ class GMTRemoteDataset(NamedTuple):
},
),
"mercury_relief": GMTRemoteDataset(
title="Mercury relief",
name="mercury_relief",
long_name="USGS Mercury relief",
description="USGS Mercury relief",
units="meters",
extra_attributes={},
resolutions={
Expand All @@ -295,9 +289,7 @@ class GMTRemoteDataset(NamedTuple):
},
),
"pluto_relief": GMTRemoteDataset(
title="Pluto relief",
name="pluto_relief",
long_name="USGS Pluto relief",
description="USGS Pluto relief",
units="meters",
extra_attributes={},
resolutions={
Expand All @@ -316,9 +308,7 @@ class GMTRemoteDataset(NamedTuple):
},
),
"venus_relief": GMTRemoteDataset(
title="Venus relief",
name="venus_relief",
long_name="NASA Magellan Venus relief",
description="NASA Magellan Venus relief",
units="meters",
extra_attributes={},
resolutions={
Expand Down Expand Up @@ -388,7 +378,7 @@ def _load_remote_dataset(
# Check resolution
if resolution not in dataset.resolutions:
raise GMTInvalidInput(
f"Invalid resolution '{resolution}' for {dataset.title} dataset. "
f"Invalid resolution '{resolution}' for {dataset.description} dataset. "
f"Available resolutions are: {', '.join(dataset.resolutions)}."
)
resinfo = dataset.resolutions[resolution]
Expand All @@ -401,7 +391,7 @@ def _load_remote_dataset(
if registration not in resinfo.registrations:
raise GMTInvalidInput(
f"{registration} registration is not available for the "
f"{resolution} {dataset.title} dataset. Only "
f"{resolution} {dataset.description} dataset. Only "
f"{resinfo.registrations[0]} registration is available."
)
else:
Expand All @@ -414,7 +404,7 @@ def _load_remote_dataset(
fname = f"@{dataset_prefix}{resolution}_{registration[0]}"
if resinfo.tiled and region is None:
raise GMTInvalidInput(
f"'region' is required for {dataset.title} resolution '{resolution}'."
f"'region' is required for {dataset.description} resolution '{resolution}'."
)

# Currently, only grids are supported. Will support images in the future.
Expand All @@ -434,8 +424,7 @@ def _load_remote_dataset(
grid.encoding["source"] = source

# Add some metadata to the grid
grid.name = dataset.name
grid.attrs["long_name"] = dataset.long_name
grid.attrs["description"] = dataset.description
if dataset.units:
grid.attrs["units"] = dataset.units
for key, value in dataset.extra_attributes.items():
Expand Down
Loading