Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harmonize CRS in the workflow #356

Merged
merged 27 commits into from
May 31, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5c30b19
Add config options for crs
davide-f May 30, 2022
7414294
Revise download_osm_data to account for desired crs
davide-f May 30, 2022
3953f3d
Revise with desired crs clean_osm_data
davide-f May 30, 2022
f8a760d
Revise build_shapes with desired crs
davide-f May 30, 2022
d58bfd2
Revise build_renewable_profiles crs
davide-f May 30, 2022
69ca2ca
Revise base_network crs
davide-f May 30, 2022
3f48b9a
Dynamic specification of crs base_network
davide-f May 30, 2022
5cc9cbc
Dynamic crs specification for build_natura_raster
davide-f May 30, 2022
774d449
Revise crs and clean build_osm_network
davide-f May 30, 2022
83bc3de
Revise crs specs in download_osm_data
davide-f May 30, 2022
15108b0
Fix crs check build_natura_raster
davide-f May 30, 2022
7a78b9a
Remove deprecation of snapshots
davide-f May 30, 2022
765d4a8
Fix base_network crs specification
davide-f May 30, 2022
726bde9
Revise renewable_profiles
davide-f May 30, 2022
157ba60
Improve description
davide-f May 30, 2022
2a6b956
Fix refuse missing crs specification
davide-f May 30, 2022
c97764d
Add release_notes
davide-f May 30, 2022
c8a4cdb
Revise crs and description
davide-f May 30, 2022
b2c4c10
Revise name and add area_crs
davide-f May 30, 2022
568375d
Revise natura raster crs
davide-f May 30, 2022
139732b
Improve crs description natura raster
davide-f May 30, 2022
d03acd3
Revise naming build_osm_network
davide-f May 30, 2022
55bb1c3
Revise build renewable profiles
davide-f May 30, 2022
8c3790c
Revise crs build_shapes
davide-f May 30, 2022
e67c7cc
Revise crs clean osm data
davide-f May 30, 2022
fd4da58
Revise crs names download osm data
davide-f May 30, 2022
83c04fa
Merge branch 'main' into verify_epsg
davide-f May 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion config.default.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ summary_dir: results
snapshots:
start: "2013-01-01"
end: "2014-01-01"
closed: "left" # end is not inclusive
inclusive: "left" # end is not inclusive

enable:
retrieve_databundle: true # Recommended 'true', for the first run. Otherwise data might be missing.
Expand All @@ -35,6 +35,12 @@ enable:
# requires cds API key https://cds.climate.copernicus.eu/api-how-to
# More information https://atlite.readthedocs.io/en/latest/introduction.html#datasets

# definition of the Coordinate Reference Systems
crs:
geo_crs: EPSG:4326 # general geographic projection, not used for metric measures. "EPSG:4326" is the standard used by OSM and google maps
distance_crs: EPSG:3857 # projection for distance measurements only. Possible recommended values are "EPSG:3857" (used by OSM and Google Maps)
area_crs: ESRI:54009 # projection for area measurements only. Possible recommended values are Global Mollweide "ESRI:54009"

# download_osm_data_nprocesses: 10 # (optional) number of threads used to download osm data

augmented_line_connection:
Expand Down
8 changes: 7 additions & 1 deletion config.tutorial.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ summary_dir: results
snapshots:
start: "2013-03-1"
end: "2013-03-14"
closed: "left" # end is not inclusive
inclusive: "left" # end is not inclusive

enable:
# prepare_links_p_nom: false
Expand All @@ -91,6 +91,12 @@ enable:
build_cutout: false
build_natura_raster: true # If True, then build_natura_raster can be run

# definition of the Coordinate Reference Systems
crs:
geo_crs: EPSG:4326 # general geographic projection, not used for metric measures. "EPSG:4326" is the standard used by OSM and google maps
distance_crs: EPSG:3857 # projection for distance measurements only. Possible recommended values are "EPSG:3857" (used by OSM and Google Maps)
area_crs: ESRI:54009 # projection for area measurements only. Possible recommended values are Global Mollweide "ESRI:54009"

electricity:
voltages: [220., 300., 380.]
co2limit: 7.75e+7 # 0.05 * 3.1e9*0.5
Expand Down
2 changes: 2 additions & 0 deletions doc/release_notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ Upcoming Release

* Add latitude_optimal to get optimal solar orientation by default `Commit 1b2466b <https://github.com/pypsa-meets-africa/pypsa-africa/commit/de7d32be8807e4fc42486a60184f45680612fd46>`_

* Harmonize CRSs by options `PR #356 <https://github.com/pypsa-meets-africa/pypsa-africa/pull/356>`_


PyPSA-Africa 0.0.2 (6th April 2022)
=====================================
Expand Down
12 changes: 5 additions & 7 deletions scripts/base_network.py
Original file line number Diff line number Diff line change
Expand Up @@ -214,20 +214,18 @@ def _set_countries_and_substations(n):
buses = n.buses

countries = snakemake.config["countries"]
country_shapes = (
gpd.read_file(snakemake.input.country_shapes)
.set_index("name")["geometry"]
.set_crs(4326)
)
country_shapes = gpd.read_file(snakemake.input.country_shapes).set_index("name")[
"geometry"
]
offshore_shapes = unary_union(
gpd.read_file(snakemake.input.offshore_shapes)["geometry"].set_crs(4326)
gpd.read_file(snakemake.input.offshore_shapes)["geometry"]
)

bus_locations = buses
bus_locations = gpd.GeoDataFrame(
bus_locations,
geometry=gpd.points_from_xy(bus_locations.x, bus_locations.y),
crs=4326,
crs=country_shapes.crs, # the workflow sets the the same crs for buses and shapes
)
# Check if bus is in shape
offshore_b = bus_locations.within(offshore_shapes)
Expand Down
29 changes: 16 additions & 13 deletions scripts/build_natura_raster.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@
_logger = logging.getLogger(__name__)
_logger.setLevel(logging.INFO)

CUTOUT_CRS = "EPSG:4326"


def get_fileshapes(list_paths, accepted_formats=(".shp",)):
"Function to parse the list of paths to include shapes included in folders, if any"
Expand All @@ -88,7 +90,7 @@ def determine_cutout_xXyY(cutout_name, out_logging):
if out_logging:
_logger.info("Stage 1/5: Determine cutout boundaries")
cutout = atlite.Cutout(cutout_name)
assert cutout.crs.to_epsg() == 4326
assert cutout.crs == CUTOUT_CRS
x, X, y, Y = cutout.extent
dx, dy = cutout.dx, cutout.dy
return [x - dx / 2.0, X + dx / 2.0, y - dy / 2.0, Y + dy / 2.0]
Expand All @@ -104,7 +106,7 @@ def get_transform_and_shape(bounds, res, out_logging):
return transform, shape


def unify_protected_shape_areas(inputs, out_logging):
def unify_protected_shape_areas(inputs, area_crs, out_logging):
"""
Iterates thorugh all snakemake rule inputs and unifies shapefiles (.shp) only.

Expand All @@ -113,7 +115,7 @@ def unify_protected_shape_areas(inputs, out_logging):

Returns
-------
unified_shape : GeoDataFrame with a unified "multishape" (crs=3035)
unified_shape : GeoDataFrame with a unified "multishape"

"""
import pandas as pd
Expand All @@ -133,16 +135,14 @@ def unify_protected_shape_areas(inputs, out_logging):
for i in shp_files:
shape = gpd.GeoDataFrame(
pd.concat([gpd.read_file(i) for i in shp_files])
).to_crs(3035)
).to_crs(area_crs)

# Removes shapely geometry with null values. Returns geoseries.
shape = shape["geometry"][shape["geometry"].is_valid]

# Create Geodataframe with crs(3035)
shape = gpd.GeoDataFrame(shape)
shape = shape.rename(columns={0: "geometry"}).set_geometry(
"geometry"
) # .set_crs(3035)
# Create Geodataframe with crs
shape = gpd.GeoDataFrame(shape, crs=area_crs)
shape = shape.rename(columns={0: "geometry"}).set_geometry("geometry")

# Unary_union makes out of i.e. 1000 shapes -> 1 unified shape
if out_logging:
Expand All @@ -152,7 +152,7 @@ def unify_protected_shape_areas(inputs, out_logging):
_logger.info(
"Stage 3/5: Unify protected shape area. Step 3: Set geometry of unified shape"
)
unified_shape = gpd.GeoDataFrame(geometry=[unified_shape_file]).set_crs(3035)
unified_shape = gpd.GeoDataFrame(geometry=[unified_shape_file], crs=area_crs)

return unified_shape

Expand All @@ -165,19 +165,22 @@ def unify_protected_shape_areas(inputs, out_logging):
snakemake = mock_snakemake("build_natura_raster")
configure_logging(snakemake)

# get crs
area_crs = snakemake.config["crs"]["area_crs"]

out_logging = True
inputs = snakemake.input
cutouts = inputs.cutouts
shapefiles = get_fileshapes(inputs)
xs, Xs, ys, Ys = zip(
*(determine_cutout_xXyY(cutout, out_logging=out_logging) for cutout in cutouts)
)
bounds = transform_bounds(4326, 3035, min(xs), min(ys), max(Xs), max(Ys))
bounds = transform_bounds(CUTOUT_CRS, area_crs, min(xs), min(ys), max(Xs), max(Ys))
transform, out_shape = get_transform_and_shape(
bounds, res=100, out_logging=out_logging
)
# adjusted boundaries
shapes = unify_protected_shape_areas(shapefiles, out_logging=out_logging)
shapes = unify_protected_shape_areas(shapefiles, area_crs, out_logging=out_logging)

if out_logging:
_logger.info("Stage 4/5: Mask geometry")
Expand All @@ -193,7 +196,7 @@ def unify_protected_shape_areas(inputs, out_logging):
dtype=rio.uint8,
count=1,
transform=transform,
crs=3035,
crs=area_crs,
compress="lzw",
width=raster.shape[1],
height=raster.shape[0],
Expand Down
Loading