Skip to content

Commit

Permalink
Merge pull request #694 from asolavi/doc_config
Browse files Browse the repository at this point in the history
* Bring top-level configuration options together at the top of the configuration file.

* Bring top-level configuration options together at the top of the tutorial configuration file.

* Top-level configuration literalinclude displayed using more robust start-at end-before Sphinx commands

* Remove depreciated options summary_dir, focus_weights, retrieve_cutout, retrieve_natura_raster, custom_busmap from top level configuration table

* Add download_osm_data on top level configuration table, and organise the table to match config.default.yaml.

* Add csv table in section run.

* Minor adjustments in the scenario section.

* Future-proof by using end-at in literalinclude instead of end-before.

* snapshot section. Improve description of available years.

* Add crs (coordinate reference system) section in the docs.

* Add augmented_line_connection section.

* Replace true/false for True/False in the table.

* Replace true/false for True/False in run.csv.

* Add cluster_options section in the configuration documentation.

* Remove space between the version and tutorial section in the config.yaml files.

* Add documentation for build_shape_options section in the configuration file.

* Add section about clean_osm_data_options in the configuration documentation.

* Add sections on build_osm_network and base_network in the configuration documentation.

* Fixing typo in base_network section of configuration documentation.

* Add load_options section in the configuration documentation.

* Update and minor fixes in electricity section of the configuration documentation.

* Fixing sections in cluster_options table in configuration documentation.

* Fixing sections in build_shape_options table in configuration documentation.

* Fixing sections in clean_osm_data_options table in configuration documentation.

* Fixing sections in build_osm_network table in configuration documentation.

* Fixing sections in base_newtwork table in configuration documentation.

* Fixing sections in load_options table in configuration documentation.

* Improve description of section cluster_options in configuration documentation.

* Update lines section in the configuration docs.

* Add description on electricity section.

* Description details for commitable cluster_options in configuration documentation.

* Improve description in load_options section of configuration documentation.

* Improve Links documentation in the configuration section.

* Improvements in the renewable section of the configuration documentation.

* Small improvement in transformer section of configuration documentation.

* Remove load section from configuration documentation. It seems to have been replaced by load_options and others.

* Improve cost section of the configuration documentation.

* Remove clustering section in configuration documentation. It got renamed to cluster_options.

* Add description in monte_carlo section of configuration documentation.

* Add description in solving section of configuration documentation.

* Add description of plotting section in configuration documentation.

* Improvements in the cluster_options section of the documentation.

* Improve renewable section of configuration documentation. Description of the extendable feature.

* Improvements in renewable section of configuration documentation. Add descriptions for hydro resource methods.

* change theme, fix some warning

* reintroduce setuptools as pypsa-eur docs is failing

* Fix old typos
---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Max Parzen <[email protected]>
Co-authored-by: ekatef <[email protected]>
  • Loading branch information
4 people authored May 6, 2023
1 parent bf3cbea commit ac5ed2d
Show file tree
Hide file tree
Showing 48 changed files with 497 additions and 351 deletions.
31 changes: 16 additions & 15 deletions config.default.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,19 @@ logging:
level: INFO
format: "%(levelname)s:%(name)s:%(message)s"

countries: ["NG", "BJ"]
# Can be replaced by country ["NG", "BJ"], continent ["Africa"] or user-specific region, see more at https://pypsa-earth.readthedocs.io/en/latest/configuration.html#top-level-configuration

enable:
retrieve_databundle: true # Recommended 'true', for the first run. Otherwise data might be missing.
retrieve_cost_data: true # true: retrieves cost data from technology data and saves in resources/costs.csv, false: uses cost data in data/costs.csv
download_osm_data: true # If 'true', OpenStreetMap data will be downloaded for the above given countries
build_natura_raster: false # If True, than an exclusion raster will be build
build_cutout: false
# If "build_cutout" : true, then environmental data is extracted according to `snapshots` date range and `countries`
# requires cds API key https://cds.climate.copernicus.eu/api-how-to
# More information https://atlite.readthedocs.io/en/latest/introduction.html#datasets

run:
name: "" # use this to keep track of runs with different settings
shared_cutouts: true # set to true to share the default cutout(s) across runs
Expand All @@ -20,24 +33,11 @@ scenario:
clusters: [10]
opts: [Co2L-3H]

countries: ["NG", "BJ"]
# Can be replaced by country ["NG", "BJ"], continent ["Africa"] or user-specific region, see more at https://pypsa-earth.readthedocs.io/en/latest/configuration.html#top-level-configuration

snapshots:
start: "2013-01-01"
end: "2014-01-01"
inclusive: "left" # end is not inclusive

enable:
retrieve_databundle: true # Recommended 'true', for the first run. Otherwise data might be missing.
retrieve_cost_data: true # true: retrieves cost data from technology data and saves in resources/costs.csv, false: uses cost data in data/costs.csv
download_osm_data: true # If 'true', OpenStreetMap data will be downloaded for the above given countries
build_natura_raster: false # If True, than an exclusion raster will be build
build_cutout: false
# If "build_cutout" : true, then environmental data is extracted according to `snapshots` date range and `countries`
# requires cds API key https://cds.climate.copernicus.eu/api-how-to
# More information https://atlite.readthedocs.io/en/latest/introduction.html#datasets

# definition of the Coordinate Reference Systems
crs:
geo_crs: EPSG:4326 # general geographic projection, not used for metric measures. "EPSG:4326" is the standard used by OSM and google maps
Expand All @@ -62,7 +62,7 @@ cluster_options:
remove_stubs: true
remove_stubs_across_borders: true
p_threshold_drop_isolated: 20 # [MW] isolated buses are being discarded if bus mean power is below the specified threshold
p_threshold_merge_isolated: 300 # [MW] isolated buses are being merged into a single isolated bus if bus mean power is above the specified threshold
p_threshold_merge_isolated: 300 # [MW] isolated buses are being merged into a single isolated bus if bus mean power is below the specified threshold
cluster_network:
algorithm: kmeans
feature: solar+onwind-time
Expand Down Expand Up @@ -297,7 +297,8 @@ renewable:
year: 2013 # (optional) year of statistics used to rescale the runoff time series. When not provided, the weather year of the snapshots is used
multiplier: 1.1 # multiplier applied after the normalization of the hydro production; default 1.0

# TODO: Needs to be adjusted for Africa
# TODO: Needs to be adjusted for Africa.
# Costs Configuration (Do not remove, needed for Sphynx documentation).
costs:
year: 2030
version: v0.5.0
Expand Down
34 changes: 17 additions & 17 deletions config.tutorial.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,6 @@ logging:
level: INFO
format: "%(levelname)s:%(name)s:%(message)s"

run:
name: ""

scenario:
simpl: ['']
ll: ['copt']
clusters: [6]
opts: [Co2L-4H]

countries: ["NG", "BJ"]

# ['DZ', 'AO', 'BJ', 'BW', 'BF', 'BI', 'CM', 'CF', 'TD', 'CG', 'CD',
Expand All @@ -31,13 +22,6 @@ countries: ["NG", "BJ"]
#["MA"] # Morroco
#["ZA"] # South Africa

summary_dir: results

snapshots:
start: "2013-03-1"
end: "2013-03-7"
inclusive: "left" # end is not inclusive

enable:
# prepare_links_p_nom: false
retrieve_databundle: true
Expand All @@ -48,6 +32,22 @@ enable:
build_cutout: false
build_natura_raster: true # If True, then build_natura_raster can be run

run:
name: ""

scenario:
simpl: ['']
ll: ['copt']
clusters: [6]
opts: [Co2L-4H]

summary_dir: results

snapshots:
start: "2013-03-1"
end: "2013-03-7"
inclusive: "left" # end is not inclusive

# definition of the Coordinate Reference Systems
crs:
geo_crs: EPSG:4326 # general geographic projection, not used for metric measures. "EPSG:4326" is the standard used by OSM and google maps
Expand Down Expand Up @@ -75,7 +75,7 @@ cluster_options:
remove_stubs: true
remove_stubs_across_borders: true
p_threshold_drop_isolated: 20 # [MW] isolated buses are being discarded if bus mean power is below the specified threshold
p_threshold_merge_isolated: 300 # [MW] isolated buses are being merged into a single isolated bus if bus mean power is above the specified threshold
p_threshold_merge_isolated: 300 # [MW] isolated buses are being merged into a single isolated bus if bus mean power is below the specified threshold
cluster_network:
algorithm: kmeans
feature: solar+onwind-time
Expand Down
34 changes: 29 additions & 5 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,16 @@
# The full version, including alpha/beta/rc tags
release = "0.1.0"

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "sphinx"

# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named "sphinx.ext.*") or your custom
# ones.
extensions = [
"myst_parser",
"sphinx.ext.autodoc",
"sphinx.ext.autosummary",
"sphinx.ext.intersphinx",
Expand Down Expand Up @@ -125,12 +129,32 @@
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = "sphinx_rtd_theme"
html_theme = "sphinx_book_theme"

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
"repository_url": "https://github.com/pypsa-meets-earth/pypsa-earth",
"use_repository_button": True,
"show_navbar_depth": 1,
}

# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
html_title = "PyPSA-Earth"

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"]
# html_static_path = ["_static"]

# A shorter title for the navigation bar. Default is the same as html_title.
html_short_title = "PyPSA-Earth"

# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = "https://github.com/pypsa-meets-earth/pypsa-meets-earth.github.io/raw/main/assets/img/logo.png"

# -- Options for Texinfo output -------------------------------------------

Expand All @@ -140,10 +164,10 @@
texinfo_documents = [
(
master_doc,
"pypsa_earth",
"pypsa-earth Documentation",
"PyPSA-Earth",
"PyPSA-Earth Documentation",
author,
"pypsa-earth",
"PyPSA-Earth",
"One line description of project.",
"Miscellaneous",
),
Expand Down
6 changes: 6 additions & 0 deletions doc/configtables/augmented_line_connection.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
,Unit,Values,Description
add_to_snakefile, bool, "{True, False}", "True: includes this rule to the workflow."
connectivity_upgrade, int, "{1, 2, 3, ...}", "Number k such that the network graph is k-edge-connected."
new_line_type,, "{[""HVAC""], [""HVDC""], [""HVAC"", ""HVDC""]}", "Type of expanded lines."
min_expansion, int or float,, "[MW] New created line capacity."
min_DC_length, int or float,, "[km] Minimum line length of HVDC line."
3 changes: 3 additions & 0 deletions doc/configtables/base_network.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
,Unit,Values,Description
min_voltage_substation_offshore, V,, "Minimum voltage magnitude in offshore substations."
min_voltage_rebase_voltage, V,, "Minimum voltage magnitude in base network."
6 changes: 6 additions & 0 deletions doc/configtables/build_osm_network.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
,Unit,Values,Description
group_close_buses, bool, "{True, False}", "True: close buses are merged and guarantee the voltage matching among line endings."
group_tolerance_buses, m,, "Tolerance in meters of the close buses to merge."
split_overpassing_lines, bool, "{True, False}", "True: lines overpassing buses are splitted and connected to the buses."
overpassing_lines_tolerance, m,, "Tolerance to identify lines overpassing buses."
force_ac, bool, "{True, False}", "True: forces all components (lines and substation) to be AC-only. To be used if DC assets create problems."
10 changes: 10 additions & 0 deletions doc/configtables/build_shape_options.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
,Unit,Values,Description
gadm_layer_id,, "{0, 1, 2}", "GADM level area used for the gadm_shapes. Codes are country-dependent but roughly: 0: country, 1: region/county-like, 2: municipality-like."
update_file, bool, "{True, False}", "True: all input files are downloaded again and replace the existing files."
out_logging, bool, "{True, False}", "True: Logging is printed in the console."
year,, "past year; e.g. YYYY", "Reference year used to derive shapes, info on population and info on GDP."
nprocesses, int,, "Number of processes to be used in build_shapes."
nchunks, int,, "Number of data chuncks for build_shapes. If not specified or smaller than nprocesses, specified as nprocesses."
worldpop_method,, "{""standard"", ""api"", false}", "Specifies how population is added to every shape: ""standard"" pulls from web 1kmx1km raster; ""api"" pulls from API 100mx100m raster; false (not ""false"") no population addition to shape. This is useful when generating only cutout."
gdp_method,, "{""standard"", false}", "Specifies how GDP is added to every shape: ""standard"" pulls from web 1x1km raster; false (not ""false"") no gdp addition to shape. This is useful when generating only cutout."
contended_flag,, "{""set_by_country"", ""drop""}", "Specifies what to do with contended countries: ""set_by_country"" assigns the contended areas to the countries according to the GADM database; ""drop"" drops the contended areas from the model."
6 changes: 6 additions & 0 deletions doc/configtables/clean_osm_data_options.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
,Unit,Values,Description
names_by_shapes, bool, "{True, False}", "True: the country name is set based on the extended country shapes."
threshold_voltage, V,, "Assets below the voltage threshold will not be considered (cable, line, generator, etc.)."
tag_substation,, "{""transmission"", ""distribution""}", "Filters only substations with the corresponding tag (""transmission"" or ""distribution"")."
add_line_endings, bool, "{True, False}", "True: line endings are added to the dataset of the substations."
generator_name_method,, "{OSM, closest_city}", "Methodology to specify the name of generators. From OpenStreetMap (OSM) or named after the closest city (closest_city)."
28 changes: 28 additions & 0 deletions doc/configtables/cluster_options.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
,Unit,Values,Description
simplify_network,,,
-- to_substations, bool, "{True, False}", "False: network is simplified to nodes with positive or negative power injection (i.e. substations or offwind connections)."
-- algorithm,, "{hac, kmeans, modularity}", "Clustering algorithm used in the simplify_network rule. Options available are Hierarchical Agglomerative Clustering (HAC), k-means, or greedy modularity."
-- feature,, "Str in the format ‘carrier1+carrier2+...+carrierN-X’, where CarrierI can be from {‘solar’, ‘onwind’, ‘offwind’, ‘ror’} and X is one of {‘cap’, ‘time’}. Examples: solar+offwind-cap, solar-time", "Only for Hierarchical Agglomerative Clustering (HAC). Feature(s) used to do the clustering."
-- exclude_carriers,, "List of Str like [ 'solar', 'onwind'] or empy list []", "Carriers not considered in the simplify_network rule. Can be any set of carriers (conventional or renewable)."
-- remove_stubs, bool, "{True, False}", "True: Stub lines and links, i.e. dead-ends of the network, are sequentially removed from the network."
-- remove_stubs_across_borders, bool, "{True, False}", "True: Stub lines and links can be removed across borders."
-- p_threshold_drop_isolated, MW, positive number, "Isolated buses are discarded if bus mean power is below the `p_threshold_drop_isolated`."
-- p_threshold_merge_isolated, MW, positive number, "Isolated buses are merged into a single isolated bus if bus mean power is below `p_threshold_merge_isolated`."
cluster_network,,,
-- algorithm,,"{hac, kmeans}", "Clustering algorithm used in the cluster_network rule. Options available are Hierarchical Agglomerative Clustering (HAC) or k-means."
-- feature,,"Str in the format ‘carrier1+carrier2+...+carrierN-X’, where CarrierI can be from {‘solar’, ‘onwind’, ‘offwind’, ‘ror’} and X is one of {‘cap’, ‘time’}. Examples: solar+offwind-cap, solar-time", "Only for Hierarchical Agglomerative Clustering (HAC). Feature(s) used to do the clustering."
-- exclude_carriers,, "List of Str like [ 'solar', 'onwind'] or empy list []", "Carriers not considered in the cluster_network rule. Can be any set of carriers (conventional or renewable)."
alternative_clustering, bool, "{True, False}", "False: use Voronoi shapes in the clustering. True: use GADM shapes in the clustering."
distribute_cluster,, "{['load'], ['pop'], ['gdp']}", "Distributes cluster nodes per country according to load (['load']), population (['pop']) or GDP (['gdp'])."
out_logging, bool, "{True, False}", "True: Logging is printed to the console."
aggregation_strategies,,,
-- generators,,,
-- -- p_nom,, "{min, mean, max, sum}", "Indicates how the p_nom of the aggregated generator is computed from the original p_nom values. For example, if sum, then all values within each cluster are summed to represent the new generator."
-- -- p_nom_max,, "{min, mean, max, sum}", "Indicates how the p_nom_max of the aggregated generator is computed from the original p_nom_max values."
-- -- p_nom_min,, "{min, mean, max, sum}", "Indicates how the p_nom_min of the aggregated generator is computed from the original p_nom_min values."
-- -- p_min_pu,, "{min, mean, max, sum}", "Indicates how the p_min_pu of the aggregated generator is computed from the original p_min_pu values."
-- -- marginal_cost,, "{min, mean, max, sum}", "Indicates how the marginal_cost of the aggregated generator is computed from the original marginal_cost values."
-- -- commitable,,"{any}", "Indicates how the commit status of the aggregated generator is set depending on the original values of the generators. Unit Commitment is currently under development, so should be left to ``any``."
-- -- ramp_limit_up,, "{min, mean, max, sum}", "Indicates how the ramp_limit_up of the aggregated generator is computed from the original ramp_limit_up values."
-- -- ramp_limit_down,, "{min, mean, max, sum}", "Indicates how the ramp_limit_down of the aggregated generator is computed from the original ramp_limit_down values."
-- -- efficiency,, "{min, mean, max, sum}", "Indicates how the efficiency of the aggregated generator is computed from the original efficiency values."
2 changes: 1 addition & 1 deletion doc/configtables/costs.csv
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
year,--,"YYYY; e.g. '2030'","Year for which to retrieve cost assumptions of ``resources/costs.csv``."
version,--,"vX.X.X; e.g. 'v0.1.0'","Version of ``technology-data`` repository to use."
rooftop_share,--,float,"Share of rooftop PV when calculating capital cost of solar (joint rooftop and utility-scale PV)."
USD2013_to_EUR2013, --, "float", "US Dollar-Euro exchange rate. Source: `ECB <ECB: https://www.ecb.europa.eu/stats/exchange/eurofxref/html/eurofxref-graph-usd.en.html>`_."
fill_values,--,float,"Default values if not specified for a technology in ``resources/costs.csv``."
capital_cost,EUR/MW,"Keys should be in the 'technology' column of ``resources/costs.csv``. Values can be any float.","For the given technologies, assumptions about their capital investment costs are set to the corresponding value. Optional; overwrites cost assumptions from ``resources/costs.csv``."
marginal_cost,EUR/MWh,"Keys should be in the 'technology' column of ``resources/costs.csv``. Values can be any float.","For the given technologies, assumptions about their marginal operating costs are set to the corresponding value. Optional; overwrites cost assumptions from ``resources/costs.csv``."
emission_prices,,,"Specify exogenous prices for emission types listed in ``network.carriers`` to marginal costs."
-- co2,EUR/t,float,"Exogenous price of carbon-dioxide added to the marginal costs of fossil-fuelled generators according to their carbon intensity. Added through the keyword ``Ep`` in the ``{opts}`` wildcard only in the rule :mod:`prepare_network``."
4 changes: 4 additions & 0 deletions doc/configtables/crs.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
,Unit,Values,Description
geo_crs,, "General geographic projection. Not used for metric measures.", "Recommended value is ‘EPSG:4326’ (used by OSM and Google Maps)."
distance_crs,, "Projection for distance measurements only.", "Recommended value is ‘EPSG:3857’ (used by OSM and Google Maps)."
area_crs,, "Projection for area measurements only.", "Recommended value is the Global Mollweide projection ‘ESRI:54009’."
Loading

0 comments on commit ac5ed2d

Please sign in to comment.