Implement HVDC lines into the model

[ekatef]

The PR is intended to resolve #286 using the following approach proposed by @davide-f (thank you for the brilliant tutorial!)

TODOs

• Check if merge_stations_lines_by_station_id_and_voltage( ) works properly for links (build_osm_network script)
• Correct a value of the "dc" column of the buses csv (build_osm_network script)
• Include converters into the model (base_network script)
• Check assumptions on p_nom_max (base_network script)
• Fix an issue with an empty links file

Checklist

• I tested my contribution locally and it seems to work fine:
  • on regional scale
    • with DC lines (CD, NA)
    • without DC lines (NG, CD on main)
  • on the whole model
• Code and workflow changes are sufficiently documented.
• Newly introduced dependencies are added to envs/environment.yaml and envs/environment.docs.yaml.
• Changes in configuration options are added in all of config.default.yaml, config.tutorial.yaml, and test/config.test1.yaml.
• Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
• A note for the release notes doc/release_notes.rst is amended in the format of previous release notes, including reference to the requested PR.

[davide-f]

@ekatef thanks for the contribution.
I added few minor recommendations.

Before merging, I think it is appropriate if you could also share the plots of the comparison between the networks (raw osm data, clean osm_data and simplify network) with the current version for countries where AC/DC lines.
After clean osm, it would be nice to differentiate between links and lines and see what happens.
A zoom may be in handy close to a station to check how components are connected.
What do you think? Those plots could be done using the already available validation plots or also using the code you have already used for the past images

[ekatef]

@davide-f, thanks for checking!
Revised and fixed.

I love your idea regarding the visual checks. Actually, a combination of a static big-picture plot and hvplot zooms gives excellent options to explore the grid structure. I have been playing with such visualisations which was quite funny :) Then I'll repeat these tests on the the revised code and share the results. Do you think it's appropriate to attach them to a PR or would it be better to create a notebook for that?

[davide-f]

@davide-f, thanks for checking! Revised and fixed.

I love your idea regarding the visual checks. Actually, a combination of a static big-picture plot and hvplot zooms gives excellent options to explore the grid structure. I have been playing with such visualisations which was quite funny :) Then I'll repeat these tests on the the revised code and share the results. Do you think it's appropriate to attach them to a PR or would it be better to create a notebook for that?

I crosschecked the comments again.
Regarding the images, you can post them here

[ekatef]

The PR implies some changes in the network topology which makes it necessary to compare the resulted network with ones from the current main branch where DC lines are treated as AC.

Results of the graphical topology check on CD look as follows.

on raw data

Both approaches give the identical results
Blue = AC, red = DC, thickness is set according to the voltage

on clean data

Blue = AC, red = DC, thickness is set according to the voltage

original approach

All DC lines were transformed to AC ones.

tested aproach

on build_network-csv data

That's a key point where OSM data are being prepared to be further used as inputs of the PyPSA network.

Lines are in blue, buses are in red

A general outline of the grid is looks the same for both approaches

A closer look on the fragments reveals some differences (results of the original approach from the left, of the tested one -- on the right)

fragment a

fragment b

fragment c

simplified network

original approach

tested approach

final network

original approach

tested approach

[ekatef]

@davide-f, could you please have a look? I think most of the issues should be fixed now. Although, I have broken the CI (sorry for that!) but it seems to be related to some minor troubles in access to https://data.open-power-system-data.org.

[davide-f]

@davide-f, could you please have a look? I think most of the issues should be fixed now. Although, I have broken the CI (sorry for that!) but it seems to be related to some minor troubles in access to https://data.open-power-system-data.org.

Don't be sorry for the CI, it happens often, the CI works is not broken with that error :) super!

To me, the images look very fine; I think we are super close to merge this PR.

Few questions:

• in the fragments, what are the red lines? Are they the fake converters?
• Still in the fragments, it seems like there is a weird additional bus that pops up but it is not connected to anything. How do you justify that?

[ekatef]

Thank you, @davide-f :)

• in the fragments, what are the red lines? Are they the fake converters?

Yes, the converters and buses are red while both AC and DC lines are blue. I'll work to improve the colour scheme for further use as hvplot seems to provide fantastic options for grid visualisations.

• Still in the fragments, it seems like there is a weird additional bus that pops up but it is not connected to anything. How do you justify that?

I think that's a switching station at Kananga city and the bus should be connected to a distribution network. There are a few similar structures along the Inga–Shaba line.

[davide-f]

Yes, the converters and buses are red while both AC and DC lines are blue. I'll work to improve the colour scheme for further use as hvplot seems to provide fantastic options for grid visualisations.

No need, I was just asking to crosschecking, perfect! no need to have more images :)

I think that's a switching station at Kananga city and the bus should be connected to a distribution network. There are a few similar structures along the Inga–Shaba line.

Ok! My thought was why in the other approach there was no such a bus, but from the point of view of the network the network seems to keep the connections. Can you confirm that?
Then I think this PR is ready, well done!! :)

[ekatef]

Ok! My thought was why in the other approach there was no such a bus, but from the point of view of the network the network seems to keep the connections. Can you confirm that? Then I think this PR is ready, well done!! :)

In the original approach there is a bus also (close to the center of this X-shape) but it's poorly visible on the picture... Have checked connectivity manually and not found any problems. It seems that PR is really working. Thank you so much for your support :)

[davide-f]

@ekatef Great work! This PR has been merged.
As next steps, would you feel like to implement converters as Link models? This is the last major task left before v0.1 :)

[ekatef]

@davide-f, very glad to know that it works! It was an amazing problem to solve :)
Yes, I'ld be happy to implement a proper converter model. I have seen that you have drafted the strategy in #385?

[davide-f]

@davide-f, very glad to know that it works! It was an amazing problem to solve :) Yes, I'ld be happy to implement a proper converter model. I have seen that you have drafted the strategy in #385?

Yes, I'll add some comments about it :)

[ekatef]

Yes, I'll add some comments about it :)

Wonderful, thank you! Now it looks for me absolutely doable :)