Resolve issues related to technical_lifetime and historical vintages in Westeros tutorials

[behnam-zakeri]

This PR resolves issues related to Westeros "baseline" tutorial, including #525, #609, #813, and #814.
This PR does the following:

1. Defines vintage-activity years for each technology based on technical_lifetime of that technology (addressing Fix handling of activity years and life time of technologies in the tutorials #525)
2. Defines vintage and activity for historical parameters, i.e., adding historical years to relevant parameters (addressing technical_lifetime not defined for historical vintages in Westeros tutorial #609, partially related to this comment in Add commodity flow of capacity variables #451, and partially related to Westeros tutorials inconsistent with changes made to baseline #624)
3. Corrects the units, makes sure they are consistent, and adds a note (partially addressing Units are not correct and consistent in Westeros tutorials #814).
4. Adds capacity-related parameters for the grid (resolving Electricity grid has no capacity-related parameters in Westeros tutorials #813). An investment cost of 800 USD/kW and fixed O&M cost of 16 USD/kW/a is added.

How to review

• Load Westeros baseline under this branch, read through the corrected text, and make sure it's understandable.
• Solve Westeros baseline under the "main" branch (and name it differently)
• Solve Westeros baseline under this branch and compare it with the older version. The results should show that the objective function is different, but the activity and capacity of coal and wind power plants must be the same in the model horizon.

PR checklist

• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update release notes.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.6%. Comparing base (0cc8102) to head (fde7729).
Report is 55 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #815     +/-   ##
=======================================
+ Coverage   95.4%   95.6%   +0.1%     
=======================================
  Files         46      46             
  Lines       4354    4335     -19     
=======================================
- Hits        4156    4145     -11     
+ Misses       198     190      -8     

Files	Coverage Δ
message_ix/testing/__init__.py	99.6% <100.0%> (ø)
message_ix/tests/test_feature_price_emission.py	100.0% <100.0%> (ø)
message_ix/tests/test_macro.py	100.0% <ø> (ø)
message_ix/tests/test_report.py	100.0% <100.0%> (ø)
message_ix/tests/test_soft_constraints.py	100.0% <100.0%> (ø)
message_ix/tests/test_tutorials.py	97.6% <ø> (ø)
message_ix/tests/util/test_tutorial.py	100.0% <100.0%> (ø)

... and 1 file with indirect coverage changes

[glatterf42]

Looks good to me, thanks :)
Aside from the ToDos you collected, I'd add this: Please adapt the objective value in this test line to the new one, which is 167594.640625:

message_ix/message_ix/tests/test_tutorials.py

Line 74 in f09e8e2

_t("w0", f"{W}_baseline", check=[("solve-objective-value", 173795.09375)]),

This seems to be the only reason the tests are failing now (the one on windows-latest-py3.9 is flaky and has nothing to do with this PR).

[behnam-zakeri]

Thanks @glatterf42 for the quick review. I hadn't changed the objective functions in those test because we need to add investment cost for the grid (one of the ToDos) which will change the objective function. I discussed with @OFR-IIASA this morning, and we believe it's not needed to update the investment cost of wind and coal power plants (the optional ToDo), because this tutorial is not representing the reality of today, or in fact any period in history, and just wants to showcase how the model works with two technologies with different investment cost. As such, I will leave the investment cost for wind and coal power plants as it is for the time being.

[glatterf42]

Sorry to backtrack here, but one clarifying question and one change request:

Does this PR really close #525, #609, #813, and #814?

And as mentioned by @khaeru in the issue he linked, these lines are not clear to new users because it's unclear what the keys are in the key_or_data argument. Could you please adjust this so that we use make_df() for this line, too, clarifying the keys?

[glatterf42]

Highjacking this PR to enable ruff for all tutorial files since I started migrating to make_df and was annoyed by the code not following our code style.

As for the question I raised earlier, I have now concluded from your PR description, @behnam-zakeri, that all issues except #814 are indeed closed by this PR since #814 is the only one where you say "partially addressing".
Assuming this is true, once we get the tests to pass, this PR is good to go :)

[behnam-zakeri]

Thanks @glatterf42 for the kind reminder and sorry for the delay. This PR indeed closes the issues #525, #609, #813, and #814. I correct the description to show that this resolves #814 fully and not partially.

[glatterf42]

Looks good to me once CI passes :)

[khaeru]

Thanks for the improvements here!

Applying the code style to the tutorials is a good step. In westeros_baseline.ipynb, this apparently result in a cell with code like:

# Create a Reporter object to describe and carry out reporting
# calculations and operations (like plotting) based on `scenario`
from message_ix.report import Reporter

# ← The line that was here

# "prepare_plots" enables several to describe reporting operations, e.g.
# "plot activity", "plot capacity", or "plot prices"
# See message_ix/util/tutorial.py for more information
from message_ix.util.tutorial import prepare_plots

rep = Reporter.from_scenario(scenario)  # ← …is now here

prepare_plots(rep)

This means the initial comment is no longer with the line it was meant to describe. This might have been due to ruff's automatic sorting of imports. More readable would be:

from message_ix.report import Reporter
from message_ix.util.tutorial import prepare_plots

# Create a Reporter object to describe and carry out reporting
# calculations and operations (like plotting) based on `scenario`
rep = Reporter.from_scenario(scenario)

# "prepare_plots" enables several to describe reporting operations, e.g.
# "plot activity", "plot capacity", or "plot prices"
# See message_ix/util/tutorial.py for more information
prepare_plots(rep)

Some other requested changes inline, as well as an important question.