Enhance calibration report

[fbenke-pik]

Purpose of this PR

Enhance output script reportCEScalib to include additional plot formats (credit goes to @0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q). Moved code to a Rmarkdown file.

An example of the new pdf report can be found here: /p/tmp/ricardar/remind/remind-develop/remind/output/SSP2EU-NPi-calibrate_2024-03-21_12.23.31/CES_calibration_report_SSP2EU-NPi-calibrate.pdf

Type of change

• Refactoring
• New feature
• Minor change (default scenarios show only small differences)

Checklist:

• My code follows the coding etiquette
• I performed a self-review of my own code
• I explained my changes within the PR, particularly in hard-to-understand areas
• All automated model tests pass (FAIL 0 in the output of make test)
• The changelog CHANGELOG.md has been updated correctly

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

One point: My original script does some shuffling to get the production factors in a sensible order in the plots.

pf_order <- read_lines(file = CES_cal_file, n_max = 1)
if (grepl('^# pf order:', pf_order)) {
    pf_order <- pf_order %>% 
        sub('^# pf order: ', '', .) %>% 
        strsplit(', ') %>% 
        unlist()
} else {
    pf_order <- d %>% 
        pull('pf') %>% 
        unique()
}

d <- d %>%
    order.levels(pf = pf_order)

The result is that production factors within the same CES nest are grouped together in the plot

instead of alphabetical order, which is rather meaningless

[ricardarosemann]

Thank you for this, very helpful. I still have some comments:

• Currently, the pdf contains an empty section "Line Plots" in the beginning, this should be deleted.
• The regions appear in a different order for the two sections "Line Plots" and "Heat map", which is a bit confusing. The "Line Plots" section appears in the order that Remind prescribes (when specifying the region set), while the "Heat map" section is alphabetical. I don't have any hard feelings which of these orders is better, but it should be the same throughout the document.
• I'm still not fully happy with the coloring of the line plots. I find iterations 1 and 10 a bit easier to tell apart now, but this could still improve. Moreover, both of these iterations can now be confused with the target. It might even make sense to give the last iteration (usually iteration 10) a specific color (rather than relying on a ggplot-color spectrum), as this usually is what will be used in the end. Maybe you can recreate the reporting pdf with my results in /p/tmp/ricardar/remind/adjH2-for-merge/remind/output/incorrectCalibration/SSP2EU-EU21-NPi-calibrate_2024-03-13_17.39.20? That's a calibration run with really ugly results, so it might be an interesting test case.

[fbenke-pik]

One point: My original script does some shuffling to get the production factors in a sensible order in the plots.

pf_order <- read_lines(file = CES_cal_file, n_max = 1)
if (grepl('^# pf order:', pf_order)) {
    pf_order <- pf_order %>% 
        sub('^# pf order: ', '', .) %>% 
        strsplit(', ') %>% 
        unlist()
} else {
    pf_order <- d %>% 
        pull('pf') %>% 
        unique()
}

d <- d %>%
    order.levels(pf = pf_order)

The result is that production factors within the same CES nest are grouped together in the plot instead of alphabetical order, which is rather meaningless

This should be fixed

[fbenke-pik]

Currently, the pdf contains an empty section "Line Plots" in the beginning, this should be deleted.

This is not an empty section, it is the heading for all the plots that follow. I have to introduce a page break, because the line plots in these section need all the space of the page, otherwise the formatting goes wrong and swallows a plot. So this is a know inconvenience that won't go away. The heading is there to allow jumping in the table of contents.

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

Currently, the pdf contains an empty section "Line Plots" in the beginning, this should be deleted.

This is not an empty section, it is the heading for all the plots that follow. I have to introduce a page break, because the line plots in these section need all the space of the page, otherwise the formatting goes wrong and swallows a plot. So this is a know inconvenience that won't go away. The heading is there to allow jumping in the table of contents.

You have a 2nd-level heading (## Line Plots), then load data (which does not show in the PDF) then a 3ed-level heading (### Quantity Outliers). No Point in that first heading. (And then a 3rd-level heading "Line Plots" further down, but that is not the one Ricarda is talking about.)

[fbenke-pik]

You have a 2nd-level heading (## Line Plots), then load data (which does not show in the PDF) then a 3ed-level heading (### Quantity Outliers). No Point in that first heading. (And then a 3rd-level heading "Line Plots" further down, but that is not the one Ricarda is talking about.)

I see, I adjusted the headings again.

[fbenke-pik]

The regions appear in a different order for the two sections "Line Plots" and "Heat map", which is a bit confusing. The "Line Plots" section appears in the order that Remind prescribes (when specifying the region set), while the "Heat map" section is alphabetical. I don't have any hard feelings which of these orders is better, but it should be the same throughout the document.

I adjusted the regions sorting to alphabetical order in the first set of plots as well, so they are unified now.

[fbenke-pik]

I'm still not fully happy with the coloring of the line plots. I find iterations 1 and 10 a bit easier to tell apart now, but this could still improve. Moreover, both of these iterations can now be confused with the target. It might even make sense to give the last iteration (usually iteration 10) a specific color (rather than relying on a ggplot-color spectrum), as this usually is what will be used in the end. Maybe you can recreate the reporting pdf with my results in /p/tmp/ricardar/remind/adjH2-for-merge/remind/output/incorrectCalibration/SSP2EU-EU21-NPi-calibrate_2024-03-13_17.39.20? That's a calibration run with really ugly results, so it might be an interesting test case.

The report can be found here: /p/tmp/ricardar/remind/adjH2-for-merge/remind/output/incorrectCalibration/SSP2EU-EU21-NPi-calibrate_2024-03-13_17.39.20/CES_calibration_report_SSP2EU-EU21-NPi-calibrate.pdf

Would any of these palettes fit better? (I picked one of the Diverging sections, "Purple-Green") https://colorspace.r-forge.r-project.org/reference/hcl_palettes.html

But yeah, happy to implement any specific instruction you find useful (I am lacking the user perspective here)

[ricardarosemann]

Would any of these palettes fit better? (I picked one of the Diverging sections, "Purple-Green") https://colorspace.r-forge.r-project.org/reference/hcl_palettes.html

I had a look and in terms of contrast I think I like the green-orange one best. These are the palettes with 10 different colors for reference:

[fbenke-pik]

How about this?

[ricardarosemann]

How about this?

I really like this, I think it is now very clearly visible which one is the final iteration and that (in this case) it closely matches the target.