Issues related to the recipe comparison tool

[schlunma]

While debugging #2900, I found several issues related to the recipe comparison tool. Most of them are related to problems with ESMValCore/ESMValTool that ultimately lead to problems when using the recipe comparison tool.

I will add these issues below, please feel free to expand this list.

[schlunma]

When using multiple experiments for a single dataset (e.g., exp: [historical, ssp585]), the resulting attributes in the cubes can change from run to run, depending on the order in which the files have been read.

Example:

:experiment = "ESM RCP8.5;ESM historical" ;
:experiment_id = "esmrcp85;esmHistorical" ;

vs.

:experiment = "ESM historical;ESM RCP8.5" ;
:experiment_id = "esmHistorical;esmrcp85" ;

Relevant code: https://github.com/ESMValGroup/ESMValCore/blob/354ee551db9bde9c5fe202f72d26388cd950dc98/esmvalcore/preprocessor/_io.py#L228

[bouweandela]

Created an issue for this: ESMValGroup/ESMValCore#1779

[schlunma]

This has been solved in ESMValGroup/ESMValCore#1783

[schlunma]

The diagnostic mlr/postprocess.py produces files where the order of the coordinates in the cell_methods attribute differ.

Example

gpp_divided_by_ref_error:cell_methods = "longitude: latitude: mean" ;

vs.

gpp_divided_by_ref_error:cell_methods = "latitude: latitude: mean" ;

[schlunma]

Fixed by #2900

[schlunma]

The MLR diagnostics (diag_script/mlr/*) write information about the machine learning model used as attributes to the output files. Since some of these options refer to memory, this can differ from run to run.

Example:

:final__regressor = "GradientBoostingRegressor(max_depth=4, n_estimators=50,\n random_state=RandomState(MT19937) at 0x7FFBA5373640,\n subsample=0.5)" ;

[schlunma]

The diagnostic mlr/preprocess.py sometimes write files with timestamps if another existing file would be duplicated. This could lead to missing/extra files in the report of the comparison tool.

Example:

work/diag_plot_cmip5_gbrt/preprocess_plot/mmm_GPP_prediction_prediction_output_20221031_165943090559.nc

Relevant code:

ESMValTool/esmvaltool/diag_scripts/mlr/preprocess.py

Lines 1071 to 1075 in 2c720df

	# Get new path
	new_path = data['filename']
	if os.path.exists(new_path):
	now = datetime.datetime.utcnow().strftime("%Y%m%d_%H%M%S%f")
	data['filename'] = new_path.replace('.nc', f'_{now}.nc')