Recipe recipe_esacci_oc.yml with ESMValTool 2.9

[Coquart]

Hi all,
I am trying to run the multi-models recipe recipe_esacci_oc.yml with the latest version of ESMValTool 2.9 .
With some datasets it works successfully, for example:
datasets:

• {dataset: NorESM2-LM, project: CMIP6, ensemble: r1i1p1f1, grid: gr}
• {dataset: MPI-ESM1-2-LR, project: CMIP6, ensemble: r1i1p1f1, grid: gn}

With some other datasets oc_chl/chl is not available so it does not work (normal), for example (chl not available for EC-Earth3):
datasets:

• {dataset: NorESM2-LM, project: CMIP6, ensemble: r1i1p1f1, grid: gr}
• {dataset: EC-Earth3, project: CMIP6, ensemble: r1i1p1f1, grid: gr}

But with some other datasets oc_chl/chl is avalaible but I have a python error (ValueError: zero-size array to reduction operation minimum which has no identity), for example with:
datasets:

• {dataset: MPI-ESM1-2-LR, project: CMIP6, ensemble: r1i1p1f1, grid: gn}
• {dataset: CNRM-ESM2-1, project: CMIP6, ensemble: r1i1p1f2, grid: gn}

Is this ValueError is a bug or is it just linked to the model I've chosen ?
I send to you my recipe, the main_log_debug.txt file for each case and the log.txt file for the last case.
main_log_debug_OK.txt
main_log_debug_chl_notfound.txt
main_log_debug_ValueError.txt
log_ValueError.txt
Thanks for your help, Best regards, Laure Coquart

[alistairsellar]

Hi @Coquart, thanks for getting in touch.

Is this ValueError is a bug or is it just linked to the model I've chosen ?

It could in theory be both: that there is something wrong or different with that model data and that ideally such a problem should be detected by the diagnostic code to provide a more informative error message. To investigate the former, I'd suggest inspecting the pre-processed data files under /data/scratch/globc/coquart/esmvaltool/output/recipe_esacci_oc_20230717_142949/preproc to see whether there are differences between the models, either in the dimensions of the data, or something radically different in the data values.

[alistairsellar]

Looking at the code, the model data array is squeeze()d before the failing function, so any dimensions of length 1 will have been removed from the array.

        model_data = cubes['model'][layer].data.squeeze()

[Coquart]

Hi @alistairsellar, thanks for your fast answer 😊
I had a look at the preproc NetCDF files when I have the ValueError message and the variable chl depends only from lat and lon, which is ok.
The problem is that chl is zero for the dataset {dataset: CNRM-ESM2-1, project: CMIP6, ensemble: r1i1p1f2, grid: gn} (not for the other datasets) which explains the message.
I will re-use your methodology for the next problem I shall meet, thanks for sharing. Best regards, Laure

[alistairsellar]

Great, I'm glad that you found the cause @Coquart!

In an ideal world, the diagnostic code would check for such cases with a useful error message. So you could open an issue to fix this - if you felt it an important case to handle.