Recipe checker could really use clearer error messages.

[ledm]

Hi,

I'm developing a new recipe from scratch and I forgot an important field in my recipe, which causes ESMValCore to break with the following message below. Here's the full message:

2021-04-23 11:02:24,699 UTC [11186] DEBUG   Checking recipe against schema /home/users/ldemora/workspace/ESMValTool/ESMValCore/esmvalcore/recipe_schema.yml
2021-04-23 11:02:25,534 UTC [11186] INFO    Maximum memory used (estimate): 0.0 GB
2021-04-23 11:02:25,535 UTC [11186] INFO    Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2021-04-23 11:02:25,719 UTC [11186] ERROR   Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
  File "/home/users/ldemora/workspace/ESMValTool/ESMValCore/esmvalcore/_main.py", line 433, in run
    fire.Fire(ESMValTool())
  File "/home/users/ldemora/miniconda3/envs/esmvaltool/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/users/ldemora/miniconda3/envs/esmvaltool/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/home/users/ldemora/miniconda3/envs/esmvaltool/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/users/ldemora/workspace/ESMValTool/ESMValCore/esmvalcore/_main.py", line 410, in run
    process_recipe(recipe_file=recipe, config_user=cfg)
  File "/home/users/ldemora/workspace/ESMValTool/ESMValCore/esmvalcore/_main.py", line 100, in process_recipe
    recipe = read_recipe_file(recipe_file, config_user)
  File "/home/users/ldemora/workspace/ESMValTool/ESMValCore/esmvalcore/_recipe.py", line 51, in read_recipe_file
    check.recipe_with_schema(filename)
  File "/home/users/ldemora/workspace/ESMValTool/ESMValCore/esmvalcore/_recipe_checks.py", line 58, in recipe_with_schema
    yamale.validate(schema, recipe, strict=False)
  File "/home/users/ldemora/miniconda3/envs/esmvaltool/lib/python3.9/site-packages/yamale/yamale.py", line 49, in validate
    raise YamaleError(results)
yamale.yamale_error.YamaleError: Error validating data '/home/users/ldemora/workspace/ESMValTool/run-brightspots/recipe_ocean_brightspots.yml' with schema '/home/users/ldemora/workspace/ESMValTool/ESMValCore/esmvalcore/recipe_schema.yml'
        diagnostics.mm_stats.scripts: Required field missing
2021-04-23 11:02:25,724 UTC [11186] INFO
If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions
If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues
To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.

This is not very helpful for new users (or for me, being honest).

Can anyone look at this error message and diagnose my problem? I suspect not as the key bit of information is missing. If this error message told me what the required missing field was and where it should be, it would be 1000x better.

[ledm]

(The answer is "scripts", but that's not the point!)

[zklaus]

I am completely with you @ledm! But for future reference and a starting point on where to look for improving this error message, the information actually is there:

diagnostics.mm_stats.scripts: Required field missing

tells us that the missing field is diagnostics.mm_stats.scripts, so is diagnostics there? Check. Does it have mm_stats? Check. Are the scripts there? No. Have to add.