Output at first timestep (of first leg) coming from ICMGG????+000000 instead of ICMGG????+YYYYMM file

[jmrgonza]

I am planing to repeat the cmorization of some historical and scenario runs produced with old releases of EC-Earth3 (3.3.0). I am using ece2cmor3 v1.8.1 for the new cmorization. I wanted to check that the contents of the new files were consistent with my previous cmorization, so I used the 'cdo diff' command to compare them. I expected some differences from issue #671 (also https://dev.ec-earth.org/issues/922), which I wanted to fix in the new cmorization. I expected those differences to affect a small number of grid points on every timestep. I found that, but also some differences in many ifs variables affecting the majority of grid points, only in the first timestep of the first leg of the historical run.
I have plotted the fields in the first timestep of the cmorized files against the grib output files produced by ifs, and I have found that the data that was written by ece2cmor v1.8.1 in the first timestep comes from the ICMGG????+000000 grib file, while in my original cmorization using ece2cmor3 v1.0.0 it was coming from ICMGG????+185001. I have only checked the clt_3hr variable, but it seems to me that for this timestep the recent ece2cmor3 releases are picking the wrong data.
There are over 90 (of 153) variables showing these extensive differences in the first timestep of my historical run.

[jmrgonza]

I created an issue and added some plots in the EC-Earth redmine portal (https://dev.ec-earth.org/issues/1024).

[goord]

Yes we have made some changes to ensure that the static fields are always taken from the ICM{GG,SH}????+000000 files, and I recall there were also some problems for the first time step of the scenarios too. The intended behavior is as follows:

• Instantaneous fields like clt_3hr should be reported on 0, 3, 6, ... hours and the 0 hour field should therefore come from the previous leg, and in the case of the first leg the ICM{GG,SH}????+000000 files,
• Averaged fields like pr_3hr we report at midpoints 1.5, 5.4, ... and those are the de-accumulated values for the 3, 6, ... hour fields in the GRIB files of the leg itself, so in some sense we shift them back half an output time step.

Can you check whether your new CMORization does this correctly for the first time step?

[jmrgonza]

In the case of clt_3hr, the first timestep contains the information from ICMGG????+000000, and the second contains the prediction at 0300 in ICMGG????+185001, so it is kind of the intended behaviour that you mention, with the only exception that the reported time for these first two timesteps in the cmorized files are 01:30 and 04:30. So it looks like instantaneous values like clt_3hr are being shifted 1:30 into the future.
Klaus linked a table in the redmine issue, indicating that some of these instantaneous fields are indeed requested as temporal means, which may be the reason for the shift.
For pr_3hr, the values in my ICMGG????+000000 file are all 0. I do not see a difference from the old ece2cmor3 to newer versions: both are consistent with the intended behaviour that you describe.

[goord]

Ok yes I see now clt is indeed requested as a temporal mean, which we can't provide of course, so they get assigned to the earlier half-output time step slice, and apparently this behavior has changed since version 1.0, with both solutions being incorrect to a some degree. Perhaps this should be somehow be communicated in an erratum...