CESM output Pressure Level Interpolation Error

[WillyChap]

Hello,

We are currently trying to run the MJO_Suite POD using standard CESM model output. We are running into an issue where mdtf is unable to interpolate the 3D U/V winds to the correct pressure levels and thus MDTF tanks. The relevant part of the error message is below.

We have tried add the PS time series to the input_data files, but this didn't seem to fix the issue.

Currently we are running mdtf v3.0 (pulled today).

Any help is appreciated!
Thanks!

@bitterbark

Preprocessing <#pdjo:MJO_suite.u200> (='U' @ 1day, 200 hPa).
DEBUG: Loaded '/glade/scratch/wchapman/mdtf/inputdata/model/f.e21.BERNER/day/f.e21.BERNER.U.day.nc'.
DEBUG: Read 2540 mb for <#pdjo:MJO_suite.u200>.
No standard_name for 'U' found in dataset; setting to 'eastward_wind'.
DEBUG: Updating units for 'time' to value 'days since 0001-01-01 00:00:00' from dataset.
DEBUG: Updating calendar for 'time' to value 'noleap' from dataset.
No standard_name for 'time_bnds' found in dataset; setting to 'time'.
No units for 'time_bnds' found in dataset; setting to 'days since 0001-01-01 00:00:00'.
DEBUG: Updating bounds for 'time' to value 'time_bnds' from dataset.
DEBUG: Updating name for 'plev' to value 'lev' from dataset.
Received event while preprocessing <#pdjo:MJO_suite.u200>: DataPreprocessEvent("Received event while parsing file metadata for <#pdjo:MJO_suite.u200>: MetadataEvent("Unexpected standard_name for variable 'lev': 'atmosphere_hybrid_sigma_pressure_coordinate' (expected 'air_pressure').").")
DEBUG: Eliminating DataKey(4) for <#pdjo:MJO_suite.u200>.
Deactivated <#pdjo:MJO_suite.u200> due to ChildFailureEvent("Deactivating <#pdjo:MJO_suite.u200> due to failure of all child objects.").
Request for <#pdjo:MJO_suite.u200> (='U' @ 1day, 200 hPa) failed; looking for alternate data.
ERROR: Deactivated <#pczP:f.e21.BERNER.MJO_suite> due to PodDataError("Requested data not available for POD <#pczP:f.e21.BERNER.MJO_suite>: No alternate data available for <#pdjo:MJO_suite.u200>.").

[wrongkindofdoctor]

@WillyChap The standarad_name for "lev" for the 4D variables required by the MJO_suite POD does not match the spec in the POD's settings file; "lev" in your data is defined as "atmosphere_hybrid_sigma_pressure_coordinate", but the POD defines it as "air_pressure".

Additionally, in the CESM/NCAR fieldlist that the framework uses for variable translation and preprocessing, the variable "lev" maps to "atmosphere_hybrid_sigma_pressure_coordinate", while "plev" maps to "air_pressure".

Unfortunately, the CESM test dataset used for this POD uses pre-defined levels, so I can't try to replicate the problem on my end without a copy of the data you are using.

You can try the following:

-If your data defines the standard_name for "plev" (or some other coordinate variable) as "air_pressure", redefine the "lev" standard_name in the MJO_suite settings file to match.

If you are still having trouble, please provide your data on globus, or via ftp. I'll also need a copy your configuration jsonc file, or you can direct me to a branch on your MDTF-diagnostics fork that is up-to-date with local copy that you are using for your tests.

[WillyChap]

@wrongkindofdoctor Thanks for the quick response ! Im not sure that I understand route 1 (-If your data defines the standard_name for "plev" (or some other coordinate variable) as "air_pressure", redefine the "lev" standard_name in the MJO_suite settings file to match.). CESM only outputs hybrid sigma pressure coordinates, thus there is no coordinate or data on pressure levels.

Typically we can specify CESM to output certain variables on specified pressure levels at runtime, but this is outside of the CMIP standard files. I think if we specified lev variable as a pressure coordinate then the interpolation would be incorrect. But I might not be understanding!

[bitterbark]

This is as close to native CESM output as we can get (since we have to make timeseries of actual CESM output). As Will says, the atmosphere_hybrid_sigma_pressure_coordinate *is* the CESM vertical coordinate. Perhaps the change to air_pressure was made without testing CESM files. I can look at the code and Will's files this afternoon and see what I can do.

…

On Fri, Jan 20, 2023 at 9:27 AM William ***@***.***> wrote: @wrongkindofdoctor <https://github.com/wrongkindofdoctor> Thanks for the quick response ! Im not sure that I understand route 1 (*-If your data defines the standard_name for "plev" (or some other coordinate variable) as "air_pressure", redefine the "lev" standard_name in the MJO_suite settings file to match.*). CESM only outputs hybrid sigma pressure coordinates, thus there is no coordinate or data on pressure levels. Typically we can specify CESM to output certain variables on specified pressure levels at runtime, but this is outside of the CMIP standard files. I think if we specified lev variable as a pressure coordinate then the interpolation would be incorrect. But I might not be understanding! — Reply to this email directly, view it on GitHub <#443 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHLJVQBW2XF6KXG6WD2P6DLWTK4INANCNFSM6AAAAAAUA4RQRQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[wrongkindofdoctor]

@WillyChap @bitterbark The framework does not interpolate data from hybrid sigma coordinates to pressure levels. The extract level function will only work on fields defined on pressure levels.

class ExtractLevelFunction(PreprocessorFunctionBase):
    """Extract a requested pressure level from a Dataset containing a 3D variable.

    .. note::

       Unit conversion on the vertical coordinate is implemented, but
       parametric vertical coordinates and coordinate interpolation are not.
       If a pressure level is requested that isn't present in the data,
       :meth:`process` raises a KeyError.
    """)

The QBOi.EXP1.AMIP.001 dataset is the one I'm referring to. The raw data was interpolated to the requisite pressure levels beforehand. For example, the ncdump of daily u850 shows the following:

netcdf QBOi.EXP1.AMIP.001.U850.day {
dimensions:
	time = 2555 ;
	lat = 192 ;
	lon = 288 ;
	nbnd = 2 ;
variables:
	double time(time) ;
		time:long_name = "time" ;
		time:units = "days since 1975-01-01 00:00:00" ;
		time:calendar = "noleap" ;
		time:bounds = "time_bnds" ;
	int date(time) ;
		date:long_name = "current date (YYYYMMDD)" ;
	double lat(lat) ;
		lat:units = "degrees_north" ;
		lat:long_name = "latitude" ;
	double lon(lon) ;
		lon:units = "degrees_east" ;
		lon:long_name = "longitude" ;
	double time_bnds(time, nbnd) ;
		time_bnds:long_name = "time interval endpoints" ;
	float U850(time, lat, lon) ;
		U850:Sampling_Sequence = "rad_lwsw" ;
		U850:units = "m/s" ;
		U850:long_name = "Zonal wind at 850 mbar pressure surface" ;
		U850:cell_methods = "time: mean                                                                                                                                                                                                                                                            " ;
}

[bitterbark]

I guess that was my misconception. Sorry about that, @WillyChap. There used to be code in the MJO_suite itself that would call the NCL vertical interpolator when supplied with a 3D field. I'll look around for that, and test it without the data preprocessor.

[bitterbark]

Here is a work around. Note that it requires

1. running with the data preprocessor turned off, as shown in the input_will.jsonc file (to replace default_tests.jsonc)
   note that you will need to change data paths in input_will
2. running once to create the wrkdir and then linking the following files ifrom inputdata dir to wrkdir by hand: u,v,PS
3. ncrename -v U,u file.U.day.nc, mv file.U.day.nc file.u.day.nc
   (and same for V)

And a note on this case, @WillyChap, the initial files are being made but the final results are failing, I think that is because there isn't enough data (as you mentioned earlier).
Feel free to look in my dir /glade/u/home/bundy/diag/mdtf/MDTF_3_main/MDTF-diagnostics and the data pointed to in src/input_will.jsonc

[bitterbark]

Tagging @judithberner