Fix soil temperature/mositure read from RRFS warm-start r…

[xyzemc]

DUE DATE for merger of this PR into develop is 2/27/2024 (six weeks after PR creation).

Description

• Undefined soil_temp and soil_moist for regional because they are not correctly read in from the RRFS' warm-sart restart file as described in the issue#677.
• Undefined or zero value of soil temp and moist will affect the calculation of surface emissivity. Surface emissivity is a factor to calculate the microwave surface channel radiance.

Resolves #677

Fix
In gsi_rfv3io_mod.f90, replace 'STC' with 'tslb', replace 'SMC' as 'smois

How has this need tested?

• This has been tested on Hera. The values of soil_temp and soil_moisture have been verified are correct now from the RRFS restart file.

[ShunLiu-NOAA]

@TingLei-NOAA and @hu5970 Could you please review this PR?

[TingLei-NOAA]

This is a fix maybe having potential significant impacts on the anaysis using sat obs. So, for being now, some preliminary comparison including the soil temp/moisture comparison between the control and the PR and the possible differences in an one cycle GSI analysis will be helpful! @xyzemc

[xyzemc]

This is a fix maybe having potential significant impacts on the anaysis using sat obs. So, for being now, some preliminary comparison including the soil temp/moisture comparison between the control and the PR and the possible differences in an one cycle GSI analysis will be helpful! @xyzemc

The soil_temp before fix is:

0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
After fix, the value becomes:
265.386530012575 265.329296455622 265.287686564725
287.122644483134 287.107760342667 287.015224838230
301.599916472329 301.599916547026 301.599916631882
285.141949310225 285.182453695278 285.130403097801
285.840519341688 285.976999112695 286.527157120589
286.559386021808 286.548110892401 285.964901381497
286.145627991059 285.544529871453 286.481316548355
288.705919813717 289.086001203200 288.306397747852
288.721856886849 288.933679407714 289.416424131676
289.044010425247 288.611918472941 288.650839559363
287.826206608543 286.875615132908 285.433724674489
284.427218789551 283.706922067357 285.783789627099
286.579713462659 287.454024079914 288.756345413178
288.713854998341 288.680328646850 288.990765426220
289.624358468952 289.111044247816 289.207870836034
289.207836156051 289.050363845361 288.473584399333
288.327136534509 287.947649848821 288.864414529597

The soil_moi before fix is:
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000
0.000000000000000E+000 0.000000000000000E+000 0.000000000000000E+000

After fix, the value becomes:
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000
0.309617316394092 0.312066345645467 0.312305934525266
0.297458054689539 0.297682165457320 0.286036729660987
0.276829110018995 0.294540172928305 0.306131131096279
0.300089082353489 0.320311654267114 0.312110829381316
0.299822899054036 0.306800763734304 0.302263842278225
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000
1.00000000000000 1.00000000000000 1.00000000000000

[hu5970]

@xyzemc Any impact to the analysis increment from satellite radiance after this change? The change is very simple. We just need an idea what it could impact. Thanks, Ming

[xyzemc]

Before the bug fix the soil temp/mois have zero value like in the figures below:
Soil Temp before bug fix

Soil Temp after bug fix

Soil Moist before bug fix

Soil Moist after bug fix

Surface Emissivity Difference with/without Soil temp/mois Bug Fix

Brightness Temperature Difference with/without Soil temp/mois Bug Fix

Before big fix, AMSUA_N19 Channel 2 data amount passed QC is 2061

After big fix, AMSUA_N19 Channel 2 data amount passed QC is 2199

[TingLei-NOAA]

@xyzemc Great to see those results. A question, what values are those points ( pointed with an ugly arrow I added as shown in below) in the soil temperature figure with bug fixed you gave

[xyzemc]

@xyzemc Great to see those results. A question, what values are those points ( pointed with an ugly arrow I added as shown in below) in the soil temperature figure with bug fixed you gave

Those dark blue point is zero, which is gsi put the zero value for soil temperature over water.

[TingLei-NOAA]

@xyzemc Thanks for the explanation. So GSI put those zero in K on water ? Are they used as some labels for water ? They will not be actually used as soil temperature in any approximation , right? Your further clarification will be helpful for me , a non-expert on this aspect.

[xyzemc]

@xyzemc Thanks for the explanation. So GSI put those zero in K on water ? Are they used as some labels for water ? They will not be actually used as soil temperature in any approximation , right? Your further clarification will be helpful for me , a non-expert on this aspect.

You are right. Only soil temperature over land will be used to calculate the land surface emissivity. All other surface types (water, ice, snow) has zero soil temperature.

[xyzemc]

Regression test on Hera
[Xiaoyan.Zhang@hfe07 build]$ ctest
Test project /scratch2/NCEPDEV/fv3-cam/Xiaoyan.Zhang/noscrub/gsi_code/issue677/soil_temp/build
Start 1: [=[global_4denvar]=]
1/7 Test #1: [=[global_4denvar]=] ............. Passed 5517.95 sec
Start 2: [=[rtma]=]
2/7 Test #2: [=[rtma]=] ....................... Passed 1692.05 sec
Start 3: [=[rrfs_3denvar_glbens]=]
3/7 Test #3: [=[rrfs_3denvar_glbens]=] ........ Passed 1451.12 sec
Start 4: [=[netcdf_fv3_regional]=]
4/7 Test #4: [=[netcdf_fv3_regional]=] ........ Passed 1566.08 sec
Start 5: [=[hafs_4denvar_glbens]=]
5/7 Test #5: [=[hafs_4denvar_glbens]=] ........ Passed 1590.82 sec
Start 6: [=[hafs_3denvar_hybens]=]
6/7 Test #6: [=[hafs_3denvar_hybens]=] ........ Passed 1826.44 sec
Start 7: [=[global_enkf]=]
7/7 Test #7: [=[global_enkf]=] ................ Passed 991.12 sec

100% tests passed, 0 tests failed out of 7

Total Test time (real) = 14635.70 sec

[TingLei-NOAA]

Thanks for taking care this issue.

[ShunLiu-NOAA]

@xyzemc Could you please sync your branch with "deveop" and complete ctest on WCOSS, Hera and Orion?

[xyzemc]

@xyzemc Could you please sync your branch with "deveop" and complete ctest on WCOSS, Hera and Orion?

I just synchronized my branch. The message is " This branch is not behind the upstream NOAA-EMC/GSI:develop. No new commits to fetch. "
I will complete the regression test on wcoss2, hera, and orion as soon as possible.

[xyzemc]

Final Regression Tests:
WCOSS2:
Test project /lfs/h2/emc/da/noscrub/Xiaoyan.Zhang/rrfs/soil_temp/build
Start 1: global_4denvar
1/7 Test #1: global_4denvar ................... Passed 1368.99 sec
Start 2: rtma
2/7 Test #2: rtma ............................. Passed 1026.66 sec
Start 3: rrfs_3denvar_glbens
3/7 Test #3: rrfs_3denvar_glbens .............. Passed 485.00 sec
Start 4: netcdf_fv3_regional
4/7 Test #4: netcdf_fv3_regional .............. Passed 482.75 sec
Start 5: hafs_4denvar_glbens
5/7 Test #5: hafs_4denvar_glbens .............. Passed 1211.29 sec
Start 6: hafs_3denvar_hybens
6/7 Test #6: hafs_3denvar_hybens .............. Passed 1209.43 sec
Start 7: global_enkf
7/7 Test #7: global_enkf ...................... Passed 1061.40 sec

100% tests passed, 0 tests failed out of 7

Total Test time (real) = 6847.97 sec

Hera:
Test #1: [=[global_4denvar]=]
Test #2: [=[rtma]=]
Test #3: [=[rrfs_3denvar_glbens]=]
Test #4: [=[netcdf_fv3_regional]=]
Test #5: [=[hafs_4denvar_glbens]=]
Test #6: [=[hafs_3denvar_hybens]=]
Test #7: [=[global_enkf]=]

Total Tests: 7
[Xiaoyan.Zhang@hfe07 build]$ ctest
Test project /scratch2/NCEPDEV/fv3-cam/Xiaoyan.Zhang/noscrub/gsi_code/issue677/soil_temp/build
Start 1: [=[global_4denvar]=]
1/7 Test #1: [=[global_4denvar]=] ............. Passed 1684.25 sec
Start 2: [=[rtma]=]
2/7 Test #2: [=[rtma]=] ....................... Passed 1210.95 sec
Start 3: [=[rrfs_3denvar_glbens]=]
3/7 Test #3: [=[rrfs_3denvar_glbens]=] ........ Passed 493.37 sec
Start 4: [=[netcdf_fv3_regional]=]
4/7 Test #4: [=[netcdf_fv3_regional]=] ........ Passed 547.09 sec
Start 5: [=[hafs_4denvar_glbens]=]
5/7 Test #5: [=[hafs_4denvar_glbens]=] ........ Passed 1527.63 sec
Start 6: [=[hafs_3denvar_hybens]=]
6/7 Test #6: [=[hafs_3denvar_hybens]=] ........ Passed 1350.04 sec
Start 7: [=[global_enkf]=]
7/7 Test #7: [=[global_enkf]=] ................ Passed 1001.01 sec

100% tests passed, 0 tests failed out of 7

Total Test time (real) = 7814.54 sec

Orion:
Orion-login-2[156] xyzhang$ ctest
Test project /work/noaa/fv3-cam/xyzhang/soil_temp/build
Start 1: [=[global_4denvar]=]
1/7 Test #1: [=[global_4denvar]=] ............. Passed 1502.55 sec
Start 2: [=[rtma]=]
2/7 Test #2: [=[rtma]=] ....................... Passed 967.94 sec
Start 3: [=[rrfs_3denvar_glbens]=]
3/7 Test #3: [=[rrfs_3denvar_glbens]=] ........ Passed 605.46 sec
Start 4: [=[netcdf_fv3_regional]=]
4/7 Test #4: [=[netcdf_fv3_regional]=] ........ Passed 482.84 sec
Start 5: [=[hafs_4denvar_glbens]=]
5/7 Test #5: [=[hafs_4denvar_glbens]=] ........ Passed 1577.19 sec
Start 6: [=[hafs_3denvar_hybens]=]
6/7 Test #6: [=[hafs_3denvar_hybens]=] ........ Passed 1335.67 sec
Start 7: [=[global_enkf]=]
Start 7: [=[global_enkf]=]
7/7 Test #7: [=[global_enkf]=] ................***Failed 488.14 sec

86% tests passed, 1 tests failed out of 7

Total Test time (real) = 6959.92 sec

The following tests FAILED:
7 - [=[global_enkf]=] (Failed)
Errors while running CTest
"The runtime for global_enkf_loproc_updat is 91.499301 seconds and is within the maximum allowable operational time of 1200 seconds,
continuing with regression test.

The runtime for global_enkf_loproc_updat is 91.499301 seconds. This has exceeded maximum allowable threshold time of 87.941901 seconds,
resulting in Failure timethresh of the regression test.

The runtime for global_enkf_hiproc_updat is 63.950762 seconds and is within the allowable threshold time of 65.882355 seconds,
continuing with regression test.

The memory for global_enkf_loproc_updat is 3633668 KBs and is within the maximum allowable memory of 4119871 KBs,
continuing with regression test.

The results between the two runs (global_enkf_loproc_updat and global_enkf_loproc_contrl) are reproducible.

The results between the two runs (global_enkf_loproc_updat and global_enkf_hiproc_updat) are reproducible"

[xyzemc]

This bug fix does affect the analysis difference. From the attached slide, we can see the temperature, wind, and the delp analysis have the very obvious difference along with the amsua-n19 radiance location.
Soil temp_mois debug.pptx