Add a RSR convert script for FY-4B GHI sensor

[yukaribbba]

• Add your name to AUTHORS.md if not there already

Original RSR comes from CMA's website. https://img.nsmc.org.cn/PORTAL/NSMC/DATASERVICE/SRF/FY4B/FY4B_GHI_SRF.zip

The hdf result. See if we can put it on zenodo:
rsr_ghi_FY-4B.zip

The images:
(rayleigh correction)

(no correction)

You see the correction is not that satisfying. I think it's because the original rsr data is kinda rough. But anyway that's the only open data I can find on CMA's website.

[simonrp84]

Hi @yukaribbba, I can't find the GHI RSRs on the NSMC website any more - your link is no longer working and I can't find an alternative (only the AGRI RSRs). Do you know where do download these now?

(only just realised I was requested to review this - doing so now!)

[simonrp84]

Looks good, one minor comment on the dates and also an updated link to the RSRs (I managed to find them!)

[yukaribbba]

@simonrp84 The work is done but not perfect.
I just found a GHI image from a NSMC promotion document on JMA website.

The color is pretty nice here. Looks like MODIS/MERSI-2. I don't know what has happened. Maybe the performance has degraded?

[simonrp84]

@yukaribbba To my knowledge the instrument hasn't degraded, the images have looked like this right from the start. I presume that a lot of post-processing was done to make their image look as it does.

Also, I just tried to run your conversion script and am getting some errors:

(base) C:\Users\Simon\PycharmProjects\pyspectral\rsr_convert_scripts>python ghi_rsr.py
Traceback (most recent call last):
  File "C:\Users\Simon\PycharmProjects\pyspectral\rsr_convert_scripts\ghi_rsr.py", line 113, in <module>
    convert_ghi()
  File "C:\Users\Simon\PycharmProjects\pyspectral\rsr_convert_scripts\ghi_rsr.py", line 106, in convert_ghi
    tohdf5(GHIRSR, 'FY-4B', FY4_GHI_BAND_NAMES)
  File "C:\Users\Simon\PycharmProjects\pyspectral\pyspectral\utils.py", line 274, in convert2hdf5
    instr = ClassIn(bandnames[0], platform_name)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Simon\PycharmProjects\pyspectral\rsr_convert_scripts\ghi_rsr.py", line 56, in __init__
    self._get_bandfilenames()
  File "C:\Users\Simon\PycharmProjects\pyspectral\pyspectral\raw_reader.py", line 66, in _get_bandfilenames
    self.options[self.platform_name + '-' +
KeyError: 'ch1'

I added this to pyspectral.yaml before running:

FY-4B-ghi:
  path: C:/Users/Simon/Downloads/FY4B_GHI_SRF/

Any idea what might be happening there?

[yukaribbba]

Nothing wrong with my yaml:

FY-4B-ghi:
    path: D:/satpy_config/FY4B_GHI_SRF
    ch1: FY4B_GHI_SRF_CH01_Pub.txt
    ch2: FY4B_GHI_SRF_CH02_Pub.txt
    ch3: FY4B_GHI_SRF_CH03_Pub.txt
    ch4: FY4B_GHI_SRF_CH04_Pub.txt
    ch5: FY4B_GHI_SRF_CH05_Pub.txt
    ch6: FY4B_GHI_SRF_CH06_Pub.txt
    ch7: FY4B_GHI_SRF_CH07_Pub.txt

[simonrp84]

Yes, you need to include that YAML in the PR :)

[yukaribbba]

oops...

[adybbroe]

Hey! I am looking at this now, looks fine. But is this the link to the original RSR data:
https://satellite.nsmc.org.cn/PortalSite/StaticContent/FileDownload.aspx?CategoryID=1&LinkID=687
?

I get a "The file does not exist" :-(

[adybbroe]

Otherwise, and in the mean time, I will add the hdf5 file to zenodo, so we could get it included. They look a bit rough yes, but this is the best we have.

[adybbroe]

@yukaribbba Can you just please fix the pre-commit CI complaints?
Then I will prepare data on Zenodo

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.00%. Comparing base (677af34) to head (479e196).
Report is 29 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #192   +/-   ##
=======================================
  Coverage   90.00%   90.00%           
=======================================
  Files          23       23           
  Lines        2571     2571           
=======================================
  Hits         2314     2314           
  Misses        257      257           

Flag	Coverage Δ
unittests	90.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[yukaribbba]

Hi @adybbroe thanks for reviewing! The link is available for me. You might want to try it several times since their website is unstable or hard to be connected sometimes. Here's the original RSR data pulled from that link.
FY4B_GHI_SRF.zip

[yukaribbba]

I remember there's a way to call pre-commit CI and let it do the fix automatically but I forgot how to do that....

[yukaribbba]

pre-commit.ci autofix

[yukaribbba]

@adybbroe Ok done.

[mraspaud]

@adybbroe could you have another look at this to see if we can merge this?

[yukaribbba]

@adybbroe could you take a moment to see if there's anything else to do for this PR?

[yukaribbba]

I've updated the link to original rsr in description and it's valid now.
https://img.nsmc.org.cn/PORTAL/NSMC/DATASERVICE/SRF/FY4B/FY4B_GHI_SRF.zip

[adybbroe]

LGTM, thanks! And sorry for the delay.

[yukaribbba]

Thanks! :D