Initial commit of NEMO NEI 2019 #54

drnimbusrain · 2024-08-30T16:50:19Z

@bbakernoaa For the NEMO NEI2019 here, I assume this is what you meant by adding new config? I updated to NEI2109 based on the [develop] branch cmaq_gfs_megan config. Also not sure if we are targeting this NEI2019 update to the [develop] or [production/AQMv7] branch.

Also required changes to NEXUS utils such as the linker script, maybe @zmoon can help here, and then changes to workflow to do a test run.

drnimbusrain · 2024-08-30T19:13:43Z

Thanks @bbakernoaa

@zmoon can you help update the NEXUS utils/python linker scripts to this PR to accommodate these updates to use the NEMO/NEI2019 emissions. You can see examples of their emission files for correct linking on Hera at: /scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/nexus/NEMO/NEI2019/v2023-03/$mm Instead of daily files, these merged 2D files are provided as four representative days (two weekdays and two weekend days) per month and could be linked accordingly for the NEXUS/HEMCO simulation date. Thanks!

zmoon · 2024-09-06T16:04:02Z

Thanks @drnimbusrain , did you want me to do it in this branch?

drnimbusrain · 2024-09-06T18:15:25Z

Yes please

…

On Fri, Sep 6, 2024, 12:04 PM Zachary Moon ***@***.***> wrote: Thanks @drnimbusrain <https://github.com/drnimbusrain> , did you want me to do it in this branch? — Reply to this email directly, view it on GitHub <#54 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGLFYNXSQKGGQLEWHNQ4MG3ZVHHARAVCNFSM6AAAAABNM2FANWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZUGM3TKOJUGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

drnimbusrain · 2024-09-24T21:29:50Z

Thanks @drnimbusrain , did you want me to do it in this branch?

@zmoon

zmoon · 2024-09-27T20:50:37Z

@drnimbusrain here's a little example of what I've currently got set up:

> mkdir tmp
> ./nexus_nei2019_linker.py -s /scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/nexus/ -w ./tmp --no_read_hemco_time -d 2023-09-27
file map size: 84
src dir: /scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/nexus
work dir: ./tmp
date: 2023-09-27, month: 9, isoweekday: 3
linking /scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/nexus/NEMO/NEI2019/v2023-03/09/NEI2019v1_0.1_degree_20190910_all.nc to ./tmp/NEMO/NEI2019/v2023-03/09/NEI2019v1_0.1_degree_20230927_all.nc

But besides adding this ability to test single dates, the CLI should be the same and you should be able to just swap for this new Python script in the workflow nexus emission Bash script.

drnimbusrain · 2024-09-30T16:48:48Z

@drnimbusrain here's a little example of what I've currently got set up:
> mkdir tmp
> ./nexus_nei2019_linker.py -s /scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/nexus/ -w ./tmp --no_read_hemco_time -d 2023-09-27
file map size: 84
src dir: /scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/nexus
work dir: ./tmp
date: 2023-09-27, month: 9, isoweekday: 3
linking /scratch1/RDARCH/rda-arl-gpu/Barry.Baker/emissions/nexus/NEMO/NEI2019/v2023-03/09/NEI2019v1_0.1_degree_20190910_all.nc to ./tmp/NEMO/NEI2019/v2023-03/09/NEI2019v1_0.1_degree_20230927_all.nc
But besides adding this ability to test single dates, the CLI should be the same and you should be able to just swap for this new Python script in the workflow nexus emission Bash script.

Ok, thank you Zach, let me test it with swapping the workflow's exaqm_nexus_emission.sh script to use this new python linker script. Ill make the necessary changes to the following places:

For changes in 2 above, I assume there is not a new "nexus_nei2016_control_tilefix.py" script, and we still use this one, correct?

Thanks again!

zmoon · 2024-09-30T17:06:33Z

not a new "nexus_nei2016_control_tilefix.py" script

May need to, let's see. It seems all it does is replace 2016$MM$DD with the target year. So for 2019 I suppose we would need to do the same, to replace the 2019 in 2019$MM$DD.

ROOT/NEMO/NEI2019/v2023-03/$MM/NEI2019v1_0.1_degree_2019$MM$DD_all.nc

But could that line not be written like

ROOT/NEMO/NEI2019/v2023-03/$MM/NEI2019v1_0.1_degree_$YYYY$MM$DD_all.nc

instead? Then the script to modify the NEXUS config shouldn't be needed?

@drnimbusrain : Yes, I it seems better that way, but for now will just add new one in case there is something else, maybe Barry can comment.

drnimbusrain · 2024-09-30T17:35:48Z

Actually best way is to remove the year specific entirely and have the config be agnostic to this (as we do for some other global inventories), and use the configs as just "NEI" not NEI2016, NEI2019 etc. We should do this in the future.

drnimbusrain · 2024-09-30T17:41:10Z

Actually best way is to remove the year specific entirely and have the config be agnostic to this (as we do for some other global inventories), and use the configs as just "NEI" not NEI2016, NEI2019 etc. We should do this in the future.

Well, not really sure about this either. I think we would want to maintain easy ways of testing any available inventory with simple switches, e.g., keeping both capabilities for NEI2016 vs. NEI2019, and going forward.

drnimbusrain · 2024-10-02T00:57:29Z

@zmoon getting the following error for a August 01-07, 2023 test on Hera:

All executables will be submitted with command \'srun -n 80 --export=ALL\'.
2023-08-01 12:00:00 2023-08-01 17:00:00
2023-08-01 12:00:00
file map size: 36
src dir: /scratch2/NCEPDEV/naqfc/RRFS_CMAQ/emissions/nexus
work dir: /scratch1/NCEPDEV/stmp4/Patrick.C.Campbell/expt_dirs/aqmna13_gfs_v16_test_aqm_dev_cmaq54_nei2019/tmp/nexus_emission_00.414949/input
date: 2023-08-01, month: 8, isoweekday: 2
error: no file found for month 8, iwd 2
FATAL ERROR:
ERROR:

drnimbusrain · 2024-10-02T00:59:47Z

Ah, hang on, I think I have another error in the workflow script with the incorrect version (-v) when running the linker script.

drnimbusrain · 2024-10-10T16:14:39Z

@zmoon @bbakernoaa The NEMO NEI2019 have been tested (against current NEI2016v1) in AQMv8 for August 01-07,2023:

Ozone Spatial Differences:

Ozone NMB Scorecard

Ozone NME Scorecard

Ozone IOA Scorecard

PM2.5 Spatial Differences

PM2.5 NMB Scorecard

PM2.5 NME Scorecard

PM2.5 IOA Scorecard

drnimbusrain · 2024-10-10T16:19:35Z

Full evaluations of NEMO NEI2019 emissions updates on AQMv8 shown here: https://drive.google.com/drive/folders/1Dxl46SCrBNX_INiFiM-aU3K_IzJYy6wL

Initial commit of NEMO NEI 2019

2469d0a

drnimbusrain requested review from zmoon and bbakernoaa August 30, 2024 16:50

drnimbusrain added the enhancement New feature or request label Aug 30, 2024

drnimbusrain linked an issue Aug 30, 2024 that may be closed by this pull request

Updating NEXUS Inputs to use NEMO NEI 2019 Emissions #55

Open

drnimbusrain marked this pull request as draft August 30, 2024 17:00

bbakernoaa approved these changes Aug 30, 2024

View reviewed changes

zmoon and others added 11 commits September 27, 2024 14:05

Start with same CLI

ba5b69c

Update for 2019 dir

608099b

Files are combined, don't need sector loop

f826e6a

Work with provided date?

9aaebb3

fmt

603ea18

Initial approach

4e72773

Add shebang

3d31590

executable

99888ce

Test with single dates

9fa7dd8

Also work for YYYY-MM-DD

f81a32b

Print just date

3abfa07

zmoon added 2 commits September 27, 2024 21:03

Also allow hypenated

30ea3d7

Check work dir exists

ed4d03e

drnimbusrain added 2 commits September 30, 2024 13:18

Need non-GFS MEGAN version config (using MERRA2) for NEI2019 update.

410ba32

NEI2019 version of control_tilefix.py script.

c41aa40

drnimbusrain added 3 commits October 2, 2024 14:02

Removing CH4_INV and CO2_INV, they are not in NEI2019 files.

8f53ee1

Added missing CL2 to NEXUS configs.

86b4d14

Added HCL, SULF, VOC_INV, UNR, and UNK to NEXUS configs.

4869f19

drnimbusrain requested a review from bbakernoaa October 10, 2024 18:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial commit of NEMO NEI 2019 #54

Initial commit of NEMO NEI 2019 #54

drnimbusrain commented Aug 30, 2024 •

edited

Loading

drnimbusrain commented Aug 30, 2024 •

edited

Loading

zmoon commented Sep 6, 2024

drnimbusrain commented Sep 6, 2024 via email

drnimbusrain commented Sep 24, 2024

zmoon commented Sep 27, 2024 •

edited

Loading

drnimbusrain commented Sep 30, 2024

zmoon commented Sep 30, 2024 •

edited

Loading

drnimbusrain commented Sep 30, 2024

drnimbusrain commented Sep 30, 2024

drnimbusrain commented Oct 2, 2024

drnimbusrain commented Oct 2, 2024

drnimbusrain commented Oct 10, 2024 •

edited

Loading

drnimbusrain commented Oct 10, 2024

Initial commit of NEMO NEI 2019 #54

Are you sure you want to change the base?

Initial commit of NEMO NEI 2019 #54

Conversation

drnimbusrain commented Aug 30, 2024 • edited Loading

drnimbusrain commented Aug 30, 2024 • edited Loading

zmoon commented Sep 6, 2024

drnimbusrain commented Sep 6, 2024 via email

drnimbusrain commented Sep 24, 2024

zmoon commented Sep 27, 2024 • edited Loading

drnimbusrain commented Sep 30, 2024

zmoon commented Sep 30, 2024 • edited Loading

drnimbusrain commented Sep 30, 2024

drnimbusrain commented Sep 30, 2024

drnimbusrain commented Oct 2, 2024

drnimbusrain commented Oct 2, 2024

drnimbusrain commented Oct 10, 2024 • edited Loading

drnimbusrain commented Oct 10, 2024

drnimbusrain commented Aug 30, 2024 •

edited

Loading

drnimbusrain commented Aug 30, 2024 •

edited

Loading

zmoon commented Sep 27, 2024 •

edited

Loading

zmoon commented Sep 30, 2024 •

edited

Loading

drnimbusrain commented Oct 10, 2024 •

edited

Loading