ERA5
***** moved from NEMO_cfgs in the NOC-MSM organisations page *****
-- all python code in this Wiki will be moved to scipts within the repository --
MAJOR UPDATE: April 2019: The ERA5 data have been extended from 1979 and are now hosted on Copernicus CDS server.
- new data are on a different grid so all data have been re downloaded on this grid
- the cumulated variables, described in the first version are now replaced by the mean variable
- the top of this documents refers to the new data but we keep the old process after (for memory)
Quick details on how to generate ECMWF ERA5 forcing for NEMO. The different steps are given below:
- Download data (if not already in our server)
- Extract your region
- Generate Forcing files
- Generate interpolation weights
- Update your namelist
Keep in mind that ERA5 reanalysis provide hourly output at 31km globally so it's HEAVY. Currently, the data are stored at NOC under /projectsa/NEMO/Forcing/ERA5/SURFACE_FORCING and can be downloaded using /projectsa/NEMO/Forcing/Download_viaCDS_ERA5_Global_PerVariable.py. The data are downloaded on the 0.25 degree regular grid with global coverage and a 1h resolution. NEMO bulk flux formulation requires a set of inputs. Below if the table of downloaded variable :
| CDS Name | ECMWF var name | Type | Units |
|---|---|---|---|
| 10m_u_component_of_wind | u10 | Instantaneous | m/s |
| 10m_v_component_of_wind | v10 | Instantaneous | m/s |
| 2m_temperature | t2m | Instantaneous | K |
| mean_sea_level_pressure | msl | Instantaneous | Pa |
| surface_pressure | sp | Instantaneous | K |
| 2m_dewpoint_temperature | d2m | Instantaneous | Pa |
| Specific Humidity | Instantaneous | % | |
| mean_surface_downward_short_wave_radiation_flux | msdwswrf | Averaged | W/m^2 |
| mean_surface_downward_long_wave_radiation_flux | msdwlwrf | Averaged | W/m^2 |
| mean_snowfall_rate | msr | Averaged | kg/m^2/s |
| mean_total_precipitation_rate | mtpr | Averaged | kg/m^2/s |
Following Valerie's comment, we now only download the mean fluxes instead of the cumulated variables that we need to decumulate (see at the end for more details of what was previously done).
The script to do this can be found under (on NOC livljobs server): /work/nibrun/DATA/ERA5/test/OFFICIAL_Generate_NEMO_Forcing_NEWERA.py
The ERA5 file are currently downloaded as one file per year per variable [which is probably not clever due to the size of the files (12Gb) but we can reshape this later on].
The first step in the script is to extract the region of interest from the global ERA5 files. You need to load the nco package on livljobs* :
module load nco/gcc/4.4.2.ncwa
Then the python script should work. Choose your lat/lon box at the beginning of the script. Precise where to store this file (they can be big so under a project name is better than /work/). Years you want to extract. I generally use one more year at beginning and end to have a perfect forcing (as the cumulated fields start at 6h so they are copied from 0 to 5h).
Year_init = 2000 ## First year to process
Year_end = 2016 ## Last one [included]
East = 19 ## East Border
West = -28 ## West Border
North = 68 ## North Border
South = 38 ## South Border
path_ERA5 = '/projectsa/NEMO/Forcing/ERA5/SURFACE_FORCING/' ## ROOT PATH OD ERA5 DATA
path_EXTRACT = './EXTRACTION/' ## WHERE TO EXTRACT YOUR REGION
path_FORCING = './FORCING/' ## NEMO FORCING
clean = False ## Clean extraction (longest bit)
sph_ON = True ## Compute specific humidity or notAfter extracting the sub-region, the scripts loads all the file in memory and generate yearly file for NEMO, applying correction when needed i.e. :
- Beginning of the full time series is copied when the data are not present (before 6h for cumulated for example)
- Fields are centred on the 1/2 time step as it seems what NEMO wants but still not fully clear in my head
- Specific humidity is computed from surface pressure and dewpoint temperature according to ECMWF documentation
You might also want to extract the land sea mask for a better representation of the coastline in NEMO. Either it's available in ERA or you download it for a few time step and assume it is constant. The new format include the ratio land/sea in each box as it comes from ERA5 atmospheric model - so you need to choose where to cut as not that many point have exactly 1 here (land only) and many have 0.99 etc; each user might want to think what to exactly use. One is store under: /projectsa/NEMO/Forcing/ERA5/era5_atmos_landseamask.nc
You can use a simple ncks command with the same lat lon defined in the python script to do so (take car to put the longitude between 0 and 360 as it is the range in the original file so add 360 to any negative longitude for ncks to work properly - this line could be added in the script...); ncks also needs lat/lon to be floats :
ncks -d latitude,38.,68. -d longitude,332.,19. /projectsa/NEMO/Forcing/ERA5/era5_atmos_landseamask.nc ./my_era5_LSM.nc
To generate the mask in NEMO format, a simple python script should do the job:
from netCDF4 import Dataset
import numpy as np
coorfile = 'ERA5_MSL_y2004.nc' ## One ERA forcing file generated previously
maskfile = 'my_era5_LSM.nc' ## Land Sea Mask from ncks
outfile = 'ERA5_LSM.nc' ## Output file
#----------------------------------------------------------------------------
## READ SRC BATHYMETRY
nc_c = Dataset( coorfile, 'r' )
lon_src = nc_c.variables[ 'lon' ][:]
lat_src = nc_c.variables[ 'lat' ][:]
nc_c.close()
print coorfile, "loaded", lon_src.shape
## READ SRC BATHYMETRY
nc_src = Dataset( maskfile, 'r' )
msk_src = nc_src.variables[ 'lsm' ][0,::-1] ## lat to be reverse as it was done in the generation of the forcing files
print maskfile, "loaded", msk_src.shape
#msk_src[(msk_src==0.)] = -1
#msk_src[(msk_src<1)] = -1
seas = msk_src < 0.5
land = msk_src >= 0.5
msk_src[seas] = -1
msk_src[land] = 1
## NETCDF OUTPUT
ncout = Dataset( outfile, 'w', format='NETCDF3_CLASSIC' )
ncout.createDimension( 'nlat', msk_src.shape[0] )
ncout.createDimension( 'nlon', msk_src.shape[1] )
lon = ncout.createVariable( 'lon', 'f4', ('nlat', 'nlon',), zlib='True' )
lat = ncout.createVariable( 'lat', 'f4', ('nlat', 'nlon',), zlib='True' )
lon[:] = lon_src; lat[:] = lat_src
bout = ncout.createVariable( "LSM", 'f4', ('nlat','nlon',), zlib='True', fill_value=-999. )
bout[:] = msk_src
ncout.close()And the namelist.cfg needs to be updated in namsbc:
&namsbc
nn_lsm = 1 ! =0 land/sea mask for input fields is not applied (keep empty land/sea mask filename field) ,
! =1:n number of iterations of land/sea mask application for input fields (fill land/sea mask filename field)
/
As long as the ERA forcing is large enough to encompass the NEMO domain, the same files can be used to force different configuration. The only thing that needs to be updated is the interpolation weights for NEMO. This can be easily done with the NEMO toolbox using SCRIP.
You can use standard files and update the namelists: namelist_reshape_bilin_atmos and namelist_reshape_bicubic_atmos; example can be found in ARCHER under /work/n01/n01/nibrun/RUNS/AMM7/INPUTS/SBC. Then you run the following:
/work/n01/n01/nibrun/NEMO/AMM15_v3_6_STABLE_package/NEMOGCM/TOOLS/WEIGHTS/scripgrid.exe namelist_reshape_bilin_atmos
/work/n01/n01/nibrun/NEMO/AMM15_v3_6_STABLE_package/NEMOGCM/TOOLS/WEIGHTS/scrip.exe namelist_reshape_bilin_atmos
/work/n01/n01/nibrun/NEMO/AMM15_v3_6_STABLE_package/NEMOGCM/TOOLS/WEIGHTS/scripshape.exe namelist_reshape_bilin_atmos
/work/n01/n01/nibrun/NEMO/AMM15_v3_6_STABLE_package/NEMOGCM/TOOLS/WEIGHTS/scripgrid.exe namelist_reshape_bicubic_atmos
/work/n01/n01/nibrun/NEMO/AMM15_v3_6_STABLE_package/NEMOGCM/TOOLS/WEIGHTS/scrip.exe namelist_reshape_bicubic_atmos
/work/n01/n01/nibrun/NEMO/AMM15_v3_6_STABLE_package/NEMOGCM/TOOLS/WEIGHTS/scripshape.exe namelist_reshape_bicubic_atmos
This generates the following files :
- weights_era5_amm7_bicubic.nc
- weights_era5_amm7_bilin.nc
!-----------------------------------------------------------------------
&namsbc_blk ! namsbc_blk generic Bulk formula (ln_blk = T)
!-----------------------------------------------------------------------
! ! bulk algorithm :
ln_NCAR = .false. ! "NCAR" algorithm (Large and Yeager 2008)
ln_COARE_3p0= .false. ! "COARE 3.0" algorithm (Fairall et al. 2003)
ln_COARE_3p5= .false. ! "COARE 3.5" algorithm (Edson et al. 2013)
ln_ECMWF = .true. ! "ECMWF" algorithm (IFS cycle 31)
!
ln_taudif = .false. ! HF tau contribution: use "mean of stress module - module of the mean stress" data
rn_zqt = 2. ! Air temperature and humidity reference height (m)
rn_zu = 10. ! Wind vector reference height (m)
rn_pfac = 1. ! multiplicative factor for precipitation (total & snow)
rn_efac = 1. ! multiplicative factor for evaporation (0. or 1.)
rn_vfac = 0. ! multiplicative factor for ocean/ice velocity
! in the calculation of the wind stress (0.=absolute winds or 1.=relative winds)
ln_Cd_L12 = .false. ! Modify the drag ice-atm and oce-atm depending on ice concentration
! This parameterization is from Lupkes et al. (JGR 2012)
cn_dir = '/work/n01/n01/nibrun/RUNS/AMM7/INPUTS/SBC/' ! root directory for the fluxes data location
!___________!_________________________!___________________!___________!_____________!________!___________!______________________________!__________!_______________!
! ! file name ! frequency (hours) ! variable ! time interp.! clim ! 'yearly'/ ! weights filename ! rotation ! land/sea mask !
! ! ! (if <0 months) ! name ! (logical) ! (T/F) ! 'monthly' ! ! pairing ! filename !
sn_humi = 'ERA5_SPH' , 1 , 'SPH' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', '' , 'ERA5_LSM'
sn_prec = 'ERA5_MTPR' , 1 , 'MTPR' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', '' , 'ERA5_LSM'
sn_qlw = 'ERA5_MSDWLWRF' , 1 ,'MSDWLWRF' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', '' , 'ERA5_LSM'
sn_qsr = 'ERA5_MSDWSWRF' , 1 ,'MSDWSWRF' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', '' , 'ERA5_LSM'
sn_snow = 'ERA5_MSR' , 1 , 'MSR' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', '' , 'ERA5_LSM'
sn_tair = 'ERA5_T2M' , 1 , 'T2M' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', '' , 'ERA5_LSM'
sn_wndi = 'ERA5_U10' , 1 , 'U10' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', 'Uwnd' , 'ERA5_LSM'
sn_wndj = 'ERA5_V10' , 1 , 'V10' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', 'Vwnd' , 'ERA5_LSM'
sn_slp = 'ERA5_MSL' , 1 , 'MSL' , .true. , .false., 'yearly' ,'weights_era5_amm7_bicubic.nc', '' , 'ERA5_LSM'
/
!-----------------------------------------------------------------------
&namsbc_core ! namsbc_core CORE bulk formulae
!-----------------------------------------------------------------------
! ! file name ! frequency (hours) ! variable ! time interp. ! clim ! 'yearly'/ ! weights ! rotation !
! ! ! (if <0 months) ! name ! (logical) ! (T/F) ! 'monthly' ! filename ! pairing !
cn_dir='/work/n01/n01/nibrun/RUNS/AMM7/INPUTS/SBC/',
ln_taudif=.false.,
rn_pfac=1.0,
rn_vfac=1.,
rn_zqt =2.,
rn_zu =10.,
sn_humi='ERA5_SPH' ,1,'SPH' ,.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','' ,'ERA5_LSM'
sn_prec='ERA5_MTPR' ,1,'MTPR' ,.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','' ,'ERA5_LSM'
sn_qlw ='ERA5_MSDWLWRF',1,'MSDWLWRF',.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','' ,'ERA5_LSM'
sn_qsr ='ERA5_MSDWSWRF',1,'MSDWSWRF',.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','' ,'ERA5_LSM'
sn_snow='ERA5_MSR' ,1,'MSR' ,.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','' ,'ERA5_LSM'
sn_tair='ERA5_T2M' ,1,'T2M' ,.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','' ,'ERA5_LSM'
sn_wndi='ERA5_U10' ,1,'U10' ,.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','Uwnd','ERA5_LSM'
sn_wndj='ERA5_V10' ,1,'V10' ,.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','Vwnd','ERA5_LSM'
/
and for the pressure:
!-----------------------------------------------------------------------
&namsbc_apr ! Atmospheric pressure used as ocean forcing or in bulk
!-----------------------------------------------------------------------
! ! file name ! frequency (hours) ! variable ! time interp. ! clim ! 'yearly'/ ! weights ! rotation ! land/sea mask !
! ! ! (if <0 months) ! name ! (logical) ! (T/F) ! 'monthly' ! filename ! pairing ! filename !
sn_apr='ERA5_MSL' ,1,'MSL' ,.true.,.false.,'yearly','weights_era5_amm7_bicubic.nc','','ERA5_LSM'
ln_ref_apr = .false. ! ref. pressure: global mean Patm (T) or a constant (F)
ln_apr_obc = .true. ! inverse barometer added to OBC ssh data
cn_dir='/work/n01/n01/nibrun/RUNS/AMM7/INPUTS/SBC/',
/
Valerie downloaded some mean fields for the cumulated variables and from some quick tests, it seems are fairly similar to the cumulated field that I de-cumulated over a period. Example below for the Surface Thermal Radiation Downwards (J/m^2) divided by 3600s (as field is cumulated over 1h so in W/m^2) versus the Mean Surface Downward Long-Wave Radiation Flux (W/m^2). The max difference over Jan. 2010 is 0.015 W/m^2 for fields varying between 250-375 W/m^2.
- Cumulated fields are divided by 1h (time step of output) - this is a change from ERA-I where the cumulation was done over the full time step. see here for more details under Mean rates and accumulations.
- snowfall and precipitation are multiplied by 1000 (convert from m to mm) [assuming: 1 kg of rain water spread over 1 square meter of surface is 1 mm in thickness i.e. 1 kg/m2 ~ 1 mm]
I have compared the fields between original ERA-interim and new ERA5 and they look similar with similar scale. More details and extreme in ERA5 as expected.