From cbe529784634f87785e072bbc298321b119201b9 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Fri, 5 Nov 2021 09:53:55 -0600 Subject: [PATCH] #1949 Corrected tyoe and applied Juloie's suggestions --- met/docs/Users_Guide/data_io.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/met/docs/Users_Guide/data_io.rst b/met/docs/Users_Guide/data_io.rst index 48139f42f9..dcca8763f9 100644 --- a/met/docs/Users_Guide/data_io.rst +++ b/met/docs/Users_Guide/data_io.rst @@ -25,7 +25,7 @@ The MET tools use following attributes and variables for input CF Compliant NetC 1. The global attribute "Conventions". -2. The ""`standard_name `_" and "`units `_" attributes for coordinate variables. The "`axis `_" attribute ("T" or "time") must exist as the time variable if the "standard_name" attribute does not exist. +2. The "`standard_name `_" and "`units `_" attributes for coordinate variables. The "`axis `_" attribute ("T" or "time") must exist as the time variable if the "standard_name" attribute does not exist. 3. The "`coordinates `_" attribute for the data variables. It contains the coordinate variable names. @@ -126,9 +126,9 @@ MET gets the valid time from the time variable and the "forecast_reference_time" Performance with NetCDF input data ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -There is no limitation on the NetCDF file size. The size of the data variables matters than the file size. NetCDF API loads the meta data first on opening the NetCDF. It's the similar on accesing data variables. It's two API calls: getting the meta data and getting the actual data. The memory is allocated and consumed at the second API call (getting actual data). +There is no limitation on the NetCDF file size. The size of the data variables matters more than the file size. The NetCDF API loads the metadata first upon opening the NetCDF file. It's similar for accessing data variables. There are two API calls: getting the metadata and getting the actual data. The memory is allocated and consumed at the second API call (getting the actual data). -The dimensions of the data variables matter. MET requests the NetCDF data need based: 1) 1oading and processing a data plane, and 2) loading and processing the next data plane. This means an extra step for slicing with one more dimension NetCDF input data. The performance is quite different if the compression is enabled with high resolution data. NetCDF does compression per variable. The variables can have different compression levels (0 to 9, 0 is not compressed). The number of decompression is the same between 1 more and 1 less dimension NetCDF input files (combined VS separated). The difference is the amount of data to be decompressed which requires more memory. For example, let's assume the time dimension is 30. NetCDF data with 1 less dimension (no time dimension) does decompression 30 times for nx by ny dataset. NetCDF with 1 more dimension does compression 30 times for 30 by nx by ny dataset & slicing for target time offset. So it's better to have multiple NetCDF files with 1 less dimension than a big file with bigger variable data if compressed. If the compression is not enabled, the file size will be much bigger (more disk space). +The dimensions of the data variables matter. MET requests the NetCDF data needs based on: 1) loading and processing a data plane, and 2) loading and processing the next data plane. This means an extra step for slicing with one more dimension in the NetCDF input data. The performance is quite different if the compression is enabled with high resolution data. NetCDF does compression per variable. The variables can have different compression levels (0 to 9). A value of 0 means no compression, and 9 is the highest level of compression possible. The number for decompression is the same between one more and one less dimension NetCDF input files (combined VS separated). The difference is the amount of data to be decompressed which requires more memory. For example, let's assume the time dimension is 30. NetCDF data with one less dimension (no time dimension) does decompression 30 times for nx by ny dataset. NetCDF with one more dimension does compression 30 times for 30 by nx by ny dataset and slicing for target time offset. So it's better to have multiple NetCDF files with one less dimension than a big file with bigger variable data if compressed. If the compression is not enabled, the file size will be much bigger requiring more disk space. .. _Intermediate data formats: