From 8429c3038f2eb5cec4b31f3e8680de4a1bb3d0f7 Mon Sep 17 00:00:00 2001 From: Russell Senior Date: Tue, 17 Nov 2015 13:58:41 -0800 Subject: [PATCH 001/126] coordinates attribute refers to an undefined variable --- apph.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/apph.adoc b/apph.adoc index e04973c6..2ee3a536 100644 --- a/apph.adoc +++ b/apph.adoc @@ -453,19 +453,19 @@ If the profile instances have the same number of elements and the vertical coord        pressure:standard_name = "air_pressure" ;        pressure:long_name = "pressure level" ;        pressure:units = "hPa" ; -        pressure:coordinates = "time lon lat altz" ; +        pressure:coordinates = "time lon lat z" ;    float temperature(profile, z) ;        temperature:standard_name = "surface_temperature" ;        temperature:long_name = "skin temperature" ;        temperature:units = "Celsius" ; -        temperature:coordinates = "time lon lat altz" ; +        temperature:coordinates = "time lon lat z" ;    float humidity(profile, z) ;        humidity:standard_name = "relative_humidity" ;        humidity:long_name = "relative humidity" ;        humidity:units = "%" ; -        humidity:coordinates = "time lon lat altz" ; +        humidity:coordinates = "time lon lat z" ; attributes:    :featureType = "profile"; From 94875c55a97596c9055174244714baaa1d9ffee4 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Sat, 22 Apr 2017 08:19:12 -0500 Subject: [PATCH 002/126] simple geometry chapter 7 and appendix e. --- appe.adoc | 59 +++++++++++++++++++++++++++++++++++- ch07.adoc | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++++ history.adoc | 3 ++ 3 files changed, 146 insertions(+), 1 deletion(-) diff --git a/appe.adoc b/appe.adoc index 35573f15..f5401859 100644 --- a/appe.adoc +++ b/appe.adoc @@ -54,5 +54,62 @@ respect to the specified dimension. | `variance` | __u^2^__ | Variance |=============== - +[[example-e.1, Example e.1, "timeseries-with-geometry"]] +[caption="Example E.1. "] +.Timeseries with geometry. +==== +---- +dimensions: + instance = 2 ; + node = 5 ; + time = 4 ; +variables: + int time(time) ; + time:units = "days since 2000-01-01" ; + double lat(instance) ; + lat:units = "degrees_north" ; + lat:standard_name = "latitude" ; + lat:geometry = "geometry_container" ; + double lon(instance) ; + lon:units = "degrees_east" ; + lon:standard_name = "longitude" ; + lon:geometry = "geometry_container" ; + int crs ; + crs:grid_mapping_name = "latitude_longitude" ; + crs:longitude_of_prime_meridian = 0.0 ; + crs:semi_major_axis = 6378137.0 ; + crs:inverse_flattening = 298.257223563 ; + int geometry_container ; + geometry_container:geometry_type = "line" ; + geometry_container:node_count = "node_count" ; + geometry_container:node_coordinates = "x y" ; + int node_count(instance) ; + double x(node) ; + x:units = "degrees_east" ; + x:standard_name = "longitude" ; + x:cf_role = "geometry_x_node" ; + double y(node) ; + y:units = "degrees_north" ; + y:standard_name = "latitude" ; + y:cf_role = "geometry_y_node" ; + double someData(instance, time) ; + someData:coordinates = "time lat lon" ; + someData:grid_mapping = "crs" ; + someData:geometry = "geometry_container" ; +// global attributes: + :Conventions = "CF-1.8" ; + :featureType = "timeSeries" ; +data: + time = 1, 2, 3, 4 ; + lat = 30, 50 ; + lon = 10, 60 ; + someData = + 1, 2, 3, 4, + 1, 2, 3, 4 ; + node_count = 3, 2 ; + x = 30, 10, 40, 50, 50 ; + y = 10, 30, 40, 60, 50 ; +---- +The time series variable, someData, is associated with line geometries via the geometry attribute. The first line geometry is comprised of three nodes, while the second has two nodes. Client applications unaware of CF geometries can fall back to the lat and lon variables to locate feature instances in space. In this example, lat and lon coordinates are identical to the first node in each line geometry, though any representative point could be used. +==== diff --git a/ch07.adoc b/ch07.adoc index e37d500e..13cc6e45 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -544,3 +544,88 @@ data: // time coordinates translated to date/time format "2000-8-1 6:00:00", "2000-9-1 6:00:00" ; ---- ==== + +[[spatial-geometries, Section 7.5, "Spatial Geometries"]] +=== Spatial Geometries + +For many geospatial applications, data values are associated with a spatial geometry (e.g., the average monthly rainfall in the UK). Although cells with an arbitrary number of multiple vertices can be described using <>, spatial geometries contain an arbitrary number of nodes for each geometry and include line and __multipart__ geometries (e.g., the different islands of the UK). The approach described here specifies how to encode such geometries following the pattern in **9.3.3 Contiguous ragged array representation** and attach them to variables in a way that is consistent with the cell bounds approach. + +A __geometry__ is usually thought to be a spatial representation of a real-world feature. It can be disjoint, having multiple parts. Geometry types are limited to __point__, __multipoint__, __line__, __multiline__, __polygon__ and __multipolygon__ types. Other types exist and may be introduced in a later version of the specification. Similar to other geospatial data formats, geometries are encoded as ordered sets of geospatial __nodes__. The connection between nodes is assumed to be linear in the coordinate reference system the nodes are defined in. Parametric geometries or otherwise curved features may be supported in the future. + +All geometries are made up of one or more nodes. The geometry type specifies the set of topological assumptions to be applied to relate the nodes. For example, __multipoint__ and __line__ geometries are nearly the same except nodes are interpreted as being connected for lines. Lines and polygons are also nearly the same except the first and last nodes must be identical for polygons. Polygons that have holes, such as waterbodies in a land unit, are encoded as a collection of polygon ring parts, each identified as __exterior__ or __interior__ polygons. Multipart geometries, such as multiple lines representing the same river or multiple islands representing the same jurisdiction, are encoded as collections of un-connected points, lines, or polygons that are logically grouped into a single geometry. + +While this geometry encoding is applicable to any variable that shares a dimension with a set of geometry, the application it was originally designed for requires that geometry be joined to the instance dimension of a Discrete Sampling Geometry `timeSeries` featureType. In this case, any data variable can be given a `geometry` attribute that is to be interpreted as the representative geometry for the quantity held in the variable. An example of this is areal average precipitation over a watershed. An example of line geometry with time series data is given in <>. + +==== Geometry Variables and Attributes + +A set of geometries can be added to a file by inserting all required data variables and a __geometry container__ variable that acts as a container for attributes that describe a set of geometries. A **`geometry`** attribute containing the name of a geometry container variable can be added to any variable that shares a dimension with the geometries. The geometry container must hold **`geometry_type`** and **`node_coordinates`** attributes. Depending on the **`geometry_type`**, the geometry container may also need to contain a **`node_count`**, **`part_node_count`**, and **`interior_ring`** attribute. These attributes are described in detail below. + +The **`geometry_type`** attribute must be carried by a geometry container variable and indicates the type of geometry present. Its allowable values are: __point__, __multipoint__, __line__, __multiline__, __polygon__, __multipolygon__. The **`node_coordinates`** attribute must be carried by a geometry container variable and contains the space delimited names of the x and y (and z) variables that contain geometry node coordinates. + +For all geometry types except __point__, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. Note that the node count may span multiple geometry parts. For __multiline__, __multipolygon__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry part. Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required. + +For __polygon__ and __multipolygon__ geometries with holes, the geometry container variable must have an **`interior_ring`** attribute that contains the name of a variable that indicates if the polygon parts are interior rings (i.e., holes) or not. The variable indicated by the **`interior_ring`** attribute should contain the value 0 to indicate an exterior ring polygon and 1 to indicate an interior ring polygon. Note that single part polygons can have interior rings; multipart polygons are distinct in that they have more than one exterior ring. + +The variables that contain geometry node coordinate data, indicated by the **`node_coordinates`** attribute on the __geometry container__ variable, are also identifiable through the use of a required **`cf_role`** attribute. Allowable values are __geometry_x_node__, __geometry_y_node__, and __geometry_z_node__. + +==== Encoding Geometries + +Geometry encoding follows a similar pattern to the contiguous ragged array approach in **9.3.3 Contiguous ragged array representation** with some modification to suit the spatial geometry use case rather than observational time series. All spatial data are encoded in the variables indicated by the **`node_coordinates`** and appropriate **`cf_role`** attribute. These node variables should be one dimensional and total number of nodes long. There are three one dimensional variables that are used to break up and interpret the node variabes: **`node_count`**, **`part_node_count`**, and **`interior_ring`**. + +For geometry types requiring a **`node_count`** attribute, the node count variable should be the number of geometries long and indicate the number of nodes per geometry. For geometry types requireing a **`part_node_count`** attribute, the part node count variable should be the number of geometry parts long and indicate the number of nodes per geometry part. For geometry types requireing an **`interior_ring`** attribute, the interior ring variable should be the number of geometry parts long and contain 0s and 1s to indicate exterior or interior. + +The ecosystem of polygon specifications and software implementations of those specifications varies in how polygons are encoded. Nodes within a polygon exterior or interior ring are typically encoded in opposite clockwise or anticlockwise direction around the polygon. This is important for operations such as caluclating area. CF requires that outer rings be encoded in anticlockwise order and interior rings be encoded in clockwise order. CF also requires that the first and last node in a polygon be identical to ensure polygon rings are complete. + +A coordinate reference system (CRS) (referred to as a grid mapping elsewhere in the CF convention) is strictly required for geometries. The normal CF practice, of attaching a **`grid_mapping`** attribute--containing the name of a CRS container variable--to a data variable, can be used and the **`grid_mapping`** CRS should be assumed to apply to the geometry. However, the normal **`grid_mapping`**, which typically applies to auxiliary coordinate variables and remains optional for use with geometries, can be overridden by attaching a **`crs`** attribute that contains the name of a CRS container variable to the geometry container variable. If a grid_mapping is not present on a data variable linked to geometry, a crs attribute is required. + +[[complete-multipolygon-example]] +[caption="Example 7.14. "] +.A multipolygon with holes +==== +This example demonstrates the use of all potential attributes and variables for encoding geometries. +---- +dimensions: + node = 25 ; + instance = 1 ; + part = 6 ; +variables: + double x(node) ; + x:units = "degrees_east" ; + x:standard_name = "longitude" ; + x:cf_role = "geometry_x_node" ; + double y(node) ; + y:units = "degrees_north" ; + y:standard_name = "latitude" ; + y:cf_role = "geometry_y_node" ; + float geometry_container ; + geometry_container:geometry_type = "multipolygon" ; + geometry_container:node_count = "node_count" ; + geometry_container:node_coordinates = "x y" ; + geometry_container:crs = "crs" ; + geometry_container:part_node_count = "part_node_count" ; + geometry_container:interior_ring = "interior_ring" ; + int node_count(instance) ; + node_count:long_name = "count of coordinates in each instance geometry" ; + int part_node_count(part) ; + part_node_count:long_name = "count of nodes in each geometry part" ; + int interior_ring(part) ; + interior_ring:long_name = "type of each geometry part" ; + float crs ; + crs:grid_mapping_name = "latitude_longitude" ; + crs:semi_major_axis = 6378137. ; + crs:inverse_flattening = 298.257223563 ; + crs:longitude_of_prime_meridian = 0. ; +// global attributes: + :Conventions = "CF-1.8" ; +data: + x = 0, 20, 20, 0, 0, 1, 10, 19, 1, 5, 7, 9, 5, 11, 13, 15, 11, 5, 9, 7, 5, + 11, 15, 13, 11 ; + y = 0, 0, 20, 20, 0, 1, 5, 1, 1, 15, 19, 15, 15, 15, 19, 15, 15, 25, 25, 29, + 25, 25, 25, 29, 25 ; + geometry_container = 0. ; + node_count = 25 ; + part_node_count = 5, 4, 4, 4, 4, 4 ; + interior_ring = 0, 1, 1, 1, 0, 0 ; + crs = 0. ; +---- +==== diff --git a/history.adoc b/history.adoc index a08b85a9..d2672384 100644 --- a/history.adoc +++ b/history.adoc @@ -156,3 +156,6 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .06 April 2017 . Ticket #70, Connecting coordinates to Grid Mapping variables: revisions in Section 5.6 and Examples 5.10 and 5.12 + +.07 April 2017 +. New Simple Geometry section 7.5 and content in appendix E. From 54fc86c8652045c0790534f5e2b63a13caf9e337 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Tue, 9 May 2017 08:11:27 -0500 Subject: [PATCH 003/126] addressing comment from @rsignell-usgs re: instance and feature dimensions --- ch07.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 13cc6e45..198699ed 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -554,11 +554,11 @@ A __geometry__ is usually thought to be a spatial representation of a real-world All geometries are made up of one or more nodes. The geometry type specifies the set of topological assumptions to be applied to relate the nodes. For example, __multipoint__ and __line__ geometries are nearly the same except nodes are interpreted as being connected for lines. Lines and polygons are also nearly the same except the first and last nodes must be identical for polygons. Polygons that have holes, such as waterbodies in a land unit, are encoded as a collection of polygon ring parts, each identified as __exterior__ or __interior__ polygons. Multipart geometries, such as multiple lines representing the same river or multiple islands representing the same jurisdiction, are encoded as collections of un-connected points, lines, or polygons that are logically grouped into a single geometry. -While this geometry encoding is applicable to any variable that shares a dimension with a set of geometry, the application it was originally designed for requires that geometry be joined to the instance dimension of a Discrete Sampling Geometry `timeSeries` featureType. In this case, any data variable can be given a `geometry` attribute that is to be interpreted as the representative geometry for the quantity held in the variable. An example of this is areal average precipitation over a watershed. An example of line geometry with time series data is given in <>. +While this geometry encoding is applicable to any variable that shares a dimension with a set of geometry, the application it was originally designed for requires that geometry be joined to the instance dimension of a Discrete Sampling Geometry `timeSeries` featureType. The instance dimension, in the more general case of features, specifies the number of features in the collection and is also referred to as the feature dimension. In this case, any data variable can be given a `geometry` attribute that is to be interpreted as the representative geometry for the quantity held in the variable. An example of this is areal average precipitation over a watershed. An example of line geometry with time series data is given in <>. ==== Geometry Variables and Attributes -A set of geometries can be added to a file by inserting all required data variables and a __geometry container__ variable that acts as a container for attributes that describe a set of geometries. A **`geometry`** attribute containing the name of a geometry container variable can be added to any variable that shares a dimension with the geometries. The geometry container must hold **`geometry_type`** and **`node_coordinates`** attributes. Depending on the **`geometry_type`**, the geometry container may also need to contain a **`node_count`**, **`part_node_count`**, and **`interior_ring`** attribute. These attributes are described in detail below. +A set of geometries can be added to a file by inserting all required data variables and a __geometry container__ variable that acts as a container for attributes that describe a set of geometries. A **`geometry`** attribute containing the name of a geometry container variable can be added to any variable that shares the feature dimension with the geometries. The geometry container must hold **`geometry_type`** and **`node_coordinates`** attributes. Depending on the **`geometry_type`**, the geometry container may also need to contain a **`node_count`**, **`part_node_count`**, and **`interior_ring`** attribute. These attributes are described in detail below. The **`geometry_type`** attribute must be carried by a geometry container variable and indicates the type of geometry present. Its allowable values are: __point__, __multipoint__, __line__, __multiline__, __polygon__, __multipolygon__. The **`node_coordinates`** attribute must be carried by a geometry container variable and contains the space delimited names of the x and y (and z) variables that contain geometry node coordinates. @@ -586,7 +586,7 @@ This example demonstrates the use of all potential attributes and variables for ---- dimensions: node = 25 ; - instance = 1 ; + feature = 1 ; part = 6 ; variables: double x(node) ; @@ -604,8 +604,8 @@ variables: geometry_container:crs = "crs" ; geometry_container:part_node_count = "part_node_count" ; geometry_container:interior_ring = "interior_ring" ; - int node_count(instance) ; - node_count:long_name = "count of coordinates in each instance geometry" ; + int node_count(feature) ; + node_count:long_name = "count of coordinates in each feature geometry" ; int part_node_count(part) ; part_node_count:long_name = "count of nodes in each geometry part" ; int interior_ring(part) ; From 5f8c0859b7158fd3ebe94a898bbd2837233a22a3 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Thu, 8 Jun 2017 09:50:17 +1000 Subject: [PATCH 004/126] updates per review --- appe.adoc | 58 -------------------------- ch07.adoc | 122 ++++++++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 99 insertions(+), 81 deletions(-) diff --git a/appe.adoc b/appe.adoc index f5401859..e98eed48 100644 --- a/appe.adoc +++ b/appe.adoc @@ -55,61 +55,3 @@ respect to the specified dimension. |=============== -[[example-e.1, Example e.1, "timeseries-with-geometry"]] -[caption="Example E.1. "] -.Timeseries with geometry. -==== ----- -dimensions: - instance = 2 ; - node = 5 ; - time = 4 ; -variables: - int time(time) ; - time:units = "days since 2000-01-01" ; - double lat(instance) ; - lat:units = "degrees_north" ; - lat:standard_name = "latitude" ; - lat:geometry = "geometry_container" ; - double lon(instance) ; - lon:units = "degrees_east" ; - lon:standard_name = "longitude" ; - lon:geometry = "geometry_container" ; - int crs ; - crs:grid_mapping_name = "latitude_longitude" ; - crs:longitude_of_prime_meridian = 0.0 ; - crs:semi_major_axis = 6378137.0 ; - crs:inverse_flattening = 298.257223563 ; - int geometry_container ; - geometry_container:geometry_type = "line" ; - geometry_container:node_count = "node_count" ; - geometry_container:node_coordinates = "x y" ; - int node_count(instance) ; - double x(node) ; - x:units = "degrees_east" ; - x:standard_name = "longitude" ; - x:cf_role = "geometry_x_node" ; - double y(node) ; - y:units = "degrees_north" ; - y:standard_name = "latitude" ; - y:cf_role = "geometry_y_node" ; - double someData(instance, time) ; - someData:coordinates = "time lat lon" ; - someData:grid_mapping = "crs" ; - someData:geometry = "geometry_container" ; -// global attributes: - :Conventions = "CF-1.8" ; - :featureType = "timeSeries" ; -data: - time = 1, 2, 3, 4 ; - lat = 30, 50 ; - lon = 10, 60 ; - someData = - 1, 2, 3, 4, - 1, 2, 3, 4 ; - node_count = 3, 2 ; - x = 30, 10, 40, 50, 50 ; - y = 10, 30, 40, 60, 50 ; ----- -The time series variable, someData, is associated with line geometries via the geometry attribute. The first line geometry is comprised of three nodes, while the second has two nodes. Client applications unaware of CF geometries can fall back to the lat and lon variables to locate feature instances in space. In this example, lat and lon coordinates are identical to the first node in each line geometry, though any representative point could be used. -==== diff --git a/ch07.adoc b/ch07.adoc index 198699ed..fb535b8c 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -546,40 +546,116 @@ data: // time coordinates translated to date/time format ==== [[spatial-geometries, Section 7.5, "Spatial Geometries"]] -=== Spatial Geometries - -For many geospatial applications, data values are associated with a spatial geometry (e.g., the average monthly rainfall in the UK). Although cells with an arbitrary number of multiple vertices can be described using <>, spatial geometries contain an arbitrary number of nodes for each geometry and include line and __multipart__ geometries (e.g., the different islands of the UK). The approach described here specifies how to encode such geometries following the pattern in **9.3.3 Contiguous ragged array representation** and attach them to variables in a way that is consistent with the cell bounds approach. - -A __geometry__ is usually thought to be a spatial representation of a real-world feature. It can be disjoint, having multiple parts. Geometry types are limited to __point__, __multipoint__, __line__, __multiline__, __polygon__ and __multipolygon__ types. Other types exist and may be introduced in a later version of the specification. Similar to other geospatial data formats, geometries are encoded as ordered sets of geospatial __nodes__. The connection between nodes is assumed to be linear in the coordinate reference system the nodes are defined in. Parametric geometries or otherwise curved features may be supported in the future. - -All geometries are made up of one or more nodes. The geometry type specifies the set of topological assumptions to be applied to relate the nodes. For example, __multipoint__ and __line__ geometries are nearly the same except nodes are interpreted as being connected for lines. Lines and polygons are also nearly the same except the first and last nodes must be identical for polygons. Polygons that have holes, such as waterbodies in a land unit, are encoded as a collection of polygon ring parts, each identified as __exterior__ or __interior__ polygons. Multipart geometries, such as multiple lines representing the same river or multiple islands representing the same jurisdiction, are encoded as collections of un-connected points, lines, or polygons that are logically grouped into a single geometry. - -While this geometry encoding is applicable to any variable that shares a dimension with a set of geometry, the application it was originally designed for requires that geometry be joined to the instance dimension of a Discrete Sampling Geometry `timeSeries` featureType. The instance dimension, in the more general case of features, specifies the number of features in the collection and is also referred to as the feature dimension. In this case, any data variable can be given a `geometry` attribute that is to be interpreted as the representative geometry for the quantity held in the variable. An example of this is areal average precipitation over a watershed. An example of line geometry with time series data is given in <>. - -==== Geometry Variables and Attributes +=== Geometries + +For many geospatial applications, data values are associated with a geometry, which is a spatial representation of a real-world feature, for instance a time-series of areal average precipitation over a watershed. +Polygonal cells with an arbitrary number of vertices can be described using <>, but in that case every cell must have the same number of vertices. +In contrast, each geometry associated with a given data variable may have a different number of nodes, the geometries may be lines (as alternatives to points and polygons), and they may be __multipart__ i.e include several disjoint parts. +The approach described here specifies how to encode such geometries following the pattern in **9.3.3 Contiguous ragged array representation** and attach them to variables in a way that is consistent with the cell bounds approach. + +All geometries are made up of one or more nodes. +The geometry type specifies the set of topological assumptions to be applied to relate the nodes. +For example, __multipoint__ and __line__ geometries are nearly the same except nodes are interpreted as being connected for lines. +Lines and polygons are also nearly the same except that it should be assumed that the first and last node are connected. +Note that CF does not require the first and last node to be identical but allows them to be coincident if desired. +Polygons that have holes, such as waterbodies in a land unit, are encoded as a collection of polygon ring parts, each identified as __exterior__ or __interior__ polygons. +Multipart geometries, such as multiple lines representing the same river or multiple islands representing the same jurisdiction, are encoded as collections of un-connected points, lines, or polygons that are logically grouped into a single geometry. + +Any data variable can be given a `geometry` attribute that indicates the geometry for the quantity held in the variable. +One of the dimensions of the data variable must be the number of geometries to which the data applies. +As shown in Example 7.14, if the data variable has a discrete sampling geometry, the number of geometries is the length of the instance dimension (Section 9.2). + +[["timeseries-with-geometry"]] +[caption="Example 7.14. "] +.Timeseries with geometry. +==== +---- +dimensions: + instance = 2 ; + node = 5 ; + time = 4 ; +variables: + int time(time) ; + time:units = "days since 2000-01-01" ; + double lat(instance) ; + lat:units = "degrees_north" ; + lat:standard_name = "latitude" ; + lat:geometry = "geometry_container" ; + double lon(instance) ; + lon:units = "degrees_east" ; + lon:standard_name = "longitude" ; + lon:geometry = "geometry_container" ; + int datum ; + datum:grid_mapping_name = "latitude_longitude" ; + datum:longitude_of_prime_meridian = 0.0 ; + datum:semi_major_axis = 6378137.0 ; + datum:inverse_flattening = 298.257223563 ; + int geometry_container ; + geometry_container:geometry_type = "line" ; + geometry_container:node_count = "node_count" ; + geometry_container:node_coordinates = "x y" ; + int node_count(instance) ; + double x(node) ; + x:units = "degrees_east" ; + x:standard_name = "longitude" ; + x:cf_role = "geometry_x_node" ; + double y(node) ; + y:units = "degrees_north" ; + y:standard_name = "latitude" ; + y:cf_role = "geometry_y_node" ; + double someData(instance, time) ; + someData:coordinates = "time lat lon" ; + someData:grid_mapping = "datum" ; + someData:geometry = "geometry_container" ; +// global attributes: + :Conventions = "CF-1.8" ; + :featureType = "timeSeries" ; +data: + time = 1, 2, 3, 4 ; + lat = 30, 50 ; + lon = 10, 60 ; + someData = + 1, 2, 3, 4, + 1, 2, 3, 4 ; + node_count = 3, 2 ; + x = 30, 10, 40, 50, 50 ; + y = 10, 30, 40, 60, 50 ; +---- +The time series variable, someData, is associated with line geometries via the geometry attribute. The first line geometry is comprised of three nodes, while the second has two nodes. Client applications unaware of CF geometries can fall back to the lat and lon variables to locate feature instances in space. In this example, lat and lon coordinates are identical to the first node in each line geometry, though any representative point could be used. +==== -A set of geometries can be added to a file by inserting all required data variables and a __geometry container__ variable that acts as a container for attributes that describe a set of geometries. A **`geometry`** attribute containing the name of a geometry container variable can be added to any variable that shares the feature dimension with the geometries. The geometry container must hold **`geometry_type`** and **`node_coordinates`** attributes. Depending on the **`geometry_type`**, the geometry container may also need to contain a **`node_count`**, **`part_node_count`**, and **`interior_ring`** attribute. These attributes are described in detail below. -The **`geometry_type`** attribute must be carried by a geometry container variable and indicates the type of geometry present. Its allowable values are: __point__, __multipoint__, __line__, __multiline__, __polygon__, __multipolygon__. The **`node_coordinates`** attribute must be carried by a geometry container variable and contains the space delimited names of the x and y (and z) variables that contain geometry node coordinates. +A __geometry container__ variable acts as a container for attributes that describe a set of geometries. +The **`geometry`** attribute of the data variable contains the name of a geometry container variable. +The geometry container variable must hold **`geometry_type`** and **`node_coordinates`** attributes. +A **`grid_mapping`** attribute, which would usually by carried by a data variable, can be carried by the geometry container variable. -For all geometry types except __point__, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. Note that the node count may span multiple geometry parts. For __multiline__, __multipolygon__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry part. Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required. +The **`geometry_type`** attribute indicates the type of geometry present. +Its allowable values are: __point__, __multipoint__, __line__, __multiline__, __polygon__, __multipolygon__. +The **`node_coordinates`** attribute contains the blank delimited names of the variables that contain geometry node coordinates (one variable for each spatial dimension). -For __polygon__ and __multipolygon__ geometries with holes, the geometry container variable must have an **`interior_ring`** attribute that contains the name of a variable that indicates if the polygon parts are interior rings (i.e., holes) or not. The variable indicated by the **`interior_ring`** attribute should contain the value 0 to indicate an exterior ring polygon and 1 to indicate an interior ring polygon. Note that single part polygons can have interior rings; multipart polygons are distinct in that they have more than one exterior ring. -The variables that contain geometry node coordinate data, indicated by the **`node_coordinates`** attribute on the __geometry container__ variable, are also identifiable through the use of a required **`cf_role`** attribute. Allowable values are __geometry_x_node__, __geometry_y_node__, and __geometry_z_node__. +The geometry node coordinate variables must each have a **`cf_role`** attribute whose allowable values are __geometry_x_node__, __geometry_y_node__, and __geometry_z_node__. +The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. +The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. +Polygon exterior rings must be put in anticlockwise order (viewed from above) and polygon interior rings in clockwise order. +They are put in opposite orders to facilitate calculation of area and consistency with the typical implementation pattern. -==== Encoding Geometries -Geometry encoding follows a similar pattern to the contiguous ragged array approach in **9.3.3 Contiguous ragged array representation** with some modification to suit the spatial geometry use case rather than observational time series. All spatial data are encoded in the variables indicated by the **`node_coordinates`** and appropriate **`cf_role`** attribute. These node variables should be one dimensional and total number of nodes long. There are three one dimensional variables that are used to break up and interpret the node variabes: **`node_count`**, **`part_node_count`**, and **`interior_ring`**. +For all geometry types except __point__, in which each geometry contains a single node, when more than one geometry is present, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. +The node count is the total nodes in all the parts. -For geometry types requiring a **`node_count`** attribute, the node count variable should be the number of geometries long and indicate the number of nodes per geometry. For geometry types requireing a **`part_node_count`** attribute, the part node count variable should be the number of geometry parts long and indicate the number of nodes per geometry part. For geometry types requireing an **`interior_ring`** attribute, the interior ring variable should be the number of geometry parts long and contain 0s and 1s to indicate exterior or interior. -The ecosystem of polygon specifications and software implementations of those specifications varies in how polygons are encoded. Nodes within a polygon exterior or interior ring are typically encoded in opposite clockwise or anticlockwise direction around the polygon. This is important for operations such as caluclating area. CF requires that outer rings be encoded in anticlockwise order and interior rings be encoded in clockwise order. CF also requires that the first and last node in a polygon be identical to ensure polygon rings are complete. +For __multiline__, __multipolygon__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute indicates a variable of the count of nodes per geometry part. +Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required. +The single dimension of the part node count variable should equal the total number of parts in all the geometries. -A coordinate reference system (CRS) (referred to as a grid mapping elsewhere in the CF convention) is strictly required for geometries. The normal CF practice, of attaching a **`grid_mapping`** attribute--containing the name of a CRS container variable--to a data variable, can be used and the **`grid_mapping`** CRS should be assumed to apply to the geometry. However, the normal **`grid_mapping`**, which typically applies to auxiliary coordinate variables and remains optional for use with geometries, can be overridden by attaching a **`crs`** attribute that contains the name of a CRS container variable to the geometry container variable. If a grid_mapping is not present on a data variable linked to geometry, a crs attribute is required. +For __polygon__ and __multipolygon__ geometries with holes, the geometry container variable must have an **`interior_ring`** attribute that contains the name of a variable that indicates if the polygon parts are interior rings (i.e., holes) or not. +This interior ring variable should contain the value 0 to indicate an exterior ring polygon and 1 to indicate an interior ring polygon. +The single dimension of the interior ring variable should be the same dimension as that of the part node count variable. [[complete-multipolygon-example]] -[caption="Example 7.14. "] +[caption="Example 7.15. "] .A multipolygon with holes ==== This example demonstrates the use of all potential attributes and variables for encoding geometries. @@ -601,7 +677,7 @@ variables: geometry_container:geometry_type = "multipolygon" ; geometry_container:node_count = "node_count" ; geometry_container:node_coordinates = "x y" ; - geometry_container:crs = "crs" ; + geometry_container:grid_mapping = "crs" ; geometry_container:part_node_count = "part_node_count" ; geometry_container:interior_ring = "interior_ring" ; int node_count(feature) ; From 030ceb4b3860edb27a0a3da0fd4128744c20cdcd Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Tue, 13 Jun 2017 20:16:50 -0500 Subject: [PATCH 005/126] update to example --- ch07.adoc | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index fb535b8c..361e8cc6 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -661,9 +661,9 @@ The single dimension of the interior ring variable should be the same dimension This example demonstrates the use of all potential attributes and variables for encoding geometries. ---- dimensions: - node = 25 ; - feature = 1 ; - part = 6 ; + node = 12 ; + feature = 2 ; + part = 4 ; variables: double x(node) ; x:units = "degrees_east" ; @@ -681,11 +681,8 @@ variables: geometry_container:part_node_count = "part_node_count" ; geometry_container:interior_ring = "interior_ring" ; int node_count(feature) ; - node_count:long_name = "count of coordinates in each feature geometry" ; int part_node_count(part) ; - part_node_count:long_name = "count of nodes in each geometry part" ; int interior_ring(part) ; - interior_ring:long_name = "type of each geometry part" ; float crs ; crs:grid_mapping_name = "latitude_longitude" ; crs:semi_major_axis = 6378137. ; @@ -694,14 +691,12 @@ variables: // global attributes: :Conventions = "CF-1.8" ; data: - x = 0, 20, 20, 0, 0, 1, 10, 19, 1, 5, 7, 9, 5, 11, 13, 15, 11, 5, 9, 7, 5, - 11, 15, 13, 11 ; - y = 0, 0, 20, 20, 0, 1, 5, 1, 1, 15, 19, 15, 15, 15, 19, 15, 15, 25, 25, 29, - 25, 25, 25, 29, 25 ; + x = 20, 10, 0, 5, 10, 15, 20, 10, 0, 50, 40, 30 ; + y = 0, 15, 0, 5, 10, 5, 20, 35, 20, 0, 15, 0 ; geometry_container = 0. ; - node_count = 25 ; - part_node_count = 5, 4, 4, 4, 4, 4 ; - interior_ring = 0, 1, 1, 1, 0, 0 ; + node_count = 9, 3 ; + part_node_count = 3, 3, 3, 3 ; + interior_ring = 0, 1, 0, 0 ; crs = 0. ; ---- ==== From c0d8cc2f25a0328f55a541237e2c09d325affcb5 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Wed, 14 Jun 2017 09:33:26 -0500 Subject: [PATCH 006/126] missed a spatial that should have been removed --- ch07.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.adoc b/ch07.adoc index 361e8cc6..71530ce6 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -545,7 +545,7 @@ data: // time coordinates translated to date/time format ---- ==== -[[spatial-geometries, Section 7.5, "Spatial Geometries"]] +[[geometries, Section 7.5, "Geometries"]] === Geometries For many geospatial applications, data values are associated with a geometry, which is a spatial representation of a real-world feature, for instance a time-series of areal average precipitation over a watershed. From d351c95af71ffd1c7d68151f2fa44bf3993c219d Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 16 Jun 2017 07:11:11 -0500 Subject: [PATCH 007/126] add lat/lon to example and update per review --- ch07.adoc | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 71530ce6..f1599b28 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -549,8 +549,9 @@ data: // time coordinates translated to date/time format === Geometries For many geospatial applications, data values are associated with a geometry, which is a spatial representation of a real-world feature, for instance a time-series of areal average precipitation over a watershed. -Polygonal cells with an arbitrary number of vertices can be described using <>, but in that case every cell must have the same number of vertices. -In contrast, each geometry associated with a given data variable may have a different number of nodes, the geometries may be lines (as alternatives to points and polygons), and they may be __multipart__ i.e include several disjoint parts. +Polygonal cells with an arbitrary number of vertices can be described using <>, but in that case every cell must have the same number of vertices and must be single polygon rings. +In contrast, each geometry may have a different number of nodes, the geometries may be lines (as alternatives to points and polygons), and they may be __multipart__ i.e include several disjoint parts. +While line and point geometries don't describe an interval along a dimension as the traditional cell bounds described above do, they do describe the extent of a geometry or real-world feature so are included in this section. The approach described here specifies how to encode such geometries following the pattern in **9.3.3 Contiguous ragged array representation** and attach them to variables in a way that is consistent with the cell bounds approach. All geometries are made up of one or more nodes. @@ -673,11 +674,20 @@ variables: y:units = "degrees_north" ; y:standard_name = "latitude" ; y:cf_role = "geometry_y_node" ; + double lat(feature) ; + lat:units = "degrees_north" ; + lat:standard_name = "latitude" ; + lat:geometry = "geometry_container" ; + double lon(feature) ; + lon:units = "degrees_east" ; + lon:standard_name = "longitude" ; + lon:geometry = "geometry_container" ; float geometry_container ; geometry_container:geometry_type = "multipolygon" ; geometry_container:node_count = "node_count" ; geometry_container:node_coordinates = "x y" ; geometry_container:grid_mapping = "crs" ; + geometry_container:coordinates = "lat lon" geometry_container:part_node_count = "part_node_count" ; geometry_container:interior_ring = "interior_ring" ; int node_count(feature) ; @@ -693,7 +703,6 @@ variables: data: x = 20, 10, 0, 5, 10, 15, 20, 10, 0, 50, 40, 30 ; y = 0, 15, 0, 5, 10, 5, 20, 35, 20, 0, 15, 0 ; - geometry_container = 0. ; node_count = 9, 3 ; part_node_count = 3, 3, 3, 3 ; interior_ring = 0, 1, 0, 0 ; From 958de4556e47ef5535b61faa6418afe92278d6bd Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 16 Jun 2017 07:19:05 -0500 Subject: [PATCH 008/126] notes related to geometry without associated data --- ch07.adoc | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index f1599b28..ebeb9a35 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -629,7 +629,7 @@ The time series variable, someData, is associated with line geometries via the g A __geometry container__ variable acts as a container for attributes that describe a set of geometries. The **`geometry`** attribute of the data variable contains the name of a geometry container variable. The geometry container variable must hold **`geometry_type`** and **`node_coordinates`** attributes. -A **`grid_mapping`** attribute, which would usually by carried by a data variable, can be carried by the geometry container variable. +A **`grid_mapping`** and **`coordinates`** attribute, which would usually by carried by a data variable, can be carried by the geometry container variable. The **`geometry_type`** attribute indicates the type of geometry present. Its allowable values are: __point__, __multipoint__, __line__, __multiline__, __polygon__, __multipolygon__. @@ -659,7 +659,7 @@ The single dimension of the interior ring variable should be the same dimension [caption="Example 7.15. "] .A multipolygon with holes ==== -This example demonstrates the use of all potential attributes and variables for encoding geometries. +This example demonstrates geometry without associated data variables and the use of all potential attributes and variables for encoding geometries. ---- dimensions: node = 12 ; @@ -686,18 +686,18 @@ variables: geometry_container:geometry_type = "multipolygon" ; geometry_container:node_count = "node_count" ; geometry_container:node_coordinates = "x y" ; - geometry_container:grid_mapping = "crs" ; + geometry_container:grid_mapping = "datum" ; geometry_container:coordinates = "lat lon" geometry_container:part_node_count = "part_node_count" ; geometry_container:interior_ring = "interior_ring" ; int node_count(feature) ; int part_node_count(part) ; int interior_ring(part) ; - float crs ; - crs:grid_mapping_name = "latitude_longitude" ; - crs:semi_major_axis = 6378137. ; - crs:inverse_flattening = 298.257223563 ; - crs:longitude_of_prime_meridian = 0. ; + float datum ; + datum:grid_mapping_name = "latitude_longitude" ; + datum:semi_major_axis = 6378137. ; + datum:inverse_flattening = 298.257223563 ; + datum:longitude_of_prime_meridian = 0. ; // global attributes: :Conventions = "CF-1.8" ; data: @@ -706,6 +706,5 @@ data: node_count = 9, 3 ; part_node_count = 3, 3, 3, 3 ; interior_ring = 0, 1, 0, 0 ; - crs = 0. ; ---- ==== From e4ef4b2415c5f7f5d2c0675c63192ba0339c055a Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 16 Jun 2017 13:21:14 -0500 Subject: [PATCH 009/126] updates to use axis instead of cf_role --- ch07.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index ebeb9a35..357ceb65 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -599,11 +599,11 @@ variables: double x(node) ; x:units = "degrees_east" ; x:standard_name = "longitude" ; - x:cf_role = "geometry_x_node" ; + x:axis = "X" ; double y(node) ; y:units = "degrees_north" ; y:standard_name = "latitude" ; - y:cf_role = "geometry_y_node" ; + y:axis = "Y" ; double someData(instance, time) ; someData:coordinates = "time lat lon" ; someData:grid_mapping = "datum" ; @@ -636,7 +636,7 @@ Its allowable values are: __point__, __multipoint__, __line__, __multiline__, __ The **`node_coordinates`** attribute contains the blank delimited names of the variables that contain geometry node coordinates (one variable for each spatial dimension). -The geometry node coordinate variables must each have a **`cf_role`** attribute whose allowable values are __geometry_x_node__, __geometry_y_node__, and __geometry_z_node__. +The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. Polygon exterior rings must be put in anticlockwise order (viewed from above) and polygon interior rings in clockwise order. @@ -669,11 +669,11 @@ variables: double x(node) ; x:units = "degrees_east" ; x:standard_name = "longitude" ; - x:cf_role = "geometry_x_node" ; + x:axis = "X" ; double y(node) ; y:units = "degrees_north" ; y:standard_name = "latitude" ; - y:cf_role = "geometry_y_node" ; + y:axis = "Y" ; double lat(feature) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; From 9dab561eda4567a4d433dc4f5c20b0b8c6bea9b4 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Thu, 13 Jul 2017 05:15:09 -0700 Subject: [PATCH 010/126] update to grid_mapping per some review --- ch07.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.adoc b/ch07.adoc index 357ceb65..88bf4b89 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -629,7 +629,7 @@ The time series variable, someData, is associated with line geometries via the g A __geometry container__ variable acts as a container for attributes that describe a set of geometries. The **`geometry`** attribute of the data variable contains the name of a geometry container variable. The geometry container variable must hold **`geometry_type`** and **`node_coordinates`** attributes. -A **`grid_mapping`** and **`coordinates`** attribute, which would usually by carried by a data variable, can be carried by the geometry container variable. +The **`grid_mapping`** and **`coordinates`** attributes can be carried by the geometry container variable provided they are also carried by the data variables associated with the container. The **`geometry_type`** attribute indicates the type of geometry present. Its allowable values are: __point__, __multipoint__, __line__, __multiline__, __polygon__, __multipolygon__. From f0cd0e46fc9df8c57fefd218f8334d42136e4db4 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 14 Aug 2017 13:02:24 -0500 Subject: [PATCH 011/126] Add table to Ch7 summarizing geometry types. --- ch07.adoc | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/ch07.adoc b/ch07.adoc index a6cc7b1c..f395e33f 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -687,13 +687,35 @@ For all geometry types except __point__, in which each geometry contains a singl The node count is the total nodes in all the parts. -For __multiline__, __multipolygon__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute indicates a variable of the count of nodes per geometry part. +For __multiline__, __multipolygon__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that indicates a variable of the count of nodes per geometry part. Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required. The single dimension of the part node count variable should equal the total number of parts in all the geometries. For __polygon__ and __multipolygon__ geometries with holes, the geometry container variable must have an **`interior_ring`** attribute that contains the name of a variable that indicates if the polygon parts are interior rings (i.e., holes) or not. This interior ring variable should contain the value 0 to indicate an exterior ring polygon and 1 to indicate an interior ring polygon. The single dimension of the interior ring variable should be the same dimension as that of the part node count variable. +The geometry types included in this convention are listed in Table 7.1. + + +[cols="4"] +|=============== +| geometry_type | Dimensions | Description of Geometry Instance | Additional required attributes on geometry container variable + +| **point** | 0 | A single location in space | + +| **multipoint** | 0 | A collection of one or more points | node_count + +| **line** | 1 | An ordered set of data points connected by linearly interpolating between points | node_count + +| **multiline** | 1 | A collection of one or more lines | node_count, part_node_count + +| **polygon** | 2 | A planar surface comprised of an exterior ring and zero or more interior rings (i.e., holes), where a ring is a closed line (i.e., the first point in the line is coincident with the last point) | node_count, part_node_count (if holes are present), interior_ring (if holes are present) + +| **multipolygon** | 2 | A collection of one or more polygons | node_count, part_node_count, interior_ring (if holes are present) +|=============== + +**Table 7.1.** Dimensionality, description, and additional required attributes for geometry_types. + [[complete-multipolygon-example]] [caption="Example 7.15. "] From cdd8fa0daeac633517ccd8d24406a89a8601ec80 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 14 Aug 2017 13:26:29 -0500 Subject: [PATCH 012/126] Add geometry attributes to Appendix A --- appa.adoc | 37 ++++++++++++++++++++++++++++++++++++- ch07.adoc | 2 +- 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/appa.adoc b/appa.adoc index 30d5482c..0024c523 100644 --- a/appa.adoc +++ b/appa.adoc @@ -156,6 +156,18 @@ formula in the definition. | <> | Identifies variables that correspond to the terms in a formula. +| **`geometry`** +| S +| C, D +| <> +| Identifies a variable that defines geometry. + +| **`geometry_type`** +| S +| - +| <> +| Indicates the type of geometry present. + | **`grid_mapping`** | S | D @@ -180,6 +192,12 @@ formula in the definition. | <> | Where the original data was produced. +| **`interior_ring`** +| S +| - +| <> +| Identifies a variable that indicates if polygon parts are interior rings (i.e., holes) or not. + | **`leap_month`** | N | C @@ -198,7 +216,6 @@ formula in the definition. | link:$$http://www.unidata.ucar.edu/netcdf/docs/netcdf.html#Attribute-Conventions$$[NUG appendix B], <> | A descriptive name that indicates a variable"s content. This name is not standardized. - | **`missing_value`** | D | C, D @@ -211,6 +228,24 @@ formula in the definition. | <> | Specifies the length of each month in a non-leap year for a user defined calendar. +| **`node_coordinates`** +| S +| - +| <> +| Identifies variables that contain geometry node coordinates. + +| **`node_count`** +| S +| - +| <> +| Identifies a variable indicating the count of nodes per geometry. + +| **`part_node_count`** +| S +| - +| <> +| Identifies a variable providing the count of nodes per geometry part. + | **`positive`** | S | C diff --git a/ch07.adoc b/ch07.adoc index f395e33f..3e21a261 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -602,7 +602,7 @@ Note that CF does not require the first and last node to be identical but allows Polygons that have holes, such as waterbodies in a land unit, are encoded as a collection of polygon ring parts, each identified as __exterior__ or __interior__ polygons. Multipart geometries, such as multiple lines representing the same river or multiple islands representing the same jurisdiction, are encoded as collections of un-connected points, lines, or polygons that are logically grouped into a single geometry. -Any data variable can be given a `geometry` attribute that indicates the geometry for the quantity held in the variable. +Any data variable can be given a **`geometry`** attribute that indicates the geometry for the quantity held in the variable. One of the dimensions of the data variable must be the number of geometries to which the data applies. As shown in Example 7.14, if the data variable has a discrete sampling geometry, the number of geometries is the length of the instance dimension (Section 9.2). From 894df8fd9fbf7443aebf09597fe34721018c75c4 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 14 Aug 2017 13:30:36 -0500 Subject: [PATCH 013/126] Change geometric dimension to dimensionality, to avoid confusion with netCDF dimensions --- ch07.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.adoc b/ch07.adoc index 3e21a261..e2bc597c 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -699,7 +699,7 @@ The geometry types included in this convention are listed in Table 7.1. [cols="4"] |=============== -| geometry_type | Dimensions | Description of Geometry Instance | Additional required attributes on geometry container variable +| geometry_type | Dimensionality | Description of Geometry Instance | Additional required attributes on geometry container variable | **point** | 0 | A single location in space | From fd74bae777bef6e8652d97ba99d2fd50fc2a2972 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 14 Aug 2017 14:53:29 -0500 Subject: [PATCH 014/126] Add data variable to example 7.15 --- ch07.adoc | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index e2bc597c..193b891f 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -721,13 +721,16 @@ The geometry types included in this convention are listed in Table 7.1. [caption="Example 7.15. "] .A multipolygon with holes ==== -This example demonstrates geometry without associated data variables and the use of all potential attributes and variables for encoding geometries. +This example demonstrates all potential attributes and variables for encoding geometries. ---- dimensions: node = 12 ; - feature = 2 ; + instance = 2 ; part = 4 ; + time = 4 ; variables: + int time(time) ; + time:units = "days since 2000-01-01" ; double x(node) ; x:units = "degrees_east" ; x:standard_name = "longitude" ; @@ -736,11 +739,11 @@ variables: y:units = "degrees_north" ; y:standard_name = "latitude" ; y:axis = "Y" ; - double lat(feature) ; + double lat(instance) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; lat:geometry = "geometry_container" ; - double lon(feature) ; + double lon(instance) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; lon:geometry = "geometry_container" ; @@ -752,7 +755,7 @@ variables: geometry_container:coordinates = "lat lon" geometry_container:part_node_count = "part_node_count" ; geometry_container:interior_ring = "interior_ring" ; - int node_count(feature) ; + int node_count(instance) ; int part_node_count(part) ; int interior_ring(part) ; float datum ; @@ -760,13 +763,24 @@ variables: datum:semi_major_axis = 6378137. ; datum:inverse_flattening = 298.257223563 ; datum:longitude_of_prime_meridian = 0. ; + double someData(instance, time) ; + someData:coordinates = "time lat lon" ; + someData:grid_mapping = "datum" ; + someData:geometry = "geometry_container" ; // global attributes: :Conventions = "CF-1.8" ; + :featureType = "timeSeries" ; data: + time = 1, 2, 3, 4 ; x = 20, 10, 0, 5, 10, 15, 20, 10, 0, 50, 40, 30 ; y = 0, 15, 0, 5, 10, 5, 20, 35, 20, 0, 15, 0 ; + lat = 25, 7 ; + lon = 10, 40 ; node_count = 9, 3 ; part_node_count = 3, 3, 3, 3 ; interior_ring = 0, 1, 0, 0 ; + someData = + 1, 2, 3, 4, + 1, 2, 3, 4 ; ---- ==== From 9929a61db098a679a32ac898244515aeac7fb41f Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Tue, 17 Oct 2017 09:16:23 -0500 Subject: [PATCH 015/126] Update history with trac number for geometry ticket --- history.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/history.adoc b/history.adoc index 56f457f0..2cb16fa4 100644 --- a/history.adoc +++ b/history.adoc @@ -172,5 +172,5 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. . Updated the links and references to NUG (The NetCDF User Guide), to refer to the current version. . Trivial updates to links for COARDS and UDUNITS in the bibliography. -.14 August 2017 -. New Simple Geometry section 7.5. \ No newline at end of file +.17 October 2017 +. Ticket #164, New Geometries section 7.5. \ No newline at end of file From c1a224a045e279363a833afbe867e9a48e22b770 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Tue, 17 Oct 2017 10:38:17 -0500 Subject: [PATCH 016/126] Revise Geometry section to combine multipart and single part geometries so that we just have point, line, and polygon. --- ch07.adoc | 50 +++++++++++++++++++++----------------------------- 1 file changed, 21 insertions(+), 29 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 193b891f..1e3dd622 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -590,24 +590,24 @@ data: // time coordinates translated to date/time format For many geospatial applications, data values are associated with a geometry, which is a spatial representation of a real-world feature, for instance a time-series of areal average precipitation over a watershed. Polygonal cells with an arbitrary number of vertices can be described using <>, but in that case every cell must have the same number of vertices and must be single polygon rings. -In contrast, each geometry may have a different number of nodes, the geometries may be lines (as alternatives to points and polygons), and they may be __multipart__ i.e include several disjoint parts. +In contrast, each geometry may have a different number of nodes, the geometries may be lines (as alternatives to points and polygons), and they may be __multipart__, i.e., include several disjoint parts. While line and point geometries don't describe an interval along a dimension as the traditional cell bounds described above do, they do describe the extent of a geometry or real-world feature so are included in this section. The approach described here specifies how to encode such geometries following the pattern in **9.3.3 Contiguous ragged array representation** and attach them to variables in a way that is consistent with the cell bounds approach. All geometries are made up of one or more nodes. The geometry type specifies the set of topological assumptions to be applied to relate the nodes. -For example, __multipoint__ and __line__ geometries are nearly the same except nodes are interpreted as being connected for lines. +For example, multipoint and line geometries are nearly the same except nodes are interpreted as being connected for lines. Lines and polygons are also nearly the same except that it should be assumed that the first and last node are connected. Note that CF does not require the first and last node to be identical but allows them to be coincident if desired. Polygons that have holes, such as waterbodies in a land unit, are encoded as a collection of polygon ring parts, each identified as __exterior__ or __interior__ polygons. -Multipart geometries, such as multiple lines representing the same river or multiple islands representing the same jurisdiction, are encoded as collections of un-connected points, lines, or polygons that are logically grouped into a single geometry. +Multipart geometries, such as multiple lines representing the same river or multiple islands representing the same jurisdiction, are encoded as collections of unconnected points, lines, or polygons that are logically grouped into a single geometry. Any data variable can be given a **`geometry`** attribute that indicates the geometry for the quantity held in the variable. One of the dimensions of the data variable must be the number of geometries to which the data applies. -As shown in Example 7.14, if the data variable has a discrete sampling geometry, the number of geometries is the length of the instance dimension (Section 9.2). +As shown in Example 7.15, if the data variable has a discrete sampling geometry, the number of geometries is the length of the instance dimension (Section 9.2). [["timeseries-with-geometry"]] -[caption="Example 7.14. "] +[caption="Example 7.15. "] .Timeseries with geometry. ==== ---- @@ -672,54 +672,46 @@ The geometry container variable must hold **`geometry_type`** and **`node_coordi The **`grid_mapping`** and **`coordinates`** attributes can be carried by the geometry container variable provided they are also carried by the data variables associated with the container. The **`geometry_type`** attribute indicates the type of geometry present. -Its allowable values are: __point__, __multipoint__, __line__, __multiline__, __polygon__, __multipolygon__. -The **`node_coordinates`** attribute contains the blank delimited names of the variables that contain geometry node coordinates (one variable for each spatial dimension). - +Its allowable values are: __point__, __line__, __polygon__. +Multipart geometries are allowed for all three geometry types. +For example, polygon geometries could include single part geometries like the State of Colorado and multipart geometries like the State of Hawaii. +The **`node_coordinates`** attribute contains the blank-separated names of the variables that contain geometry node coordinates (one variable for each spatial dimension). The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. Polygon exterior rings must be put in anticlockwise order (viewed from above) and polygon interior rings in clockwise order. They are put in opposite orders to facilitate calculation of area and consistency with the typical implementation pattern. +When more than one geometry instance is present, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. +The node count is the total number of nodes in all the parts. +The exception is when all geometries are single part point geometries, in which case a node count is not needed since each geometry contains a single node. -For all geometry types except __point__, in which each geometry contains a single node, when more than one geometry is present, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. -The node count is the total nodes in all the parts. - - -For __multiline__, __multipolygon__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that indicates a variable of the count of nodes per geometry part. -Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required. +For multipart __lines__, multipart __polygons__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that indicates a variable of the count of nodes per geometry part. +Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required for __point__ geometry types. The single dimension of the part node count variable should equal the total number of parts in all the geometries. -For __polygon__ and __multipolygon__ geometries with holes, the geometry container variable must have an **`interior_ring`** attribute that contains the name of a variable that indicates if the polygon parts are interior rings (i.e., holes) or not. +For __polygon__ geometries with holes, the geometry container variable must have an **`interior_ring`** attribute that contains the name of a variable that indicates if the polygon parts are interior rings (i.e., holes) or not. This interior ring variable should contain the value 0 to indicate an exterior ring polygon and 1 to indicate an interior ring polygon. The single dimension of the interior ring variable should be the same dimension as that of the part node count variable. The geometry types included in this convention are listed in Table 7.1. - [cols="4"] |=============== | geometry_type | Dimensionality | Description of Geometry Instance | Additional required attributes on geometry container variable -| **point** | 0 | A single location in space | +| **point** | 0 | A collection of one or more points, where a point is a single location in space | node_count (if multipart geometries are present) -| **multipoint** | 0 | A collection of one or more points | node_count +| **line** | 1 | A collection of one or more lines, where a line is an ordered set of data points connected by linearly interpolating between points | node_count, part_node_count (if multipart geometries are present) -| **line** | 1 | An ordered set of data points connected by linearly interpolating between points | node_count - -| **multiline** | 1 | A collection of one or more lines | node_count, part_node_count - -| **polygon** | 2 | A planar surface comprised of an exterior ring and zero or more interior rings (i.e., holes), where a ring is a closed line (i.e., the first point in the line is coincident with the last point) | node_count, part_node_count (if holes are present), interior_ring (if holes are present) - -| **multipolygon** | 2 | A collection of one or more polygons | node_count, part_node_count, interior_ring (if holes are present) +| **polygon** | 2 | A collection of one or more polygons, where a polygon is a planar surface comprised of an exterior ring and zero or more interior rings (i.e., holes), where a ring is a closed line (i.e., the first point in the line is coincident with the last point) | node_count, part_node_count (if holes or multipart geometries are present), interior_ring (if holes are present) |=============== **Table 7.1.** Dimensionality, description, and additional required attributes for geometry_types. - [[complete-multipolygon-example]] -[caption="Example 7.15. "] -.A multipolygon with holes +[caption="Example 7.16. "] +.Polygons with holes ==== This example demonstrates all potential attributes and variables for encoding geometries. ---- @@ -748,7 +740,7 @@ variables: lon:standard_name = "longitude" ; lon:geometry = "geometry_container" ; float geometry_container ; - geometry_container:geometry_type = "multipolygon" ; + geometry_container:geometry_type = "polygon" ; geometry_container:node_count = "node_count" ; geometry_container:node_coordinates = "x y" ; geometry_container:grid_mapping = "datum" ; From e3ff90a6ab5bfef16872c386c9421b82397ce636 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Fri, 20 Oct 2017 09:07:36 -0500 Subject: [PATCH 017/126] Reference Table 7.1 when introducing geometry types --- ch07.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch07.adoc b/ch07.adoc index 1e3dd622..b7f8eec5 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -595,7 +595,7 @@ While line and point geometries don't describe an interval along a dimension as The approach described here specifies how to encode such geometries following the pattern in **9.3.3 Contiguous ragged array representation** and attach them to variables in a way that is consistent with the cell bounds approach. All geometries are made up of one or more nodes. -The geometry type specifies the set of topological assumptions to be applied to relate the nodes. +The geometry type specifies the set of topological assumptions to be applied to relate the nodes (see Table 7.1). For example, multipoint and line geometries are nearly the same except nodes are interpreted as being connected for lines. Lines and polygons are also nearly the same except that it should be assumed that the first and last node are connected. Note that CF does not require the first and last node to be identical but allows them to be coincident if desired. From b6316f4257d452f65f6db413748461ecffc04c43 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Fri, 20 Oct 2017 10:01:34 -0500 Subject: [PATCH 018/126] Use must instead of should to indicate requirements --- ch07.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index b7f8eec5..2bfde0a4 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -597,7 +597,7 @@ The approach described here specifies how to encode such geometries following th All geometries are made up of one or more nodes. The geometry type specifies the set of topological assumptions to be applied to relate the nodes (see Table 7.1). For example, multipoint and line geometries are nearly the same except nodes are interpreted as being connected for lines. -Lines and polygons are also nearly the same except that it should be assumed that the first and last node are connected. +Lines and polygons are also nearly the same except that the first and last nodes are assumed to be connected for polygons. Note that CF does not require the first and last node to be identical but allows them to be coincident if desired. Polygons that have holes, such as waterbodies in a land unit, are encoded as a collection of polygon ring parts, each identified as __exterior__ or __interior__ polygons. Multipart geometries, such as multiple lines representing the same river or multiple islands representing the same jurisdiction, are encoded as collections of unconnected points, lines, or polygons that are logically grouped into a single geometry. @@ -689,11 +689,11 @@ The exception is when all geometries are single part point geometries, in which For multipart __lines__, multipart __polygons__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that indicates a variable of the count of nodes per geometry part. Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required for __point__ geometry types. -The single dimension of the part node count variable should equal the total number of parts in all the geometries. +The single dimension of the part node count variable must equal the total number of parts in all the geometries. For __polygon__ geometries with holes, the geometry container variable must have an **`interior_ring`** attribute that contains the name of a variable that indicates if the polygon parts are interior rings (i.e., holes) or not. -This interior ring variable should contain the value 0 to indicate an exterior ring polygon and 1 to indicate an interior ring polygon. -The single dimension of the interior ring variable should be the same dimension as that of the part node count variable. +This interior ring variable must contain the value 0 to indicate an exterior ring polygon and 1 to indicate an interior ring polygon. +The single dimension of the interior ring variable must be the same dimension as that of the part node count variable. The geometry types included in this convention are listed in Table 7.1. [cols="4"] From 56667696362b0bf4ce8f7d517f556bf3d62b3656 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Wed, 13 Dec 2017 16:38:50 -0600 Subject: [PATCH 019/126] Implement suggestions from trac #164; be consistent about not requiring first and last polygon nodes to be equivalent --- appa.adoc | 16 ++++++++-------- ch07.adoc | 6 +++--- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/appa.adoc b/appa.adoc index ee2cdbaa..c489522b 100644 --- a/appa.adoc +++ b/appa.adoc @@ -6,7 +6,7 @@ All CF attributes are listed here except for those that are used to describe grid mappings. See Appendix F for the grid mapping attributes. -The "Type" values are **S** for string, **N** for numeric, and **D** for the type of the data variable. The "Use" values are **G** for global, **C** for variables containing coordinate data, **D** for variables containing non-coordinate data, and **-** for variables with a special purpose. "Links" indicates the location of the attribute"s original definition (first link) and sections where the attribute is discussed in this document (additional links as necessary). +The "Type" values are **S** for string, **N** for numeric, and **D** for the type of the data variable. The "Use" values are **G** for global, **C** for variables containing coordinate data, **D** for variables containing non-coordinate data, **M** for geometry container variables, and **-** for variables with a special purpose. "Links" indicates the location of the attribute"s original definition (first link) and sections where the attribute is discussed in this document (additional links as necessary). [[table-attributes]] .Attributes @@ -107,7 +107,7 @@ formula in the definition. | **`coordinates`** | S -| D +| D, M | <>, <>, <> | Identifies auxiliary coordinate variables, label variables, and alternate coordinate variables. @@ -164,13 +164,13 @@ formula in the definition. | **`geometry_type`** | S -| - +| M | <> | Indicates the type of geometry present. | **`grid_mapping`** | S -| D +| D, M | <> | Identifies a variable that defines a grid mapping. @@ -194,7 +194,7 @@ formula in the definition. | **`interior_ring`** | S -| - +| M | <> | Identifies a variable that indicates if polygon parts are interior rings (i.e., holes) or not. @@ -230,19 +230,19 @@ formula in the definition. | **`node_coordinates`** | S -| - +| M | <> | Identifies variables that contain geometry node coordinates. | **`node_count`** | S -| - +| M | <> | Identifies a variable indicating the count of nodes per geometry. | **`part_node_count`** | S -| - +| M | <> | Identifies a variable providing the count of nodes per geometry part. diff --git a/ch07.adoc b/ch07.adoc index 2bfde0a4..6bbc8bc2 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -1,7 +1,7 @@  == Data Representative of Cells -When gridded data does not represent the point values of a field but instead represents some characteristic of the field within cells of finite "volume," a complete description of the variable should include metadata that describes the domain or extent of each cell, and the characteristic of the field that the cell values represent. It is possible for a single data value to be the result of an operation whose domain is a disjoint set of cells. This is true for many types of climatological averages, for example, the mean January temperature for the years 1970-2000. The methods that we present below for describing cells only provides an association of a grid point with a single cell, not with a collection of cells. However, climatological statistics are of such importance that we provide special methods for describing their associated computational domains in <>. +When gridded data does not represent the point values of a field but instead represents some characteristic of the field within cells of finite "volume," a complete description of the variable should include metadata that describes the domain or extent of each cell, and the characteristic of the field that the cell values represent. It is possible for a single data value to be the result of an operation whose domain is a disjoint set of cells. This is true for many types of climatological averages, for example, the mean January temperature for the years 1970-2000. The methods that we present below for describing cells only provides an association of a grid point with a single cell, not with a collection of cells. However, climatological statistics are of such importance that we provide special methods for describing their associated computational domains in <>. For cases when data pertain to geospatial features with highly variable geometry node counts such as river lines or watershed boundaries, we provide < as an alternative to bounds. @@ -589,7 +589,7 @@ data: // time coordinates translated to date/time format === Geometries For many geospatial applications, data values are associated with a geometry, which is a spatial representation of a real-world feature, for instance a time-series of areal average precipitation over a watershed. -Polygonal cells with an arbitrary number of vertices can be described using <>, but in that case every cell must have the same number of vertices and must be single polygon rings. +Polygonal cells with an arbitrary number of vertices can be described using <>, but in that case every cell must have the same number of vertices and must be a single polygon ring. In contrast, each geometry may have a different number of nodes, the geometries may be lines (as alternatives to points and polygons), and they may be __multipart__, i.e., include several disjoint parts. While line and point geometries don't describe an interval along a dimension as the traditional cell bounds described above do, they do describe the extent of a geometry or real-world feature so are included in this section. The approach described here specifies how to encode such geometries following the pattern in **9.3.3 Contiguous ragged array representation** and attach them to variables in a way that is consistent with the cell bounds approach. @@ -704,7 +704,7 @@ The geometry types included in this convention are listed in Table 7.1. | **line** | 1 | A collection of one or more lines, where a line is an ordered set of data points connected by linearly interpolating between points | node_count, part_node_count (if multipart geometries are present) -| **polygon** | 2 | A collection of one or more polygons, where a polygon is a planar surface comprised of an exterior ring and zero or more interior rings (i.e., holes), where a ring is a closed line (i.e., the first point in the line is coincident with the last point) | node_count, part_node_count (if holes or multipart geometries are present), interior_ring (if holes are present) +| **polygon** | 2 | A collection of one or more polygons, where a polygon is a planar surface comprised of an exterior ring and zero or more interior rings (i.e., holes), where a ring is a closed line (i.e., the last point in the line is assumed to be connected to the first point) | node_count, part_node_count (if holes or multipart geometries are present), interior_ring (if holes are present) |=============== **Table 7.1.** Dimensionality, description, and additional required attributes for geometry_types. From ef7c9cc1a8d8c18dfc3b16592a19a019faf6479e Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Wed, 13 Dec 2017 16:41:29 -0600 Subject: [PATCH 020/126] update history --- history.adoc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/history.adoc b/history.adoc index 7b446669..ddf1e9b9 100644 --- a/history.adoc +++ b/history.adoc @@ -177,3 +177,6 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .17 October 2017 . Ticket #164, New Geometries section 7.5. + +.13 December 2017 +. Ticket #164, Implement suggestions from trac ticket comments. From 7975d39ac7cef0ec2923563f347168d393013bd7 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Wed, 24 Jan 2018 11:43:30 -0600 Subject: [PATCH 021/126] If coordinates attribute is carried by geometry container, require coordinate variables which correspond to node coordinate variables to have the corresponding axis attribute. --- ch07.adoc | 3 +++ history.adoc | 3 +++ 2 files changed, 6 insertions(+) diff --git a/ch07.adoc b/ch07.adoc index 6bbc8bc2..8c10f980 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -678,6 +678,7 @@ For example, polygon geometries could include single part geometries like the St The **`node_coordinates`** attribute contains the blank-separated names of the variables that contain geometry node coordinates (one variable for each spatial dimension). The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. +If a **`coordinates`** attribute is carried by the geometry container variable, then those coordinate variables which correspond to node coordinate variables must also have the **`axis`** attribute. The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. Polygon exterior rings must be put in anticlockwise order (viewed from above) and polygon interior rings in clockwise order. @@ -735,10 +736,12 @@ variables: lat:units = "degrees_north" ; lat:standard_name = "latitude" ; lat:geometry = "geometry_container" ; + lat:axis = "Y" ; double lon(instance) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; lon:geometry = "geometry_container" ; + lon:axis = "X" ; float geometry_container ; geometry_container:geometry_type = "polygon" ; geometry_container:node_count = "node_count" ; diff --git a/history.adoc b/history.adoc index ddf1e9b9..ac180d8f 100644 --- a/history.adoc +++ b/history.adoc @@ -180,3 +180,6 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .13 December 2017 . Ticket #164, Implement suggestions from trac ticket comments. + +.24 January 2018 +. Ticket #164, If coordinates attribute is carried by geometry container, require coordinate variables which correspond to node coordinate variables to have the corresponding axis attribute. From edab7e8b6a834ef1db2f5b04899fd85d368c21b1 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Thu, 15 Feb 2018 16:02:08 -0600 Subject: [PATCH 022/126] Remove geometry attribute from lat/lon variables in examples. --- ch07.adoc | 4 ---- 1 file changed, 4 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 8c10f980..d3eca36e 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -621,11 +621,9 @@ variables: double lat(instance) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; - lat:geometry = "geometry_container" ; double lon(instance) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; - lon:geometry = "geometry_container" ; int datum ; datum:grid_mapping_name = "latitude_longitude" ; datum:longitude_of_prime_meridian = 0.0 ; @@ -735,12 +733,10 @@ variables: double lat(instance) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; - lat:geometry = "geometry_container" ; lat:axis = "Y" ; double lon(instance) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; - lon:geometry = "geometry_container" ; lon:axis = "X" ; float geometry_container ; geometry_container:geometry_type = "polygon" ; From d8ea261b9313639699ad7bdbd68ba8069eed1132 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Thu, 15 Feb 2018 16:04:27 -0600 Subject: [PATCH 023/126] Update history --- history.adoc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/history.adoc b/history.adoc index ac180d8f..ef6abd3c 100644 --- a/history.adoc +++ b/history.adoc @@ -183,3 +183,6 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .24 January 2018 . Ticket #164, If coordinates attribute is carried by geometry container, require coordinate variables which correspond to node coordinate variables to have the corresponding axis attribute. + +.24 January 2018 +. Ticket #164, Remove geometry attribute from lat/lon variables in examples. From 2535a5fcfbc1154a847901db7dbf767300975e51 Mon Sep 17 00:00:00 2001 From: Ethan Davis Date: Tue, 20 Feb 2018 17:37:55 -0700 Subject: [PATCH 024/126] Add definition of 'name_strlen' dimension where missing in Appendix H CDL examples. --- apph.adoc | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/apph.adoc b/apph.adoc index b3536fe6..7f8131d9 100644 --- a/apph.adoc +++ b/apph.adoc @@ -76,6 +76,8 @@ If the time series instances have the same number of elements and the time value dimensions:   station = 10 ;  // measurement locations   time = UNLIMITED ; +  name_strlen = 23 ; + variables:   float humidity(station,time) ;     humidity:standard_name = "specific humidity" ; @@ -122,6 +124,7 @@ Much of the simplicity of the orthogonal multidimensional representation can be dimensions:    station = UNLIMITED ;    obs = 13 ; + name_strlen = 23 ; variables:    float lon(station) ; @@ -296,6 +299,7 @@ When the time series have different lengths and the data values for entire time dimensions:    station = 23 ;    obs = 1234 ; + name_strlen = 23 ; variables:    float lon(station) ; @@ -370,6 +374,7 @@ When time series with different lengths are written incrementally, the indexed r dimensions:    station = 23 ;    obs = UNLIMITED ; + name_strlen = 23 ; variables:    float lon(station) ; @@ -700,6 +705,7 @@ When storing multiple trajectories in the same file, and the number of elements dimensions:    obs = 1000 ;    trajectory = 77 ; + name_strlen = 23 ; variables:    char trajectory(trajectory, name_strlen) ; @@ -760,6 +766,7 @@ When a single trajectory is stored in the data variable, there is no need for th ---- dimensions:    time = 42; + name_strlen = 23 ; variables:    char trajectory(name_strlen) ; @@ -817,6 +824,7 @@ When the number of elements for each trajectory varies, and one can control the dimensions:    obs = 3443;    trajectory = 77 ; + name_strlen = 23 ; variables:    char trajectory(trajectory, name_strlen) ; @@ -877,6 +885,7 @@ When the number of elements at each trajectory vary, and the elements cannot be dimensions:    obs = UNLIMITED ;    trajectory = 77 ; + name_strlen = 23 ; variables:    char trajectory(trajectory, name_strlen) ; @@ -950,6 +959,7 @@ When storing time series of profiles at multiple stations in the same data varia    station = 22 ;    profile = 3002 ;    z = 42 ; + name_strlen = 23 ; variables:    float lon(station) ; @@ -1056,6 +1066,7 @@ If there is only one station in the data variable, there is no need for the stat dimensions:    profile = 30 ;    z = 42 ; + name_strlen = 23 ; variables:    float lon ; @@ -1123,6 +1134,7 @@ When the number of profiles and levels for each station varies, one can use a ra    obs = UNLIMITED ;    profiles = 1420 ;    stations = 42; + name_strlen = 23 ; variables:    float lon(station) ; From f15f449ca363ca16d283259f5e3f41cd7245826c Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 20 Apr 2018 08:09:40 -0500 Subject: [PATCH 025/126] update authors --- cf-conventions.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 86bf3afd..3ab79ba5 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -30,6 +30,8 @@ include::toc-extra.adoc[] * Martin Juckes, BADC * Martin Raspaud, SMHI * Randy Horne, Excalibur Laboratories, Inc., Melbourne Beach Florida USA +* Timothy Whiteaker, University of Texas +* David Blodgett, USGS Many others have contributed to the development of CF through their participation in discussions about proposed changes. From 64bde6679c5da2c7095fed4a9c5b521799829698 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Wed, 9 May 2018 16:25:18 -0500 Subject: [PATCH 026/126] Replace axis with bounds for coordinate variables related to geometry node variables. --- ch07.adoc | 6 +++--- history.adoc | 6 ++++++ 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index d3eca36e..93d20558 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -676,7 +676,7 @@ For example, polygon geometries could include single part geometries like the St The **`node_coordinates`** attribute contains the blank-separated names of the variables that contain geometry node coordinates (one variable for each spatial dimension). The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. -If a **`coordinates`** attribute is carried by the geometry container variable, then those coordinate variables which correspond to node coordinate variables must also have the **`axis`** attribute. +If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables which correspond to node coordinate variables must have a **`bounds`** attribute that names the corresponding node coordinate. The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. Polygon exterior rings must be put in anticlockwise order (viewed from above) and polygon interior rings in clockwise order. @@ -733,11 +733,11 @@ variables: double lat(instance) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; - lat:axis = "Y" ; + lat:bounds = "y" ; double lon(instance) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; - lon:axis = "X" ; + lon:bounds = "x" ; float geometry_container ; geometry_container:geometry_type = "polygon" ; geometry_container:node_count = "node_count" ; diff --git a/history.adoc b/history.adoc index ef6abd3c..a28a82c0 100644 --- a/history.adoc +++ b/history.adoc @@ -186,3 +186,9 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .24 January 2018 . Ticket #164, Remove geometry attribute from lat/lon variables in examples. + +.20 April 2018 +. Ticket #164, Add Tim Whiteaker and Dave Blodgett at authors. + +.09 May 2018 +. Ticket #164, Replace axis with bounds for coordinate variables related to geometry node variables. From 6572f7d6dde72caedbb5bbcc58a8e1dbb505fe49 Mon Sep 17 00:00:00 2001 From: marqh Date: Wed, 20 Jun 2018 14:02:50 +0000 Subject: [PATCH 027/126] Update ch05.adoc --- ch05.adoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/ch05.adoc b/ch05.adoc index 9ea834e7..5619b03a 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -183,9 +183,8 @@ _This section has been superseded by the treatment of time series as a type of d [[grid-mappings-and-projections, Section 5.6, "Horizontal Coordinate Reference Systems, Grid Mappings, and Projections"]] === Horizontal Coordinate Reference Systems, Grid Mappings, and Projections -When the coordinate variables for a horizontal grid are not longitude and latitude, it is required -that the true latitude and longitude coordinates be supplied via the **`coordinates`** attribute. A -__grid mapping variable__ is required if, in addition, it is desired to describe the mapping between the +When the coordinate variables for a horizontal grid are not longitude and latitude, the true latitude and longitude coordinates may be supplied via the **`coordinates`** attribute. +A __grid mapping variable__ is required if, in addition, it is desired to describe the mapping between the given coordinate variables and the true latitude and longitude coordinates, or to describe the figure of the Earth used to define the latitude and longitude coordinates, or to describe another coordinate reference system definition used by some coordinates or auxiliary coordinates. Such a From 16c9b0114a695c44f1a20d7b80236e00f6bf204f Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 25 Jun 2018 09:07:38 -0500 Subject: [PATCH 028/126] Add bounds to first geometry example. Correct typo in history. Mention right-hand rule for polygon rings, and that interior rings must occur after the exterior ring that contains them, as per discussion at June netCDF workshop. --- ch07.adoc | 6 +++++- history.adoc | 5 ++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 93d20558..3d748486 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -621,9 +621,11 @@ variables: double lat(instance) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; + lat:bounds = "y" ; double lon(instance) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; + lon:bounds = "x" ; int datum ; datum:grid_mapping_name = "latitude_longitude" ; datum:longitude_of_prime_meridian = 0.0 ; @@ -679,7 +681,9 @@ The geometry node coordinate variables must each have an **`axis`** attribute wh If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables which correspond to node coordinate variables must have a **`bounds`** attribute that names the corresponding node coordinate. The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. -Polygon exterior rings must be put in anticlockwise order (viewed from above) and polygon interior rings in clockwise order. +Polygon exterior rings must be stored before any interior rings they may contain. +Nodes for polygon exterior rings must be ordered using the right-hand rule, e.g., anticlockwise in the lon-lat plane as viewed from above. +Polygon interior rings must be in clockwise order. They are put in opposite orders to facilitate calculation of area and consistency with the typical implementation pattern. When more than one geometry instance is present, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. diff --git a/history.adoc b/history.adoc index a28a82c0..a67f6915 100644 --- a/history.adoc +++ b/history.adoc @@ -188,7 +188,10 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. . Ticket #164, Remove geometry attribute from lat/lon variables in examples. .20 April 2018 -. Ticket #164, Add Tim Whiteaker and Dave Blodgett at authors. +. Ticket #164, Add Tim Whiteaker and Dave Blodgett as authors. .09 May 2018 . Ticket #164, Replace axis with bounds for coordinate variables related to geometry node variables. + +.25 June 2018 +. Ticket #164, Add bounds attribute to first geometry CDL example. From 7ce6b4a077aa5da1a2749b9619ad622205c3b069 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Wed, 18 Jul 2018 20:07:19 -0500 Subject: [PATCH 029/126] change bounds to nodes and qualify their use --- ch07.adoc | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 3d748486..86c2b949 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -621,11 +621,11 @@ variables: double lat(instance) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; - lat:bounds = "y" ; + lat:nodes = "y" ; double lon(instance) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; - lon:bounds = "x" ; + lon:nodes = "x" ; int datum ; datum:grid_mapping_name = "latitude_longitude" ; datum:longitude_of_prime_meridian = 0.0 ; @@ -678,7 +678,8 @@ For example, polygon geometries could include single part geometries like the St The **`node_coordinates`** attribute contains the blank-separated names of the variables that contain geometry node coordinates (one variable for each spatial dimension). The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. -If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables which correspond to node coordinate variables must have a **`bounds`** attribute that names the corresponding node coordinate. +If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables which have a meaningful correspondence with node coordinates may have a **`nodes`** attribute that names the corresponding node coordinates. +Whether linked to normal CF space-time coordinates with a **`nodes`** attribute or not, inclusion of such coordinates is recommended to maintain backward compatibility with software that has not implemented geometry capabilities. The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. Polygon exterior rings must be stored before any interior rings they may contain. @@ -737,11 +738,11 @@ variables: double lat(instance) ; lat:units = "degrees_north" ; lat:standard_name = "latitude" ; - lat:bounds = "y" ; + lat:nodes = "y" ; double lon(instance) ; lon:units = "degrees_east" ; lon:standard_name = "longitude" ; - lon:bounds = "x" ; + lon:nodes = "x" ; float geometry_container ; geometry_container:geometry_type = "polygon" ; geometry_container:node_count = "node_count" ; From 3d42587de2b6588a679e366db6066e70b041223a Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Mon, 21 May 2018 14:17:44 -0500 Subject: [PATCH 030/126] draft github CONTRIBUTING guidelines --- .github/CONTRIBUTING.md | 73 ++++++++++++++++++++++++++++++++ .github/PULL_REQUEST_TEMPLATE.md | 8 +++- CODE_OF_CONDUCT.md | 25 +++++++++++ 3 files changed, 105 insertions(+), 1 deletion(-) create mode 100644 .github/CONTRIBUTING.md create mode 100644 CODE_OF_CONDUCT.md diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 00000000..4a90cb01 --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,73 @@ +# Contributing to the NetCDF-CF conventions + +Dear CF community member, + +Thank you for taking the time to consider making a contribution to the +[cf-conventions](http://cfconventions.org/). +The NetCDF Climate and Forecasting conventions are a product by and for a broad +community and your contribution is the key to their usefulness. + +This set of guidelines provides a brief overview of the practices and procedures +that should be used in fixing, updating, or adding to the conventions. It builds +on the [rules for CF Convention changes.](http://cfconventions.org/rules.html) + +As a prerequisite to this guide, please review the community's code of +[conduct.](https://github.com/cf-convention/cf-conventions/blob/master/CODE_OF_CONDUCT.md) +The CF community takes great pride in respectful and collegial +discourse. Any disrespectful or otherwise derogatory communication will not be +tolerated. + +As of the creation of this version of these guidelines, contributions can also be +made through the [cf-trac](https://cf-trac.llnl.gov/trac/) infrastructure. +[Procedures for using trac](http://cfconventions.org/rules.html) are not covered +here and it is expected that the trac system will be deprecated once the +community is comfortable using github for the purpose. + +## General Guidelines + +1. **A given proposal should be discussed as one issue.** It shouldn't fork or +be superseded by another one, unless that reflects what has happened to the +proposal. This is so that it's easy to trace the discussion which led to a +given agreed proposal. + +2. **A proposal should convey the reasoning and effect on all relevant +sections of the specification.** An overview of all actual changes and the +impact the changes have on the specification should be clear. Depending on the +length and nature of the proposal, this may require different approaches as +described below. + +3. **In general, issues should be used for discussion of proposed changes and +pull requests should be used for review of agreed upon changes.** In other words, +if the content or concept of what is being proposed needs to be vetted by the +community it should be vetted in an issue. If the proposal is non-controversial +(such as a typo correction) or has been agreed to in concept in an issue, then +detailed review of the text may take place in a pull request. + +## Issues and Pull Requests + +The following cases describe potential patterns of use for Issues. + +1. **Typo Fix** If the change is a non-controversial fix such as a typo, +no issue is required. A pull request with the fix can be submitted directly. +Contributors not familiar with github are encouraged to submit issues for typos +and similar issues for others to fix. + +2. **Single Section Change** In the case of a change concerning one to a few +paragraphs, an issue should be opened that describes the problem and proposed +fix. The problem text should be pasted in the body of the issue and proposed fix +included. A link to the line where the problem exists could also be included. +If the modification is non-controversial, a pull request could be opened +simultaneously if the reporter is familiar with git and github. Discussion of +the proposal should take place in one issue. Once accepted, a pull request could +be submitted implementing the suggested change. Final review should take place +in the pull request and the issue closed when the pull request is merged. + +3. **Changes Spanning Multiple Sections** If reasonable, changes concerning +multiple sections should follow the patter described Single Section Change. +If explicitly listing proposed changes is not practical, general guideline +2 should be followed to document the proposal. Depending on the nature of the +proposal, interested community members can decide what the most effective tool +is for development and review of specification changes. Some proposals have +been developed in google docs and contributed as a pull request after general +consensus has been reached, but the specific tools used for development of +significant changes is up to those contributing and reviewing it. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 1441f946..57114603 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,7 +1,13 @@ -See http://cf-trac.llnl.gov/trac/ticket/XXX +_If trac was used for review of contribution:_ +See http://cf-trac.llnl.gov/trac/ticket/XXX - [ ] Added link from trac ticket to this PR? > Applied via ​https://github.com/cf-convention/cf-conventions/pull/XXX - [ ] Updated "Revision History"? (Use the date you applied the changes.) + +_If github issues were used for review:_ +See issue #XXX + + - [ ] Updated "Revision History"? (Use the date you applied the changes.) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..af36991f --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,25 @@ +# Contributor Code of Conduct + +As contributors and maintainers of this project, we pledge to respect all people who +contribute through reporting issues, posting feature requests, updating +documentation, submitting pull requests or patches, and other activities. + +We are committed to making participation in this project a harassment-free experience for +everyone, regardless of level of experience, gender, gender identity and expression, +sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion. + +Examples of unacceptable behavior by participants include the use of sexual language or +imagery, derogatory comments or personal attacks, trolling, public or private harassment, +insults, or other unprofessional conduct. + +Project maintainers have the right and responsibility to remove, edit, or reject comments, +commits, code, wiki edits, issues, and other contributions that are not aligned to this +Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed +from the project team. + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by +opening an issue or contacting one or more of the project maintainers. + +This Code of Conduct is adapted from the Contributor Covenant +(http:contributor-covenant.org), version 1.0.0, available at +http://contributor-covenant.org/version/1/0/0/ From a5495b4dcc64f31c2f25cefa72695d7ac1f5f52b Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Mon, 21 May 2018 19:55:11 -0500 Subject: [PATCH 031/126] minor changes per review --- .github/CONTRIBUTING.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 4a90cb01..37371966 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -21,7 +21,7 @@ As of the creation of this version of these guidelines, contributions can also b made through the [cf-trac](https://cf-trac.llnl.gov/trac/) infrastructure. [Procedures for using trac](http://cfconventions.org/rules.html) are not covered here and it is expected that the trac system will be deprecated once the -community is comfortable using github for the purpose. +community is comfortable using github for this purpose. ## General Guidelines @@ -45,12 +45,12 @@ detailed review of the text may take place in a pull request. ## Issues and Pull Requests -The following cases describe potential patterns of use for Issues. +The following cases describe potential patterns of use for issues and pull requests. 1. **Typo Fix** If the change is a non-controversial fix such as a typo, no issue is required. A pull request with the fix can be submitted directly. -Contributors not familiar with github are encouraged to submit issues for typos -and similar issues for others to fix. +Contributors not familiar with github can submit issues for typos and similar +issues for others to fix. 2. **Single Section Change** In the case of a change concerning one to a few paragraphs, an issue should be opened that describes the problem and proposed From dd39d1505d576a12e881dfde87f155d94b876dc1 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Thu, 7 Jun 2018 07:27:27 -0500 Subject: [PATCH 032/126] labels --- .github/CONTRIBUTING.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 37371966..0ecdbc4a 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -43,6 +43,10 @@ community it should be vetted in an issue. If the proposal is non-controversial (such as a typo correction) or has been agreed to in concept in an issue, then detailed review of the text may take place in a pull request. +4. **Use [labels](https://github.com/cf-convention/cf-conventions/labels) on issues +and pull requests.** Please attempt to present contributions as enhancements, +defects, or typos and label them accordingly. + ## Issues and Pull Requests The following cases describe potential patterns of use for issues and pull requests. From 63288ba885ea97dbc09321de8acafd3ff7b0e68b Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Wed, 18 Jul 2018 20:35:42 -0500 Subject: [PATCH 033/126] reflow and edit contributing --- .github/CONTRIBUTING.md | 92 ++++++++++++++--------------------------- 1 file changed, 32 insertions(+), 60 deletions(-) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 0ecdbc4a..d7c6163a 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -2,76 +2,48 @@ Dear CF community member, -Thank you for taking the time to consider making a contribution to the -[cf-conventions](http://cfconventions.org/). -The NetCDF Climate and Forecasting conventions are a product by and for a broad -community and your contribution is the key to their usefulness. +Thank you for taking the time to consider making a contribution to the [cf-conventions](http://cfconventions.org/). +The NetCDF Climate and Forecasting conventions are a product by and for a broad community and your contribution is the key to their usefulness. -This set of guidelines provides a brief overview of the practices and procedures -that should be used in fixing, updating, or adding to the conventions. It builds -on the [rules for CF Convention changes.](http://cfconventions.org/rules.html) +This set of guidelines provides a brief overview of the practices and procedures that should be used in fixing, updating, or adding to the conventions. +It builds on the [rules for CF Convention changes.](http://cfconventions.org/rules.html) -As a prerequisite to this guide, please review the community's code of -[conduct.](https://github.com/cf-convention/cf-conventions/blob/master/CODE_OF_CONDUCT.md) -The CF community takes great pride in respectful and collegial -discourse. Any disrespectful or otherwise derogatory communication will not be -tolerated. - -As of the creation of this version of these guidelines, contributions can also be -made through the [cf-trac](https://cf-trac.llnl.gov/trac/) infrastructure. -[Procedures for using trac](http://cfconventions.org/rules.html) are not covered -here and it is expected that the trac system will be deprecated once the -community is comfortable using github for this purpose. +As a prerequisite to this guide, please review the community's code of [conduct.](https://github.com/cf-convention/cf-conventions/blob/master/CODE_OF_CONDUCT.md) +The CF community takes great pride in respectful and collegial discourse. Any disrespectful or otherwise derogatory communication will not be tolerated. ## General Guidelines -1. **A given proposal should be discussed as one issue.** It shouldn't fork or -be superseded by another one, unless that reflects what has happened to the -proposal. This is so that it's easy to trace the discussion which led to a -given agreed proposal. +1. **A given proposal should be discussed as one issue.** It shouldn't fork or be superseded by another one, unless that reflects what has happened to the +proposal. +This is so it is easy to trace the discussion what led to a given agreed proposal. -2. **A proposal should convey the reasoning and effect on all relevant -sections of the specification.** An overview of all actual changes and the -impact the changes have on the specification should be clear. Depending on the -length and nature of the proposal, this may require different approaches as -described below. +2. **A proposal should convey the reasoning and effect on all relevant sections of the specification.** +An overview of all actual changes and the impact the changes have on the specification should be clear. +Depending on the length and nature of the proposal, this may require different approaches as described below. -3. **In general, issues should be used for discussion of proposed changes and -pull requests should be used for review of agreed upon changes.** In other words, -if the content or concept of what is being proposed needs to be vetted by the -community it should be vetted in an issue. If the proposal is non-controversial -(such as a typo correction) or has been agreed to in concept in an issue, then -detailed review of the text may take place in a pull request. +3. **In general, issues should be used for discussion of proposed changes and pull requests should be used for review of agreed upon changes.** +In other words, if the content or concept of what is being proposed needs to be vetted by the community it should be vetted in an issue. +If the proposal is non-controversial (such as a typo correction) or has been agreed to in concept in an issue, then detailed review of the text may take place in a pull request. -4. **Use [labels](https://github.com/cf-convention/cf-conventions/labels) on issues -and pull requests.** Please attempt to present contributions as enhancements, -defects, or typos and label them accordingly. +4. **Use [labels](https://github.com/cf-convention/cf-conventions/labels) on issues and pull requests.** +Please attempt to present contributions as enhancements, defects, or typos and label them accordingly. ## Issues and Pull Requests The following cases describe potential patterns of use for issues and pull requests. -1. **Typo Fix** If the change is a non-controversial fix such as a typo, -no issue is required. A pull request with the fix can be submitted directly. -Contributors not familiar with github can submit issues for typos and similar -issues for others to fix. - -2. **Single Section Change** In the case of a change concerning one to a few -paragraphs, an issue should be opened that describes the problem and proposed -fix. The problem text should be pasted in the body of the issue and proposed fix -included. A link to the line where the problem exists could also be included. -If the modification is non-controversial, a pull request could be opened -simultaneously if the reporter is familiar with git and github. Discussion of -the proposal should take place in one issue. Once accepted, a pull request could -be submitted implementing the suggested change. Final review should take place -in the pull request and the issue closed when the pull request is merged. - -3. **Changes Spanning Multiple Sections** If reasonable, changes concerning -multiple sections should follow the patter described Single Section Change. -If explicitly listing proposed changes is not practical, general guideline -2 should be followed to document the proposal. Depending on the nature of the -proposal, interested community members can decide what the most effective tool -is for development and review of specification changes. Some proposals have -been developed in google docs and contributed as a pull request after general -consensus has been reached, but the specific tools used for development of -significant changes is up to those contributing and reviewing it. +1. **Typo Fix** If the change is a non-controversial fix such as a typo, no issue is required. +A pull request with the fix can be submitted directly. +Contributors not familiar with github can submit issues for typos and similar issues for others to fix. + +2. **Single Section Change** In the case of a change concerning one to a few paragraphs, an issue should be opened that describes the problem and proposed fix. +If important to the issue, the problem text should be pasted in the body of the issue and proposed fix included. +A link to the line where the problem exists could also be included. +If the modification is non-controversial, a pull request could be opened simultaneously. +Discussion of the proposal should take place in one issue. +Final review should take place in the pull request and the issue closed when the pull request is merged. + +3. **Changes Spanning Multiple Sections** If reasonable, changes concerning multiple sections should follow the patter described in Single Section Change. +If explicitly listing proposed changes is not practical, general guideline 2 should be followed to document the proposal. +Depending on the nature of the proposal, interested community members can decide what the most effective tool is for development and review of specification changes. +Tools used for development of significant changes are up to those contributing and reviewing it. From 591cb882fd0693ce7aeda2ac04a8eeb4a235ec4e Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 20 Jul 2018 13:59:06 -0500 Subject: [PATCH 034/126] appa and text update per review --- appa.adoc | 6 ++++++ ch07.adoc | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/appa.adoc b/appa.adoc index c489522b..b7eecaa4 100644 --- a/appa.adoc +++ b/appa.adoc @@ -240,6 +240,12 @@ formula in the definition. | <> | Identifies a variable indicating the count of nodes per geometry. +| **`nodes`** +| S +| C +| <> +| Identifies a coordinate node variable. + | **`part_node_count`** | S | M diff --git a/ch07.adoc b/ch07.adoc index 86c2b949..ddfdaa0f 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -678,7 +678,7 @@ For example, polygon geometries could include single part geometries like the St The **`node_coordinates`** attribute contains the blank-separated names of the variables that contain geometry node coordinates (one variable for each spatial dimension). The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. -If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables which have a meaningful correspondence with node coordinates may have a **`nodes`** attribute that names the corresponding node coordinates. +If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables that have a meaningful correspondence with node coordinates are indicated as such by a **`nodes`** attribute that names the corresponding node coordinates. Whether linked to normal CF space-time coordinates with a **`nodes`** attribute or not, inclusion of such coordinates is recommended to maintain backward compatibility with software that has not implemented geometry capabilities. The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. From d17c4bab8d69ec027146686b9e4631e2d2e179f6 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Mon, 23 Jul 2018 14:48:46 -0500 Subject: [PATCH 035/126] some review changes from @JonathanGregory --- .github/CONTRIBUTING.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index d7c6163a..76f5f63f 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -13,9 +13,8 @@ The CF community takes great pride in respectful and collegial discourse. Any di ## General Guidelines -1. **A given proposal should be discussed as one issue.** It shouldn't fork or be superseded by another one, unless that reflects what has happened to the -proposal. -This is so it is easy to trace the discussion what led to a given agreed proposal. +1. **A given proposal should be discussed as one issue.** It shouldn't fork or be superseded by another one, unless that reflects what has happened to the proposal. +This is so it is easy to trace the discussion tha led to a given agreed proposal. 2. **A proposal should convey the reasoning and effect on all relevant sections of the specification.** An overview of all actual changes and the impact the changes have on the specification should be clear. @@ -24,15 +23,16 @@ Depending on the length and nature of the proposal, this may require different a 3. **In general, issues should be used for discussion of proposed changes and pull requests should be used for review of agreed upon changes.** In other words, if the content or concept of what is being proposed needs to be vetted by the community it should be vetted in an issue. If the proposal is non-controversial (such as a typo correction) or has been agreed to in concept in an issue, then detailed review of the text may take place in a pull request. +Practically all changes should be documented and discussed in an issue fixed in a related pull request. 4. **Use [labels](https://github.com/cf-convention/cf-conventions/labels) on issues and pull requests.** -Please attempt to present contributions as enhancements, defects, or typos and label them accordingly. +Contributions must be presented as enhancements, defects, or typos and labeled accordingly. ## Issues and Pull Requests The following cases describe potential patterns of use for issues and pull requests. -1. **Typo Fix** If the change is a non-controversial fix such as a typo, no issue is required. +1. **Typo Fix** If the change is a non-controversial fix such as a typo, no issue is required as these changes do not appear in the convention history. A pull request with the fix can be submitted directly. Contributors not familiar with github can submit issues for typos and similar issues for others to fix. @@ -43,7 +43,8 @@ If the modification is non-controversial, a pull request could be opened simulta Discussion of the proposal should take place in one issue. Final review should take place in the pull request and the issue closed when the pull request is merged. -3. **Changes Spanning Multiple Sections** If reasonable, changes concerning multiple sections should follow the patter described in Single Section Change. +3. **Changes Spanning Multiple Sections** If reasonable, changes concerning multiple sections should follow the pattern described in Single Section Change. If explicitly listing proposed changes is not practical, general guideline 2 should be followed to document the proposal. Depending on the nature of the proposal, interested community members can decide what the most effective tool is for development and review of specification changes. -Tools used for development of significant changes are up to those contributing and reviewing it. +Tools used for development of significant changes are up to those contributing and reviewing it. +Note that their is a rendered "rich-diff" view of a pull request that can be helpful for review of large contributions. From 38ee5214650dea818ad00fb91c006c9cb561603d Mon Sep 17 00:00:00 2001 From: Ethan Davis Date: Tue, 24 Jul 2018 15:33:18 -0600 Subject: [PATCH 036/126] Fix for bad reference to an example in section 6.1 "Labels". Also update numbering of examples in chapter 6. --- ch06.adoc | 9 +++++---- toc-extra.adoc | 4 ++-- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/ch06.adoc b/ch06.adoc index 57bbcac1..0969d909 100644 --- a/ch06.adoc +++ b/ch06.adoc @@ -17,8 +17,9 @@ element of an axis. This is particularly useful for discrete axes observational data from a number of observing stations, it may be convenient to provide the names of the stations as labels for the elements of the station dimension (<>). - <> illustrates another application for labels. - +There are several other uses for labels in CF. For instance, +<> shows the use of labels +to indicate geographic regions. Character strings labelling the elements of an axis are regarded as string-valued auxiliary coordinate variables. The **`coordinates`** @@ -41,7 +42,7 @@ to the data variable. This is a convenience feature. When data is representative of geographic regions which can be identified by names but which have complex boundaries that cannot practically be specified using longitude and latitude boundary coordinates, a labeled axis should be used to identify the regions. We recommend that the names be chosen from the list of link:$$http://cfconventions.org/Data/cf-standard-names/docs/standardized-region-names.html$$[standardized region names] whenever possible. To indicate that the label values are standardized the variable that contains the labels must be given the **`standard_name`** attribute with the value `region`. [[northward-heat-transport-in-atlantic-ocean-ex]] -[caption="Example 6.2. "] +[caption="Example 6.1. "] .Northward heat transport in Atlantic Ocean ==== @@ -83,7 +84,7 @@ data: In some situations a dimension may have alternative sets of coordinates values. Since there can only be one coordinate variable for the dimension (the variable with the same name as the dimension), any alternative sets of values have to be stored in auxiliary coordinate variables. For such alternative coordinate variables, there are no mandatory attributes, but they may have any of the attributes allowed for coordinate variables. [[model-level-numbers-ex]] -[caption="Example 6.3. "] +[caption="Example 6.2. "] .Model level numbers ==== diff --git a/toc-extra.adoc b/toc-extra.adoc index 055172c4..e771a3ef 100644 --- a/toc-extra.adoc +++ b/toc-extra.adoc @@ -35,8 +35,8 @@ F.1. <> 5.11. <> 5.12. <> 5.13. <> -6.2. <> -6.3. <> +6.1. <> +6.2. <> 7.1. <> 7.2. <> 7.3. <> From e56da4407c3d855b8e59b16d528cf0dc7bac2556 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Wed, 1 Aug 2018 16:26:23 -0700 Subject: [PATCH 037/126] Revise Daniel's draft. Revert authors and version. --- cf-conventions.adoc | 7 +++-- ch01.adoc | 10 ++++++- ch02.adoc | 70 ++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 83 insertions(+), 4 deletions(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 86bf3afd..e049e686 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -1,6 +1,6 @@ = NetCDF Climate and Forecast (CF) Metadata Conventions -Brian Eaton; Jonathan Gregory; Bob Drach; Karl Taylor; Steve Hankin; Jon Blower; John Caron; Rich Signell; Phil Bentley; Greg Rappa; Heinke Höck; Alison Pamment; Martin Juckes; Martin Raspaud -Version 1.7 +Brian Eaton; Jonathan Gregory; Bob Drach; Karl Taylor; Steve Hankin; Jon Blower; John Caron; Rich Signell; Phil Bentley; Greg Rappa; Heinke Höck; Alison Pamment; Martin Juckes; Martin Raspaud; Randy Horne +Version 1.8 :sectanchors: :toc: macro :toclevels: 3 @@ -94,6 +94,9 @@ include::appg.adoc[] :numbered: include::apph.adoc[] +:numbered: +include::appi.adoc[] + :numbered!: include::history.adoc[] diff --git a/ch01.adoc b/ch01.adoc index 1e0b9e1b..fff35464 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -48,12 +48,20 @@ grid mapping variable:: A variable used as a container for attributes that defin latitude dimension:: A dimension of a netCDF variable that has an associated latitude coordinate variable. +local apex group:: the nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. + longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. +most proximal identifier:: The identifier of the element that can be reached via the shortest traversal of the file from the referring element following the rules set forth in the <>. + multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. +out-of-group reference:: a reference to an element that is not contained in the referring group. + recommendation:: Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. +referring group:: the group in which a reference to an identifier occurs. + scalar coordinate variable:: A scalar variable (i.e. one with no dimensions) that contains coordinate data. Depending on context, it may be functionally equivalent either to a size-one coordinate variable (<>) or to a size-one auxiliary coordinate @@ -71,7 +79,7 @@ vertical dimension:: A dimension of a netCDF variable that has an associated ver No variable or dimension names are standardized by this convention. Instead we follow the lead of the NUG and standardize only the names of attributes and some of the values taken by those attributes. The overview provided in this section will be followed with more complete descriptions in following sections. <> contains a summary of all the attributes used in this convention. -We recommend that the NUG defined attribute **`Conventions`** be given the string value "**`CF-1.7`**" to identify datasets that conform to these conventions. +We recommend that the NUG defined attribute **`Conventions`** be given the string value "**`CF-1.8`**" to identify datasets that conform to these conventions. The general description of a file's contents should be contained in the following attributes: **`title`** , **`history`** , **`institution`** , **`source`** , **`comment`** and **`references`** ( <> ). For backwards compatibility with COARDS none of these attributes is required, but their use is recommended to provide human readable documentation of the file contents. diff --git a/ch02.adoc b/ch02.adoc index 19d80681..083b7005 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -113,7 +113,7 @@ This standard describes many attributes (some mandatory, others optional), but a We recommend that netCDF files that follow these conventions indicate this by setting the NUG defined global attribute **`Conventions`** to -the string value "**`CF-1.7`**". The string is interpreted as a +the string value "**`CF-1.8`**". The string is interpreted as a directory name relative to a directory that is a repository of documents describing sets of discipline-specific conventions. The conventions directory name is currently interpreted relative to the directory @@ -163,5 +163,73 @@ The global **`external_variables`** attribute is a blank-separated list of the n These variables are to be found in other files (called "external files") but CF does not provide conventions for identifying the files concerned. The only attribute for which CF standardises the use of external variables is **`cell_measures`**. +[[groups, Section 2.7, "Groups"]] +=== Groups +Groups provide a powerful mechanism to structure data hierarchically. +This convention does not standardize group names. +It may be of benefit to name groups in such a way that human readers can interpret them. +However, files that conform to this standard shall not require software to interpret or decode information from group names. +Instead, references to out-of-group elements shall be found by applying the scoping rules outlined below. + +==== Scope + +Objects can be referred to as long as they are visible within the scope from which they are used. +Any object can be referred to, as long as it can be found with one of the following search strategies: + +* Search by absolute path +* Search by relative path +* Search by proximity + +These strategies are explained in detail in the following sections. + +===== Search by absolute path + +An identifier specified with an absolute path (i.e., with a leading slash "/") resolves to the indicated location. +For example, a `coordinates` attribute of `/g1/lat` refers to the `lat` variable in group `/g1`. +The dataset producer must ensure that dimension(s) utilized by out-of-group objects are the same as those used by the referring object. +For example, the nearest dimension `lat` to the current (referring) group must be the same as the dimension `lat` nearest in scope to the out-of-group coordinates. +Thus a variable `/g2/temperature` with coordinates attribute containing `/g1/lat` is permissible so long as there is no other `lat` dimension "in between" those locations, e.g., `/lat`. + +===== Search by relative path + +An identifier specified with a relative path (i.e., with an internal but no leading slash "/") resolves to the indicated location. +For example, a `coordinates` attribute of `g1/lat` refers to the `lat` variable in subgroup `g1` of the current (referring) group. +Upward path traversals from the current group are indicated with the UNIX convention. +For example, `../g1/lat` refers to the `lat` variable in the sibling group `g1` of the current (referring) group. + +===== Search by proximity + +An identifier specified with no path resolves to the most proximal identifier of the same name. +In all cases, if the named object is present in the referring group, that object is the most proximal. +For example, a `coordinates` attribute of `lat` refers to the `lat` variable (if any) in the present group. +If the object is not in the referring group then the identifier is termed an out-of-group reference. + +Out-of-group references are resolved by searching all direct ancestors until the specified identifier is found. + +[NOTE] +==== +The special lateral search strategy described below is for use with variables that reference coordinates that are outside of the referring group's direct ancestors. +This placement of coordinates and lateral search strategy to find them are discouraged. They are allowed mainly for backwards-compatibility with existing datasets, and may be deprecated in future versions of the standard. +==== + +A special case exists for coordinate variables. +Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the local apex group is reached. +If the identifier in question cannot be found within the local apex group or any of the intermediate groups between it and the referring group, a lateral (width-wise) search is initiated from the local apex group. +The lateral search proceeds downwards from the local apex group through each level of groups until the sought identifier is found. + +==== Application of attributes + +The following attributes are considered global attributes: + +* `title` +* `history` +* `Conventions` + +These attributes are considered global attributes and should not be duplicated in child groups. +They should not be attached to any variable and should not appear in any child groups of the root group. + +Other attributes may be present within groups without being attached to a variable. +Such attributes apply to the group where they are defined, and to that group's descendants, but not to ancestor or sibling groups. +If a group attribute is defined in a parent group, and one of the child group redefines the same attribute, the definition within the child group applies for the child and all of its descendants. From d29466b4c48fc6465fc56cca00fdd3fce7fafb78 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Wed, 1 Aug 2018 19:00:01 -0700 Subject: [PATCH 038/126] eliminate Randy Horne --- cf-conventions.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index e049e686..840cd19c 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -1,5 +1,5 @@ = NetCDF Climate and Forecast (CF) Metadata Conventions -Brian Eaton; Jonathan Gregory; Bob Drach; Karl Taylor; Steve Hankin; Jon Blower; John Caron; Rich Signell; Phil Bentley; Greg Rappa; Heinke Höck; Alison Pamment; Martin Juckes; Martin Raspaud; Randy Horne +Brian Eaton; Jonathan Gregory; Bob Drach; Karl Taylor; Steve Hankin; Jon Blower; John Caron; Rich Signell; Phil Bentley; Greg Rappa; Heinke Höck; Alison Pamment; Martin Juckes; Martin Raspaud Version 1.8 :sectanchors: :toc: macro From 452fd0a69163d1ba5d419fd3cc83b5952df62b30 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Wed, 1 Aug 2018 19:41:42 -0700 Subject: [PATCH 039/126] Define per-variable attributes --- ch02.adoc | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/ch02.adoc b/ch02.adoc index 083b7005..b82a5198 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -220,7 +220,8 @@ The lateral search proceeds downwards from the local apex group through each lev ==== Application of attributes -The following attributes are considered global attributes: +The attributes marked in the NUG conventions +(link:$$http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html$$[NUG Appendix A, Attribute Conventions]) as global attributes may only appear in the root group. These include * `title` * `history` @@ -228,8 +229,21 @@ The following attributes are considered global attributes: These attributes are considered global attributes and should not be duplicated in child groups. They should not be attached to any variable and should not appear in any child groups of the root group. +All other attributes defined in the NUG conventions are per-variable attributes. These include + +* `units` +* `long_name` +* `_FillValue` +* `missing_value` +* `valid_min` +* `valid_max` +* `valid_range` +* `scale_factor` +* `add_offset` +* `Coordinates` + +Per-variable attributes should be attached to variables and should not appear as group-level metadata. Other attributes may be present within groups without being attached to a variable. Such attributes apply to the group where they are defined, and to that group's descendants, but not to ancestor or sibling groups. If a group attribute is defined in a parent group, and one of the child group redefines the same attribute, the definition within the child group applies for the child and all of its descendants. - From 0017f41e38ec04d81e1c39b72dbd74dcbd3bf51d Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Mon, 25 Jun 2018 21:19:33 -0600 Subject: [PATCH 040/126] Add group terms to Terminology --- ch01.adoc | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/ch01.adoc b/ch01.adoc index 1e0b9e1b..9c190784 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -48,12 +48,21 @@ grid mapping variable:: A variable used as a container for attributes that defin latitude dimension:: A dimension of a netCDF variable that has an associated latitude coordinate variable. +local apex group:: the nearest ancestor group to a referring group. The local apex contains the definition of the referred identifier. + longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. +// TODO: Which section? Make a proper reference. +most proximal identifier:: The identifier of an element which can be reached via the shortest traversal of the file from the referring element following the rules set forth in the Scoping section. + multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. +out of group reference:: a reference to an element which is not contained in the referring group. + recommendation:: Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. +referring group:: the group in which a reference to an identifier occurs. + scalar coordinate variable:: A scalar variable (i.e. one with no dimensions) that contains coordinate data. Depending on context, it may be functionally equivalent either to a size-one coordinate variable (<>) or to a size-one auxiliary coordinate From ccf4c4a5bca5cdd0867a4de5c2f9e52247ac4f55 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Thu, 28 Jun 2018 12:35:11 -0600 Subject: [PATCH 041/126] Finalize new definitions --- ch01.adoc | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/ch01.adoc b/ch01.adoc index 9c190784..19a8d9ed 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -48,16 +48,15 @@ grid mapping variable:: A variable used as a container for attributes that defin latitude dimension:: A dimension of a netCDF variable that has an associated latitude coordinate variable. -local apex group:: the nearest ancestor group to a referring group. The local apex contains the definition of the referred identifier. +local apex group:: the nearest ancestor group to a referring group in which any of the dimensions underlying an out-of-group coordinate is defined. longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. -// TODO: Which section? Make a proper reference. -most proximal identifier:: The identifier of an element which can be reached via the shortest traversal of the file from the referring element following the rules set forth in the Scoping section. +most proximal identifier:: The identifier of an element which can be reached via the shortest traversal of the file from the referring element following the rules set forth in the <>. multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. -out of group reference:: a reference to an element which is not contained in the referring group. +out-of-group reference:: a reference to an element which is not contained in the referring group. recommendation:: Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. From 2e088018c052d72ec5bd3aa1f87fff6da107b9d2 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Thu, 28 Jun 2018 12:35:39 -0600 Subject: [PATCH 042/126] Insert section on groups (2.7) --- ch02.adoc | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/ch02.adoc b/ch02.adoc index 19d80681..1d66a8a7 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -163,5 +163,74 @@ The global **`external_variables`** attribute is a blank-separated list of the n These variables are to be found in other files (called "external files") but CF does not provide conventions for identifying the files concerned. The only attribute for which CF standardises the use of external variables is **`cell_measures`**. +[[groups, Section 2.7, "Groups"]] +=== Groups +Groups provide a powerful mechanism to structure data hierarchically. +This convention does not standardize group names. +It may be of benefit to name groups in such a way that human readers can interpret them. +However, files which conform to this standard shall not require software to interpret names of groups. +Instead, references to out-of-group elements shall be found by applying the scoping rules outlined below. + +==== Scope + +Objects can be referred to as long as they are visible within the scope from which they are used. +This means that any object can be referred to, as long as it can be found with one of the following search strategies: + +* Search by absolute path +* Search by relative path +* Search by proximity + +These strategies are explained in detail in the following sections. + +===== Search by absolute path + +An identifier specified with an absolute path (i.e., with a leading slash "/") resolves to the indicated location. +For example, a `coordinates` attribute of `/g1/lat` refers to the `lat` variable in group `/g1`. +The dataset producer must ensure that dimension(s) utilized by out-of-group objects are the same as those used by the referring object. +For example, the nearest dimension `lat` to the current (referring) group must be the same as the dimension `lat` nearest in scope to the out-of-group coordinates. +Thus a variable `/g2/temperature` with coordinates attribute containing `/g1/lat` is permissible so long as there is no other lat dimension "in between" those locations, e.g., `/lat`. + +===== Search by relative path + +An identifier specified with a relative path (i.e., with an internal but no leading slash "/") resolves to the indicated location. +For example, a `coordinates` attribute of `g1/lat` refers to the `lat` variable in subgroup `g1` of the current (referring) group. +Upward path traversals from the current group are indicated with the UNIX convention. +For example, `../g1/lat` refers to the `lat` variable in the sibling group `g1` of the current (referring) group. + +===== Search by proximity + +An identifier specified with no path resolves to the most proximal identifier of the same name. +In all cases, if the named object is present in the referring group, that object is the most proximal. +For example, a `coordinates` attribute of `lat` refers to the `lat` variable (if any) in the present group. +If the object is not in the referring group then the identifier is termed an out-of-group reference. + +Out-of-group references are resolved by searching all direct ancestors until the specified identifier is found. + +[NOTE] +==== +The special search strategy described below is for use with coordinate variables which reference identifiers that are not found within the caller's direct ancestors. +This strategy is deprecated and may be removed from future versions of the standard. +==== + +A special case exists for coordinate variables. +Because coordinate variables must share dimensions with the variables which reference them, the ancestor search is executed only until the local apex group is reached. +If the identifier in question cannot be found within the local apex group or any of the intermediate groups between it and the referring group, a width-wise search is initiated from the local apex group. +The width-wise search proceeds downwards from the local apex group through each level of groups until the sought identifier is found. +This means that variables may use coordinates which must be found via proximity search, and which are not located in the referring group or any of its ancestors. + +==== Application of attributes + +The following attributes are considered global attributes: + +* `title` +* `history` +* `Conventions` + +These attributes are considered global attributes and should not be duplicated in child groups. +They should not be attached to any variable and should not appear in any child groups of the root group. + +Other attributes may be present within groups without being attached to a variable. +Such attributes apply to the group where they are defined, and to that group's descendants, but not to ancestor or sibling groups. +If a group attribute is defined in a parent group, and one of the child group redefines the same attribute, the definition within the child group applies for the child and all of its descendants. From 0b52c87acd857c8e43d497a94f919a53b8064284 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Thu, 19 Jul 2018 19:50:00 +0200 Subject: [PATCH 043/126] Add Appendix I --- appi.adoc | 404 ++++++++++++++++++++++++++++++++++++++++++++ cf-conventions.adoc | 3 + history.adoc | 5 +- toc-extra.adoc | 5 + 4 files changed, 416 insertions(+), 1 deletion(-) create mode 100644 appi.adoc diff --git a/appi.adoc b/appi.adoc new file mode 100644 index 00000000..e49f7df8 --- /dev/null +++ b/appi.adoc @@ -0,0 +1,404 @@ +[[appendix-examples-groups, Appendix I, Good Practices and Annotated Examples for Using Groups]] + +[appendix] +== Good Practices and Annotated Examples for Using Groups + +This appendix contains good practices for using netCDF-4 groups in CF compliant data, as well as annotated examples of existing use cases for representing data using groups. + +=== Good practices + +. The use of attributes attached to a group in order to store metadata normally attached directly to variables is discouraged. +This includes, for example, replacing per-variable attributes like `_FillValue`, `scale_factor`, `valid_min`, with group-level equivalents. +Although group attributes might be more concise, it is likely to create problems with downstream software and reduce interoperability. + +. Each group's attributes, dimensions and variables should be self-contained in combination with the group metadata and any inherited properties (e.g., dimension sizes, coordinates). +This ensures that if a group is renamed or extracted (with any inherited properties) into a new file, its information content is preserved. +For example, storage of ensembles as sibling groups (as described below) is often clearer when the realization number is encoded in the group name. +In this case, the `Realization` group attribute retains the realization number even if the group is renamed. +Other commonly enumerated group names, such as station identifiers, buoy numbers, or channel wavelengths are analogous. +It is fine to enumerate or itemize names so long as the number or name is redundantly stored as a group attribute. + +. Renaming or moving a group or self-contained branch of groups to a new location should not affect the interpretation of data. +Since relative and absolute paths (containing "/") of coordinates, dimensions, and attributes are fragile, it is preferred they not be present in attributes. +Instead, named objects resolve to the most proximal object (i.e., dimension or variable) of that name that has the referring attribute within its scope. +This makes files which use groups amenable to sub-setting and aggregation. +For example, the CF `coordinates` attribute identifies a variable's coordinates in a whitespace-separated list such as `"lat lon"`. +When the coordinates are outside the group that contains the `coordinates` attribute, it is tempting to store the coordinate locations as full, unambiguous paths such as `"/g1/lat /g1/lon"`, or as relative paths such as `"g1/lat g1/lon"`. +However, paths that contain slashes must be explicitly altered when the variable is subset into a new file with a different group hierarchy, or when the hierarchy is flattened. +CF attributes affected by this practice include `ancillary_variables`, `bounds`, `cell_measures`, `climatology`, `coordinates`, `formula_terms`, and `grid_mapping`. +The simpler `"lat lon"` specification works without alteration in all situations for out-of-group locations so long as it is understood to mean the nearest identifiers that have the referring variable in their scope. + +. When using a grid mapping which is referenced in multiple groups, the reference to the coordinates themselves should be maintained as an attribute of the referencing variables, and not as an attribute of the grid mapping itself. This way the coordinates can resolve to the correct coordinates which are "owned" by the referencing variable, and which are not shared amongst variables, while the grid mapping which is used to interpret the coordinates is stored in only one place. Both coordinates and grid mapping are thus found via regular scoping mechanisms. ++ +A common use case for this is data produced on board a geostationary platform. Different swaths are stored in different groups, each with their own coordinates, but all of which share a common grid mapping for the entire full disk view. + +=== Use cases + +==== Collections + +Group datasets are well-suited when users might benefit from storing related datasets (collections of variables) in a single location (file). +Loose collections might comprise different sets of distinct variables with a common purpose, e.g., multiple sensor observations at a single location. +For example, a model and satellite retrieval of a temperature field might be combined with an in situ temperature sensor as follows: + +[[example-i.1]] +[caption="Example I.1. "] +.Collection of related datasets sharing a single set of global metadata and segregated using groups. +==== +---- +netcdf clc { + :Conventions = "CF-1.8"; + :history = "Tue Apr 25 12:46:10 PDT 2017: ncgen -k netCDF-4 -b -o ~/nco/data/clc.nc ~/nco/data/clc.cdl"; + :Purpose = "Demonstrate a collection of related datasets stored in hierarchical format"; + + group: model { + :Source = "Model simulations, e.g., of temperature"; + dimensions: + lat=2; + lon=3; + time=unlimited; + variables: + float temperature(time,lat,lon); + double time(time); // Variable attributes omitted for clarity + double lat(lat); + double lon(lon); + data: + lat=-90,90.; + lon=0.,120.,240.; + temperature=273.,273.,273.,273.,273.,273.; + time=1.; + } // end model + + group: measurements_remote_sensing { + :Source = "Satellite measurements of same region as modelled, and on a different spatio-temporal grid"; + dimensions: + lat=3; + lon=4; + time=unlimited; + variables: + float temperature(time,lat,lon); + double time(time); // Variable attributes omitted for clarity + double lat(lat); + double lon(lon); + data: + lat=-90,0.,90.; + lon=0.,90.,180.,270.; + temperature=273.,273.,273.,273.,273.,273.,273.,273.,273.,273.,273.,273.; + time=1.; + } // end measurements_remote_sensing + + group: measurements_in_situ { + :Source = "In situ measurements, e.g., from an automated weather station with its own time-frequency"; + dimensions: + time=unlimited; + variables: + float temperature_10m(time); + double time(time); // Variable attributes omitted for clarity + data: + temperature_10m=271,272,273,274; + time=1.,2.,3.,4.; + } // end measurements_in_situ +} // end root group +---- +The namespace separation provided by groups allows variable and dimension names to be re-used and axes lengths to be re-defined. +In this example two groups contain a `temperature` variable, and the third contains a temperature at 10 m height. +Each group has its own spatio-temporal grid that re-uses the same coordinate names (`lat`, `lon`, `time`) as the other groups without conflict. +==== + +==== Ensembles + +Geoscientists use the label "ensemble" for collections of realizations of individual models or measurements of the same phenomena. +It is particularly important for models to repeat simulations of nonlinear systems multiple times (with slightly perturbed initial conditions) in order to characterize the statistical properties of systems with internal variability. +The namespace separation provided by groups ensures that variable names can be re-used. +Axis lengths can be re-defined if distinct realizations employ different spatio-temporal resolutions. +Multiple realizations of a single model temperature field might be stored as: + +[[example-i.2]] +[caption="Example I.2. "] +.Various members of an ensemble forecast stored as individual groups. +==== +---- +netcdf nsm { + :Conventions = "CF-1.8"; + :history = "Tue Apr 25 12:46:10 PDT 2017: ncgen -k netCDF-4 -b -o ~/nco/data/clc.nc ~/nco/data/clc.cdl"; + :Purpose = "Demonstrate a model ensemble stored in hierarchical format"; + + group: cesm_01 { + :Scenario = "Historical"; + :Model = "CESM"; + :Realization = "1"; + + dimensions: + time=unlimited; + variables: + float temperature(time); + double time(time); + data: + temperature=272.1,272.1,272.1,272.1; + time=1.,2.,3.,4.; + } // cesm_01 + + group: cesm_02 { + :Scenario = "Historical"; + :Model = "CESM"; + :Realization = "2"; + + dimensions: + time=unlimited; + variables: + float temperature(time); + double time(time); + data: + temperature=272.2,272.2,272.2,272.2; + time=1.,2.,3.,4.; + } // cesm_02 + + group: cesm_03 { + :Scenario = "Historical"; + :Model = "CESM"; + :Realization = "3"; + + dimensions: + time=unlimited; + variables: + float temperature(time); + double time(time); + data: + temperature=272.3,272.3,272.3,272.3; + time=1.,2.,3.,4.; + } // cesm_03 +} // root group +---- +Here each group contains a different realization of the same model, and the group names are suffixed with a numerical identifier, as well as containing a numerically valued group attribute named `Realization`. +This attribute would be carried with its group should the group ever be renamed or extracted into a new file, thus preserving the identity of the original realization. +It is allowed to include numeric metadata in group names so long as the information is redundantly stored as group metadata (e.g., `Realization`). +To accommodate the potential need of downstream software to deconstruct a number-containing group name into its original components it is suggested that the numeric portion be encoded as a fixed-width string separated by a non-alphanumeric character, such as "`_03`" above. +==== + +==== Discrete Sampling Geometries + +CF describes a powerful syntax for encoding spatiotemporal data from multiple locations into multidimensional flat-file formats. +The patterns of the spatiotemporal data are encapsulated into several features, each labeled with a distinct `featureType` that must be either `point`, `timeSeries`, `profile`, `trajectory`, `timeSeriesProfile`, or `trajectoryProfile`. +These features use an instance dimension to span a collection of like features. +One-dimensional variables that have only the instance dimension in a Discrete Geometry CF file are called instance variables. +Common instance variables include `lat(station)` and `station_name(station, name_len)`. +Here the `station` dimension enumerates the stations in the collection. + +It is also possible to use groups as an extended form of discrete sampling features where groups replace the instance dimension in Discrete Sampling Geometries. +Instead of a `station` dimension, feature collections may designate a group to contain the feature for each station. +Typically the group name would be the same as the `station_name`. +A `timeSeries` collection might appear like this: + +[[example-i.3]] +[caption="Example I.3. "] +.Orthogonal time series stored using groups. +==== +---- +netcdf tms { + :Conventions = "CF-1.8"; + :history = "Thu Jun 22 17:45:12 PDT 2017: ncgen -k netCDF-4 -b -o ~/nco/data/tms.nc ~/nco/data/tms.cdl"; + :Purpose = "Demonstrate a collection of DSG timeSeries featureType stored in hierarchical format"; + :featureType = "timeSeries"; + + dimensions: + time=unlimited; + + variables: + double time(time) ; + time:standard_name = "time"; + time:long_name = "time of measurement" ; + time:units = "days since 1970-01-01 00:00:00" ; + + group: irvine { + variables: + + float humidity(time) ; + humidity:standard_name = "specific humidity" ; + humidity:coordinates = "lat lon alt station_name" ; + humidity:_FillValue = -999.9f; + + float lon ; + lon:standard_name = "longitude"; + lon:long_name = "station longitude"; + lon:units = "degrees_east"; + + float lat ; + lat:standard_name = "latitude"; + lat:long_name = "station latitude" ; + lat:units = "degrees_north" ; + + float alt ; + alt:long_name = "vertical distance above the surface" ; + alt:standard_name = "height" ; + alt:units = "m"; + alt:positive = "up"; + alt:axis = "Z"; + + string station_name; + station_name:long_name = "station name" ; + station_name:cf_role = "timeseries_id"; + } // irvine + + group: boulder { + // Variables/dimensions repeated, omitted for clarity + } // boulder +} // root group +---- +Placement of the `time` dimension depends upon the characteristics of the sensor network, and is key to economically represent the collection. +If sensors at different locations measure values at the same time, then a single `time` coordinate may be placed in the root directory. +Each station (group) inherits this coordinate. +This is the case for an orthogonal multidimensional array representations. +==== + +When stations measure with distinct time coordinates amongst themselves, it is recommended that the `time` coordinates be stored locally within each group: + +[[example-i.4]] +[caption="Example I.4. "] +.Ragged array of station time series stored using groups. +==== +---- +netcdf tms { + + // Global metadata omitted for clarity + + group: irvine { + dimensions: + time=unlimited; + variables: + double time(time) ; + time:standard_name = "time"; + time:long_name = "time of measurement" ; + time:units = "days since 1970-01-01 00:00:00" ; + // Variables besides time as before, omitted for clarity + } // irvine + + group: boulder { + dimensions: + time=unlimited; + + variables: + double time(time) ; + time:standard_name = "time"; + time:long_name = "time of measurement" ; + time:units = "days since 1970-01-01 00:00:00" ; + // Variables besides time as before, omitted for clarity + } // boulder +} // root group +---- +This accommodates the common situation where different sensors have different observation times. +An alternative approach the use of an incomplete multidimensional array representation, which increases the rank and size of the `time` coordinate, so that each station must allocate space for all observation times used anywhere in the collection. +Groups can avoid this complexity by employing a station-specific `time` coordinate within each group. +This saves space relative to the incomplete multidimensional array representation since the representation using groups avoids padding the missing data. + +This formalism of station-specific `time` coordinates also naturally handles timeseries with time-varying deviations from a nominal point spatial location and offers an alternative to contiguous and indexed ragged arrays. +==== + +==== Remote sensing channels + +In satellite remote sensing, hierarchical datasets can be useful for storing low-level data, such as payload data, engineering data or instrument data for processing into geophysical variables. +While it is useful to store all sensed data from a single satellite or instrument in one unified file, many applications require only a subset of this data in order to produce higher-level products. +Additionally, some applications require data concerning the state of the vehicle or instrument, while others do not. +Therefore it is useful to split the observations from different channels and/or instruments into different groups within the netCDF file, as follows (for the sake of simplicity, a reduced, hypothetical file is shown): + +[[example-i.5]] +[caption="Example I.5. "] +.Rich remote sensing data represented hierarchically using groups. +==== +---- +netcdf nextgen-satellite { + // global attributes: + :title = "EUMETSAT EPS-SG IASI-NG Level 1c data" ; + :summary = "Demonstrate a Level 1 satellite product stored using groups"; + :Conventions = "CF-1.8"; + :orbit_start = 5 ; + :orbit_end = 6 ; + + group: status { + group: satellite { + dimensions: + manoeuvre_items = 0 ; + variables: + int manoeuvre_start_time_utc(manoeuvre_items); + int manoeuvre_end_time_utc(manoeuvre_items); + } // group satellite + } // group status + + group: data { + group: instrument_01 { + :instrument_identifier = "IASI-NG" ; + + dimensions: time = 1 ; + dimensions: nrows = 1 ; + dimensions: ncols = 1 ; + + variables: + float lat(nrows, ncols) ; + lat:units = "degrees_north" ; + lat:standard_name = "latitude" ; + float lon(nrows, ncols) ; + lon:units = "degrees_east" ; + lon:standard_name = "longitude" ; + double time(time) ; + time:standard_name = "time" ; + time:units = "seconds since 2000-01-01 00:00.00Z" ; + time:calendar = "gregorian" ; + + group: band_01 { + :sensor_band_identifier = "IASI-NG Channel 1" ; + + group: radiances { + dimensions: + n_wavenumbers = 1 ; + + variables: + int wavenumber(n_wavenumbers) ; + wavenumber:standard_name = "sensor_band_central_radiation_wavenumber" ; + double spectrum(nrows, ncols, n_wn) ; + spectrum:standard_name = "toa_outgoing_radiance_per_unit_wavenumber" ; + } // group radiances + + group: quality { + variables: + int number_of_missing_samples(nrows, ncols) ; + } // group quality + + } // group band_01 + + group: band_02{ + :sensor_band_identifier = "IASI-NG Channel 2" ; + } // group band_02 + } // group instrument_01 + + group: instrument_02 { + :instrument_identifier = "IASI-TLA" ; + } // group instrument_02 + } // group data +---- +A real example would be much more complex, but already this contrived example demonstrates the flexibility gained through the use of groups. +In this case, all observations from a given orbital dump are stored in a single file. +Subsets of this file can easily be produced, however, which contain observations only from certain instruments or certain bands of various instruments. +This can greatly reduce the volume of data which must be transferred between production facilities and thus increase timeliness for near-real-time products without sacrificing metadata integrity for archival purposes. +==== + +=== Mapping between Hierarchical and Flat files + +Files which use groups can be mapped to a set of flat files. +This procedure involves separating the group hierarchy tree into multiple distinct, self-contained, flat files, and is called _dismembering_. +A related procedure, _flattening_, collapses an entire hierarchical file into a single flat file. + +Files constructed in accord with the best practices outlined in this document can be dismembered without loss of information. +This allows dismembered files to be used with software aware of only flat files. +However, dismemberment often destroys the logical associations between data embodied in the original hierarchical file. + +Files using groups can be flattened without loss or alteration of information only in cases where none of the groups or their contents re-use name identifiers. +When name identifiers are re-used, a flattening algorithm must disambiguate the namespace conflicts in the flattened file, and this results in metadata alteration. +To guarantee resolution of such namespace conflicts, the flattening procedure must rename conflicting variables, dimensions, and group attributes. +For example, variables that share a name in separate groups in a hierarchical file (e.g., `/g1/v1` and `/g2/v1`) can be renamed by concatenating their names with their original group paths, with forward-slash path separators eliminated or replaced by a special character string in the flattened version (e.g., `g1_v1` and `g2_v1`). +A similar procedure must be followed to resolve namespace conflicts for group metadata and for dimension names. + +The link:$$https://www.opendap.org/software/hyrax-data-server$$[OPeNDAP Hyrax Data Server] implements such an algorithm when flattening hierarchical files. +Hyrax goes further in that it renames all variables beneath the root group by prepending the former full path name (with slashes represented as spaces) to the original short name. +Hyrax preserves the original name and group path of the variables in new attributes named `origname` and `fullnamepath`. +Their preservation ensures that a suitably programmed "inflation" tool could reverse the flattening and re-construct a hierarchical file with all the original names. +To our knowledge, no such inflator is yet available. +Software to flatten hierarchical files without namespace conflicts is available (e.g., link:$$http://nco.sf.net/nco.html#flatten$$[NCO]). +It is anticipated that tools (such as flatteners and inflators) that facilitate interoperability of files using groups will become more mature in the future. diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 86bf3afd..4dd74a6e 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -94,6 +94,9 @@ include::appg.adoc[] :numbered: include::apph.adoc[] +:numbered: +include::appi.adoc[] + :numbered!: include::history.adoc[] diff --git a/history.adoc b/history.adoc index 7ed7b636..4764fa70 100644 --- a/history.adoc +++ b/history.adoc @@ -1,6 +1,9 @@ [[revhistory, Revision History]] == Revision History +.TBD +. Added section on using groups. + .14 June 2004 . Added <>. . <> : Added **`latitude_of_projection_origin`** map parameter. @@ -173,4 +176,4 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. . Trivial updates to links for COARDS and UDUNITS in the bibliography. .10 August 2017 -. Updated use of WKT-CRS syntax. \ No newline at end of file +. Updated use of WKT-CRS syntax. diff --git a/toc-extra.adoc b/toc-extra.adoc index 055172c4..d311e223 100644 --- a/toc-extra.adoc +++ b/toc-extra.adoc @@ -75,3 +75,8 @@ H.19. <> H.20. <> H.21. <> H.22. <> +I.1. <> +I.2. <> +I.3. <> +I.4. <> +I.5. <> From 7a8faefa5d5f447d4986bc458aab17d13176b8b4 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Wed, 1 Aug 2018 16:26:23 -0700 Subject: [PATCH 044/126] Revise Daniel's draft. --- ch01.adoc | 6 +++--- ch02.adoc | 17 ++++++++--------- 2 files changed, 11 insertions(+), 12 deletions(-) diff --git a/ch01.adoc b/ch01.adoc index 19a8d9ed..54eb4fd3 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -48,15 +48,15 @@ grid mapping variable:: A variable used as a container for attributes that defin latitude dimension:: A dimension of a netCDF variable that has an associated latitude coordinate variable. -local apex group:: the nearest ancestor group to a referring group in which any of the dimensions underlying an out-of-group coordinate is defined. +local apex group:: the nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. -most proximal identifier:: The identifier of an element which can be reached via the shortest traversal of the file from the referring element following the rules set forth in the <>. +most proximal identifier:: The identifier of the element that can be reached via the shortest traversal of the file from the referring element following the rules set forth in the <>. multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. -out-of-group reference:: a reference to an element which is not contained in the referring group. +out-of-group reference:: a reference to an element that is not contained in the referring group. recommendation:: Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. diff --git a/ch02.adoc b/ch02.adoc index 1d66a8a7..b0f0128f 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -169,13 +169,13 @@ The only attribute for which CF standardises the use of external variables is ** Groups provide a powerful mechanism to structure data hierarchically. This convention does not standardize group names. It may be of benefit to name groups in such a way that human readers can interpret them. -However, files which conform to this standard shall not require software to interpret names of groups. +However, files that conform to this standard shall not require software to interpret or decode information from group names. Instead, references to out-of-group elements shall be found by applying the scoping rules outlined below. ==== Scope Objects can be referred to as long as they are visible within the scope from which they are used. -This means that any object can be referred to, as long as it can be found with one of the following search strategies: +Any object can be referred to, as long as it can be found with one of the following search strategies: * Search by absolute path * Search by relative path @@ -189,7 +189,7 @@ An identifier specified with an absolute path (i.e., with a leading slash "/") r For example, a `coordinates` attribute of `/g1/lat` refers to the `lat` variable in group `/g1`. The dataset producer must ensure that dimension(s) utilized by out-of-group objects are the same as those used by the referring object. For example, the nearest dimension `lat` to the current (referring) group must be the same as the dimension `lat` nearest in scope to the out-of-group coordinates. -Thus a variable `/g2/temperature` with coordinates attribute containing `/g1/lat` is permissible so long as there is no other lat dimension "in between" those locations, e.g., `/lat`. +Thus a variable `/g2/temperature` with coordinates attribute containing `/g1/lat` is permissible so long as there is no other `lat` dimension "in between" those locations, e.g., `/lat`. ===== Search by relative path @@ -209,15 +209,14 @@ Out-of-group references are resolved by searching all direct ancestors until the [NOTE] ==== -The special search strategy described below is for use with coordinate variables which reference identifiers that are not found within the caller's direct ancestors. -This strategy is deprecated and may be removed from future versions of the standard. +The special lateral search strategy described below is for use with variables that reference coordinates that are outside of the referring group's direct ancestors. +This placement of coordinates and lateral search strategy to find them are discouraged. They are allowed mainly for backwards-compatibility with existing datasets, and may be deprecated in future versions of the standard. ==== A special case exists for coordinate variables. -Because coordinate variables must share dimensions with the variables which reference them, the ancestor search is executed only until the local apex group is reached. -If the identifier in question cannot be found within the local apex group or any of the intermediate groups between it and the referring group, a width-wise search is initiated from the local apex group. -The width-wise search proceeds downwards from the local apex group through each level of groups until the sought identifier is found. -This means that variables may use coordinates which must be found via proximity search, and which are not located in the referring group or any of its ancestors. +Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the local apex group is reached. +If the identifier in question cannot be found within the local apex group or any of the intermediate groups between it and the referring group, a lateral (width-wise) search is initiated from the local apex group. +The lateral search proceeds downwards from the local apex group through each level of groups until the sought identifier is found. ==== Application of attributes From 5cdddf7fe23a95663c4398109ce8f71d78c1ae10 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Wed, 1 Aug 2018 19:41:42 -0700 Subject: [PATCH 045/126] Define per-variable attributes --- ch02.adoc | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/ch02.adoc b/ch02.adoc index b0f0128f..f64a423b 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -220,7 +220,8 @@ The lateral search proceeds downwards from the local apex group through each lev ==== Application of attributes -The following attributes are considered global attributes: +The attributes marked in the NUG conventions +(link:$$http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html$$[NUG Appendix A, Attribute Conventions]) as global attributes may only appear in the root group. These include * `title` * `history` @@ -228,8 +229,21 @@ The following attributes are considered global attributes: These attributes are considered global attributes and should not be duplicated in child groups. They should not be attached to any variable and should not appear in any child groups of the root group. +All other attributes defined in the NUG conventions are per-variable attributes. These include + +* `units` +* `long_name` +* `_FillValue` +* `missing_value` +* `valid_min` +* `valid_max` +* `valid_range` +* `scale_factor` +* `add_offset` +* `Coordinates` + +Per-variable attributes should be attached to variables and should not appear as group-level metadata. Other attributes may be present within groups without being attached to a variable. Such attributes apply to the group where they are defined, and to that group's descendants, but not to ancestor or sibling groups. If a group attribute is defined in a parent group, and one of the child group redefines the same attribute, the definition within the child group applies for the child and all of its descendants. - From a7454aa835600ea5b1ebdb9ca76ec7bbaefe47df Mon Sep 17 00:00:00 2001 From: marqh Date: Fri, 3 Aug 2018 10:18:05 +0000 Subject: [PATCH 046/126] Conditional: grid_mapping or aux_coords --- ch05.adoc | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/ch05.adoc b/ch05.adoc index 5619b03a..c62c73ec 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -183,12 +183,16 @@ _This section has been superseded by the treatment of time series as a type of d [[grid-mappings-and-projections, Section 5.6, "Horizontal Coordinate Reference Systems, Grid Mappings, and Projections"]] === Horizontal Coordinate Reference Systems, Grid Mappings, and Projections -When the coordinate variables for a horizontal grid are not longitude and latitude, the true latitude and longitude coordinates may be supplied via the **`coordinates`** attribute. -A __grid mapping variable__ is required if, in addition, it is desired to describe the mapping between the -given coordinate variables and the true latitude and longitude coordinates, or to describe the +A __grid mapping variable__ may be referenced by a data variable, to describe the mapping between the +given coordinate variables and the true latitude and longitude coordinates, to describe the figure of the Earth used to define the latitude and longitude coordinates, or to describe another -coordinate reference system definition used by some coordinates or auxiliary coordinates. Such a -grid mapping variable provides the description of the mapping via a collection of attached +coordinate reference system definition used by some coordinates or auxiliary coordinates. + +When the coordinate variables for a horizontal grid are not longitude and latitude, it is required that further information is provided to geo-locate the horizontal position. A __grid mapping variable__ provides this information. + +If no __grid mapping variable__ is provided and the coordinate variables for a horizontal grid are not longitude and latitude, then it is required that the true latitude and longitude coordinates are supplied via the **`coordinates`** attribute. + +A grid mapping variable provides the description of the mapping via a collection of attached attributes. It is of arbitrary type since it contains no data. Its purpose is to act as a container for the attributes that define the mapping. The one attribute that all grid mapping variables must have is grid_mapping_name, which takes a string value that contains the mapping's name. The other From e25cd7c65d816f2a05480d074dc2808773a36abd Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 3 Aug 2018 13:05:53 +0200 Subject: [PATCH 047/126] Update revision history --- history.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/history.adoc b/history.adoc index 4764fa70..f512e353 100644 --- a/history.adoc +++ b/history.adoc @@ -1,8 +1,8 @@ [[revhistory, Revision History]] == Revision History -.TBD -. Added section on using groups. +.3 August 2018 +. Added <> and <>. .14 June 2004 . Added <>. From 36b797c524528b158854a511000ff191fe20881f Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Sat, 11 Aug 2018 19:31:47 -0500 Subject: [PATCH 048/126] template updates --- .github/ISSUE_TEMPLATE/issue_template.md | 9 +++++++++ .github/PULL_REQUEST_TEMPLATE.md | 14 +------------- 2 files changed, 10 insertions(+), 13 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/issue_template.md diff --git a/.github/ISSUE_TEMPLATE/issue_template.md b/.github/ISSUE_TEMPLATE/issue_template.md new file mode 100644 index 00000000..eeef3e3e --- /dev/null +++ b/.github/ISSUE_TEMPLATE/issue_template.md @@ -0,0 +1,9 @@ +Change proposals should include the following information as applicable. + +**Title:** Short and descriptive. +**Moderator:** @user +**Requirement Summary:** A few sentence functional summary +**Technical Proposal Summary:** Brief proposal overview +**Benefits:** Who or what will benefit from this proposal? +**Status Quo:** Discussion of the current state CF and other standards. +**Detailed Proposal:** Complete proposal \ No newline at end of file diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 57114603..9117eddc 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,13 +1 @@ -_If trac was used for review of contribution:_ -See http://cf-trac.llnl.gov/trac/ticket/XXX - - - [ ] Added link from trac ticket to this PR? - - > Applied via ​https://github.com/cf-convention/cf-conventions/pull/XXX - - - [ ] Updated "Revision History"? (Use the date you applied the changes.) - -_If github issues were used for review:_ -See issue #XXX - - - [ ] Updated "Revision History"? (Use the date you applied the changes.) +See issue #XXX for discussion of these changes. \ No newline at end of file From 1fcd06a1de4879e2cda4c68b1028c8bb559d461a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Guilherme=20Castel=C3=A3o?= Date: Sun, 12 Aug 2018 15:41:02 -0700 Subject: [PATCH 049/126] Typos (plural dimensions) in section H There are few examples on section H with inconsistent dimensions alternating between plural and singular: - profiles x profile - station x stations --- apph.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/apph.adoc b/apph.adoc index b3536fe6..95cec5df 100644 --- a/apph.adoc +++ b/apph.adoc @@ -628,7 +628,7 @@ When the number of vertical levels for each profile varies, and one cannot write ---- dimensions:    obs = UNLIMITED ; -    profiles = 142 ; +    profile = 142 ; variables:    int profile(profile) ; @@ -1121,8 +1121,8 @@ When the number of profiles and levels for each station varies, one can use a ra ---- dimensions:    obs = UNLIMITED ; -    profiles = 1420 ; -    stations = 42; +    profile = 1420 ; +    station = 42; variables:    float lon(station) ; @@ -1341,7 +1341,7 @@ When the number of profiles and levels for each trajectory varies, one can use a ---- dimensions:    obs = UNLIMITED ; -    profiles = 142 ; +    profile = 142 ; variables:    int trajectory(trajectory) ; From 23c0db289c3422b2fbf5a618f987f836577b490d Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Wed, 15 Aug 2018 13:23:30 -0500 Subject: [PATCH 050/126] minor contributing update --- .github/CONTRIBUTING.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 76f5f63f..ae7ef606 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -30,6 +30,9 @@ Contributions must be presented as enhancements, defects, or typos and labeled a ## Issues and Pull Requests +Issues should attempt to follow the guidelines here and in the issue template as much as possible. +All new pull requests should be submitted to the master branch and will be merged or closed as soon as agreement has been reached. +Use of other branches is at the discretion of the repository administrators. The following cases describe potential patterns of use for issues and pull requests. 1. **Typo Fix** If the change is a non-controversial fix such as a typo, no issue is required as these changes do not appear in the convention history. From 241c20721824a5fc7c5cd6829fbb64104a33cce1 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Sat, 25 Aug 2018 10:58:19 -0500 Subject: [PATCH 051/126] typos --- .github/CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index ae7ef606..8e6e5714 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -14,7 +14,7 @@ The CF community takes great pride in respectful and collegial discourse. Any di ## General Guidelines 1. **A given proposal should be discussed as one issue.** It shouldn't fork or be superseded by another one, unless that reflects what has happened to the proposal. -This is so it is easy to trace the discussion tha led to a given agreed proposal. +This is so it is easy to trace the discussion that led to a given agreed proposal. 2. **A proposal should convey the reasoning and effect on all relevant sections of the specification.** An overview of all actual changes and the impact the changes have on the specification should be clear. @@ -50,4 +50,4 @@ Final review should take place in the pull request and the issue closed when the If explicitly listing proposed changes is not practical, general guideline 2 should be followed to document the proposal. Depending on the nature of the proposal, interested community members can decide what the most effective tool is for development and review of specification changes. Tools used for development of significant changes are up to those contributing and reviewing it. -Note that their is a rendered "rich-diff" view of a pull request that can be helpful for review of large contributions. +Note that there is a rendered "rich-diff" view of a pull request that can be helpful for review of large contributions. From 60655c89686698ba56311a040a7c3e1460062f49 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Fri, 7 Sep 2018 15:13:51 -0700 Subject: [PATCH 052/126] lazy commit --- appi.adoc | 25 +++++++++---------------- 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/appi.adoc b/appi.adoc index e49f7df8..8af68280 100644 --- a/appi.adoc +++ b/appi.adoc @@ -1,15 +1,15 @@ -[[appendix-examples-groups, Appendix I, Good Practices and Annotated Examples for Using Groups]] +[[appendix-examples-groups, Appendix I, Recommended Practices and Annotated Examples for Using Groups]] [appendix] -== Good Practices and Annotated Examples for Using Groups +== Recommended Practices and Annotated Examples for Using Groups -This appendix contains good practices for using netCDF-4 groups in CF compliant data, as well as annotated examples of existing use cases for representing data using groups. +This appendix contains recommended practices for using netCDF-4 groups in CF compliant data, as well as annotated examples of existing use cases for representing data using groups. -=== Good practices +=== Recommended practices . The use of attributes attached to a group in order to store metadata normally attached directly to variables is discouraged. This includes, for example, replacing per-variable attributes like `_FillValue`, `scale_factor`, `valid_min`, with group-level equivalents. -Although group attributes might be more concise, it is likely to create problems with downstream software and reduce interoperability. +Although group attributes might be more concise, it is likely to create problems with downstream software and to reduce interoperability. . Each group's attributes, dimensions and variables should be self-contained in combination with the group metadata and any inherited properties (e.g., dimension sizes, coordinates). This ensures that if a group is renamed or extracted (with any inherited properties) into a new file, its information content is preserved. @@ -19,7 +19,8 @@ Other commonly enumerated group names, such as station identifiers, buoy numbers It is fine to enumerate or itemize names so long as the number or name is redundantly stored as a group attribute. . Renaming or moving a group or self-contained branch of groups to a new location should not affect the interpretation of data. -Since relative and absolute paths (containing "/") of coordinates, dimensions, and attributes are fragile, it is preferred they not be present in attributes. +Relative and absolute paths (containing "/") of coordinates, dimensions, and attributes are fragile in that they may break if groups are moved or renamed. +To prevent this, it is preferred that paths not be present in attribute values. Instead, named objects resolve to the most proximal object (i.e., dimension or variable) of that name that has the referring attribute within its scope. This makes files which use groups amenable to sub-setting and aggregation. For example, the CF `coordinates` attribute identifies a variable's coordinates in a whitespace-separated list such as `"lat lon"`. @@ -30,7 +31,7 @@ The simpler `"lat lon"` specification works without alteration in all situations . When using a grid mapping which is referenced in multiple groups, the reference to the coordinates themselves should be maintained as an attribute of the referencing variables, and not as an attribute of the grid mapping itself. This way the coordinates can resolve to the correct coordinates which are "owned" by the referencing variable, and which are not shared amongst variables, while the grid mapping which is used to interpret the coordinates is stored in only one place. Both coordinates and grid mapping are thus found via regular scoping mechanisms. + -A common use case for this is data produced on board a geostationary platform. Different swaths are stored in different groups, each with their own coordinates, but all of which share a common grid mapping for the entire full disk view. +A common use case for this is data produced onboard a geostationary platform. Different swaths are stored in different groups, each with their own coordinates, but all of which share a common grid mapping for the entire full disk view. === Use cases @@ -381,7 +382,7 @@ This can greatly reduce the volume of data which must be transferred between pro === Mapping between Hierarchical and Flat files -Files which use groups can be mapped to a set of flat files. +Files that use groups can be mapped to a set of flat files. This procedure involves separating the group hierarchy tree into multiple distinct, self-contained, flat files, and is called _dismembering_. A related procedure, _flattening_, collapses an entire hierarchical file into a single flat file. @@ -394,11 +395,3 @@ When name identifiers are re-used, a flattening algorithm must disambiguate the To guarantee resolution of such namespace conflicts, the flattening procedure must rename conflicting variables, dimensions, and group attributes. For example, variables that share a name in separate groups in a hierarchical file (e.g., `/g1/v1` and `/g2/v1`) can be renamed by concatenating their names with their original group paths, with forward-slash path separators eliminated or replaced by a special character string in the flattened version (e.g., `g1_v1` and `g2_v1`). A similar procedure must be followed to resolve namespace conflicts for group metadata and for dimension names. - -The link:$$https://www.opendap.org/software/hyrax-data-server$$[OPeNDAP Hyrax Data Server] implements such an algorithm when flattening hierarchical files. -Hyrax goes further in that it renames all variables beneath the root group by prepending the former full path name (with slashes represented as spaces) to the original short name. -Hyrax preserves the original name and group path of the variables in new attributes named `origname` and `fullnamepath`. -Their preservation ensures that a suitably programmed "inflation" tool could reverse the flattening and re-construct a hierarchical file with all the original names. -To our knowledge, no such inflator is yet available. -Software to flatten hierarchical files without namespace conflicts is available (e.g., link:$$http://nco.sf.net/nco.html#flatten$$[NCO]). -It is anticipated that tools (such as flatteners and inflators) that facilitate interoperability of files using groups will become more mature in the future. From 0809906da7972d6483fe1a838fa8d53d2720b538 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Fri, 21 Sep 2018 11:45:01 -0700 Subject: [PATCH 053/126] lazy commit --- ch01.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch01.adoc b/ch01.adoc index fff35464..de7c4db3 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -48,7 +48,7 @@ grid mapping variable:: A variable used as a container for attributes that defin latitude dimension:: A dimension of a netCDF variable that has an associated latitude coordinate variable. -local apex group:: the nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. +local apex group:: The nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. The word "apex" refers to position of this group at the upper vertex of the triangle of groups formed by it, the referring group, and the group where a coordinate is located. longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. From 3ceadfcadecf71267d39422af683e507572c86c2 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Fri, 21 Sep 2018 17:06:17 -0700 Subject: [PATCH 054/126] Add a bunch of definitions. Replace object/element with variable and/or dimension as appropriate. --- ch01.adoc | 14 +++++++++++--- ch02.adoc | 11 +++++------ 2 files changed, 16 insertions(+), 9 deletions(-) diff --git a/ch01.adoc b/ch01.adoc index de7c4db3..99dee361 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -23,6 +23,8 @@ We have also striven to maximize conformance to the COARDS standard, that is, wh The terms in this document that refer to components of a netCDF file are defined in the NetCDF User's Guide (NUG) <> NUG. Some of those definitions are repeated below for convenience. +ancestor group:: A group from which the referring group is descended via direct parent-child relationships + auxiliary coordinate variable:: Any netCDF variable that contains coordinate data, but is not a coordinate variable (in the sense of that term defined by the NUG and used by this standard - see below). Unlike coordinate variables, there is no relationship between the name of an auxiliary coordinate variable and the name(s) of its dimension(s). boundary variable:: A boundary variable is associated with a variable that contains coordinate data. When a data value provides information about conditions in a cell occupying a region of space/time or some other dimension, the boundary variable provides a description of cell extent. @@ -44,11 +46,13 @@ coordinate variable:: We use this term precisely as it is defined in the link:$$http://www.unidata.ucar.edu/software/netcdf/docs/netcdf_data_set_components.html#coordinate_variables$$[NUG section on coordinate variables]. It is a one-dimensional variable with the same name as its dimension [e.g., **`time(time)`** ], and it is defined as a numeric data type with values that are ordered monotonically. Missing values are not allowed in coordinate variables. +element:: A group, dimension, variable, or attribute. + grid mapping variable:: A variable used as a container for attributes that define a specific grid mapping. The type of the variable is arbitrary since it contains no data. latitude dimension:: A dimension of a netCDF variable that has an associated latitude coordinate variable. -local apex group:: The nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. The word "apex" refers to position of this group at the upper vertex of the triangle of groups formed by it, the referring group, and the group where a coordinate is located. +local apex group:: The nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. The word "apex" refers to position of this group at the vertex of the tree of groups formed by it, the referring group, and the group where a coordinate is located. longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. @@ -56,17 +60,21 @@ most proximal identifier:: The identifier of the element that can be reached via multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. -out-of-group reference:: a reference to an element that is not contained in the referring group. +nearest dimension `dim_name`:: The dimension named `dim_name` in whose scope the referring group lies. The scoping rules of netCDF dictate that the nearest dimension is always in the referring group or an ancestor of it. In cases where all multiple ancestor groups define dimensions name `dim_name`, the nearest dimension `dim_name` is the closest such ancestor to the referring group. + +out-of-group reference:: A reference to an element that is not contained in the referring group. recommendation:: Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. -referring group:: the group in which a reference to an identifier occurs. +referring group:: The group in which a reference to an identifier occurs. scalar coordinate variable:: A scalar variable (i.e. one with no dimensions) that contains coordinate data. Depending on context, it may be functionally equivalent either to a size-one coordinate variable (<>) or to a size-one auxiliary coordinate variable (<> and <>). +sibling group:: Any group with the same parent group as the referring group + spatiotemporal dimension:: A dimension of a netCDF variable that is used to identify a location in time and/or space. time dimension:: A dimension of a netCDF variable that has an associated time coordinate variable. diff --git a/ch02.adoc b/ch02.adoc index b82a5198..c8fcea92 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -170,12 +170,11 @@ Groups provide a powerful mechanism to structure data hierarchically. This convention does not standardize group names. It may be of benefit to name groups in such a way that human readers can interpret them. However, files that conform to this standard shall not require software to interpret or decode information from group names. -Instead, references to out-of-group elements shall be found by applying the scoping rules outlined below. +Instead, references to out-of-group variable and dimensions shall be found by applying the scoping rules outlined below. ==== Scope -Objects can be referred to as long as they are visible within the scope from which they are used. -Any object can be referred to, as long as it can be found with one of the following search strategies: +Any variable or dimension can be referred to, as long as it can be found with one of the following search strategies: * Search by absolute path * Search by relative path @@ -187,9 +186,9 @@ These strategies are explained in detail in the following sections. An identifier specified with an absolute path (i.e., with a leading slash "/") resolves to the indicated location. For example, a `coordinates` attribute of `/g1/lat` refers to the `lat` variable in group `/g1`. -The dataset producer must ensure that dimension(s) utilized by out-of-group objects are the same as those used by the referring object. -For example, the nearest dimension `lat` to the current (referring) group must be the same as the dimension `lat` nearest in scope to the out-of-group coordinates. -Thus a variable `/g2/temperature` with coordinates attribute containing `/g1/lat` is permissible so long as there is no other `lat` dimension "in between" those locations, e.g., `/lat`. +The dataset producer must ensure that dimension(s) utilized by out-of-group variables are the same as those used by the referring variable. +For example, the dimension `lat` underlying variables in the current (referring) group must be the same as the dimension `lat` nearest in scope to the out-of-group coordinates for those variables. +Thus a variable `/g2/temperature` with `coordinates` attribute containing `/g1/lat` is permissible so long as there is no other `lat` dimension defined "in between" those locations, e.g., `/lat`. ===== Search by relative path From 751de504e08b6ec9af4f548312f5167114cb7cc5 Mon Sep 17 00:00:00 2001 From: Charlie Zender Date: Wed, 26 Sep 2018 16:45:33 -0700 Subject: [PATCH 055/126] Finish definitions and text responses to #144. Will post tomorrow after sleeping on it. --- ch01.adoc | 10 ++++++---- ch02.adoc | 16 ++++++++-------- 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/ch01.adoc b/ch01.adoc index 99dee361..b01b232f 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -54,19 +54,21 @@ latitude dimension:: A dimension of a netCDF variable that has an associated lat local apex group:: The nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. The word "apex" refers to position of this group at the vertex of the tree of groups formed by it, the referring group, and the group where a coordinate is located. -longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. +location:: The position or path in a group hierarchy of a variable or dimension. -most proximal identifier:: The identifier of the element that can be reached via the shortest traversal of the file from the referring element following the rules set forth in the <>. +longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. +nearest coordinate or variable:: The variable that can be reached via the shortest traversal of the file from the referring group following the rules set forth in the <>. + nearest dimension `dim_name`:: The dimension named `dim_name` in whose scope the referring group lies. The scoping rules of netCDF dictate that the nearest dimension is always in the referring group or an ancestor of it. In cases where all multiple ancestor groups define dimensions name `dim_name`, the nearest dimension `dim_name` is the closest such ancestor to the referring group. -out-of-group reference:: A reference to an element that is not contained in the referring group. +out-of-group reference:: A reference to a variable or dimension that is not contained in the referring group. recommendation:: Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. -referring group:: The group in which a reference to an identifier occurs. +referring group:: The group in which a reference to a variable or dimension occurs. scalar coordinate variable:: A scalar variable (i.e. one with no dimensions) that contains coordinate data. Depending on context, it may be functionally equivalent either to a size-one diff --git a/ch02.adoc b/ch02.adoc index c8fcea92..61cadae8 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -184,7 +184,7 @@ These strategies are explained in detail in the following sections. ===== Search by absolute path -An identifier specified with an absolute path (i.e., with a leading slash "/") resolves to the indicated location. +An variable or dimension specified with an absolute path (i.e., with a leading slash "/") is at the indicated location. For example, a `coordinates` attribute of `/g1/lat` refers to the `lat` variable in group `/g1`. The dataset producer must ensure that dimension(s) utilized by out-of-group variables are the same as those used by the referring variable. For example, the dimension `lat` underlying variables in the current (referring) group must be the same as the dimension `lat` nearest in scope to the out-of-group coordinates for those variables. @@ -192,19 +192,19 @@ Thus a variable `/g2/temperature` with `coordinates` attribute containing `/g1/l ===== Search by relative path -An identifier specified with a relative path (i.e., with an internal but no leading slash "/") resolves to the indicated location. +An variable or dimension specified with a relative path (i.e., with an internal but no leading slash "/") is at the location obtained by affixing the relative path to the absolute path of the referring attribute. For example, a `coordinates` attribute of `g1/lat` refers to the `lat` variable in subgroup `g1` of the current (referring) group. Upward path traversals from the current group are indicated with the UNIX convention. For example, `../g1/lat` refers to the `lat` variable in the sibling group `g1` of the current (referring) group. ===== Search by proximity -An identifier specified with no path resolves to the most proximal identifier of the same name. -In all cases, if the named object is present in the referring group, that object is the most proximal. +A variable or dimension specified with no path is the nearest variable or dimension of the same name. +In all cases, if the named variable or dimension is present in the referring group, that object is the nearest. For example, a `coordinates` attribute of `lat` refers to the `lat` variable (if any) in the present group. -If the object is not in the referring group then the identifier is termed an out-of-group reference. +If the variable or dimension is not in the referring group then it is termed an out-of-group reference. -Out-of-group references are resolved by searching all direct ancestors until the specified identifier is found. +Out-of-group references are resolved by searching all direct ancestors until the specified variable or dimension is found. [NOTE] ==== @@ -214,8 +214,8 @@ This placement of coordinates and lateral search strategy to find them are disco A special case exists for coordinate variables. Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the local apex group is reached. -If the identifier in question cannot be found within the local apex group or any of the intermediate groups between it and the referring group, a lateral (width-wise) search is initiated from the local apex group. -The lateral search proceeds downwards from the local apex group through each level of groups until the sought identifier is found. +If the coordinate in question cannot be found within the local apex group or any of the intermediate groups between it and the referring group, a lateral (width-wise) search is initiated from the local apex group. +The lateral search proceeds downwards from the local apex group through each level of groups until the sought coordinate is found. ==== Application of attributes From 8ae9b6b6665a0f95200887b28d8e77cce09f6b27 Mon Sep 17 00:00:00 2001 From: David Hassell Date: Mon, 8 Oct 2018 21:43:03 +0100 Subject: [PATCH 056/126] Update README.md Test - ignore --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b54b0ba6..2681049c 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ process can be seen at: http://cfconventions.org/cf-conventions/cf-conventions.h For the official web site please visit: http://cfconventions.org/, and the corresponding GitHub organisation: https://github.com/cf-convention. -### Building the HTML +### Building the HTML To convert the AsciiDoc files into the resulting HTML file: From d2dbf92de9cf1d5b5bf604b26a7bfeb8190bb1f8 Mon Sep 17 00:00:00 2001 From: David Hassell Date: Mon, 8 Oct 2018 21:43:24 +0100 Subject: [PATCH 057/126] Update README.md Test 2 (ignore) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2681049c..b54b0ba6 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ process can be seen at: http://cfconventions.org/cf-conventions/cf-conventions.h For the official web site please visit: http://cfconventions.org/, and the corresponding GitHub organisation: https://github.com/cf-convention. -### Building the HTML +### Building the HTML To convert the AsciiDoc files into the resulting HTML file: From b436ba63977b6f78cdd6cbe581c9f6f38949add6 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 10 Oct 2018 15:42:20 +0200 Subject: [PATCH 058/126] Remove "element" from glossary --- ch01.adoc | 2 -- 1 file changed, 2 deletions(-) diff --git a/ch01.adoc b/ch01.adoc index b01b232f..6d05ae58 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -46,8 +46,6 @@ coordinate variable:: We use this term precisely as it is defined in the link:$$http://www.unidata.ucar.edu/software/netcdf/docs/netcdf_data_set_components.html#coordinate_variables$$[NUG section on coordinate variables]. It is a one-dimensional variable with the same name as its dimension [e.g., **`time(time)`** ], and it is defined as a numeric data type with values that are ordered monotonically. Missing values are not allowed in coordinate variables. -element:: A group, dimension, variable, or attribute. - grid mapping variable:: A variable used as a container for attributes that define a specific grid mapping. The type of the variable is arbitrary since it contains no data. latitude dimension:: A dimension of a netCDF variable that has an associated latitude coordinate variable. From f683db943c46b444188aee95b188911284340d66 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 10 Oct 2018 15:45:45 +0200 Subject: [PATCH 059/126] Replace "local apex" with "root" in lateral search --- ch02.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ch02.adoc b/ch02.adoc index 61cadae8..6435527a 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -213,9 +213,9 @@ This placement of coordinates and lateral search strategy to find them are disco ==== A special case exists for coordinate variables. -Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the local apex group is reached. -If the coordinate in question cannot be found within the local apex group or any of the intermediate groups between it and the referring group, a lateral (width-wise) search is initiated from the local apex group. -The lateral search proceeds downwards from the local apex group through each level of groups until the sought coordinate is found. +Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the root group is reached. +If the coordinate in question cannot be found within the root group or any of the intermediate groups between it and the referring group, a lateral (width-wise) search is initiated from the root group. +The lateral search proceeds downwards from the root group through each level of groups until the sought coordinate is found. ==== Application of attributes From 73a31520c5875049d153bbc5619bf93d5f230226 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 10 Oct 2018 16:20:59 +0200 Subject: [PATCH 060/126] Clarify dimensions in out-of-group variables This also involved moving the text to another section and deleting a confusing example. --- ch02.adoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/ch02.adoc b/ch02.adoc index 6435527a..d6746474 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -182,13 +182,12 @@ Any variable or dimension can be referred to, as long as it can be found with on These strategies are explained in detail in the following sections. +When referencing out-of-group variables, it is the responsibility of the dataset producer to ensure that the dimension(s) utilized by the out-of-group variable are the same as those used by the referring variable. + ===== Search by absolute path An variable or dimension specified with an absolute path (i.e., with a leading slash "/") is at the indicated location. For example, a `coordinates` attribute of `/g1/lat` refers to the `lat` variable in group `/g1`. -The dataset producer must ensure that dimension(s) utilized by out-of-group variables are the same as those used by the referring variable. -For example, the dimension `lat` underlying variables in the current (referring) group must be the same as the dimension `lat` nearest in scope to the out-of-group coordinates for those variables. -Thus a variable `/g2/temperature` with `coordinates` attribute containing `/g1/lat` is permissible so long as there is no other `lat` dimension defined "in between" those locations, e.g., `/lat`. ===== Search by relative path From cd8931dff390d3b1c7a868c40e7e15f88901ebe2 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 10 Oct 2018 16:31:32 +0200 Subject: [PATCH 061/126] Clarify search by proximity algorithm --- ch02.adoc | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/ch02.adoc b/ch02.adoc index d6746474..00f37fc5 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -198,12 +198,10 @@ For example, `../g1/lat` refers to the `lat` variable in the sibling group `g1` ===== Search by proximity -A variable or dimension specified with no path is the nearest variable or dimension of the same name. -In all cases, if the named variable or dimension is present in the referring group, that object is the nearest. +A variable or dimension specified with no path is the variable or dimension of the same name which can be found via a search of direct ancestor groups with the shortest number of intermediate groups. For example, a `coordinates` attribute of `lat` refers to the `lat` variable (if any) in the present group. If the variable or dimension is not in the referring group then it is termed an out-of-group reference. - -Out-of-group references are resolved by searching all direct ancestors until the specified variable or dimension is found. +Out-of-group references are resolved by searching all direct ancestors, starting from the direct ancestor and proceeding toward the root group, until the specified variable or dimension is found. [NOTE] ==== From 43e8896c135e3c6acb11cc81cc8d5fce593efc85 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 5 Nov 2018 09:48:45 -0800 Subject: [PATCH 062/126] move CONTIRBUTING out of .github dir --- .github/CONTRIBUTING.md => CONTRIBUTING.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename .github/CONTRIBUTING.md => CONTRIBUTING.md (100%) diff --git a/.github/CONTRIBUTING.md b/CONTRIBUTING.md similarity index 100% rename from .github/CONTRIBUTING.md rename to CONTRIBUTING.md From 8643935686a82b3668ccf1cd7f79b283c510276f Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 9 Nov 2018 06:28:40 -0600 Subject: [PATCH 063/126] Update issue templates --- .../ISSUE_TEMPLATE/cf-proposed-change-request.md | 16 ++++++++++++++++ .../ISSUE_TEMPLATE/proposed-change-request.md | 16 ++++++++++++++++ 2 files changed, 32 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/cf-proposed-change-request.md create mode 100644 .github/ISSUE_TEMPLATE/proposed-change-request.md diff --git a/.github/ISSUE_TEMPLATE/cf-proposed-change-request.md b/.github/ISSUE_TEMPLATE/cf-proposed-change-request.md new file mode 100644 index 00000000..265f98f8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/cf-proposed-change-request.md @@ -0,0 +1,16 @@ +--- +name: CF Proposed Change Request +about: This template should be used as applicable for any proposed change to the CF + convention. + +--- + +Change proposals should include the following information as applicable. + +**Title:** Short and descriptive. +**Moderator:** @user +**Requirement Summary:** A few sentence functional summary +**Technical Proposal Summary:** Brief proposal overview +**Benefits:** Who or what will benefit from this proposal? +**Status Quo:** Discussion of the current state CF and other standards. +**Detailed Proposal:** Complete proposal diff --git a/.github/ISSUE_TEMPLATE/proposed-change-request.md b/.github/ISSUE_TEMPLATE/proposed-change-request.md new file mode 100644 index 00000000..b6e27ef7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/proposed-change-request.md @@ -0,0 +1,16 @@ +--- +name: Proposed Change Request +about: This template should be used as applicable for any proposed change to the CF + convention. + +--- + +Change proposals should include the following information as applicable. + +**Title:** Short and descriptive. +**Moderator:** @user +**Requirement Summary:** A few sentence functional summary +**Technical Proposal Summary:** Brief proposal overview +**Benefits:** Who or what will benefit from this proposal? +**Status Quo:** Discussion of the current state CF and other standards. +**Detailed Proposal:** Complete proposal From 17802853ef54077f3c9286bfa9aa63e619ede49e Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 9 Nov 2018 06:29:30 -0600 Subject: [PATCH 064/126] delete extra template --- .../ISSUE_TEMPLATE/cf-proposed-change-request.md | 16 ---------------- 1 file changed, 16 deletions(-) delete mode 100644 .github/ISSUE_TEMPLATE/cf-proposed-change-request.md diff --git a/.github/ISSUE_TEMPLATE/cf-proposed-change-request.md b/.github/ISSUE_TEMPLATE/cf-proposed-change-request.md deleted file mode 100644 index 265f98f8..00000000 --- a/.github/ISSUE_TEMPLATE/cf-proposed-change-request.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -name: CF Proposed Change Request -about: This template should be used as applicable for any proposed change to the CF - convention. - ---- - -Change proposals should include the following information as applicable. - -**Title:** Short and descriptive. -**Moderator:** @user -**Requirement Summary:** A few sentence functional summary -**Technical Proposal Summary:** Brief proposal overview -**Benefits:** Who or what will benefit from this proposal? -**Status Quo:** Discussion of the current state CF and other standards. -**Detailed Proposal:** Complete proposal From 6e2b01e8ab78a3691febeb3713f0276ba17cad6a Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 9 Nov 2018 06:29:43 -0600 Subject: [PATCH 065/126] delete extra template --- .github/ISSUE_TEMPLATE/issue_template.md | 9 --------- 1 file changed, 9 deletions(-) delete mode 100644 .github/ISSUE_TEMPLATE/issue_template.md diff --git a/.github/ISSUE_TEMPLATE/issue_template.md b/.github/ISSUE_TEMPLATE/issue_template.md deleted file mode 100644 index eeef3e3e..00000000 --- a/.github/ISSUE_TEMPLATE/issue_template.md +++ /dev/null @@ -1,9 +0,0 @@ -Change proposals should include the following information as applicable. - -**Title:** Short and descriptive. -**Moderator:** @user -**Requirement Summary:** A few sentence functional summary -**Technical Proposal Summary:** Brief proposal overview -**Benefits:** Who or what will benefit from this proposal? -**Status Quo:** Discussion of the current state CF and other standards. -**Detailed Proposal:** Complete proposal \ No newline at end of file From 56ef27381362cb486f4a94e8baa98b089b742c61 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 9 Nov 2018 06:33:15 -0600 Subject: [PATCH 066/126] add link to contributing guide and rules. --- .github/ISSUE_TEMPLATE/proposed-change-request.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/proposed-change-request.md b/.github/ISSUE_TEMPLATE/proposed-change-request.md index b6e27ef7..60cbb5eb 100644 --- a/.github/ISSUE_TEMPLATE/proposed-change-request.md +++ b/.github/ISSUE_TEMPLATE/proposed-change-request.md @@ -5,6 +5,8 @@ about: This template should be used as applicable for any proposed change to the --- +Before submitting an issue be sure you have read and understand the [github contributing guidelines](https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md) and the [rules for CF changes](http://cfconventions.org/rules.html) + Change proposals should include the following information as applicable. **Title:** Short and descriptive. From 3651d6792fb85f510b5529505fb0af1c604cbc3a Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 9 Nov 2018 06:35:24 -0600 Subject: [PATCH 067/126] better links --- .github/ISSUE_TEMPLATE/proposed-change-request.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/proposed-change-request.md b/.github/ISSUE_TEMPLATE/proposed-change-request.md index 60cbb5eb..511ff972 100644 --- a/.github/ISSUE_TEMPLATE/proposed-change-request.md +++ b/.github/ISSUE_TEMPLATE/proposed-change-request.md @@ -5,7 +5,7 @@ about: This template should be used as applicable for any proposed change to the --- -Before submitting an issue be sure you have read and understand the [github contributing guidelines](https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md) and the [rules for CF changes](http://cfconventions.org/rules.html) +Before submitting an issue be sure you have read and understand the github contributing guidelines: https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md and the rules for CF changes: http://cfconventions.org/rules.html Change proposals should include the following information as applicable. From 3291284d908e9917ed52c5806774ca71aa5d7a79 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 9 Nov 2018 06:39:52 -0600 Subject: [PATCH 068/126] markdown formatting --- .github/ISSUE_TEMPLATE/proposed-change-request.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/proposed-change-request.md b/.github/ISSUE_TEMPLATE/proposed-change-request.md index 511ff972..54e5639b 100644 --- a/.github/ISSUE_TEMPLATE/proposed-change-request.md +++ b/.github/ISSUE_TEMPLATE/proposed-change-request.md @@ -12,7 +12,7 @@ Change proposals should include the following information as applicable. **Title:** Short and descriptive. **Moderator:** @user **Requirement Summary:** A few sentence functional summary -**Technical Proposal Summary:** Brief proposal overview -**Benefits:** Who or what will benefit from this proposal? -**Status Quo:** Discussion of the current state CF and other standards. -**Detailed Proposal:** Complete proposal +**Technical Proposal Summary:** Brief proposal overview +**Benefits:** Who or what will benefit from this proposal? +**Status Quo:** Discussion of the current state CF and other standards. +**Detailed Proposal:** Complete proposal From 2c9e48d65b9ca3008e8ccd84405f96c77fb8a21b Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Thu, 3 Jan 2019 18:17:58 +0100 Subject: [PATCH 069/126] Clarifications as per discussing with Jonathan Gregory in #144 --- ch02.adoc | 51 +++++++++++++++++---------------------------------- 1 file changed, 17 insertions(+), 34 deletions(-) diff --git a/ch02.adoc b/ch02.adoc index 00f37fc5..20760255 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -186,60 +186,43 @@ When referencing out-of-group variables, it is the responsibility of the dataset ===== Search by absolute path -An variable or dimension specified with an absolute path (i.e., with a leading slash "/") is at the indicated location. +A variable or dimension specified with an absolute path (i.e., with a leading slash "/") is at the indicated location. For example, a `coordinates` attribute of `/g1/lat` refers to the `lat` variable in group `/g1`. ===== Search by relative path -An variable or dimension specified with a relative path (i.e., with an internal but no leading slash "/") is at the location obtained by affixing the relative path to the absolute path of the referring attribute. +An variable or dimension specified with a relative path (i.e., containing a slash but not with a leading slash, e.g. `child/lat`) is at the location obtained by affixing the relative path to the absolute path of the referring attribute. For example, a `coordinates` attribute of `g1/lat` refers to the `lat` variable in subgroup `g1` of the current (referring) group. Upward path traversals from the current group are indicated with the UNIX convention. For example, `../g1/lat` refers to the `lat` variable in the sibling group `g1` of the current (referring) group. ===== Search by proximity -A variable or dimension specified with no path is the variable or dimension of the same name which can be found via a search of direct ancestor groups with the shortest number of intermediate groups. -For example, a `coordinates` attribute of `lat` refers to the `lat` variable (if any) in the present group. +A variable or dimension specified with no path (for example, `lat`) refers to the variable or dimension of that name, if there is one, in the referring group. If the variable or dimension is not in the referring group then it is termed an out-of-group reference. Out-of-group references are resolved by searching all direct ancestors, starting from the direct ancestor and proceeding toward the root group, until the specified variable or dimension is found. +A special case exists for coordinate variables. +Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the local apex group is reached. +For coordinate variables that are not found in the referring group or its ancestors, a further strategy is provided, called lateral search. +The lateral search proceeds downwards from the local apex group width-wise through each level of groups until the sought coordinate is found. + [NOTE] ==== -The special lateral search strategy described below is for use with variables that reference coordinates that are outside of the referring group's direct ancestors. -This placement of coordinates and lateral search strategy to find them are discouraged. They are allowed mainly for backwards-compatibility with existing datasets, and may be deprecated in future versions of the standard. +This use of the lateral search strategy to find them is discouraged. +They are allowed mainly for backwards-compatibility with existing datasets, and may be deprecated in future versions of the standard. ==== -A special case exists for coordinate variables. -Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the root group is reached. -If the coordinate in question cannot be found within the root group or any of the intermediate groups between it and the referring group, a lateral (width-wise) search is initiated from the root group. -The lateral search proceeds downwards from the root group through each level of groups until the sought coordinate is found. - ==== Application of attributes -The attributes marked in the NUG conventions -(link:$$http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html$$[NUG Appendix A, Attribute Conventions]) as global attributes may only appear in the root group. These include +The following attributes may only be used in the root group and shall not be duplicated or overridden in child groups: * `title` * `history` -* `Conventions` - -These attributes are considered global attributes and should not be duplicated in child groups. -They should not be attached to any variable and should not appear in any child groups of the root group. -All other attributes defined in the NUG conventions are per-variable attributes. These include - -* `units` -* `long_name` -* `_FillValue` -* `missing_value` -* `valid_min` -* `valid_max` -* `valid_range` -* `scale_factor` -* `add_offset` -* `Coordinates` - -Per-variable attributes should be attached to variables and should not appear as group-level metadata. - -Other attributes may be present within groups without being attached to a variable. -Such attributes apply to the group where they are defined, and to that group's descendants, but not to ancestor or sibling groups. +* `conventions` + +Furthermore, per-variable attributes must be attached to the variables to which they refer. +They may not be attached to a group, even if all variables within that group use the same attribute and value. + +If attributes are present within groups without being attached to a variable, they attributes apply to the group where they are defined, and to that group's descendants, but not to ancestor or sibling groups. If a group attribute is defined in a parent group, and one of the child group redefines the same attribute, the definition within the child group applies for the child and all of its descendants. From 9ca5ba76c35b7af5ed05fdbe7fa4c3f3135d0ef8 Mon Sep 17 00:00:00 2001 From: David Hassell Date: Mon, 4 Feb 2019 17:28:32 +0100 Subject: [PATCH 070/126] Update ch02.adoc As per input from @davidhassel Co-Authored-By: erget --- ch02.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch02.adoc b/ch02.adoc index 20760255..15f6eebf 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -219,7 +219,7 @@ The following attributes may only be used in the root group and shall not be dup * `title` * `history` -* `conventions` +* `Conventions` Furthermore, per-variable attributes must be attached to the variables to which they refer. They may not be attached to a group, even if all variables within that group use the same attribute and value. From 9e5fd7205a70140b4c1c09234e32eac217efc7b4 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Mon, 4 Feb 2019 17:30:55 +0100 Subject: [PATCH 071/126] Update based on review by @davidhassel --- appi.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appi.adoc b/appi.adoc index 8af68280..0f038deb 100644 --- a/appi.adoc +++ b/appi.adoc @@ -16,7 +16,7 @@ This ensures that if a group is renamed or extracted (with any inherited propert For example, storage of ensembles as sibling groups (as described below) is often clearer when the realization number is encoded in the group name. In this case, the `Realization` group attribute retains the realization number even if the group is renamed. Other commonly enumerated group names, such as station identifiers, buoy numbers, or channel wavelengths are analogous. -It is fine to enumerate or itemize names so long as the number or name is redundantly stored as a group attribute. +It is fine to enumerate or itemize names so long as the number or name is also stored as a group attribute. . Renaming or moving a group or self-contained branch of groups to a new location should not affect the interpretation of data. Relative and absolute paths (containing "/") of coordinates, dimensions, and attributes are fragile in that they may break if groups are moved or renamed. From fcad2b7cddc264b2ccb7bf38876d0a3b4f5a5257 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 6 Feb 2019 19:01:08 +0100 Subject: [PATCH 072/126] Remove Appendix I at request by @JonathanGregory --- appi.adoc | 397 ------------------------------------------------------ 1 file changed, 397 deletions(-) delete mode 100644 appi.adoc diff --git a/appi.adoc b/appi.adoc deleted file mode 100644 index 0f038deb..00000000 --- a/appi.adoc +++ /dev/null @@ -1,397 +0,0 @@ -[[appendix-examples-groups, Appendix I, Recommended Practices and Annotated Examples for Using Groups]] - -[appendix] -== Recommended Practices and Annotated Examples for Using Groups - -This appendix contains recommended practices for using netCDF-4 groups in CF compliant data, as well as annotated examples of existing use cases for representing data using groups. - -=== Recommended practices - -. The use of attributes attached to a group in order to store metadata normally attached directly to variables is discouraged. -This includes, for example, replacing per-variable attributes like `_FillValue`, `scale_factor`, `valid_min`, with group-level equivalents. -Although group attributes might be more concise, it is likely to create problems with downstream software and to reduce interoperability. - -. Each group's attributes, dimensions and variables should be self-contained in combination with the group metadata and any inherited properties (e.g., dimension sizes, coordinates). -This ensures that if a group is renamed or extracted (with any inherited properties) into a new file, its information content is preserved. -For example, storage of ensembles as sibling groups (as described below) is often clearer when the realization number is encoded in the group name. -In this case, the `Realization` group attribute retains the realization number even if the group is renamed. -Other commonly enumerated group names, such as station identifiers, buoy numbers, or channel wavelengths are analogous. -It is fine to enumerate or itemize names so long as the number or name is also stored as a group attribute. - -. Renaming or moving a group or self-contained branch of groups to a new location should not affect the interpretation of data. -Relative and absolute paths (containing "/") of coordinates, dimensions, and attributes are fragile in that they may break if groups are moved or renamed. -To prevent this, it is preferred that paths not be present in attribute values. -Instead, named objects resolve to the most proximal object (i.e., dimension or variable) of that name that has the referring attribute within its scope. -This makes files which use groups amenable to sub-setting and aggregation. -For example, the CF `coordinates` attribute identifies a variable's coordinates in a whitespace-separated list such as `"lat lon"`. -When the coordinates are outside the group that contains the `coordinates` attribute, it is tempting to store the coordinate locations as full, unambiguous paths such as `"/g1/lat /g1/lon"`, or as relative paths such as `"g1/lat g1/lon"`. -However, paths that contain slashes must be explicitly altered when the variable is subset into a new file with a different group hierarchy, or when the hierarchy is flattened. -CF attributes affected by this practice include `ancillary_variables`, `bounds`, `cell_measures`, `climatology`, `coordinates`, `formula_terms`, and `grid_mapping`. -The simpler `"lat lon"` specification works without alteration in all situations for out-of-group locations so long as it is understood to mean the nearest identifiers that have the referring variable in their scope. - -. When using a grid mapping which is referenced in multiple groups, the reference to the coordinates themselves should be maintained as an attribute of the referencing variables, and not as an attribute of the grid mapping itself. This way the coordinates can resolve to the correct coordinates which are "owned" by the referencing variable, and which are not shared amongst variables, while the grid mapping which is used to interpret the coordinates is stored in only one place. Both coordinates and grid mapping are thus found via regular scoping mechanisms. -+ -A common use case for this is data produced onboard a geostationary platform. Different swaths are stored in different groups, each with their own coordinates, but all of which share a common grid mapping for the entire full disk view. - -=== Use cases - -==== Collections - -Group datasets are well-suited when users might benefit from storing related datasets (collections of variables) in a single location (file). -Loose collections might comprise different sets of distinct variables with a common purpose, e.g., multiple sensor observations at a single location. -For example, a model and satellite retrieval of a temperature field might be combined with an in situ temperature sensor as follows: - -[[example-i.1]] -[caption="Example I.1. "] -.Collection of related datasets sharing a single set of global metadata and segregated using groups. -==== ----- -netcdf clc { - :Conventions = "CF-1.8"; - :history = "Tue Apr 25 12:46:10 PDT 2017: ncgen -k netCDF-4 -b -o ~/nco/data/clc.nc ~/nco/data/clc.cdl"; - :Purpose = "Demonstrate a collection of related datasets stored in hierarchical format"; - - group: model { - :Source = "Model simulations, e.g., of temperature"; - dimensions: - lat=2; - lon=3; - time=unlimited; - variables: - float temperature(time,lat,lon); - double time(time); // Variable attributes omitted for clarity - double lat(lat); - double lon(lon); - data: - lat=-90,90.; - lon=0.,120.,240.; - temperature=273.,273.,273.,273.,273.,273.; - time=1.; - } // end model - - group: measurements_remote_sensing { - :Source = "Satellite measurements of same region as modelled, and on a different spatio-temporal grid"; - dimensions: - lat=3; - lon=4; - time=unlimited; - variables: - float temperature(time,lat,lon); - double time(time); // Variable attributes omitted for clarity - double lat(lat); - double lon(lon); - data: - lat=-90,0.,90.; - lon=0.,90.,180.,270.; - temperature=273.,273.,273.,273.,273.,273.,273.,273.,273.,273.,273.,273.; - time=1.; - } // end measurements_remote_sensing - - group: measurements_in_situ { - :Source = "In situ measurements, e.g., from an automated weather station with its own time-frequency"; - dimensions: - time=unlimited; - variables: - float temperature_10m(time); - double time(time); // Variable attributes omitted for clarity - data: - temperature_10m=271,272,273,274; - time=1.,2.,3.,4.; - } // end measurements_in_situ -} // end root group ----- -The namespace separation provided by groups allows variable and dimension names to be re-used and axes lengths to be re-defined. -In this example two groups contain a `temperature` variable, and the third contains a temperature at 10 m height. -Each group has its own spatio-temporal grid that re-uses the same coordinate names (`lat`, `lon`, `time`) as the other groups without conflict. -==== - -==== Ensembles - -Geoscientists use the label "ensemble" for collections of realizations of individual models or measurements of the same phenomena. -It is particularly important for models to repeat simulations of nonlinear systems multiple times (with slightly perturbed initial conditions) in order to characterize the statistical properties of systems with internal variability. -The namespace separation provided by groups ensures that variable names can be re-used. -Axis lengths can be re-defined if distinct realizations employ different spatio-temporal resolutions. -Multiple realizations of a single model temperature field might be stored as: - -[[example-i.2]] -[caption="Example I.2. "] -.Various members of an ensemble forecast stored as individual groups. -==== ----- -netcdf nsm { - :Conventions = "CF-1.8"; - :history = "Tue Apr 25 12:46:10 PDT 2017: ncgen -k netCDF-4 -b -o ~/nco/data/clc.nc ~/nco/data/clc.cdl"; - :Purpose = "Demonstrate a model ensemble stored in hierarchical format"; - - group: cesm_01 { - :Scenario = "Historical"; - :Model = "CESM"; - :Realization = "1"; - - dimensions: - time=unlimited; - variables: - float temperature(time); - double time(time); - data: - temperature=272.1,272.1,272.1,272.1; - time=1.,2.,3.,4.; - } // cesm_01 - - group: cesm_02 { - :Scenario = "Historical"; - :Model = "CESM"; - :Realization = "2"; - - dimensions: - time=unlimited; - variables: - float temperature(time); - double time(time); - data: - temperature=272.2,272.2,272.2,272.2; - time=1.,2.,3.,4.; - } // cesm_02 - - group: cesm_03 { - :Scenario = "Historical"; - :Model = "CESM"; - :Realization = "3"; - - dimensions: - time=unlimited; - variables: - float temperature(time); - double time(time); - data: - temperature=272.3,272.3,272.3,272.3; - time=1.,2.,3.,4.; - } // cesm_03 -} // root group ----- -Here each group contains a different realization of the same model, and the group names are suffixed with a numerical identifier, as well as containing a numerically valued group attribute named `Realization`. -This attribute would be carried with its group should the group ever be renamed or extracted into a new file, thus preserving the identity of the original realization. -It is allowed to include numeric metadata in group names so long as the information is redundantly stored as group metadata (e.g., `Realization`). -To accommodate the potential need of downstream software to deconstruct a number-containing group name into its original components it is suggested that the numeric portion be encoded as a fixed-width string separated by a non-alphanumeric character, such as "`_03`" above. -==== - -==== Discrete Sampling Geometries - -CF describes a powerful syntax for encoding spatiotemporal data from multiple locations into multidimensional flat-file formats. -The patterns of the spatiotemporal data are encapsulated into several features, each labeled with a distinct `featureType` that must be either `point`, `timeSeries`, `profile`, `trajectory`, `timeSeriesProfile`, or `trajectoryProfile`. -These features use an instance dimension to span a collection of like features. -One-dimensional variables that have only the instance dimension in a Discrete Geometry CF file are called instance variables. -Common instance variables include `lat(station)` and `station_name(station, name_len)`. -Here the `station` dimension enumerates the stations in the collection. - -It is also possible to use groups as an extended form of discrete sampling features where groups replace the instance dimension in Discrete Sampling Geometries. -Instead of a `station` dimension, feature collections may designate a group to contain the feature for each station. -Typically the group name would be the same as the `station_name`. -A `timeSeries` collection might appear like this: - -[[example-i.3]] -[caption="Example I.3. "] -.Orthogonal time series stored using groups. -==== ----- -netcdf tms { - :Conventions = "CF-1.8"; - :history = "Thu Jun 22 17:45:12 PDT 2017: ncgen -k netCDF-4 -b -o ~/nco/data/tms.nc ~/nco/data/tms.cdl"; - :Purpose = "Demonstrate a collection of DSG timeSeries featureType stored in hierarchical format"; - :featureType = "timeSeries"; - - dimensions: - time=unlimited; - - variables: - double time(time) ; - time:standard_name = "time"; - time:long_name = "time of measurement" ; - time:units = "days since 1970-01-01 00:00:00" ; - - group: irvine { - variables: - - float humidity(time) ; - humidity:standard_name = "specific humidity" ; - humidity:coordinates = "lat lon alt station_name" ; - humidity:_FillValue = -999.9f; - - float lon ; - lon:standard_name = "longitude"; - lon:long_name = "station longitude"; - lon:units = "degrees_east"; - - float lat ; - lat:standard_name = "latitude"; - lat:long_name = "station latitude" ; - lat:units = "degrees_north" ; - - float alt ; - alt:long_name = "vertical distance above the surface" ; - alt:standard_name = "height" ; - alt:units = "m"; - alt:positive = "up"; - alt:axis = "Z"; - - string station_name; - station_name:long_name = "station name" ; - station_name:cf_role = "timeseries_id"; - } // irvine - - group: boulder { - // Variables/dimensions repeated, omitted for clarity - } // boulder -} // root group ----- -Placement of the `time` dimension depends upon the characteristics of the sensor network, and is key to economically represent the collection. -If sensors at different locations measure values at the same time, then a single `time` coordinate may be placed in the root directory. -Each station (group) inherits this coordinate. -This is the case for an orthogonal multidimensional array representations. -==== - -When stations measure with distinct time coordinates amongst themselves, it is recommended that the `time` coordinates be stored locally within each group: - -[[example-i.4]] -[caption="Example I.4. "] -.Ragged array of station time series stored using groups. -==== ----- -netcdf tms { - - // Global metadata omitted for clarity - - group: irvine { - dimensions: - time=unlimited; - variables: - double time(time) ; - time:standard_name = "time"; - time:long_name = "time of measurement" ; - time:units = "days since 1970-01-01 00:00:00" ; - // Variables besides time as before, omitted for clarity - } // irvine - - group: boulder { - dimensions: - time=unlimited; - - variables: - double time(time) ; - time:standard_name = "time"; - time:long_name = "time of measurement" ; - time:units = "days since 1970-01-01 00:00:00" ; - // Variables besides time as before, omitted for clarity - } // boulder -} // root group ----- -This accommodates the common situation where different sensors have different observation times. -An alternative approach the use of an incomplete multidimensional array representation, which increases the rank and size of the `time` coordinate, so that each station must allocate space for all observation times used anywhere in the collection. -Groups can avoid this complexity by employing a station-specific `time` coordinate within each group. -This saves space relative to the incomplete multidimensional array representation since the representation using groups avoids padding the missing data. - -This formalism of station-specific `time` coordinates also naturally handles timeseries with time-varying deviations from a nominal point spatial location and offers an alternative to contiguous and indexed ragged arrays. -==== - -==== Remote sensing channels - -In satellite remote sensing, hierarchical datasets can be useful for storing low-level data, such as payload data, engineering data or instrument data for processing into geophysical variables. -While it is useful to store all sensed data from a single satellite or instrument in one unified file, many applications require only a subset of this data in order to produce higher-level products. -Additionally, some applications require data concerning the state of the vehicle or instrument, while others do not. -Therefore it is useful to split the observations from different channels and/or instruments into different groups within the netCDF file, as follows (for the sake of simplicity, a reduced, hypothetical file is shown): - -[[example-i.5]] -[caption="Example I.5. "] -.Rich remote sensing data represented hierarchically using groups. -==== ----- -netcdf nextgen-satellite { - // global attributes: - :title = "EUMETSAT EPS-SG IASI-NG Level 1c data" ; - :summary = "Demonstrate a Level 1 satellite product stored using groups"; - :Conventions = "CF-1.8"; - :orbit_start = 5 ; - :orbit_end = 6 ; - - group: status { - group: satellite { - dimensions: - manoeuvre_items = 0 ; - variables: - int manoeuvre_start_time_utc(manoeuvre_items); - int manoeuvre_end_time_utc(manoeuvre_items); - } // group satellite - } // group status - - group: data { - group: instrument_01 { - :instrument_identifier = "IASI-NG" ; - - dimensions: time = 1 ; - dimensions: nrows = 1 ; - dimensions: ncols = 1 ; - - variables: - float lat(nrows, ncols) ; - lat:units = "degrees_north" ; - lat:standard_name = "latitude" ; - float lon(nrows, ncols) ; - lon:units = "degrees_east" ; - lon:standard_name = "longitude" ; - double time(time) ; - time:standard_name = "time" ; - time:units = "seconds since 2000-01-01 00:00.00Z" ; - time:calendar = "gregorian" ; - - group: band_01 { - :sensor_band_identifier = "IASI-NG Channel 1" ; - - group: radiances { - dimensions: - n_wavenumbers = 1 ; - - variables: - int wavenumber(n_wavenumbers) ; - wavenumber:standard_name = "sensor_band_central_radiation_wavenumber" ; - double spectrum(nrows, ncols, n_wn) ; - spectrum:standard_name = "toa_outgoing_radiance_per_unit_wavenumber" ; - } // group radiances - - group: quality { - variables: - int number_of_missing_samples(nrows, ncols) ; - } // group quality - - } // group band_01 - - group: band_02{ - :sensor_band_identifier = "IASI-NG Channel 2" ; - } // group band_02 - } // group instrument_01 - - group: instrument_02 { - :instrument_identifier = "IASI-TLA" ; - } // group instrument_02 - } // group data ----- -A real example would be much more complex, but already this contrived example demonstrates the flexibility gained through the use of groups. -In this case, all observations from a given orbital dump are stored in a single file. -Subsets of this file can easily be produced, however, which contain observations only from certain instruments or certain bands of various instruments. -This can greatly reduce the volume of data which must be transferred between production facilities and thus increase timeliness for near-real-time products without sacrificing metadata integrity for archival purposes. -==== - -=== Mapping between Hierarchical and Flat files - -Files that use groups can be mapped to a set of flat files. -This procedure involves separating the group hierarchy tree into multiple distinct, self-contained, flat files, and is called _dismembering_. -A related procedure, _flattening_, collapses an entire hierarchical file into a single flat file. - -Files constructed in accord with the best practices outlined in this document can be dismembered without loss of information. -This allows dismembered files to be used with software aware of only flat files. -However, dismemberment often destroys the logical associations between data embodied in the original hierarchical file. - -Files using groups can be flattened without loss or alteration of information only in cases where none of the groups or their contents re-use name identifiers. -When name identifiers are re-used, a flattening algorithm must disambiguate the namespace conflicts in the flattened file, and this results in metadata alteration. -To guarantee resolution of such namespace conflicts, the flattening procedure must rename conflicting variables, dimensions, and group attributes. -For example, variables that share a name in separate groups in a hierarchical file (e.g., `/g1/v1` and `/g2/v1`) can be renamed by concatenating their names with their original group paths, with forward-slash path separators eliminated or replaced by a special character string in the flattened version (e.g., `g1_v1` and `g2_v1`). -A similar procedure must be followed to resolve namespace conflicts for group metadata and for dimension names. From 782a15cb704620b2e99abfa14169a5ab95898704 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 6 Feb 2019 19:06:38 +0100 Subject: [PATCH 073/126] Update authors list --- cf-conventions.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 840cd19c..b8c50287 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -1,5 +1,5 @@ = NetCDF Climate and Forecast (CF) Metadata Conventions -Brian Eaton; Jonathan Gregory; Bob Drach; Karl Taylor; Steve Hankin; Jon Blower; John Caron; Rich Signell; Phil Bentley; Greg Rappa; Heinke Höck; Alison Pamment; Martin Juckes; Martin Raspaud +Brian Eaton; Jonathan Gregory; Bob Drach; Karl Taylor; Steve Hankin; Jon Blower; John Caron; Rich Signell; Phil Bentley; Greg Rappa; Heinke Höck; Alison Pamment; Martin Juckes; Martin Raspaud; Charlie Zender; Daniel Lee Version 1.8 :sectanchors: :toc: macro From 8480c30666d0988447d1c177b3df74fb25fd85ef Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 6 Feb 2019 19:13:25 +0100 Subject: [PATCH 074/126] Update definitions for clarity --- ch01.adoc | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/ch01.adoc b/ch01.adoc index 6d05ae58..87a2bbd5 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -52,15 +52,11 @@ latitude dimension:: A dimension of a netCDF variable that has an associated lat local apex group:: The nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. The word "apex" refers to position of this group at the vertex of the tree of groups formed by it, the referring group, and the group where a coordinate is located. -location:: The position or path in a group hierarchy of a variable or dimension. - longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. -nearest coordinate or variable:: The variable that can be reached via the shortest traversal of the file from the referring group following the rules set forth in the <>. - -nearest dimension `dim_name`:: The dimension named `dim_name` in whose scope the referring group lies. The scoping rules of netCDF dictate that the nearest dimension is always in the referring group or an ancestor of it. In cases where all multiple ancestor groups define dimensions name `dim_name`, the nearest dimension `dim_name` is the closest such ancestor to the referring group. +nearest variable:: The variable that can be reached via the shortest traversal of the file from the referring group following the rules set forth in the <>. out-of-group reference:: A reference to a variable or dimension that is not contained in the referring group. From f376a3b7ad688595aa9492d6fd0a98cd30098a77 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 6 Feb 2019 19:25:10 +0100 Subject: [PATCH 075/126] Adopt suggestions from @JonathanGregory --- ch02.adoc | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/ch02.adoc b/ch02.adoc index 15f6eebf..b2bde205 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -170,7 +170,7 @@ Groups provide a powerful mechanism to structure data hierarchically. This convention does not standardize group names. It may be of benefit to name groups in such a way that human readers can interpret them. However, files that conform to this standard shall not require software to interpret or decode information from group names. -Instead, references to out-of-group variable and dimensions shall be found by applying the scoping rules outlined below. +References to out-of-group variable and dimensions shall be found by applying the scoping rules outlined below. ==== Scope @@ -182,16 +182,16 @@ Any variable or dimension can be referred to, as long as it can be found with on These strategies are explained in detail in the following sections. -When referencing out-of-group variables, it is the responsibility of the dataset producer to ensure that the dimension(s) utilized by the out-of-group variable are the same as those used by the referring variable. +If any dimension of an out-of-group variable has the same name as a dimension of the referring variable, the two must be the same dimension (i.e. they must have the same netCDF dimension ID). ===== Search by absolute path -A variable or dimension specified with an absolute path (i.e., with a leading slash "/") is at the indicated location. +A variable or dimension specified with an absolute path (i.e., with a leading slash "/") is at the indicated location relative to the root group, as in a UNIX-style file convention. For example, a `coordinates` attribute of `/g1/lat` refers to the `lat` variable in group `/g1`. ===== Search by relative path -An variable or dimension specified with a relative path (i.e., containing a slash but not with a leading slash, e.g. `child/lat`) is at the location obtained by affixing the relative path to the absolute path of the referring attribute. +As in a UNIX-style file convention, a variable or dimension specified with a relative path (i.e., containing a slash but not with a leading slash, e.g. `child/lat`) is at the location obtained by affixing the relative path to the absolute path of the referring attribute. For example, a `coordinates` attribute of `g1/lat` refers to the `lat` variable in subgroup `g1` of the current (referring) group. Upward path traversals from the current group are indicated with the UNIX convention. For example, `../g1/lat` refers to the `lat` variable in the sibling group `g1` of the current (referring) group. @@ -199,8 +199,7 @@ For example, `../g1/lat` refers to the `lat` variable in the sibling group `g1` ===== Search by proximity A variable or dimension specified with no path (for example, `lat`) refers to the variable or dimension of that name, if there is one, in the referring group. -If the variable or dimension is not in the referring group then it is termed an out-of-group reference. -Out-of-group references are resolved by searching all direct ancestors, starting from the direct ancestor and proceeding toward the root group, until the specified variable or dimension is found. +If not, the ancestors of the referring group are searched for it, starting from the direct ancestor and proceeding toward the root group, until it is found. A special case exists for coordinate variables. Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the local apex group is reached. From 5332e8cfc5258bb153217718c46541e51f567f71 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 6 Feb 2019 19:37:35 +0100 Subject: [PATCH 076/126] Update to respect scope --- appa.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appa.adoc b/appa.adoc index 073e36d3..aa620763 100644 --- a/appa.adoc +++ b/appa.adoc @@ -130,7 +130,7 @@ formula in the definition. | S | G | <> -| Specifies the type of discrete sampling geometry to which the data in the file belongs, and implies that all data variables in the file contain collections of features of that type. +| Specifies the type of discrete sampling geometry to which the data in the scope of this attribute belongs, and implies that all data variables in the scope of this attribute contain collections of features of that type. | **`flag_masks`** | D From 31a16e5c5783de59616e30ea16ff2f334cb3c5aa Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Thu, 7 Feb 2019 18:57:15 +0100 Subject: [PATCH 077/126] Remove "nearest variable" --- ch01.adoc | 2 -- 1 file changed, 2 deletions(-) diff --git a/ch01.adoc b/ch01.adoc index 87a2bbd5..2283e0f3 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -56,8 +56,6 @@ longitude dimension:: A dimension of a netCDF variable that has an associated lo multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. -nearest variable:: The variable that can be reached via the shortest traversal of the file from the referring group following the rules set forth in the <>. - out-of-group reference:: A reference to a variable or dimension that is not contained in the referring group. recommendation:: Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. From c3286fec69a9707003b3d66c4c72819ad0ff568f Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Thu, 7 Feb 2019 18:59:55 +0100 Subject: [PATCH 078/126] Forbid use of below root --- ch02.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/ch02.adoc b/ch02.adoc index b2bde205..b94c976f 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -219,6 +219,7 @@ The following attributes may only be used in the root group and shall not be dup * `title` * `history` * `Conventions` +* `external_variables` Furthermore, per-variable attributes must be attached to the variables to which they refer. They may not be attached to a group, even if all variables within that group use the same attribute and value. From 530d2a0c3b779d50f7b5864b179afc1a0e2dce4b Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 15 Feb 2019 09:22:07 +0100 Subject: [PATCH 079/126] Revert "Remove "nearest variable"" This reverts commit 31a16e5c5783de59616e30ea16ff2f334cb3c5aa. --- ch01.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ch01.adoc b/ch01.adoc index 2283e0f3..87a2bbd5 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -56,6 +56,8 @@ longitude dimension:: A dimension of a netCDF variable that has an associated lo multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. +nearest variable:: The variable that can be reached via the shortest traversal of the file from the referring group following the rules set forth in the <>. + out-of-group reference:: A reference to a variable or dimension that is not contained in the referring group. recommendation:: Recommendations in this convention are meant to provide advice that may be helpful for reducing common mistakes. In some cases we have recommended rather than required particular attributes in order to maintain backwards compatibility with COARDS. An application must not depend on a dataset's adherence to recommendations. From bb9ce2581a7bdd751ef54035209fa772b0efc924 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 15 Feb 2019 09:22:44 +0100 Subject: [PATCH 080/126] Update "nearest" definition to include groups This makes the lateral search algorithm clearer and encompasses finding variables or groups based on proximity, as required by the lateral search algorithm. --- ch01.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch01.adoc b/ch01.adoc index 87a2bbd5..b4cab9bc 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -56,7 +56,7 @@ longitude dimension:: A dimension of a netCDF variable that has an associated lo multidimensional coordinate variable:: An auxiliary coordinate variable that is multidimensional. -nearest variable:: The variable that can be reached via the shortest traversal of the file from the referring group following the rules set forth in the <>. +nearest item:: The item (variable or group) that can be reached via the shortest traversal of the file from the referring group following the rules set forth in the <>. out-of-group reference:: A reference to a variable or dimension that is not contained in the referring group. From 94ddcc386d502a98a1d76554a3d3560ed9cb7661 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 15 Feb 2019 10:04:39 +0100 Subject: [PATCH 081/126] Forbid lateral search for aux coordinates Lateral search for NUG coordinates is still allowed. We might consider extending this in the future, but further discussion would be needed. --- ch02.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/ch02.adoc b/ch02.adoc index b94c976f..8fdece65 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -205,6 +205,7 @@ A special case exists for coordinate variables. Because coordinate variables must share dimensions with the variables that reference them, the ancestor search is executed only until the local apex group is reached. For coordinate variables that are not found in the referring group or its ancestors, a further strategy is provided, called lateral search. The lateral search proceeds downwards from the local apex group width-wise through each level of groups until the sought coordinate is found. +The lateral search algorithm may only be used for NUG coordinate variables; it shall not be used for auxiliary coordinate variables. [NOTE] ==== From 03326d2bed6d82cd5d66c2f4828d7974d90d2f16 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 15 Feb 2019 10:36:27 +0100 Subject: [PATCH 082/126] Provide additional guidance on attributes --- ch02.adoc | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/ch02.adoc b/ch02.adoc index 8fdece65..3097a61e 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -215,15 +215,24 @@ They are allowed mainly for backwards-compatibility with existing datasets, and ==== Application of attributes -The following attributes may only be used in the root group and shall not be duplicated or overridden in child groups: +The following attributes are optional for non-root groups. +They are allowed in order to provide additional provenance and description of the subsidiary data. +They do not override attributes from parent groups. * `title` * `history` * `Conventions` + +If these attributes are present, they may be applied additively to the parent attributes of the same name. +If a file containing groups is modified, the user or application need only update these attributes in the root group, rather than traversing all groups and updating all attributes that are found with the same name. +In the case of conflicts, the root group attribute takes precedence over per-group instances of these attributes. + +The following attributes may only be used in the root group and shall not be duplicated or overridden in child groups: + * `external_variables` Furthermore, per-variable attributes must be attached to the variables to which they refer. They may not be attached to a group, even if all variables within that group use the same attribute and value. -If attributes are present within groups without being attached to a variable, they attributes apply to the group where they are defined, and to that group's descendants, but not to ancestor or sibling groups. +If attributes are present within groups without being attached to a variable, these attributes apply to the group where they are defined, and to that group's descendants, but not to ancestor or sibling groups. If a group attribute is defined in a parent group, and one of the child group redefines the same attribute, the definition within the child group applies for the child and all of its descendants. From ca41968d3b0f92b6242d6b8dcfa90d9e774dfb07 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 4 Mar 2019 14:40:11 -0600 Subject: [PATCH 083/126] Add geometry_dimension attribute to geometry container Fixes #155 --- ch07.adoc | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index ddfdaa0f..7db1a719 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -633,6 +633,7 @@ variables: datum:inverse_flattening = 298.257223563 ; int geometry_container ; geometry_container:geometry_type = "line" ; + geometry_container:geometry_dimension = "instance" ; geometry_container:node_count = "node_count" ; geometry_container:node_coordinates = "x y" ; int node_count(instance) ; @@ -668,7 +669,7 @@ The time series variable, someData, is associated with line geometries via the g A __geometry container__ variable acts as a container for attributes that describe a set of geometries. The **`geometry`** attribute of the data variable contains the name of a geometry container variable. -The geometry container variable must hold **`geometry_type`** and **`node_coordinates`** attributes. +The geometry container variable must hold **`geometry_type`**, **`geometry_dimension`**, and **`node_coordinates`** attributes. The **`grid_mapping`** and **`coordinates`** attributes can be carried by the geometry container variable provided they are also carried by the data variables associated with the container. The **`geometry_type`** attribute indicates the type of geometry present. @@ -676,6 +677,10 @@ Its allowable values are: __point__, __line__, __polygon__. Multipart geometries are allowed for all three geometry types. For example, polygon geometries could include single part geometries like the State of Colorado and multipart geometries like the State of Hawaii. +The **`geometry_dimension`** attribute stores the name of the instance dimension +for the geometries. +This dimension must be shared by the data variable related to the geometry container. + The **`node_coordinates`** attribute contains the blank-separated names of the variables that contain geometry node coordinates (one variable for each spatial dimension). The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables that have a meaningful correspondence with node coordinates are indicated as such by a **`nodes`** attribute that names the corresponding node coordinates. @@ -689,7 +694,10 @@ They are put in opposite orders to facilitate calculation of area and consistenc When more than one geometry instance is present, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. The node count is the total number of nodes in all the parts. -The exception is when all geometries are single part point geometries, in which case a node count is not needed since each geometry contains a single node. +The exception is when all geometries are single part point geometries, in which +case a node count is not needed since each geometry contains a single node. +The variable indicated by the **`node_count`** attribute must use the same +dimension as specified by the **`geometry_dimension`** attribute. For multipart __lines__, multipart __polygons__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that indicates a variable of the count of nodes per geometry part. Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required for __point__ geometry types. @@ -745,6 +753,7 @@ variables: lon:nodes = "x" ; float geometry_container ; geometry_container:geometry_type = "polygon" ; + geometry_container:geometry_dimension = "instance" ; geometry_container:node_count = "node_count" ; geometry_container:node_coordinates = "x y" ; geometry_container:grid_mapping = "datum" ; From bfad9fde08a00ea6fec78a3ed4ec9a4adddcbf8a Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 4 Mar 2019 14:52:38 -0600 Subject: [PATCH 084/126] Geometry container may have alternate grid_mapping from data variable --- ch07.adoc | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 7db1a719..7d288d03 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -683,8 +683,20 @@ This dimension must be shared by the data variable related to the geometry conta The **`node_coordinates`** attribute contains the blank-separated names of the variables that contain geometry node coordinates (one variable for each spatial dimension). The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. -If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables that have a meaningful correspondence with node coordinates are indicated as such by a **`nodes`** attribute that names the corresponding node coordinates. -Whether linked to normal CF space-time coordinates with a **`nodes`** attribute or not, inclusion of such coordinates is recommended to maintain backward compatibility with software that has not implemented geometry capabilities. + +If a **`coordinates`** attribute is carried by the geometry container variable +or its parent data variable, then those coordinate variables that have a +meaningful correspondence with node coordinates are indicated as such by a +**`nodes`** attribute that names the corresponding node coordinates, but only if +the **`grid_mapping`** attribute on the geometry container variable names the +same variable as the **`grid_mapping`** attribute on the associated data +variable. If a different grid mapping variable is named, then the provided +coordinates must not have the **`nodes`** attribute. + +Whether linked to normal CF space-time coordinates with a **`nodes`** attribute +or not, inclusion of such coordinates is recommended to maintain backward +compatibility with software that has not implemented geometry capabilities. + The geometry node coordinate variables must all have the same single dimension, which is the total number of nodes in all the geometries. The nodes must be stored consecutively for each geometry and in the order of the geometries, and within each multipart geometry the nodes must be stored consecutively for each part and in the order of the parts. Polygon exterior rings must be stored before any interior rings they may contain. From c0ade9cafa28bf1df5b22fc1f651ee06306af8c5 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 4 Mar 2019 15:09:17 -0600 Subject: [PATCH 085/126] Update history with geometry updates --- history.adoc | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/history.adoc b/history.adoc index a67f6915..ad0b1e85 100644 --- a/history.adoc +++ b/history.adoc @@ -195,3 +195,7 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .25 June 2018 . Ticket #164, Add bounds attribute to first geometry CDL example. + +.4 March 2019 +. GitHub Issue #155, #156. Allow alternate grid mappings for geometry +containers. Require geometry_dimension attribute on geometry container From 664e10f60b5f179344519fd02c6bfa4d01d8ccb9 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 8 Mar 2019 11:38:56 +0100 Subject: [PATCH 086/126] Remove reference to removed Appendix I Appendix I contains examples pertaining to the use of groups as specified in Chapter 2 and it is hoped that it can be added in the future in order to provide useful information, guidance and clarification to the standard. --- cf-conventions.adoc | 3 --- toc-extra.adoc | 5 ----- 2 files changed, 8 deletions(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index b8c50287..953fd3e6 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -94,9 +94,6 @@ include::appg.adoc[] :numbered: include::apph.adoc[] -:numbered: -include::appi.adoc[] - :numbered!: include::history.adoc[] diff --git a/toc-extra.adoc b/toc-extra.adoc index d311e223..055172c4 100644 --- a/toc-extra.adoc +++ b/toc-extra.adoc @@ -75,8 +75,3 @@ H.19. <> H.20. <> H.21. <> H.22. <> -I.1. <> -I.2. <> -I.3. <> -I.4. <> -I.5. <> From 9649b0eb5352b78dfc451e965ecc098523387b89 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 8 Mar 2019 11:39:55 +0100 Subject: [PATCH 087/126] Reorder entry in revision log --- history.adoc | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/history.adoc b/history.adoc index f512e353..9f0e9eda 100644 --- a/history.adoc +++ b/history.adoc @@ -1,9 +1,6 @@ [[revhistory, Revision History]] == Revision History -.3 August 2018 -. Added <> and <>. - .14 June 2004 . Added <>. . <> : Added **`latitude_of_projection_origin`** map parameter. @@ -177,3 +174,7 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .10 August 2017 . Updated use of WKT-CRS syntax. + +.3 August 2018 +. Added <> and <>. + From f3f467546c2b817e8b2cc1b99771b71de745071b Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 8 Mar 2019 11:40:13 +0100 Subject: [PATCH 088/126] Add reference to netCDF docs as proposed by @marqh --- ch02.adoc | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/ch02.adoc b/ch02.adoc index 3097a61e..8968fb0a 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -174,6 +174,11 @@ References to out-of-group variable and dimensions shall be found by applying th ==== Scope +The scoping mechanism is in keeping with the following principal: + +[quote, 'https://www.unidata.ucar.edu/software/netcdf/docs/groups.html[The NetCDF Data Model: Groups]'] +"Dimensions are scoped such that they are visible to all child groups. For example, you can define a dimension in the root group, and use its dimension id when defining a variable in a sub-group." + Any variable or dimension can be referred to, as long as it can be found with one of the following search strategies: * Search by absolute path From 6600fadc5dde2d69c0345a5ff703a0b554e05285 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Fri, 8 Mar 2019 11:42:33 +0100 Subject: [PATCH 089/126] Move date of revision from Aug 2018 to Mar 2019 Also remove reference to Appendix I, which will hopefully be included in the future. --- history.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/history.adoc b/history.adoc index 9f0e9eda..3929081f 100644 --- a/history.adoc +++ b/history.adoc @@ -175,6 +175,6 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .10 August 2017 . Updated use of WKT-CRS syntax. -.3 August 2018 -. Added <> and <>. +.8 March 2019 +. Added <>. From 5d122f71453acbad2ddf70e044e0ce7b0787df46 Mon Sep 17 00:00:00 2001 From: David Hassell Date: Tue, 12 Mar 2019 11:47:54 +0000 Subject: [PATCH 090/126] Update README.md Changed status from 'unofficial' to 'official' --- README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/README.md b/README.md index b54b0ba6..81d53ba3 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,7 @@ ### Status -This is an *unofficial* work-in-progress to show how the CF metadata conventions -could be managed via GitHub, Travis-CI, and asciidoctor. The output from this -process can be seen at: http://cfconventions.org/cf-conventions/cf-conventions.html +This repository contains the official source of the CF metadata conventions. The source files are built into HTML (see below), thw output of which can be seen at: http://cfconventions.org/cf-conventions/cf-conventions.html. For the official web site please visit: http://cfconventions.org/, and the corresponding GitHub organisation: https://github.com/cf-convention. From cf5e4fe7f1376438a24935c995b28497d5611c0b Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Fri, 22 Mar 2019 11:28:16 -0500 Subject: [PATCH 091/126] revise per discussion in #156 Make the comparison between grid mappings, between the node and coordinate variables. --- ch07.adoc | 5 ++--- history.adoc | 4 ++-- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 7d288d03..66a82ce0 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -688,9 +688,8 @@ If a **`coordinates`** attribute is carried by the geometry container variable or its parent data variable, then those coordinate variables that have a meaningful correspondence with node coordinates are indicated as such by a **`nodes`** attribute that names the corresponding node coordinates, but only if -the **`grid_mapping`** attribute on the geometry container variable names the -same variable as the **`grid_mapping`** attribute on the associated data -variable. If a different grid mapping variable is named, then the provided +the **`grid_mapping`** associated with the geometry node variables is the same as that of +the coordinate variables. If a different grid mapping is used, then the provided coordinates must not have the **`nodes`** attribute. Whether linked to normal CF space-time coordinates with a **`nodes`** attribute diff --git a/history.adoc b/history.adoc index ad0b1e85..70fd54fe 100644 --- a/history.adoc +++ b/history.adoc @@ -196,6 +196,6 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .25 June 2018 . Ticket #164, Add bounds attribute to first geometry CDL example. -.4 March 2019 +.22 March 2019 . GitHub Issue #155, #156. Allow alternate grid mappings for geometry -containers. Require geometry_dimension attribute on geometry container +containers. Require geometry_dimension attribute on geometry container. From be5a5a6818e23fb9796116716ac46d0b2e673cab Mon Sep 17 00:00:00 2001 From: Klaus Zimmermann Date: Mon, 15 Apr 2019 16:06:15 +0200 Subject: [PATCH 092/126] Fixed misplaced bibliography tag (fixes #135) --- bibliography.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bibliography.adoc b/bibliography.adoc index 207946ae..9351302e 100644 --- a/bibliography.adoc +++ b/bibliography.adoc @@ -1,6 +1,6 @@ -[bibliography] == Bibliography +[bibliography] === References - [[[COARDS]]] link:$$http://www.ferret.noaa.gov/noaa_coop/coop_cdf_profile.html$$[ Conventions for the standardization of NetCDF Files ] . From 2babd92f39f4cf0470745636f01469a1cd8d72da Mon Sep 17 00:00:00 2001 From: Klaus Zimmermann Date: Mon, 15 Apr 2019 16:13:57 +0200 Subject: [PATCH 093/126] Improvements targeting primarily pdf version - Using non breakable spaces avoids newlines in the middle of author names - :doctype: book with :media: prepress makes odd pages appear correctly on the right side of a double page - In pdf, only a limited number of entities (&;) is supported. This replaces the unsupported ≤ with the equivalent <= and the fragile literal < with <, consistent with the usage in the rest of the document --- appf.adoc | 2 +- cf-conventions.adoc | 6 ++++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/appf.adoc b/appf.adoc index 431288a8..04568a3a 100644 --- a/appf.adoc +++ b/appf.adoc @@ -622,7 +622,7 @@ to a reference direction. | **`straight_vertical_longitude_from_pole`** | N | The longitude (degrees_east) to be oriented straight up from the North or - South Pole. Domain: **`-180.0 ≤ straight_vertical_longitude_from_pole< 180.0`** + South Pole. Domain: **`-180.0 <= straight_vertical_longitude_from_pole < 180.0`** | **`towgs84`** | N | This indicates a list of up diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 3ab79ba5..0a0161ad 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -1,6 +1,8 @@ -= NetCDF Climate and Forecast (CF) Metadata Conventions -Brian Eaton; Jonathan Gregory; Bob Drach; Karl Taylor; Steve Hankin; Jon Blower; John Caron; Rich Signell; Phil Bentley; Greg Rappa; Heinke Höck; Alison Pamment; Martin Juckes; Martin Raspaud += NetCDF Climate and Forecast (CF) Metadata Conventions +Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; Jon{nbsp}Blower; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud Version 1.7 +:doctype: book +:media: prepress :sectanchors: :toc: macro :toclevels: 3 From 78d0a27c2f4be3513036b4f91c0848145af995b2 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Thu, 18 Apr 2019 17:57:49 +0200 Subject: [PATCH 094/126] Implement requests from @JonathanGregory Related to #144 --- appa.adoc | 6 +++--- ch01.adoc | 2 +- ch02.adoc | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/appa.adoc b/appa.adoc index aa620763..cdbfebc9 100644 --- a/appa.adoc +++ b/appa.adoc @@ -6,7 +6,7 @@ All CF attributes are listed here except for those that are used to describe grid mappings. See Appendix F for the grid mapping attributes. -The "Type" values are **S** for string, **N** for numeric, and **D** for the type of the data variable. The "Use" values are **G** for global, **C** for variables containing coordinate data, **D** for variables containing non-coordinate data, and **-** for variables with a special purpose. "Links" indicates the location of the attribute"s original definition (first link) and sections where the attribute is discussed in this document (additional links as necessary). +The "Type" values are **S** for string, **N** for numeric, and **D** for the type of the data variable. The "Use" values are **G** for global, **Gr** for applying to groups, **C** for variables containing coordinate data, **D** for variables containing non-coordinate data, and **-** for variables with a special purpose. "Links" indicates the location of the attribute"s original definition (first link) and sections where the attribute is discussed in this document (additional links as necessary). [[table-attributes]] .Attributes @@ -164,7 +164,7 @@ formula in the definition. | **`history`** | S -| G +| G, Gr | link:$$http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html$$[NUG Appendix A, "Attribute Conventions"] | List of the applications that have modified the original data. @@ -255,7 +255,7 @@ formula in the definition. | **`title`** | S -| G +| G, Gr | link:$$http://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html$$[NUG Appendix A, "Attribute Conventions"] | Short description of the file contents. diff --git a/ch01.adoc b/ch01.adoc index b4cab9bc..3e89132a 100644 --- a/ch01.adoc +++ b/ch01.adoc @@ -50,7 +50,7 @@ grid mapping variable:: A variable used as a container for attributes that defin latitude dimension:: A dimension of a netCDF variable that has an associated latitude coordinate variable. -local apex group:: The nearest (to a referring group) ancestor group in which a dimension underlying an out-of-group coordinate is defined. The word "apex" refers to position of this group at the vertex of the tree of groups formed by it, the referring group, and the group where a coordinate is located. +local apex group:: The nearest (to a referring group) ancestor group in which a dimension of an out-of-group coordinate is defined. The word "apex" refers to position of this group at the vertex of the tree of groups formed by it, the referring group, and the group where a coordinate is located. longitude dimension:: A dimension of a netCDF variable that has an associated longitude coordinate variable. diff --git a/ch02.adoc b/ch02.adoc index 8968fb0a..42b4e3e1 100644 --- a/ch02.adoc +++ b/ch02.adoc @@ -226,7 +226,6 @@ They do not override attributes from parent groups. * `title` * `history` -* `Conventions` If these attributes are present, they may be applied additively to the parent attributes of the same name. If a file containing groups is modified, the user or application need only update these attributes in the root group, rather than traversing all groups and updating all attributes that are found with the same name. @@ -234,6 +233,7 @@ In the case of conflicts, the root group attribute takes precedence over per-gro The following attributes may only be used in the root group and shall not be duplicated or overridden in child groups: +* `Conventions` * `external_variables` Furthermore, per-variable attributes must be attached to the variables to which they refer. From 2bcff6dfc88207a945d9c67a4bdb7d89a8aaf379 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Mon, 4 Mar 2019 14:40:11 -0600 Subject: [PATCH 095/126] Revert "Add geometry_dimension attribute to geometry container" This reverts commit ca41968d3b0f92b6242d6b8dcfa90d9e774dfb07. --- ch07.adoc | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 66a82ce0..e46527a8 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -633,7 +633,6 @@ variables: datum:inverse_flattening = 298.257223563 ; int geometry_container ; geometry_container:geometry_type = "line" ; - geometry_container:geometry_dimension = "instance" ; geometry_container:node_count = "node_count" ; geometry_container:node_coordinates = "x y" ; int node_count(instance) ; @@ -669,7 +668,7 @@ The time series variable, someData, is associated with line geometries via the g A __geometry container__ variable acts as a container for attributes that describe a set of geometries. The **`geometry`** attribute of the data variable contains the name of a geometry container variable. -The geometry container variable must hold **`geometry_type`**, **`geometry_dimension`**, and **`node_coordinates`** attributes. +The geometry container variable must hold **`geometry_type`** and **`node_coordinates`** attributes. The **`grid_mapping`** and **`coordinates`** attributes can be carried by the geometry container variable provided they are also carried by the data variables associated with the container. The **`geometry_type`** attribute indicates the type of geometry present. @@ -677,10 +676,6 @@ Its allowable values are: __point__, __line__, __polygon__. Multipart geometries are allowed for all three geometry types. For example, polygon geometries could include single part geometries like the State of Colorado and multipart geometries like the State of Hawaii. -The **`geometry_dimension`** attribute stores the name of the instance dimension -for the geometries. -This dimension must be shared by the data variable related to the geometry container. - The **`node_coordinates`** attribute contains the blank-separated names of the variables that contain geometry node coordinates (one variable for each spatial dimension). The geometry node coordinate variables must each have an **`axis`** attribute whose allowable values are __X__, __Y__, and __Z__. @@ -705,10 +700,7 @@ They are put in opposite orders to facilitate calculation of area and consistenc When more than one geometry instance is present, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. The node count is the total number of nodes in all the parts. -The exception is when all geometries are single part point geometries, in which -case a node count is not needed since each geometry contains a single node. -The variable indicated by the **`node_count`** attribute must use the same -dimension as specified by the **`geometry_dimension`** attribute. +The exception is when all geometries are single part point geometries, in which case a node count is not needed since each geometry contains a single node. For multipart __lines__, multipart __polygons__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that indicates a variable of the count of nodes per geometry part. Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required for __point__ geometry types. @@ -764,7 +756,6 @@ variables: lon:nodes = "x" ; float geometry_container ; geometry_container:geometry_type = "polygon" ; - geometry_container:geometry_dimension = "instance" ; geometry_container:node_count = "node_count" ; geometry_container:node_coordinates = "x y" ; geometry_container:grid_mapping = "datum" ; From 243140617f8c9b626facda33c92151b2ecb1bc48 Mon Sep 17 00:00:00 2001 From: Tim Whiteaker Date: Wed, 15 May 2019 16:22:22 -0500 Subject: [PATCH 096/126] Update per discussion in #155 When node_count attribute is missing, require the dimension of the node coordinate variables to be one of the dimensions of the data variable. --- ch07.adoc | 6 +++++- history.adoc | 5 +++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index e46527a8..96cf19f1 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -700,7 +700,11 @@ They are put in opposite orders to facilitate calculation of area and consistenc When more than one geometry instance is present, the geometry container variable must have a **`node_count`** attribute that contains the name of a variable indicating the count of nodes per geometry. The node count is the total number of nodes in all the parts. -The exception is when all geometries are single part point geometries, in which case a node count is not needed since each geometry contains a single node. +The exception is when all geometries are single part point geometries, in which +case a node count is not needed since each geometry contains a single node. +However in that case, the dimension of the node coordinate variables must be one +of the dimensions of the data variable (because it serves also as the instance +dimension for geometries). For multipart __lines__, multipart __polygons__, and __polygons__ with holes, the geometry container variable must have a **`part_node_count`** attribute that indicates a variable of the count of nodes per geometry part. Note that because multipoint geometries always have a single node per part, the **`part_node_count`** is not required for __point__ geometry types. diff --git a/history.adoc b/history.adoc index 70fd54fe..0427dfff 100644 --- a/history.adoc +++ b/history.adoc @@ -196,6 +196,7 @@ Replaced first para in Section 9.6. "Missing Data". Added verbiage to Section 2. .25 June 2018 . Ticket #164, Add bounds attribute to first geometry CDL example. -.22 March 2019 +.15 May 2019 . GitHub Issue #155, #156. Allow alternate grid mappings for geometry -containers. Require geometry_dimension attribute on geometry container. +containers. When node_count attribute is missing, require the dimension of the +node coordinate variables to be one of the dimensions of the data variable. \ No newline at end of file From d03f939bc54caaff5079cc25051be4aa448616f9 Mon Sep 17 00:00:00 2001 From: Klaus Zimmermann Date: Tue, 11 Jun 2019 14:10:48 +0200 Subject: [PATCH 097/126] Replaced prepress option with pdf-folio-placement This results in the correct placement of recto and verso pages without the introduction of empty pages. --- cf-conventions.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 0a0161ad..6edf57f8 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -2,7 +2,7 @@ Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; Jon{nbsp}Blower; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud Version 1.7 :doctype: book -:media: prepress +:pdf-folio-placement: physical :sectanchors: :toc: macro :toclevels: 3 From 46104c822f1b40a4531747a366ce3c98df435481 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Sat, 15 Jun 2019 16:11:29 -0500 Subject: [PATCH 098/126] Update issue templates I migrated the existing tag descriptions over to be issue descriptions and set it up so the tags get applied by default. --- .github/ISSUE_TEMPLATE/defect-issue.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/defect.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/enhancement-issue.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/github-usage-issue.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/github-usage.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/help-wanted-issue.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/help-wanted.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/proposed-change-request.md | 5 ++++- .github/ISSUE_TEMPLATE/question-issue.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/question.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/style-issue.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/style.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/typo-issue.md | 10 ++++++++++ .github/ISSUE_TEMPLATE/typo.md | 10 ++++++++++ 14 files changed, 134 insertions(+), 1 deletion(-) create mode 100644 .github/ISSUE_TEMPLATE/defect-issue.md create mode 100644 .github/ISSUE_TEMPLATE/defect.md create mode 100644 .github/ISSUE_TEMPLATE/enhancement-issue.md create mode 100644 .github/ISSUE_TEMPLATE/github-usage-issue.md create mode 100644 .github/ISSUE_TEMPLATE/github-usage.md create mode 100644 .github/ISSUE_TEMPLATE/help-wanted-issue.md create mode 100644 .github/ISSUE_TEMPLATE/help-wanted.md create mode 100644 .github/ISSUE_TEMPLATE/question-issue.md create mode 100644 .github/ISSUE_TEMPLATE/question.md create mode 100644 .github/ISSUE_TEMPLATE/style-issue.md create mode 100644 .github/ISSUE_TEMPLATE/style.md create mode 100644 .github/ISSUE_TEMPLATE/typo-issue.md create mode 100644 .github/ISSUE_TEMPLATE/typo.md diff --git a/.github/ISSUE_TEMPLATE/defect-issue.md b/.github/ISSUE_TEMPLATE/defect-issue.md new file mode 100644 index 00000000..88316785 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/defect-issue.md @@ -0,0 +1,10 @@ +--- +name: Defect issue +about: Use to document inconsistencies or errors +title: '' +labels: defect +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/defect.md b/.github/ISSUE_TEMPLATE/defect.md new file mode 100644 index 00000000..d86d5a83 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/defect.md @@ -0,0 +1,10 @@ +--- +name: Defect +about: Use to document inconsistencies or errors +title: '' +labels: defect +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/enhancement-issue.md b/.github/ISSUE_TEMPLATE/enhancement-issue.md new file mode 100644 index 00000000..049f7b28 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/enhancement-issue.md @@ -0,0 +1,10 @@ +--- +name: Enhancement issue +about: Use for issues that add new capabilities or improve existing ones +title: '' +labels: enhancement +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/github-usage-issue.md b/.github/ISSUE_TEMPLATE/github-usage-issue.md new file mode 100644 index 00000000..15cab978 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/github-usage-issue.md @@ -0,0 +1,10 @@ +--- +name: Github usage issue +about: Use to document an issue with how CF uses github +title: '' +labels: GitHubProblem +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/github-usage.md b/.github/ISSUE_TEMPLATE/github-usage.md new file mode 100644 index 00000000..3cb8f5f8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/github-usage.md @@ -0,0 +1,10 @@ +--- +name: Github Usage +about: Use to document an issue with how CF uses github +title: '' +labels: GitHubProblem +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/help-wanted-issue.md b/.github/ISSUE_TEMPLATE/help-wanted-issue.md new file mode 100644 index 00000000..4ab37d19 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/help-wanted-issue.md @@ -0,0 +1,10 @@ +--- +name: Help Wanted Issue +about: Use to describe an action that someone needs to take +title: '' +labels: help wanted +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/help-wanted.md b/.github/ISSUE_TEMPLATE/help-wanted.md new file mode 100644 index 00000000..7f5e1acb --- /dev/null +++ b/.github/ISSUE_TEMPLATE/help-wanted.md @@ -0,0 +1,10 @@ +--- +name: Help Wanted +about: Use to describe an action that someone needs to take +title: '' +labels: help wanted +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/proposed-change-request.md b/.github/ISSUE_TEMPLATE/proposed-change-request.md index 54e5639b..3db6f9c7 100644 --- a/.github/ISSUE_TEMPLATE/proposed-change-request.md +++ b/.github/ISSUE_TEMPLATE/proposed-change-request.md @@ -2,6 +2,9 @@ name: Proposed Change Request about: This template should be used as applicable for any proposed change to the CF convention. +title: '' +labels: '' +assignees: '' --- @@ -15,4 +18,4 @@ Change proposals should include the following information as applicable. **Technical Proposal Summary:** Brief proposal overview **Benefits:** Who or what will benefit from this proposal? **Status Quo:** Discussion of the current state CF and other standards. -**Detailed Proposal:** Complete proposal +**Detailed Proposal:** Complete proposal diff --git a/.github/ISSUE_TEMPLATE/question-issue.md b/.github/ISSUE_TEMPLATE/question-issue.md new file mode 100644 index 00000000..a7f2ec25 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/question-issue.md @@ -0,0 +1,10 @@ +--- +name: Question Issue +about: Use for questions you want help with +title: '' +labels: question +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/question.md b/.github/ISSUE_TEMPLATE/question.md new file mode 100644 index 00000000..b3b7b6ef --- /dev/null +++ b/.github/ISSUE_TEMPLATE/question.md @@ -0,0 +1,10 @@ +--- +name: Question +about: Use for questions you want help with +title: '' +labels: question +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/style-issue.md b/.github/ISSUE_TEMPLATE/style-issue.md new file mode 100644 index 00000000..456f1587 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/style-issue.md @@ -0,0 +1,10 @@ +--- +name: Style Issue +about: Use for issues concerning formatting or document syntax +title: '' +labels: style +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/style.md b/.github/ISSUE_TEMPLATE/style.md new file mode 100644 index 00000000..722ec2e0 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/style.md @@ -0,0 +1,10 @@ +--- +name: Style +about: Use for issues concerning formatting or document syntax +title: '' +labels: style +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/typo-issue.md b/.github/ISSUE_TEMPLATE/typo-issue.md new file mode 100644 index 00000000..f6d275e1 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/typo-issue.md @@ -0,0 +1,10 @@ +--- +name: Typo Issue +about: Use for small errors in spelling or grammar +title: '' +labels: typo +assignees: '' + +--- + + diff --git a/.github/ISSUE_TEMPLATE/typo.md b/.github/ISSUE_TEMPLATE/typo.md new file mode 100644 index 00000000..cc3a1e62 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/typo.md @@ -0,0 +1,10 @@ +--- +name: Typo +about: Use for small errors in spelling or grammar +title: '' +labels: typo +assignees: '' + +--- + + From c5cb875ee8e9ef8f99f5591fb5bb3b9dd987f1c4 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Sat, 15 Jun 2019 16:13:06 -0500 Subject: [PATCH 099/126] Update proposed-change-request.md --- .github/ISSUE_TEMPLATE/proposed-change-request.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/proposed-change-request.md b/.github/ISSUE_TEMPLATE/proposed-change-request.md index 3db6f9c7..ef3fbf56 100644 --- a/.github/ISSUE_TEMPLATE/proposed-change-request.md +++ b/.github/ISSUE_TEMPLATE/proposed-change-request.md @@ -3,7 +3,7 @@ name: Proposed Change Request about: This template should be used as applicable for any proposed change to the CF convention. title: '' -labels: '' +labels: 'enhancement' assignees: '' --- From cc73442532a559328abeb4df12e4f4fb61af1604 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Thu, 20 Jun 2019 13:04:03 -0500 Subject: [PATCH 100/126] remove duplicate issue templates --- .github/ISSUE_TEMPLATE/defect.md | 10 ---------- .github/ISSUE_TEMPLATE/github-usage.md | 10 ---------- .github/ISSUE_TEMPLATE/help-wanted.md | 10 ---------- .github/ISSUE_TEMPLATE/question.md | 10 ---------- .github/ISSUE_TEMPLATE/style.md | 10 ---------- .github/ISSUE_TEMPLATE/typo.md | 10 ---------- 6 files changed, 60 deletions(-) delete mode 100644 .github/ISSUE_TEMPLATE/defect.md delete mode 100644 .github/ISSUE_TEMPLATE/github-usage.md delete mode 100644 .github/ISSUE_TEMPLATE/help-wanted.md delete mode 100644 .github/ISSUE_TEMPLATE/question.md delete mode 100644 .github/ISSUE_TEMPLATE/style.md delete mode 100644 .github/ISSUE_TEMPLATE/typo.md diff --git a/.github/ISSUE_TEMPLATE/defect.md b/.github/ISSUE_TEMPLATE/defect.md deleted file mode 100644 index d86d5a83..00000000 --- a/.github/ISSUE_TEMPLATE/defect.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -name: Defect -about: Use to document inconsistencies or errors -title: '' -labels: defect -assignees: '' - ---- - - diff --git a/.github/ISSUE_TEMPLATE/github-usage.md b/.github/ISSUE_TEMPLATE/github-usage.md deleted file mode 100644 index 3cb8f5f8..00000000 --- a/.github/ISSUE_TEMPLATE/github-usage.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -name: Github Usage -about: Use to document an issue with how CF uses github -title: '' -labels: GitHubProblem -assignees: '' - ---- - - diff --git a/.github/ISSUE_TEMPLATE/help-wanted.md b/.github/ISSUE_TEMPLATE/help-wanted.md deleted file mode 100644 index 7f5e1acb..00000000 --- a/.github/ISSUE_TEMPLATE/help-wanted.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -name: Help Wanted -about: Use to describe an action that someone needs to take -title: '' -labels: help wanted -assignees: '' - ---- - - diff --git a/.github/ISSUE_TEMPLATE/question.md b/.github/ISSUE_TEMPLATE/question.md deleted file mode 100644 index b3b7b6ef..00000000 --- a/.github/ISSUE_TEMPLATE/question.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -name: Question -about: Use for questions you want help with -title: '' -labels: question -assignees: '' - ---- - - diff --git a/.github/ISSUE_TEMPLATE/style.md b/.github/ISSUE_TEMPLATE/style.md deleted file mode 100644 index 722ec2e0..00000000 --- a/.github/ISSUE_TEMPLATE/style.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -name: Style -about: Use for issues concerning formatting or document syntax -title: '' -labels: style -assignees: '' - ---- - - diff --git a/.github/ISSUE_TEMPLATE/typo.md b/.github/ISSUE_TEMPLATE/typo.md deleted file mode 100644 index cc3a1e62..00000000 --- a/.github/ISSUE_TEMPLATE/typo.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -name: Typo -about: Use for small errors in spelling or grammar -title: '' -labels: typo -assignees: '' - ---- - - From 3e06dd59e11e8f7aad0b70dec8b5c8c46d52b319 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Thu, 20 Jun 2019 14:32:07 -0500 Subject: [PATCH 101/126] remove un-needed template --- .github/ISSUE_TEMPLATE/help-wanted-issue.md | 10 ---------- 1 file changed, 10 deletions(-) delete mode 100644 .github/ISSUE_TEMPLATE/help-wanted-issue.md diff --git a/.github/ISSUE_TEMPLATE/help-wanted-issue.md b/.github/ISSUE_TEMPLATE/help-wanted-issue.md deleted file mode 100644 index 4ab37d19..00000000 --- a/.github/ISSUE_TEMPLATE/help-wanted-issue.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -name: Help Wanted Issue -about: Use to describe an action that someone needs to take -title: '' -labels: help wanted -assignees: '' - ---- - - From e838aae55fc4ae52c4e77650428c41815614e944 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Tue, 9 Jul 2019 12:25:02 -0500 Subject: [PATCH 102/126] update contributing fixes #170 --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8e6e5714..c136c2d8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -26,7 +26,7 @@ If the proposal is non-controversial (such as a typo correction) or has been agr Practically all changes should be documented and discussed in an issue fixed in a related pull request. 4. **Use [labels](https://github.com/cf-convention/cf-conventions/labels) on issues and pull requests.** -Contributions must be presented as enhancements, defects, or typos and labeled accordingly. +Currently this is achieved by using an appropriate issue template when creating a [new issue](https://github.com/cf-convention/cf-conventions/issues/new/choose). ## Issues and Pull Requests From 095ef8ba221afd1915b3ace58e00647430662d01 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Wed, 17 Jul 2019 10:35:52 -0700 Subject: [PATCH 103/126] fix asciidoctor build for #175 --- bibliography.adoc | 2 -- 1 file changed, 2 deletions(-) diff --git a/bibliography.adoc b/bibliography.adoc index 207946ae..de3bc6d9 100644 --- a/bibliography.adoc +++ b/bibliography.adoc @@ -1,8 +1,6 @@ [bibliography] == Bibliography -=== References - - [[[COARDS]]] link:$$http://www.ferret.noaa.gov/noaa_coop/coop_cdf_profile.html$$[ Conventions for the standardization of NetCDF Files ] . Sponsored by the "Cooperative Ocean/Atmosphere Research Data From 59abb3f30be1063647db64b585049688fb0a244f Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Wed, 17 Jul 2019 10:42:08 -0700 Subject: [PATCH 104/126] try updated version of ruby --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index c9dd2d19..f1452ce3 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,6 +1,6 @@ language: ruby rvm: - - 2.1.2 + - 2.6 install: - gem install asciidoctor From d1688f7da502969208771bfee771fc8ce676daf7 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Wed, 17 Jul 2019 11:10:47 -0700 Subject: [PATCH 105/126] fix formatted text failing in pdf fixes #175 --- appf.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/appf.adoc b/appf.adoc index 431288a8..f25c14e0 100644 --- a/appf.adoc +++ b/appf.adoc @@ -622,7 +622,7 @@ to a reference direction. | **`straight_vertical_longitude_from_pole`** | N | The longitude (degrees_east) to be oriented straight up from the North or - South Pole. Domain: **`-180.0 ≤ straight_vertical_longitude_from_pole< 180.0`** + South Pole. Domain: **`-180.0 > straight_vertical_longitude_from_pole < 180.0`** | **`towgs84`** | N | This indicates a list of up From 0d2d7ee4859528538fc82d33fa968fa652592f57 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 17 Jul 2019 14:11:07 -0700 Subject: [PATCH 106/126] Update history.adoc --- history.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/history.adoc b/history.adoc index 33087c40..e1415e4c 100644 --- a/history.adoc +++ b/history.adoc @@ -202,4 +202,4 @@ containers. When node_count attribute is missing, require the dimension of the node coordinate variables to be one of the dimensions of the data variable. .17 July 2019 -. #144 - Add <>. +. #144 - Add <>. From 446e37b57b7924015292783de9686181ffce4a79 Mon Sep 17 00:00:00 2001 From: Daniel Lee Date: Wed, 17 Jul 2019 16:01:28 -0700 Subject: [PATCH 107/126] Add authors of Groups proposal --- cf-conventions.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index e8ed2ddf..b9b51d01 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -32,6 +32,8 @@ include::toc-extra.adoc[] * Randy Horne, Excalibur Laboratories, Inc., Melbourne Beach Florida USA * Timothy Whiteaker, University of Texas * David Blodgett, USGS +* Charlie Zender, University of California, Irvine +* Daniel Lee, EUMETSAT Many others have contributed to the development of CF through their participation in discussions about proposed changes. From 4df9d186d10c31b48c54d8834c35fa69c1d17ad2 Mon Sep 17 00:00:00 2001 From: marqh Date: Thu, 18 Jul 2019 14:34:15 +0000 Subject: [PATCH 108/126] adapt text at the start of chapter 5 as well --- ch05.adoc | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/ch05.adoc b/ch05.adoc index c62c73ec..6e966682 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -25,7 +25,11 @@ If the longitude, latitude, vertical or time coordinate is multi-valued, varies If an **`axis`** attribute is attached to an auxiliary coordinate variable, it can be used by applications in the same way the **`axis`** attribute attached to a coordinate variable is used. However, it is not permissible for a data variable to have both a coordinate variable and an auxiliary coordinate variable, or more than one of either type of variable, having an **`axis`** attribute with any given value e.g. there must be no more than one **`axis`** attribute for **`X`** for any data variable. Note that if the **`axis`** attribute is not specified for an auxiliary coordinate variable, it may still be possible to determine if it is a spatiotemporal dimension from its own units or standard_name, or from the units and standard_name of the coordinate variable corresponding to its dimensions (see <>). For instance, auxiliary coordinate variables which lie on the horizontal surface can be identified as such by their dimensions being horizontal. Horizontal dimensions are those whose coordinate variables have an **`axis`** attribute of **`X`** or **`Y`**, or a **`units`** attribute indicating latitude and longitude. -If the coordinate variables for a horizontal grid are not longitude and latitude, it is recommended that they be supplied __in addition__ to the required coordinates. For example, the Cartesian coordinates of a map projection should be supplied as coordinate variables in addition to the required two-dimensional latitude and longitude variables that are identified via the **`coordinates`** attribute. The use of the **`axis`** attribute with values **`X`** and **`Y`** is recommended for the coordinate variables (see <>). +To geo-reference data horizontally with respect to the Earth (or other body), a __grid mapping variable__ may be provided by the data variable, using the **`grid_mapping`** attribute. +If the coordinate variables for a horizontal grid are not longitude and latitude, then a grid_mapping variable provides the information required to derive longitude and latitude values for each grid location, if required. +If no __grid mapping variable__ is referenced by a data variable, then longitude and latitude coordinate values shall be supplied __in addition__ to the required coordinates. +For example, the Cartesian coordinates of a map projection may be supplied as coordinate variables and, in addition, two-dimensional latitude and longitude variables may be supplied, that are identified by a data variable using the **`coordinates`** attribute. +The use of the **`axis`** attribute with values **`X`** and **`Y`** is recommended for the coordinate variables (see <>). It is sometimes not practical to specify the latitude-longitude location of data which is representative of geographic regions with complex boundaries. For this purpose, provision is made in <> for indicating the region by a standardized name. From 0cda56fb15eb41d7e02dd547b99041308d764530 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Thu, 18 Jul 2019 09:55:07 -0700 Subject: [PATCH 109/126] add content into issue templates --- .github/ISSUE_TEMPLATE/defect-issue.md | 10 +++++++++ .github/ISSUE_TEMPLATE/enhancement-issue.md | 10 +++++++++ .github/ISSUE_TEMPLATE/github-usage-issue.md | 2 +- .../ISSUE_TEMPLATE/proposed-change-request.md | 21 ------------------- .github/ISSUE_TEMPLATE/question-issue.md | 2 +- .github/ISSUE_TEMPLATE/style-issue.md | 2 +- .github/ISSUE_TEMPLATE/typo-issue.md | 2 +- 7 files changed, 24 insertions(+), 25 deletions(-) delete mode 100644 .github/ISSUE_TEMPLATE/proposed-change-request.md diff --git a/.github/ISSUE_TEMPLATE/defect-issue.md b/.github/ISSUE_TEMPLATE/defect-issue.md index 88316785..77ff65d5 100644 --- a/.github/ISSUE_TEMPLATE/defect-issue.md +++ b/.github/ISSUE_TEMPLATE/defect-issue.md @@ -7,4 +7,14 @@ assignees: '' --- +Before submitting an issue be sure you have read and understand the github contributing guidelines: https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md and the rules for CF changes: http://cfconventions.org/rules.html +Change proposals should include the following information as applicable. + +**Title:** Short and descriptive. +**Moderator:** @user +**Requirement Summary:** A few sentence functional summary +**Technical Proposal Summary:** Brief proposal overview +**Benefits:** Who or what will benefit from this proposal? +**Status Quo:** Discussion of the current state CF and other standards. +**Detailed Proposal:** Complete proposal diff --git a/.github/ISSUE_TEMPLATE/enhancement-issue.md b/.github/ISSUE_TEMPLATE/enhancement-issue.md index 049f7b28..def7c13c 100644 --- a/.github/ISSUE_TEMPLATE/enhancement-issue.md +++ b/.github/ISSUE_TEMPLATE/enhancement-issue.md @@ -7,4 +7,14 @@ assignees: '' --- +Before submitting an issue be sure you have read and understand the github contributing guidelines: https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md and the rules for CF changes: http://cfconventions.org/rules.html +Change proposals should include the following information as applicable. + +**Title:** Short and descriptive. +**Moderator:** @user +**Requirement Summary:** A few sentence functional summary +**Technical Proposal Summary:** Brief proposal overview +**Benefits:** Who or what will benefit from this proposal? +**Status Quo:** Discussion of the current state CF and other standards. +**Detailed Proposal:** Complete proposal diff --git a/.github/ISSUE_TEMPLATE/github-usage-issue.md b/.github/ISSUE_TEMPLATE/github-usage-issue.md index 15cab978..8a659448 100644 --- a/.github/ISSUE_TEMPLATE/github-usage-issue.md +++ b/.github/ISSUE_TEMPLATE/github-usage-issue.md @@ -7,4 +7,4 @@ assignees: '' --- - +Before submitting an issue be sure you have read and understand the github contributing guidelines: https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md and the rules for CF changes: http://cfconventions.org/rules.html diff --git a/.github/ISSUE_TEMPLATE/proposed-change-request.md b/.github/ISSUE_TEMPLATE/proposed-change-request.md deleted file mode 100644 index ef3fbf56..00000000 --- a/.github/ISSUE_TEMPLATE/proposed-change-request.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Proposed Change Request -about: This template should be used as applicable for any proposed change to the CF - convention. -title: '' -labels: 'enhancement' -assignees: '' - ---- - -Before submitting an issue be sure you have read and understand the github contributing guidelines: https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md and the rules for CF changes: http://cfconventions.org/rules.html - -Change proposals should include the following information as applicable. - -**Title:** Short and descriptive. -**Moderator:** @user -**Requirement Summary:** A few sentence functional summary -**Technical Proposal Summary:** Brief proposal overview -**Benefits:** Who or what will benefit from this proposal? -**Status Quo:** Discussion of the current state CF and other standards. -**Detailed Proposal:** Complete proposal diff --git a/.github/ISSUE_TEMPLATE/question-issue.md b/.github/ISSUE_TEMPLATE/question-issue.md index a7f2ec25..77e88cb9 100644 --- a/.github/ISSUE_TEMPLATE/question-issue.md +++ b/.github/ISSUE_TEMPLATE/question-issue.md @@ -7,4 +7,4 @@ assignees: '' --- - +Before submitting an issue be sure you have read and understand the github contributing guidelines: https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md and the rules for CF changes: http://cfconventions.org/rules.html diff --git a/.github/ISSUE_TEMPLATE/style-issue.md b/.github/ISSUE_TEMPLATE/style-issue.md index 456f1587..6954cc67 100644 --- a/.github/ISSUE_TEMPLATE/style-issue.md +++ b/.github/ISSUE_TEMPLATE/style-issue.md @@ -7,4 +7,4 @@ assignees: '' --- - +Before submitting an issue be sure you have read and understand the github contributing guidelines: https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md and the rules for CF changes: http://cfconventions.org/rules.html diff --git a/.github/ISSUE_TEMPLATE/typo-issue.md b/.github/ISSUE_TEMPLATE/typo-issue.md index f6d275e1..913dff95 100644 --- a/.github/ISSUE_TEMPLATE/typo-issue.md +++ b/.github/ISSUE_TEMPLATE/typo-issue.md @@ -7,4 +7,4 @@ assignees: '' --- - +Before submitting an issue be sure you have read and understand the github contributing guidelines: https://github.com/cf-convention/cf-conventions/blob/master/CONTRIBUTING.md and the rules for CF changes: http://cfconventions.org/rules.html From eff2cdc68f6381d8694964c41729100991e22f21 Mon Sep 17 00:00:00 2001 From: marqh Date: Fri, 19 Jul 2019 12:06:18 +0000 Subject: [PATCH 110/126] Text revisions following reviews --- ch05.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/ch05.adoc b/ch05.adoc index 6e966682..268dfc41 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -25,10 +25,10 @@ If the longitude, latitude, vertical or time coordinate is multi-valued, varies If an **`axis`** attribute is attached to an auxiliary coordinate variable, it can be used by applications in the same way the **`axis`** attribute attached to a coordinate variable is used. However, it is not permissible for a data variable to have both a coordinate variable and an auxiliary coordinate variable, or more than one of either type of variable, having an **`axis`** attribute with any given value e.g. there must be no more than one **`axis`** attribute for **`X`** for any data variable. Note that if the **`axis`** attribute is not specified for an auxiliary coordinate variable, it may still be possible to determine if it is a spatiotemporal dimension from its own units or standard_name, or from the units and standard_name of the coordinate variable corresponding to its dimensions (see <>). For instance, auxiliary coordinate variables which lie on the horizontal surface can be identified as such by their dimensions being horizontal. Horizontal dimensions are those whose coordinate variables have an **`axis`** attribute of **`X`** or **`Y`**, or a **`units`** attribute indicating latitude and longitude. -To geo-reference data horizontally with respect to the Earth (or other body), a __grid mapping variable__ may be provided by the data variable, using the **`grid_mapping`** attribute. +To geo-reference data horizontally with respect to the Earth, a __grid mapping variable__ may be provided by the data variable, using the **`grid_mapping`** attribute. If the coordinate variables for a horizontal grid are not longitude and latitude, then a grid_mapping variable provides the information required to derive longitude and latitude values for each grid location, if required. If no __grid mapping variable__ is referenced by a data variable, then longitude and latitude coordinate values shall be supplied __in addition__ to the required coordinates. -For example, the Cartesian coordinates of a map projection may be supplied as coordinate variables and, in addition, two-dimensional latitude and longitude variables may be supplied, that are identified by a data variable using the **`coordinates`** attribute. +For example, the Cartesian coordinates of a map projection may be supplied as coordinate variables and, in addition, two-dimensional latitude and longitude variables may be supplied via the **`coordinates`** attribute on a data variable. The use of the **`axis`** attribute with values **`X`** and **`Y`** is recommended for the coordinate variables (see <>). It is sometimes not practical to specify the latitude-longitude location of data which is representative of geographic regions with complex boundaries. For this purpose, provision is made in <> for indicating the region by a standardized name. @@ -187,8 +187,8 @@ _This section has been superseded by the treatment of time series as a type of d [[grid-mappings-and-projections, Section 5.6, "Horizontal Coordinate Reference Systems, Grid Mappings, and Projections"]] === Horizontal Coordinate Reference Systems, Grid Mappings, and Projections -A __grid mapping variable__ may be referenced by a data variable, to describe the mapping between the -given coordinate variables and the true latitude and longitude coordinates, to describe the +A __grid mapping variable__ may be referenced by a data variable: to describe the mapping between the +given coordinate variables and explicit latitude and longitude coordinates (if supplied), to describe the figure of the Earth used to define the latitude and longitude coordinates, or to describe another coordinate reference system definition used by some coordinates or auxiliary coordinates. From d0fc46f6ab4db8d6306eac30d92cd0014d8530f5 Mon Sep 17 00:00:00 2001 From: marqh Date: Fri, 19 Jul 2019 12:06:59 +0000 Subject: [PATCH 111/126] remove 'if required' --- ch05.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch05.adoc b/ch05.adoc index 268dfc41..720505f4 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -26,7 +26,7 @@ If the longitude, latitude, vertical or time coordinate is multi-valued, varies If an **`axis`** attribute is attached to an auxiliary coordinate variable, it can be used by applications in the same way the **`axis`** attribute attached to a coordinate variable is used. However, it is not permissible for a data variable to have both a coordinate variable and an auxiliary coordinate variable, or more than one of either type of variable, having an **`axis`** attribute with any given value e.g. there must be no more than one **`axis`** attribute for **`X`** for any data variable. Note that if the **`axis`** attribute is not specified for an auxiliary coordinate variable, it may still be possible to determine if it is a spatiotemporal dimension from its own units or standard_name, or from the units and standard_name of the coordinate variable corresponding to its dimensions (see <>). For instance, auxiliary coordinate variables which lie on the horizontal surface can be identified as such by their dimensions being horizontal. Horizontal dimensions are those whose coordinate variables have an **`axis`** attribute of **`X`** or **`Y`**, or a **`units`** attribute indicating latitude and longitude. To geo-reference data horizontally with respect to the Earth, a __grid mapping variable__ may be provided by the data variable, using the **`grid_mapping`** attribute. -If the coordinate variables for a horizontal grid are not longitude and latitude, then a grid_mapping variable provides the information required to derive longitude and latitude values for each grid location, if required. +If the coordinate variables for a horizontal grid are not longitude and latitude, then a grid_mapping variable provides the information required to derive longitude and latitude values for each grid location. If no __grid mapping variable__ is referenced by a data variable, then longitude and latitude coordinate values shall be supplied __in addition__ to the required coordinates. For example, the Cartesian coordinates of a map projection may be supplied as coordinate variables and, in addition, two-dimensional latitude and longitude variables may be supplied via the **`coordinates`** attribute on a data variable. The use of the **`axis`** attribute with values **`X`** and **`Y`** is recommended for the coordinate variables (see <>). From 1f41406bc8edb675ae13b5267b3bb203c6db1d53 Mon Sep 17 00:00:00 2001 From: markh Date: Fri, 19 Jul 2019 15:42:42 +0000 Subject: [PATCH 112/126] update bot token --- .travis.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index f1452ce3..266a0fe4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -38,7 +38,7 @@ after_success: git add $TARGET_DIR/index.html; fi - git commit -m "Auto-generated from ${TRAVIS_REPO_SLUG}@${TRAVIS_COMMIT}" - - git push https://${GH_TOKEN}@github.com/${TRAVIS_REPO_SLUG}.git gh-pages >/dev/null 2>&1 + - git push https://${GHT_TOKEN}@github.com/${TRAVIS_REPO_SLUG}.git gh-pages >/dev/null 2>&1 branches: only: @@ -49,6 +49,7 @@ branches: env: global: # Values for cf-conventions-bot/cf-conventions + - secure: "B0zZZ1d69+JRxsi2wzbqJspUuGPhE+XRxW63vCcXHqQw3ehHRgb5lZuDZyGfUnfAKblciDi/NJmHXuY5KAU3zUZjk/SISdvoygcrwkqtO9JrZ8H2e3U/VxAIpM4gwE6qHclAa5vbJtf0H9vowy8dlSUtC8BVCRABEomJJD1jaQUL/ebqpua01FlN8/T53KQiCPEnB0OuxmNAAxPgUOliteU/6XriW9YBm7GA/pH4wcvxyibu1SyyCLyYJDtJqSzTN+M9nZvXCB8B2Vh3UfF2Ma6Par2pYscNwkWdL7aZTWZwjObB9aWYmegDtKIjUtMSGJhgbh2umBxWCpwiyheq/cxPQ/eQmuZycpV+/9AypcPFYxIy/YyI8EFybr9s9pb4MvI1fedVafAUEJKm0UDFvQsYRPd0guTGVxT08mdWVHcuqkBjwBFrpuFO0cm9t2F6hIbz0VOuu7pcG8PJdkkp+tnzS5gLG++48xDZTQxjKHBrpkcB3nAsycjDYZUgJSGDVHHroykIp0zLMyUxDOBEqOktTEe/Sr/rdXzsjm3NLJOANv+2IsoEEZi5xa8ZJsOK4bCasFcEh58kJZQ9Lcyn8ASbJ/cVN7H65AL53IPwR7vJTjRG713Lk/VplISrXmFkWxgX8NoTcUTXlBfFwR7xpMJDYzHPcgJHxDlCE5p1cn8=" - secure: "d+R5tzKSdhytTnjrIW3oAqEnoiVY1I/pqcfNQDBrHDNE+FuCuaGCvjrQuRCil50SAMiZ0/80aN8t9I7iwY2hCcUGJoyfgzTjPi/x0RytcQR4q5qylJw33L3Va1r7g91Y7Fi78c5rXLqXZf/wqhiyTw4uLfMum0ka8TLGn0U960mD9PO8uhavJNZbUsF4puN2q5s7PWBhayVm6S5WiGpKINH3lhH4aFz5+oEBidtBRHMZtT4w5hmjXpU0nd6CfF9/+reMmTjJnbNGCbtTRN3UodIU2OY7ToeLD73f7aBoIOVS2LSjtyWZzCvWszcH7y2U2HlmIiyUA2/DMAnYKYuxHa4P0R5xpSZWUSMb517/YVRWDrJVcQaoWdB4hYQsSYo6Nv/Hj35vf1E7sXiwZCPzSaEWRes31E5mf22Ug5+5R1GwTv0FZ2BxdJ+EZThVvWkm2Cvb4yrOObCq+jC2Z1iJ/f9P0q7DNKzgePiv5MsMWzNFfJ5nwRG9cRl5tbghnScKFntf2LSMVtmHUFHYty1TgxgePko4hUOD3IB14zv71dJLu07qdRuyjZYOjM/qwGfW+FaEcuNC1DHKU+eJPGP9GJjaf+iCM/FQiYiepuofp5ys7k4yqLLlW/FCWGNIPZEdKt4ZggHAl9rCmUDMZF5gClr9YjrbYtCl8mOVYKCVPUg=" - secure: "3KygyBd58pedh3zsDMAI8HZY7G5w7Y4ZYDvAERBPoZQmM23XD0Ot8TPUM3RGAVBsAhxwomrVaBJNRaMc9/yr8cvarD+o8G/YfgITNVDsSbXTb6cBkA499bTawDK1ocKDGXUYTwGbaLSV9kLiD/0NXptkFEQFzsNtRwsvUXXHOp+j4UBJutCZKZhD/1OfHxk23JmKhkAgeKcOGJ8F3xv3NPwp5DBJ3z3/l31aVjckft+og0EPARJOFxlLEA3/pKwJpuV4LdvuiWK5slDc+IVMgAmJm+5BlqaluH+zBxMol3/uXL57JX0rNIOTKwXQj6Xy+r1iQMlrazK1qy0Z/xQN5IBGjdu4CaDxhg1Kh0yfkZ+TtnamnjbdXYW1/AJW3kuVDFJAdWdRnuZAI4NIWPJTVDDgDVCnqyzduM2O/q8lqvA2U1eTpTnGcExDJZO3zMCj+/opAj1o00HBdxS7hFhN/phBuARwCz+Vj1hoC6p+jC515X+1mhtuP1ghiGVwrmJTyqY1Xc8xJF6xqqhTKR6SGphgeM8OnHeoJeCBspS1ehJkHm9Z7ENmB+bIV082aw4Ti95v0rdRtUOF4L9UUsPJvq/uwxFVAMYr6nbZM4pd+kko+tQsbUoC6g6N+Zn1vcNFQYDkK1StBESxLQ8SxZXXt+Xbjv1bvT88isyKpXlxAQw=" - secure: "LpRVpYe0hphtelWP5U5o7GMicgorl1TSwmedPG7Sa0TUvqNQltRXXbkGodxtu7fUKrw+SBp+6CGVmgU5RP+Z+R5xETRxNY0IyA8up4uKKACjWvVi5eLp3fKR2R6I0l1LF2GfAtIgV6K2QQWjwar2zrVeN8hs29eI9HqEWQpwgqDxEnHmHpaL84OIcVtNUJkuiIJRkepOHthLe1N3V+xmqYnjXYjmAyN7yyYC+x8KIxqjJSKRUsOmhJwnwpUw27r9QhsVc033KDz+cSm/hG87gnRTkr3iD4jxArKi9d5Oat9ahip6XxLs7MzDpDbb69KTMpf3y4SARuVJ7GW0Gget3GjqF4Ly0r/TLYp4ztWy6aDPvI13w9PJ2NO+L169Vn8HvtRVB45Hu4svFIjpMow80QC10BqUAmKk/AaxSdwQzkZEFPzvYgTGTJjkBy9Gk5isnNoA1GOz2g2b+NZc/H/TuvkvFZBAT4Bl4I1btr9qnYrp5Hinkz+jkUmhGkxD/K0BBa8Cd+bbj+2HyKjAhRTH8g65nIKmgyO97308BBwKeOl2g8kEil6qYydbqLrM9zVJeyYhyTzDFzsXrqWE8g1PT8kO3+5O2FRrZ6otKbCbRUTlkzw/Wzz9leFpOtqYsh59i8sxq1uGVEH3lrbXvfYkBaHnrxXT6FJw5wdcLLu6lW0=" From a884f3e5532b8fb280106d8f7d1f93d2b8b42ab1 Mon Sep 17 00:00:00 2001 From: marqh Date: Fri, 19 Jul 2019 15:54:27 +0000 Subject: [PATCH 113/126] misprint in secure token --- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 266a0fe4..9b28c61d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -49,7 +49,7 @@ branches: env: global: # Values for cf-conventions-bot/cf-conventions - - secure: "B0zZZ1d69+JRxsi2wzbqJspUuGPhE+XRxW63vCcXHqQw3ehHRgb5lZuDZyGfUnfAKblciDi/NJmHXuY5KAU3zUZjk/SISdvoygcrwkqtO9JrZ8H2e3U/VxAIpM4gwE6qHclAa5vbJtf0H9vowy8dlSUtC8BVCRABEomJJD1jaQUL/ebqpua01FlN8/T53KQiCPEnB0OuxmNAAxPgUOliteU/6XriW9YBm7GA/pH4wcvxyibu1SyyCLyYJDtJqSzTN+M9nZvXCB8B2Vh3UfF2Ma6Par2pYscNwkWdL7aZTWZwjObB9aWYmegDtKIjUtMSGJhgbh2umBxWCpwiyheq/cxPQ/eQmuZycpV+/9AypcPFYxIy/YyI8EFybr9s9pb4MvI1fedVafAUEJKm0UDFvQsYRPd0guTGVxT08mdWVHcuqkBjwBFrpuFO0cm9t2F6hIbz0VOuu7pcG8PJdkkp+tnzS5gLG++48xDZTQxjKHBrpkcB3nAsycjDYZUgJSGDVHHroykIp0zLMyUxDOBEqOktTEe/Sr/rdXzsjm3NLJOANv+2IsoEEZi5xa8ZJsOK4bCasFcEh58kJZQ9Lcyn8ASbJ/cVN7H65AL53IPwR7vJTjRG713Lk/VplISrXmFkWxgX8NoTcUTXlBfFwR7xpMJDYzHPcgJHxDlCE5p1cn8=" + - secure: "AduHBhrgWc2XOSXUdCzQ8W7z12iHhQjzZgodVbulftW+Uwn2tGx5bPBc0MrrfPJ0AaoE5e1ZeCGk6ZGYVMnOAQEq95NpTJWuqKtfGFvYVuhbVYQ989ZacSugxEWY8zp2yTulQCQ4iuL/HnSgRVEAlAbkIbrjG8mwgBFeXeyUB9TRn11huw/P9PfOOKOVdxgcYmnYzQSD5DVWreLGag6oXMQ/3o3SJUDGpXToZeO+yQZXnKpjEKKIMey0fkCeDeZbviTKVO1A70iiBk9Zc5WEf8FzyJ/sUDr9qjI3gxp70CPV+32aKzqXoxNclssOFPYDoWBZDs8rYS7OBEgyJvE5At5ZhGAmcLn0UGjCSepruJeDdOEYv3tc3QfKm+ZkqNDOjTaHS3gLqHofXqk6C9ppLRzZSxCDbNP7N1nQ/fGMp3dPMrBlH+GG+BNpsQ+wEFY0G9O2veRKfMZnIz5/Lqn20SUOV/0x2FlgYfm1PUBzuqnkwcc+YSOzKNwf4LpCZhUQVQlGWxTmrq9mcSXxi1KzJ7N67+SDqu4x5xC0VI1yrQBGVATCBJYC7XYIe+sABO2gBM9tkO7FpVSzbNDWCnDAPELvBbB1KZfBpV6MI5F9Qx8c2ExnepeOHRIpvcvDJoQfihX8fGXZYbl/XLLnoHmnPJGA8CexMnBHXGHLF6uxdWI=" - secure: "d+R5tzKSdhytTnjrIW3oAqEnoiVY1I/pqcfNQDBrHDNE+FuCuaGCvjrQuRCil50SAMiZ0/80aN8t9I7iwY2hCcUGJoyfgzTjPi/x0RytcQR4q5qylJw33L3Va1r7g91Y7Fi78c5rXLqXZf/wqhiyTw4uLfMum0ka8TLGn0U960mD9PO8uhavJNZbUsF4puN2q5s7PWBhayVm6S5WiGpKINH3lhH4aFz5+oEBidtBRHMZtT4w5hmjXpU0nd6CfF9/+reMmTjJnbNGCbtTRN3UodIU2OY7ToeLD73f7aBoIOVS2LSjtyWZzCvWszcH7y2U2HlmIiyUA2/DMAnYKYuxHa4P0R5xpSZWUSMb517/YVRWDrJVcQaoWdB4hYQsSYo6Nv/Hj35vf1E7sXiwZCPzSaEWRes31E5mf22Ug5+5R1GwTv0FZ2BxdJ+EZThVvWkm2Cvb4yrOObCq+jC2Z1iJ/f9P0q7DNKzgePiv5MsMWzNFfJ5nwRG9cRl5tbghnScKFntf2LSMVtmHUFHYty1TgxgePko4hUOD3IB14zv71dJLu07qdRuyjZYOjM/qwGfW+FaEcuNC1DHKU+eJPGP9GJjaf+iCM/FQiYiepuofp5ys7k4yqLLlW/FCWGNIPZEdKt4ZggHAl9rCmUDMZF5gClr9YjrbYtCl8mOVYKCVPUg=" - secure: "3KygyBd58pedh3zsDMAI8HZY7G5w7Y4ZYDvAERBPoZQmM23XD0Ot8TPUM3RGAVBsAhxwomrVaBJNRaMc9/yr8cvarD+o8G/YfgITNVDsSbXTb6cBkA499bTawDK1ocKDGXUYTwGbaLSV9kLiD/0NXptkFEQFzsNtRwsvUXXHOp+j4UBJutCZKZhD/1OfHxk23JmKhkAgeKcOGJ8F3xv3NPwp5DBJ3z3/l31aVjckft+og0EPARJOFxlLEA3/pKwJpuV4LdvuiWK5slDc+IVMgAmJm+5BlqaluH+zBxMol3/uXL57JX0rNIOTKwXQj6Xy+r1iQMlrazK1qy0Z/xQN5IBGjdu4CaDxhg1Kh0yfkZ+TtnamnjbdXYW1/AJW3kuVDFJAdWdRnuZAI4NIWPJTVDDgDVCnqyzduM2O/q8lqvA2U1eTpTnGcExDJZO3zMCj+/opAj1o00HBdxS7hFhN/phBuARwCz+Vj1hoC6p+jC515X+1mhtuP1ghiGVwrmJTyqY1Xc8xJF6xqqhTKR6SGphgeM8OnHeoJeCBspS1ehJkHm9Z7ENmB+bIV082aw4Ti95v0rdRtUOF4L9UUsPJvq/uwxFVAMYr6nbZM4pd+kko+tQsbUoC6g6N+Zn1vcNFQYDkK1StBESxLQ8SxZXXt+Xbjv1bvT88isyKpXlxAQw=" - secure: "LpRVpYe0hphtelWP5U5o7GMicgorl1TSwmedPG7Sa0TUvqNQltRXXbkGodxtu7fUKrw+SBp+6CGVmgU5RP+Z+R5xETRxNY0IyA8up4uKKACjWvVi5eLp3fKR2R6I0l1LF2GfAtIgV6K2QQWjwar2zrVeN8hs29eI9HqEWQpwgqDxEnHmHpaL84OIcVtNUJkuiIJRkepOHthLe1N3V+xmqYnjXYjmAyN7yyYC+x8KIxqjJSKRUsOmhJwnwpUw27r9QhsVc033KDz+cSm/hG87gnRTkr3iD4jxArKi9d5Oat9ahip6XxLs7MzDpDbb69KTMpf3y4SARuVJ7GW0Gget3GjqF4Ly0r/TLYp4ztWy6aDPvI13w9PJ2NO+L169Vn8HvtRVB45Hu4svFIjpMow80QC10BqUAmKk/AaxSdwQzkZEFPzvYgTGTJjkBy9Gk5isnNoA1GOz2g2b+NZc/H/TuvkvFZBAT4Bl4I1btr9qnYrp5Hinkz+jkUmhGkxD/K0BBa8Cd+bbj+2HyKjAhRTH8g65nIKmgyO97308BBwKeOl2g8kEil6qYydbqLrM9zVJeyYhyTzDFzsXrqWE8g1PT8kO3+5O2FRrZ6otKbCbRUTlkzw/Wzz9leFpOtqYsh59i8sxq1uGVEH3lrbXvfYkBaHnrxXT6FJw5wdcLLu6lW0=" From cd1180ba8de2ca97647dd731bae711867db48451 Mon Sep 17 00:00:00 2001 From: David Blodgett Date: Fri, 19 Jul 2019 09:03:44 -0700 Subject: [PATCH 114/126] update readme and add 1.8 (draft) to title --- README.md | 4 +++- cf-conventions.adoc | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 81d53ba3..a9d9f22d 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,9 @@ ### Status -This repository contains the official source of the CF metadata conventions. The source files are built into HTML (see below), thw output of which can be seen at: http://cfconventions.org/cf-conventions/cf-conventions.html. +This repository contains the official source of the CF metadata conventions. +The source files are built into HTML automatically when changes are merged into this repository. +The latest build of the specfication is held in the `gh-pages` branch and can be seen at: http://cfconventions.org/cf-conventions/cf-conventions.html. For the official web site please visit: http://cfconventions.org/, and the corresponding GitHub organisation: https://github.com/cf-convention. diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 1d06ec77..18f066c2 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -1,6 +1,6 @@ = NetCDF Climate and Forecast (CF) Metadata Conventions Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; Jon{nbsp}Blower; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud; Randy{nbsp}Horne; Timothy{nbsp}Whiteaker; David{nbsp}Blodgett; Charlie{nbsp}Zender; Daniel{nbsp}Lee -Version 1.7 +Version 1.8 (draft) :doctype: book :pdf-folio-placement: physical :sectanchors: From fc90b42495250651aa182c05353a4ac5f14b8ff5 Mon Sep 17 00:00:00 2001 From: marqh Date: Fri, 19 Jul 2019 16:04:06 +0000 Subject: [PATCH 115/126] update version string to 1.8 (draft) --- cf-conventions.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 1d06ec77..18f066c2 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -1,6 +1,6 @@ = NetCDF Climate and Forecast (CF) Metadata Conventions Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; Jon{nbsp}Blower; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud; Randy{nbsp}Horne; Timothy{nbsp}Whiteaker; David{nbsp}Blodgett; Charlie{nbsp}Zender; Daniel{nbsp}Lee -Version 1.7 +Version 1.8 (draft) :doctype: book :pdf-folio-placement: physical :sectanchors: From 72471d0754aedd6af1d5f6b3fb50501b72b5bec6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Guilherme=20Castel=C3=A3o?= Date: Sun, 21 Jul 2019 18:28:22 -0700 Subject: [PATCH 116/126] Issue #136: Missing dimension trajectory in H.22 Variable trajectory uses dimension trajectory, which was missing. --- apph.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/apph.adoc b/apph.adoc index 907c7b4f..48d3c34e 100644 --- a/apph.adoc +++ b/apph.adoc @@ -1353,6 +1353,7 @@ When the number of profiles and levels for each trajectory varies, one can use a ---- dimensions:    obs = UNLIMITED ; + trajectory = 22 ;    profile = 142 ; variables: From 1b454ed26dd1d9bc9c39956754857bca93e4aa78 Mon Sep 17 00:00:00 2001 From: Rosalyn Hatcher Date: Tue, 23 Jul 2019 09:51:57 +0100 Subject: [PATCH 117/126] Minor corrections - #186 --- appf.adoc | 2 +- ch05.adoc | 4 ++-- ch09.adoc | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/appf.adoc b/appf.adoc index 04568a3a..9cd9ccd5 100644 --- a/appf.adoc +++ b/appf.adoc @@ -467,7 +467,7 @@ to a reference direction. Corresponds to an OGC WKT VERT_DATUM name. This attribute and **`geoid_name`** cannot both be specified. -| **`grid_mapping_name`** | N +| **`grid_mapping_name`** | S | The name used to identify the grid mapping. | **`grid_north_pole_latitude`** | N diff --git a/ch05.adoc b/ch05.adoc index 9ea834e7..624391b5 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -390,11 +390,11 @@ dimensions: variables: double x(x) ; x:standard_name = "projection_x_coordinate" ; - x:long_name = "Easting" + x:long_name = "Easting" ; x:units = "m" ; double y(y) ; y:standard_name = "projection_y_coordinate" ; - y:long_name = "Northing" + y:long_name = "Northing" ; y:units = "m" ; double z(z) ; z:standard_name = "height_above_reference_ellipsoid" ; diff --git a/ch09.adoc b/ch09.adoc index 8fc75a31..6faeffef 100644 --- a/ch09.adoc +++ b/ch09.adoc @@ -488,7 +488,7 @@ Every feature within a Discrete Geometry CF file must be unambiguously associate -Where feasible a variable with the attribute **cf_role** should be included.  The only acceptable values of cf_role for Discrete Geometry CF data sets are **`timeseries_id`** , **`profile_id`** , and **`trajectory_id`** .   The variable carrying the cf_role attribute may have any data type.  When a variable is assigned this attribute, it must provide a unique identifier for each feature instance.   CF files that contain timeSeries, profile or trajectory featureTypes, should include only a single occurrence of a **`cf_role`** attribute;  CF files that contain timeSeriesProfile or trajectoryProfile may contain two occurrences, corresponding to the two levels of structure in these feature types. +Where feasible a coordinate or auxiliary coordinate variable with the attribute **cf_role** should be included.  The only acceptable values of cf_role for Discrete Geometry CF data sets are **`timeseries_id`** , **`profile_id`** , and **`trajectory_id`** .   The variable carrying the cf_role attribute may have any data type.  When a variable is assigned this attribute, it must provide a unique identifier for each feature instance.   CF files that contain timeSeries, profile or trajectory featureTypes, should include only a single occurrence of a **`cf_role`** attribute;  CF files that contain timeSeriesProfile or trajectoryProfile may contain two occurrences, corresponding to the two levels of structure in these feature types. From d256c23bb792e5cdb27afd9b2061477bfe7a2bad Mon Sep 17 00:00:00 2001 From: marqh Date: Wed, 24 Jul 2019 15:17:29 +0000 Subject: [PATCH 118/126] review changes --- ch05.adoc | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/ch05.adoc b/ch05.adoc index 720505f4..15c6350b 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -187,10 +187,8 @@ _This section has been superseded by the treatment of time series as a type of d [[grid-mappings-and-projections, Section 5.6, "Horizontal Coordinate Reference Systems, Grid Mappings, and Projections"]] === Horizontal Coordinate Reference Systems, Grid Mappings, and Projections -A __grid mapping variable__ may be referenced by a data variable: to describe the mapping between the -given coordinate variables and explicit latitude and longitude coordinates (if supplied), to describe the -figure of the Earth used to define the latitude and longitude coordinates, or to describe another -coordinate reference system definition used by some coordinates or auxiliary coordinates. +A __grid mapping variable__ may be referenced by a data variable in order to explicitly declare the coordinate reference system (CRS) used for the spatial coordinate values. +For example, if the spatial coordinates are latitude and longitude, the grid mapping variable can be used to declare the figure of the earth (WGS84 ellipsoid, sphere, etc.) they are based on. If the spatial coordinates are easting, northing, and elevation in a map projection, the grid mapping variable declares the map projection CRS used and provides the information needed to calculate latitude and longitude from easting and northing. When the coordinate variables for a horizontal grid are not longitude and latitude, it is required that further information is provided to geo-locate the horizontal position. A __grid mapping variable__ provides this information. From 7a93c31f1008ab8de595070ec1cbc6b652476764 Mon Sep 17 00:00:00 2001 From: Rosalyn Hatcher Date: Thu, 25 Jul 2019 11:25:29 +0100 Subject: [PATCH 119/126] Make font for cf_role consistent. #189 --- ch09.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch09.adoc b/ch09.adoc index 6faeffef..bb5d17c3 100644 --- a/ch09.adoc +++ b/ch09.adoc @@ -488,7 +488,7 @@ Every feature within a Discrete Geometry CF file must be unambiguously associate -Where feasible a coordinate or auxiliary coordinate variable with the attribute **cf_role** should be included.  The only acceptable values of cf_role for Discrete Geometry CF data sets are **`timeseries_id`** , **`profile_id`** , and **`trajectory_id`** .   The variable carrying the cf_role attribute may have any data type.  When a variable is assigned this attribute, it must provide a unique identifier for each feature instance.   CF files that contain timeSeries, profile or trajectory featureTypes, should include only a single occurrence of a **`cf_role`** attribute;  CF files that contain timeSeriesProfile or trajectoryProfile may contain two occurrences, corresponding to the two levels of structure in these feature types. +Where feasible a coordinate or auxiliary coordinate variable with the attribute **cf_role** should be included.  The only acceptable values of **`cf_role`** for Discrete Geometry CF data sets are **`timeseries_id`** , **`profile_id`** , and **`trajectory_id`** .   The variable carrying the **`cf_role`** attribute may have any data type.  When a variable is assigned this attribute, it must provide a unique identifier for each feature instance.   CF files that contain timeSeries, profile or trajectory featureTypes, should include only a single occurrence of a **`cf_role`** attribute;  CF files that contain timeSeriesProfile or trajectoryProfile may contain two occurrences, corresponding to the two levels of structure in these feature types. From 9ca630e1cc773b63f3bd2724005345fb8f104f2e Mon Sep 17 00:00:00 2001 From: Rosalyn Hatcher Date: Thu, 1 Aug 2019 09:02:46 +0100 Subject: [PATCH 120/126] Additional corrections to Ch5 examples - #186 --- ch05.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ch05.adoc b/ch05.adoc index 624391b5..71a0ea27 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -258,7 +258,7 @@ variables: float rlat(rlat) ; rlat:long_name = "latitude in rotated pole grid" ; rlat:units = "degrees" ; - rlon:standard_name = "grid_latitude"; + rlat:standard_name = "grid_latitude"; float lev(lev) ; lev:long_name = "pressure level" ; lev:units = "hPa" ; @@ -412,10 +412,10 @@ dimensions: temp:coordinates = "lat lon" ; temp:grid_mapping = "crsOSGB: x y crsWGS84: lat lon" ; float pres(z, y, x) ; - temp:standard_name = "air_pressure" ; - temp:units = "Pa" ; - temp:coordinates = "lat lon" ; - temp:grid_mapping = "crsOSGB: x y crsWGS84: lat lon" ; + pres:standard_name = "air_pressure" ; + pres:units = "Pa" ; + pres:coordinates = "lat lon" ; + pres:grid_mapping = "crsOSGB: x y crsWGS84: lat lon" ; int crsOSGB ; crsOSGB:grid_mapping_name = "transverse_mercator"; crsOSGB:semi_major_axis = 6377563.396 ; From 67004e953a0f06ce1708217818f9acc32ab84946 Mon Sep 17 00:00:00 2001 From: marqh Date: Thu, 1 Aug 2019 12:58:25 +0000 Subject: [PATCH 121/126] changes based on review --- ch05.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/ch05.adoc b/ch05.adoc index 15c6350b..af888d6b 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -187,12 +187,12 @@ _This section has been superseded by the treatment of time series as a type of d [[grid-mappings-and-projections, Section 5.6, "Horizontal Coordinate Reference Systems, Grid Mappings, and Projections"]] === Horizontal Coordinate Reference Systems, Grid Mappings, and Projections -A __grid mapping variable__ may be referenced by a data variable in order to explicitly declare the coordinate reference system (CRS) used for the spatial coordinate values. -For example, if the spatial coordinates are latitude and longitude, the grid mapping variable can be used to declare the figure of the earth (WGS84 ellipsoid, sphere, etc.) they are based on. If the spatial coordinates are easting, northing, and elevation in a map projection, the grid mapping variable declares the map projection CRS used and provides the information needed to calculate latitude and longitude from easting and northing. +A __grid mapping variable__ may be referenced by a data variable in order to explicitly declare the coordinate reference system (CRS) used for the hrizontal spatial coordinate values. +For example, if the horizontal spatial coordinates are latitude and longitude, the grid mapping variable can be used to declare the figure of the earth (WGS84 ellipsoid, sphere, etc.) they are based on. If the horizontal spatial coordinates are easting and northing in a map projection, the grid mapping variable declares the map projection CRS used and provides the information needed to calculate latitude and longitude from easting and northing. -When the coordinate variables for a horizontal grid are not longitude and latitude, it is required that further information is provided to geo-locate the horizontal position. A __grid mapping variable__ provides this information. +When the horizontal spatial coordinate variables are not longitude and latitude, it is required that further information is provided to geo-locate the horizontal position. A __grid mapping variable__ provides this information. -If no __grid mapping variable__ is provided and the coordinate variables for a horizontal grid are not longitude and latitude, then it is required that the true latitude and longitude coordinates are supplied via the **`coordinates`** attribute. +If no __grid mapping variable__ is provided and the coordinate variables for a horizontal grid are not longitude and latitude, then it is required that the latitude and longitude coordinates are supplied via the **`coordinates`** attribute. Such coordinates may be provided in addition to the provision of a __grid mapping variable__, but that is not required. A grid mapping variable provides the description of the mapping via a collection of attached attributes. It is of arbitrary type since it contains no data. Its purpose is to act as a container From 29a2d5c2766384fd5fded26a6e6450e9a5a100c4 Mon Sep 17 00:00:00 2001 From: marqh Date: Thu, 1 Aug 2019 13:01:20 +0000 Subject: [PATCH 122/126] ... --- ch05.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ch05.adoc b/ch05.adoc index af888d6b..a03f7233 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -187,7 +187,7 @@ _This section has been superseded by the treatment of time series as a type of d [[grid-mappings-and-projections, Section 5.6, "Horizontal Coordinate Reference Systems, Grid Mappings, and Projections"]] === Horizontal Coordinate Reference Systems, Grid Mappings, and Projections -A __grid mapping variable__ may be referenced by a data variable in order to explicitly declare the coordinate reference system (CRS) used for the hrizontal spatial coordinate values. +A __grid mapping variable__ may be referenced by a data variable in order to explicitly declare the coordinate reference system (CRS) used for the horizontal spatial coordinate values. For example, if the horizontal spatial coordinates are latitude and longitude, the grid mapping variable can be used to declare the figure of the earth (WGS84 ellipsoid, sphere, etc.) they are based on. If the horizontal spatial coordinates are easting and northing in a map projection, the grid mapping variable declares the map projection CRS used and provides the information needed to calculate latitude and longitude from easting and northing. When the horizontal spatial coordinate variables are not longitude and latitude, it is required that further information is provided to geo-locate the horizontal position. A __grid mapping variable__ provides this information. From fbb5541f0cfa526ecbe687a4e6d0307c686f4cea Mon Sep 17 00:00:00 2001 From: Martin Date: Thu, 15 Aug 2019 18:06:50 +0100 Subject: [PATCH 123/126] Clarification of use of standard region names in "region" variables -- new example See #198 --- ch03.adoc | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/ch03.adoc b/ch03.adoc index 8c9de0eb..f814b1b9 100644 --- a/ch03.adoc +++ b/ch03.adoc @@ -6,8 +6,6 @@ The attributes described in this section are used to provide a description of th The **`standard_name`** attribute can be used to identify variables that contain coordinate data. But since it is an optional attribute, applications that implement these standards must continue to be able to identify coordinate types based on the COARDS conventions. - - [[units, Section 3.1, "Units"]] === Units @@ -60,7 +58,8 @@ standard name:: The name used to identify the physical quantity. A standard name canonical units:: Representative units of the physical quantity. Unless it is dimensionless, a variable with a **`standard_name`** attribute must have units which are physically equivalent (not necessarily identical) to the canonical units, possibly modified by an operation specified by the standard name modifier (see below and <>) or by the **`cell_methods`** attribute (see <> and <>) or both. -description:: The description is meant to clarify the qualifiers of the fundamental quantities such as which surface a quantity is defined on or what the flux sign conventions are. We don"t attempt to provide precise definitions of fundumental physical quantities (e.g., temperature) which may be found in the literature. +description:: The description is meant to clarify the qualifiers of the fundamental quantities such as which surface a quantity is defined on or what the flux sign conventions are. We don't attempt to provide precise definitions of fundumental physical quantities (e.g., temperature) which may be found in the literature. +The description may define rules on the variable type, attributes and coordinates which must be complied with by any variable carrying that standard name (such as in example 3.4). When appropriate, the table entry also contains the corresponding GRIB parameter code(s) (from ECMWF and NCEP) and AMIP identifiers. @@ -189,8 +188,26 @@ the flagged conditions may be **`true`**, depending on the variable bit settings. The following example illustrates the use of **`flag_masks`** to express six sensor status conditions. -[[flag-variable-flag-masks-ex]] +[[region-variable-flag-masks-ex]] [caption="Example 3.4. "] +.A region variable, using **`flag_values`** +==== + +---- +int basin(lat, lon); + standard_name: region; + flag_values: 1, 2, 3; + flag_meanings:"atlantic_arctic_ocean indo_pacific_ocean global_ocean"; +data: + basin: 1, 1, 1, 1, 2, ..... ; +---- + +==== + +A variable with standard name of `region`, `area_type` or any other standard name which requires string-valued values from a defined list may use flags together with `flag_values` and `flag_meanings` attributes to record the translation to the string values. Example 3.4 illustrates this using integer flag values for a variable with standard name `region` and `flag_values` selected from the link:$$http://cfconventions.org/Data/cf-standard-names/docs/standardized-region-names.html$$[standardized region names] (see section 6.1.1). + +[[flag-variable-flag-masks-ex]] +[caption="Example 3.5. "] .A flag variable, using **`flag_masks`** ==== @@ -226,7 +243,7 @@ number of status conditions with different **`flag_values`**. The conditions and one enumerated status code. [[flag-variable-flag-masks-flag-values-ex]] -[caption="Example 3.5. "] +[caption="Example 3.6. "] .A flag variable, using **`flag_masks`** and **`flag_values`** ==== @@ -284,3 +301,4 @@ with the variable. When a result is equal to the corresponding **`flag_masks`** enable a simple mechanism for clients to detect all possible conditions. + From 0d1dd669e96630b6806aa3f15c7195ff09fd90e0 Mon Sep 17 00:00:00 2001 From: Martin Date: Thu, 15 Aug 2019 18:07:23 +0100 Subject: [PATCH 124/126] Clarification of use of standard region names in "region" variables -- updating index to examples See #198 --- toc-extra.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/toc-extra.adoc b/toc-extra.adoc index e771a3ef..3f649f1e 100644 --- a/toc-extra.adoc +++ b/toc-extra.adoc @@ -17,7 +17,8 @@ F.1. <> 3.2. <> 3.3. <> 3.4. <> -3.5. <> +3.5. <> +3.6. <> 4.1. <> 4.2. <> 4.3. <> From e560f34669362fbcdf8cefe0347ab28e190a8ab5 Mon Sep 17 00:00:00 2001 From: Martin Date: Wed, 21 Aug 2019 08:56:15 +0100 Subject: [PATCH 125/126] Updates to enhancement issue template Updates to enhancement issue template in line with modified rules on moderation as discussed in cf-convention/cf-conventions#151 --- .github/ISSUE_TEMPLATE/enhancement-issue.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/ISSUE_TEMPLATE/enhancement-issue.md b/.github/ISSUE_TEMPLATE/enhancement-issue.md index def7c13c..4e6c76c7 100644 --- a/.github/ISSUE_TEMPLATE/enhancement-issue.md +++ b/.github/ISSUE_TEMPLATE/enhancement-issue.md @@ -13,6 +13,7 @@ Change proposals should include the following information as applicable. **Title:** Short and descriptive. **Moderator:** @user +**Moderator Status Review [last updated: YY/MM/DD]:** Brief comment on current status, update periodically **Requirement Summary:** A few sentence functional summary **Technical Proposal Summary:** Brief proposal overview **Benefits:** Who or what will benefit from this proposal? From 3593e515311caf23c180097c761fcad58e8a2c0a Mon Sep 17 00:00:00 2001 From: lbdreyer Date: Thu, 22 Aug 2019 17:35:45 +0100 Subject: [PATCH 126/126] Fix Section 7 examples numbering in the list of examples. --- ch07.adoc | 3 ++- toc-extra.adoc | 29 ++++++++++++++++------------- 2 files changed, 18 insertions(+), 14 deletions(-) diff --git a/ch07.adoc b/ch07.adoc index 96cf19f1..bc214fb8 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -17,6 +17,7 @@ If a parametric coordinate variable with a **`formula_terms`** attribute (sectio Whenever a **`formula_terms`** attribute is attached to a boundary variable, the formula terms may additionally be identified using a second method: variables appearing in the vertical coordinates' **`formula_terms`** may be declared to be coordinate, scalar coordinate or auxiliary coordinate variables, and those coordinates may have **`bounds`** attributes that identify their boundary variables. In that case, the **`bounds`** attribute of a formula terms variable must be consistent with the **`formula_terms`** attribute of the boundary variable. Software digesting legacy datasets (constructed prior to version 1.7 of this standard) may have to rely in some cases on the first method of identifying the formula term variables and in other cases, on the second. Starting from version 1.7, however, the first method will be sufficient. +[[specifying-formula_terms-ex]] [caption="Example 7.1. "] .Specifying **`formula_terms`** when a parametric coordinate variable has bounds. ==== @@ -606,7 +607,7 @@ Any data variable can be given a **`geometry`** attribute that indicates the geo One of the dimensions of the data variable must be the number of geometries to which the data applies. As shown in Example 7.15, if the data variable has a discrete sampling geometry, the number of geometries is the length of the instance dimension (Section 9.2). -[["timeseries-with-geometry"]] +[[timeseries-with-geometry]] [caption="Example 7.15. "] .Timeseries with geometry. ==== diff --git a/toc-extra.adoc b/toc-extra.adoc index e771a3ef..fce3b02c 100644 --- a/toc-extra.adoc +++ b/toc-extra.adoc @@ -37,19 +37,22 @@ F.1. <> 5.13. <> 6.1. <> 6.2. <> -7.1. <> -7.2. <> -7.3. <> -7.4. <> -7.5. <> -7.6. <> -7.7. <> -7.8. <> -7.9. <> -7.10. <> -7.11. <> -7.12. <> -7.13. <> +7.1. <> +7.2. <> +7.3. <> +7.4. <> +7.5. <> +7.6. <> +7.7. <> +7.8. <> +7.9. <> +7.10. <> +7.11. <> +7.12. <> +7.13. <> +7.14. <> +7.15. <> +7.16. <> 8.1. <> 8.2. <> B.1. <>