Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSW / Support pluggable outputSchema #1280

Merged

Conversation

fxprunayre
Copy link
Member

Currently, the CSW implementation only support a fixed set of outputSchema:

For ISO19139 profiles was added a custom value "own" which was allowing to retrieve the record in its original form.

This proposal add to the SchemaPlugin a CSWPlugin interface with a method to return the list of typeNames supported by a standards (cf. CSWPlugin#getCswTypeNames).

When schema plugins are loaded by schema manager the list of available typenames are collected from all loaded plugins.

When querying CSW capabilities, instead of providing a hard coded list of typenames and outputschema, the list is created dynamically based on loaded plugins. For GetRecords, GetRecordsById operation the method to add those parameters are now similar.

      <ows:Parameter xmlns:gfc="http://www.isotc211.org/2005/gfc" name="outputSchema">
        <ows:Value>http://www.opengis.net/cat/csw/2.0.2</ows:Value>
        <ows:Value>http://www.isotc211.org/2005/gfc</ows:Value>
        <ows:Value>http://www.isotc211.org/2005/gmd</ows:Value>
      </ows:Parameter>
      <ows:Parameter xmlns:gfc="http://www.isotc211.org/2005/gfc" name="typeNames">
        <ows:Value>csw:Record</ows:Value>
        <ows:Value>gfc:FC_FeatureCatalogue</ows:Value>
        <ows:Value>gmd:MD_Metadata</ows:Value>
      </ows:Parameter>

Note: As before, typeNames is still not used to subset record search on a specific schema.

When querying CSW records, if an invalid outputSchema is provided, the following errors is returned and indicate supported values:

<?xml version="1.0" encoding="UTF-8"?>
<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd">
  <ows:Exception exceptionCode="InvalidParameterValue" locator="outputSchema">
    <ows:ExceptionText>'aa' schema is not valid. Supported values are [http://www.opengis.net/cat/csw/2.0.2, http://www.isotc211.org/2005/gfc, http://www.isotc211.org/2005/gmd]</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>

Note: csw:Record, csw:IsoRecord and own value are still supported for backward compatibility even if not advertised in the error message.

A schema plugin (eg. ISO19110SchemaPlugin) declare all typeNames and outputSchema using the following:

allTypenames = ImmutableMap.<String, Namespace>builder()
                .put("csw:Record", Namespace.getNamespace("csw", "http://www.opengis.net/cat/csw/2.0.2"))
                .put("gfc:FC_FeatureCatalogue", ISO19110Namespaces.GFC)
                .build();

This means that http://www.opengis.net/cat/csw/2.0.2 or http://www.isotc211.org/2005/gfc can be used to return a feature catalog record. To retrieve the record:

<?xml version="1.0"?>
<csw:GetRecordById xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" 
service="CSW" version="2.0.2" 
outputSchema="http://www.opengis.net/cat/csw/2.0.2">
    <csw:Id>bb7b8035-1fbd-482c-b1dd-0df9ace75e46</csw:Id>
</csw:GetRecordById>
<?xml version="1.0"?>
<csw:GetRecordById xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" 
service="CSW" version="2.0.2" 
outputSchema="http://www.isotc211.org/2005/gfc">
    <csw:Id>bb7b8035-1fbd-482c-b1dd-0df9ace75e46</csw:Id>
</csw:GetRecordById>
  • in ISO19139 using http://www.isotc211.org/2005/gmd is not supported anymore because there is no transformation from FeatureCatlaog to MD_Metadata. In such query, the following exception is returned indicating that no XSL transformation is available:
<?xml version="1.0" encoding="UTF-8"?>
<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd">
  <ows:Exception exceptionCode="NoApplicableCode">
    <ows:ExceptionText>org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: 
code=NoApplicableCode, locator=null, message=Raised exception while getting metadata 
:org.fao.geonet.csw.common.exceptions.InvalidParameterValueEx: code=InvalidParameterValue,
 locator=OutputSchema 'gmd' not supported for metadata with '13217' (iso19110). 
Corresponding XSL transformation '/path/WEB-INF/data/config/schema_plugins/iso19110/present/csw/gmd-summary.xsl'
 does not exist., message=gmd</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>

Same happen when the record is part of a GetRecord response. The record is not transformed and a warning is displayed in the log:

2015-10-30 09:32:32,681 WARN  [jeeves.webapp.csw] - OutputSchema 'gmd' not supported for metadata with '14065' (dublin-core). 
Corresponding XSL transformation '/path/WEB-INF/data/config/schema_plugins/dublin-core/present/csw/gmd-summary.xsl' does not exist. The record will not be returned

To retrieve all records in different schema in a GetRecord response, use http://www.opengis.net/cat/csw/2.0.2 to retrieve Dublin Core record or own to retrieve records in their original form.

The XSL transformation applied based on the resultType is still stored in the /present/csw folder but file are renammed using the prefix of the output schema parameter in order to make it pluggable. eg. ogc-full.xsl is renamed to csw-full.xsl. To plug a new output schema (eg. http://www.isotc211.org/2005/gfc), the plugin only needs to add gfc-full.xsl.

This proposal:

  • increase conformity to the CSW standards,
  • improve pluggability of schema plugins and is required for better support of ISO19115-3,
  • remove the need of the "own" outputSchema hack value.

For schema plugin maintainer, the 2 changes are:

  • add allTypenames in XYZSchemaPlugin class to register supported typenames
  • rename CSW XSLT transformation ogc => csw, iso => gmd

François Prunayre added 5 commits October 30, 2015 07:34
Currently, the CSW implementation only support a fixed set of outputSchema:
* csw:Record
* csw:IsoRecord
* http://www.opengis.net/cat/csw/2.0.2
* http://www.isotc211.org/2005/gmd

For ISO19139 profiles was added a custom value "own" which was allowing to retrieve the record in its original form.

This proposal add to the SchemaPlugin a CSWPlugin interface with a method to return the list of typeNames supported by a standards (cf. CSWPlugin#getCswTypeNames).

When schema plugins are loaded by schema manager the list of available typenames are collected from all loaded plugins.

When querying CSW capabilities, instead of providing a hard coded list of typenames and outputschema, the list is created dynamically based on loaded plugins. For GetRecords, GetRecordsById operation the method to add those parameters are now similar.

```
      <ows:Parameter xmlns:gfc="http://www.isotc211.org/2005/gfc" name="outputSchema">
        <ows:Value>http://www.opengis.net/cat/csw/2.0.2</ows:Value>
        <ows:Value>http://www.isotc211.org/2005/gfc</ows:Value>
        <ows:Value>http://www.isotc211.org/2005/gmd</ows:Value>
      </ows:Parameter>
      <ows:Parameter xmlns:gfc="http://www.isotc211.org/2005/gfc" name="typeNames">
        <ows:Value>csw:Record</ows:Value>
        <ows:Value>gfc:FC_FeatureCatalogue</ows:Value>
        <ows:Value>gmd:MD_Metadata</ows:Value>
      </ows:Parameter>
```

Note: As before, typeNames is still not used to subset record search on a specific schema.

When querying CSW records, if an invalid outputSchema is provided, the following errors is returned and indicate supported values:

```
<?xml version="1.0" encoding="UTF-8"?>
<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd">
  <ows:Exception exceptionCode="InvalidParameterValue" locator="outputSchema">
    <ows:ExceptionText>'aa' schema is not valid. Supported values are [http://www.opengis.net/cat/csw/2.0.2, http://www.isotc211.org/2005/gfc, http://www.isotc211.org/2005/gmd]</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>
```

Note: csw:Record, csw:IsoRecord and own value are still supported for backward compatibility even if not advertised in the error message.

A schema plugin (eg. ISO19110SchemaPlugin) declare all typeNames and outputSchema using the following:
```
allTypenames = ImmutableMap.<String, Namespace>builder()
                .put("csw:Record", Namespace.getNamespace("csw", "http://www.opengis.net/cat/csw/2.0.2"))
                .put("gfc:FC_FeatureCatalogue", ISO19110Namespaces.GFC)
                .build();
```
This means that http://www.opengis.net/cat/csw/2.0.2 or http://www.isotc211.org/2005/gfc can be used to return a feature catalog record. To retrieve the record:

* in dublin core use http://www.opengis.net/cat/csw/2.0.2 (Record, csw:Record is also supported)
```
<?xml version="1.0"?>
<csw:GetRecordById xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" service="CSW" version="2.0.2" outputSchema="http://www.opengis.net/cat/csw/2.0.2">
    <csw:Id>bb7b8035-1fbd-482c-b1dd-0df9ace75e46</csw:Id>
</csw:GetRecordById>
```
* in ISO19110 use http://www.isotc211.org/2005/gfc (own is also supported)
```
<?xml version="1.0"?>
<csw:GetRecordById xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" service="CSW" version="2.0.2" outputSchema="http://www.isotc211.org/2005/gfc">
    <csw:Id>bb7b8035-1fbd-482c-b1dd-0df9ace75e46</csw:Id>
</csw:GetRecordById>
```
* in ISO19139 using http://www.isotc211.org/2005/gmd is not supported anymore because there is no transformation from FeatureCatlaog to MD_Metadata. In such query, the following exception is returned indicating that no XSL transformation is available:
```
<?xml version="1.0" encoding="UTF-8"?>
<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd">
  <ows:Exception exceptionCode="NoApplicableCode">
    <ows:ExceptionText>org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message=Raised exception while getting metadata :org.fao.geonet.csw.common.exceptions.InvalidParameterValueEx: code=InvalidParameterValue, locator=OutputSchema 'gmd' not supported for metadata with '13217' (iso19110). Corresponding XSL transformation '/home/francois/dev/core-geonetwork/web/src/main/webapp/WEB-INF/data/config/schema_plugins/iso19110/present/csw/gmd-summary.xsl' does not exist., message=gmd</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>
```

Same happen when the record is part of a GetRecord response. The record is not transformed and a warning is displayed in the log:
```
2015-10-30 09:32:32,681 WARN  [jeeves.webapp.csw] - OutputSchema 'gmd' not supported for metadata with '14065' (dublin-core). Corresponding XSL transformation '/home/francois/dev/core-geonetwork/web/src/main/webapp/WEB-INF/data/config/schema_plugins/dublin-core/present/csw/gmd-summary.xsl' does not exist. The record will not be returned
```
To retrieve all records in different schema in a GetRecord response, use http://www.opengis.net/cat/csw/2.0.2 to retrieve Dublin Core record or own to retrieve records in their original form.

The XSL transformation applied based on the resultType is still stored in the <schemaplugin>/present/csw folder but file are renammed using the prefix of the output schema parameter in order to make it pluggable. eg. ogc-full.xsl is renamed to csw-full.xsl. To plug a new output schema (eg. http://www.isotc211.org/2005/gfc), the plugin only needs to add gfc-full.xsl.

This proposal:
* increase conformity to the CSW standards,
* improve pluggability of schema plugins and is required for better support of ISO19115-3,
* remove the need of the "own" outputSchema hack value.
Namespace ns = typenames.get(typeName);
String typename = typeName;
// TODO: Schema plugin schema should be published in
// /web/geonetwork/xml/validation/csw/2.0.2 for validation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the implications about this TODO. I understand the schema xsd should be copied manually to that folder? Can be this improved to do it automatically?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was not done previously but I added a comment as it should probably be done if we would like better support of the DescribeRecord operation with TypeName (which I've never used and does not look much used in any case). This request should return the XSD for the schema. I'm not sure we should really implement it for schema plugin, so I added the TODO.

See CSW request example: https://github.com/geonetwork/core-geonetwork/blob/3.0.x/web/src/main/webapp/xml/csw/test/csw-DescribeRecordWithMD_Metadata.xml

@fxprunayre fxprunayre self-assigned this Nov 6, 2015
fxprunayre pushed a commit that referenced this pull request Nov 6, 2015
…tputschema

CSW / Support pluggable outputSchema
@fxprunayre fxprunayre merged commit 7124865 into geonetwork:3.0.x Nov 6, 2015
fxprunayre pushed a commit to metadata101/iso19115-3 that referenced this pull request Nov 10, 2015
josegar74 added a commit to OpenBfS/core-geonetwork that referenced this pull request Mar 30, 2016
mlechner pushed a commit to OpenBfS/core-geonetwork that referenced this pull request Oct 11, 2016
Conflicts:
	schemas/iso19139.bfs/pom.xml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants