Ogcapi records - better catalog objects #1

davidblasby · 2024-09-17T19:53:02Z

This improves JSON output for the landing page (/), /collections, and /collections/<collectionid>.

In the output of these I inject the OGCAPI Records catalog.yaml object.

A GN Portal (full portal or sub-portal) is an OGCAPI Catalog.

Getting info

This will retrieve the GN ServiceRecord that's linked to a GN Source DB row. This can be done in the GN GUI: Admin Console-Settings-sources for sub-portals.

There is no GUI to set this for the "main" portal - do this in GN GUI: Admin Console-Settings-Settings-Catalog Service for the Web (CSW)-Record to use for GetCapabilities (system/csw/capabilityRecordUuid).

Basically, this PR will retrieve this linked record via the Elastic Index (JSON). This is processed in GN via the index-utils.xsl.

If there isn't a linked service record, then the catalog details will not be added (i.e. unchanged from how GN ogcapi records used to work).

New Model Objects

I have put in a bunch of new model objects - OgcApi*. These are 1:1 with the OGCAPI object model - defined in the spec as well as the .yaml files. I've copy-and-pasted in the documentation from the .yaml files for readability.

NOTE: I also moved the old Link object to OgcApiLink for consistency.

Landing page (/)

The landing page JSON also includes a new property systemInfo which contains a catalog.yaml object in it.
I'm not sure if this is allowed in the spec, but it allows for more metadata about the entire GN system ("ogcapi-records" server). This is useful for making a nicer landing page (cf. pygeoapi's landing page).

A few details

Most of the work is done in ElasticIndexJson2CollectionInfo - this converts the Elastic Index JSON object to the ogcapi catalog.yaml object. There's also a simple test case in ElasticIndexJson2CollectionInfoTest.

Example

Underlying XML linked Service Record (xml)

9bac358b-11ec-4293-aeef-5a077b778412.txt

Elastic Index (JSON):

elastic-index-record.txt

OGCAPI catalog.yaml output (JSON):
ogcpai-catalog.json

josegar74 · 2024-09-18T10:03:20Z

...-records/src/main/java/org/fao/geonet/ogcapi/records/controller/CollectionApiController.java

            .buildFromSource(source, language, requestBaseUrl,
-                configuration.getFormat(mediaType), configuration);
+                configuration.getFormat(mediaType), configuration,request);


Suggested change

configuration.getFormat(mediaType), configuration,request);

configuration.getFormat(mediaType), configuration, request);

Thank for the review, jose. I've run the code through an intellij reformat "cleanup". The first time I did this, it changed my javadocs and the checkstyle got very upset. However, I just re-ran it and it didn't make changes - all these types of problems should be fixed now.

josegar74 · 2024-09-18T10:21:17Z

...vices/ogc-api-records/src/main/java/org/fao/geonet/ogcapi/records/controller/model/Root.java

+  @JsonProperty("systemInfo")
+  @JacksonXmlProperty(localName = "systemInfo")
+  @JsonInclude(Include.NON_EMPTY)
+  private CollectionInfo systemInfo;


I can't find this property in https://docs.ogc.org/is/17-069r3/17-069r3.html#_api_landing_page, is this a custom extension?

Yes - I mentioned it, above, in the PR description. I've added java doc to make it more explicit in the code.

josegar74 · 2024-09-18T10:27:08Z

...s/services/ogc-api-records/src/main/java/org/fao/geonet/ogcapi/records/model/OgcApiLink.java

+  @JsonInclude(Include.NON_EMPTY)
+  @XmlElementWrapper(name = "href")
+  @XmlElement(name = "href")
+  private String href;


This property is not defined in https://github.com/opengeospatial/ogcapi-records/blob/master/core/openapi/schemas/linkBase.yaml, but looks odd that is missing in the specification.

yes - this looks like a mistake. I've put some comments in the code about this - and a link to another specification of link.

josegar74 · 2024-09-18T10:30:10Z

...es/ogc-api-records/src/main/java/org/fao/geonet/ogcapi/records/controller/model/CrsEnum.java

@@ -0,0 +1,42 @@
+package org.fao.geonet.ogcapi.records.controller.model;


Add file header

done. NOTE: most of the files don't have the header.

josegar74 · 2024-09-18T10:32:00Z

...les/services/ogc-api-records/src/main/java/org/fao/geonet/ogcapi/records/util/JsonUtils.java

+
+public class JsonUtils {
+  /**
+   * todo - be language aware (send in desired language).  Move to utility class.


I see a TODO. Is this required to be implemented?

I think its ok at the moment - I'll have to see if I can find some multi-lingual iso19193 files.

josegar74 · 2024-09-18T11:05:06Z

...api-records/src/main/java/org/fao/geonet/ogcapi/records/controller/model/CollectionInfo.java

  private List<String> crs = null;

+  /** A list of contacts qualified by their role(s) in association


It's missing recordLanguages, but not sure how relevant it is. See https://docs.ogc.org/DRAFTS/20-004.html#collection-properties-table

I will add this. I also see that there are a few others in the written spec versus whats in the .yml. Thanks for noticing this. It was on my todo list of things to check.

I added the extra objects - the "Schema" object is VERY VERY complex. I didn't fully include it (because of the pattern properties). I've left TODOs for when this is necessary (its not at the moment). I notice that pygeoapi seems to include this.

josegar74 · 2024-09-18T11:07:42Z

...ecords/src/main/java/org/fao/geonet/ogcapi/records/controller/CapabilitiesApiController.java

+  CollectionInfoBuilder collectionInfoBuilder;
+
+  @Autowired
+  RecordService recordService;


It seems not used.

davidblasby · 2024-09-18T23:15:12Z

Added some documentation - modelled off docs/ in GN core.

david-blasby added 5 commits September 17, 2024 09:21

adding ogcapi 'catalog.yaml' output for catalogs

0005bce

cleaning up code and documenting the ogc .yaml objects

8496c6e

refactor to use OgcApi* objects

f8e9ce6

refactor Elastic Index JSON parsing to its own class

9909a0e

add test case for ElasticIndexJson2CollectionInfo

6623ea7

davidblasby changed the title ~~Ogcapi records~~ Ogcapi records - better catalog objects Sep 17, 2024

josegar74 reviewed Sep 18, 2024

View reviewed changes

david-blasby added 2 commits September 18, 2024 11:36

fixes from joses review

dd1db6e

added documentation

acfc284

davidblasby mentioned this pull request Sep 23, 2024

Ogcapi records - better "catalog.yaml" objects geonetwork/geonetwork-microservices#115

Merged

david-blasby added 12 commits September 25, 2024 10:11

CORS - allow all connections

1014da7

add link to collection (catalog or sub-portal) for its icon

734a97d

minor formating changes

dbf1eb1

better elastic query

dad79a6

initial queryables implementation

51fbde4

add hash/idx to Link (as per jose)

39136d8

refactor to use the parsed elastic index model

4f4017e

add hash/idx to Link (as per jose)

5304f8b

Merge remote-tracking branch 'db/_ogcapi_records' into _ogcapi_records

fc6d900

handle more types of objects

77b988f

test case fix

db02e4f

fix for test cases

7b9f4df

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ogcapi records - better catalog objects #1

Ogcapi records - better catalog objects #1

davidblasby commented Sep 17, 2024 •

edited

Loading

josegar74 Sep 18, 2024

davidblasby Sep 18, 2024

josegar74 Sep 18, 2024

davidblasby Sep 18, 2024

josegar74 Sep 18, 2024

davidblasby Sep 18, 2024

josegar74 Sep 18, 2024

davidblasby Sep 18, 2024

josegar74 Sep 18, 2024

davidblasby Sep 18, 2024

josegar74 Sep 18, 2024

davidblasby Sep 18, 2024 •

edited

Loading

davidblasby Sep 18, 2024 •

edited

Loading

josegar74 Sep 18, 2024

davidblasby Sep 18, 2024

davidblasby commented Sep 18, 2024

	configuration.getFormat(mediaType), configuration,request);
	configuration.getFormat(mediaType), configuration, request);

		@@ -0,0 +1,42 @@
		package org.fao.geonet.ogcapi.records.controller.model;

		private List<String> crs = null;

		/** A list of contacts qualified by their role(s) in association

Ogcapi records - better catalog objects #1

Are you sure you want to change the base?

Ogcapi records - better catalog objects #1

Conversation

davidblasby commented Sep 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidblasby Sep 18, 2024 • edited Loading

Choose a reason for hiding this comment

davidblasby Sep 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidblasby commented Sep 18, 2024

davidblasby commented Sep 17, 2024 •

edited

Loading

davidblasby Sep 18, 2024 •

edited

Loading

davidblasby Sep 18, 2024 •

edited

Loading