GNIP 96: Implement relations between resources

[etj]

GNIP - Implement relations between resources

Overview

At the moment linked resources are only available:

• between dataset and maps
• between documents and datasets
• between documents and maps

We want to create generic references between resources.

Proposed By

• @etj - Emanuele Tajariol

Assigned to Release

This proposal is for GeoNode 4.2.

State

• Under Discussion
• In Progress
• Completed
• Rejected
• Deferred

Motivation

• There is no way to find out which resources are linked to Dashboards or GeoStories
• Possibility to have links between multiple document translations
• ...

Proposal

Requirements

We want to implement a generic model for relating any resource type taking into account the following:

• The client should explicitly list the resources employed inside Dashboards and Geostories when they're saved. This is what we already do for MapLayers: the client includes the MapLayers payload when it saves a map, without the backend having to inspect the blob content
• Users should be able to manually link additional resources to Maps, Dashboards, and Documents, beyond those set automatically by the client

The linked_resources API endpoint will be extended to take into account the new link model.

Proposal

We need a simple "source resource" --> "target resource" relationship between resources, in order to point out any basic coupling between such resources.
At first sight, no other info is needed -- further fields are listed in the "future evolution" section.
We may have two different kinds of relationships:

• internal:
  • resources that are linked by the client's inner logic (layers belonging to a map, documents used by a geoapp, ...)
  • Internal links may only be set using the API, leaving to the external app (or client app) the task to create or delete them.
• external:
  • links created by the user, adding them to the metadata editing page.
  • The metadata editing page will only be able to view, add or delete "external" links.

The viewing pages may have panels listing the resources as:

• "This <resource|dataset|map|...> uses:"
  • list of resources linked by internal links
    • items will have as info
      • type (dataset, map, doc, ...)
      • title
      • link to the resource
• "This <resource|dataset|map|...> references:"
  • list of resources linked by external links
• "This <resource|dataset|map|...> is referenced by:"
  • list of resources that have this resource linked by either internal or external links

Current status

At the moment we have two classes for setting relationships:

• MapLayer
• DocumentResourceLink

API changes

/resources/{id}/linked_resources/

Payload

The resources entry should be replaced by:

• linked_to: very similar to resources, also adding the boolean internal attribute. It contains all the LinkedResource records where the current resource is the source object.
• linked_by: It contains all the LinkedResource records where the current resource is the target object

The old resource entry will be first deprecated and then kept long enough to allow time for the client part to use the new data.

Pagination

This entrypoint at the moment is paginated.

Since we'll have two different lists, the pagination would be confusing. The idea is to remove the pagination for this call (also, not so many linked resources are expected -- 640kresources ought to be enough for anybody ;) )
Also, the client at the moment is requesting page=1&page_size=99999, so pagination is not really used.

The pagination could be removed right away -- maybe by placing fake values in the items if they are needed to exist.

Filtering

Since we do expect few linked resources, filtering could be done client side when/if needed.

Example

See sample output in #11494 (comment)

Backwards Compatibility

Documents extents

Reading comments from older commits, it seems that a signal occurred when a map was saved, in order to update the extents of the documents linked to it.
However, it seems to be a dead feature, since the logic supposedly implementing it does nothing.
We're not going to re-implement this feature since the dependency graph among the various resources may have loops that may cause infinite signals loop

Class DocumentResourceLink

Declare its Backwards Compatibility.

Future evolution

• Relations info can be augmented with
  • Notes
  • Relationship type
  • client_owner: a string representing the owner of the internal relation; for instance when a GeoStory references a dataset, the LinkedResource record would have internal=true and client_owner='mapstore'

Feedback

Update this section with relevant feedbacks, if any.

Voting

Project Steering Committee:

• Alessio Fabiani: 👍
• Francesco Bartoli:
• Giovanni Allegri: +1
• Simone Dalmasso:
• Toni Schoenbuchner:
• Florian Hoedt: 👍

Links

Remove unused links below.

• Email Discussion
• Pull Request
• Mail Discussion
• Linked Issue

[mwallschlaeger]

Hey @etj i really like your idea on this topic. I like to add the idea to have relationships between datasets themself also.

[afabiani]

+1

[etj]

@mwallschlaeger datasets are resources, so this GNIP is including such relationships as well.

[etj]

WIP

Rationale

Basically we want a simple "source resource" --> "target resource" relationship between resources, in order to point out any basic coupling between such resources.
At first sight, no other info are needed -- maybe in the future a note about the relation may be added, but it's out of scope right know.
We may have two different kinds of relation:

• internal:
  • resources that are linked by the client inner logic (layers belonging to a map, documents used by a geoapp, ...)
  • Internal links may only be set using the API, leaving to the external app (or client app) the task to create or delete them.
• external:
  • links created by the user, adding them in the metadata editing page.
  • The metadata editing page will only be able to view, add or delete "external" links.

The viewing pages may have panels listing the resources as:

• "This <resource|dataset|map|...> uses:"
  • list of resources linked by internal links
    • items will have as info
      • type (dataset, map, doc, ...)
      • title
      • link to the resource
• "This <resource|dataset|map|...> references:"
  • list of resources linked by external links
• "This <resource|dataset|map|...> is referenced by:"
  • list of resources that have this resources linked by either internal or external links

Current status

At the moment we have two classes for setting relationships:

• MapLayer
• DocumentResourceLink

[gannebamm]

I would like to have a list of relationship types. Something like those from DataCite: https://www.dublincore.org/specifications/dublin-core/rdf-relation-types/

What do you think?

[giohappy]

@gannebamm at the moment the idea is just to implement the simple relationship model. We can extend it to include additional properties in the future, but this will require also extending the editor. This is something I wouldn't do until the metadata editor has not been migrated to the new client. I have almost completed the design of a proposal for the new metadata editor, but no timeline at the moment since.

[etj]

@gannebamm added "Relationship type" in "Future evolution" section.

[etj]

Sample output for http://localhost:8000/api/v2/resources/5/linked_resource?page=1

There are 3 links:

• 5 --> 1
• 5 --> 5
• 6 --> 5

{
    "WARNINGS": {
        "DEPRECATION": "'resources' field is deprecated, please use 'linked_to'",
        "PAGINATION": "Pagination is not supported on this call"
    },
    "resources": [
        {
            "pk": 1,
            "title": "pat_po",
            "resource_type": "dataset",
            "detail_url": "/catalogue/#/dataset/1",
            "thumbnail_url": "http://localhost/uploaded/thumbs/dataset-7fff231c-97ae-4d45-9299-c6286c2dc13f-thumb-91b8b715-8210-4ce8-ab94-dd61359320f4.jpg",
        },
        {
            "pk": 5,
            "title": "URHIA_500x200.jpg",
            "resource_type": "document",
            "detail_url": "/catalogue/#/document/5",
            "thumbnail_url": "http://localhost:8000/uploaded/thumbs/document-87dad54c-7e2c-49ca-9269-e0d73993f0a9-thumb-e905e5ec-7b22-44cc-be40-0e13181e1f98.jpg",
        }
    ],
    "linked_to": [
        {
            "internal": false,
            "pk": 1,
            "title": "pat_po",
            "resource_type": "dataset",
            "detail_url": "/catalogue/#/dataset/1",
            "thumbnail_url": "http://localhost/uploaded/thumbs/dataset-7fff231c-97ae-4d45-9299-c6286c2dc13f-thumb-91b8b715-8210-4ce8-ab94-dd61359320f4.jpg"
        },
        {
            "internal": false,
            "pk": 5,
            "title": "URHIA_500x200.jpg",
            "resource_type": "document",
            "detail_url": "/catalogue/#/document/5",
            "thumbnail_url": "http://localhost:8000/uploaded/thumbs/document-87dad54c-7e2c-49ca-9269-e0d73993f0a9-thumb-e905e5ec-7b22-44cc-be40-0e13181e1f98.jpg"
        }
    ],
    "linked_by": [
        {
            "internal": false,
            "pk": 5,
            "title": "URHIA_500x200.jpg",
            "resource_type": "document",
            "detail_url": "/catalogue/#/document/5",
            "thumbnail_url": "http://localhost:8000/uploaded/thumbs/document-87dad54c-7e2c-49ca-9269-e0d73993f0a9-thumb-e905e5ec-7b22-44cc-be40-0e13181e1f98.jpg"
        },
        {
            "internal": false,
            "pk": 6,
            "title": "AADA.png",
            "resource_type": "document",
            "detail_url": "/catalogue/#/document/6",
            "thumbnail_url": "http://localhost:8000/uploaded/thumbs/document-33bdee0f-b27e-4f08-9b41-3b7fddc5bffd-thumb-728a6ad5-85d4-476c-9e2a-a60577bc2234.jpg"
        }
    ]
}

[giohappy]

@etj the linked_resources endpoint will be also available under the resource specific endpoint, right? E.g. /datasets, /documents, etc.

[etj]

@giohappy

the linked_resources endpoint will be also available under the resource specific endpoint, right?

I don't know if it's really needed now that the base resource exposes the entrypoint.
In a first revision, in order to maintain compatibility with the client I'd leave it as it is.
On the long run, probably the endpoint doesnt need to be replicated for each resource type.

Anyway, any idea why

• resources/ID/linked_resource returns a simplified versione of the resources (id, title, type) , while
• documents/ID/linked_resource returns the full resources attribute set?