Consume mandatees from LMB

[lagartoverde]

Overview

This implements the LDES consumer which takes mandatees created on LMB and adds them to GN, In the corresponding public and private graphs

connected issues and PRs:

GN-4980

Setup

You will need to replace the extra headers env variable, ask me for the auth token.
Also is probably your dev setup of virtuoso doesn't have enough ram, if you are running into out of memory problems increase it

How to test/reproduce

Consume the data, check that now you have mandatees in http://mu.semte.ch/graphs/lmb-data-public, you should be able to find these mandatees in the frontend if you go to the corresponding administrative units

Challenges/uncertainties

Waiting for a Niels answer related to in which graphs should be put de data

Checks PR readiness

• UI: works on smaller screen sizes
• UI: feedback for any loading/error states
• Check cancel/go-back flows
• Check database state correct when deleting/updating (especially regarding relationships)
• changelog
• no new deprecations

[elpoelma]

The mandatees and their related data seem to be ingested well :) Really nice work. They also correctly show up on the frontend.
Some notes:

• I had to use a different virtuoso.ini files (with different query and RAM limits), which I got from @lagartoverde
• We should probably clear the cache after the postprocessing step

I have not yet been able to test what happens when adding a new mandatee to the LMB application.

[elpoelma]

Additionally, you should probably also add the following entry {"variables":[],"name":"lmb-public"} to the DEFAULT_MU_AUTH_ALLOWED_GROUPS_HEADER env var.

[elpoelma]

Tested, with the additional DEFAULT_MU_AUTH_ALLOWED_GROUPS_HEADER:

• It seems that adding a new mandatee to the lmb application seems to more-or-less go through (there still seem to be some issues on the side of lmb). As expected, I do have to restart the cache
• Unfortunately, it also seems the database contains a lot of duplicate triples:
  

[abeforgit]

this now works on my machine
It threw an error earlier today, but I tried again with a fresh qa backup and it ran without issue, so 🤷

[elpoelma]

This now works well on my end (with the beefy virtuoso.ini of course).
For this feature to work correctly a beefier virtuoso.ini is needed, as the service needs to be able to make queries/updates above 10000 triples.

I can correctly see the new mandatees appearing in the frontend.

I did notice there are still two ununused queries:

• moveIdentifiersToPersonStagingQuery
• moveBirthdaysToPersonStagingQuery

I would still omit the identifier property (even from the private graph), as we do not seem to require it.