Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a dedicated counter representing the number of missing contexts #1438

Open
marekhorst opened this issue Oct 12, 2023 · 0 comments
Open

Comments

@marekhorst
Copy link
Member

This is a #1434 follow-up.

We already expose the number of imported contexts, as a counter exported to prometheus, but we might also want to export the number of missing contexts. Currently this fact is simply logged and we decided not to interrupt the job if a context is missing but we could do more in order to make this information more visible just in case missing context is really relevant.

Originally mentioned here: https://support.openaire.eu/issues/9059#note-6:

I was thinking today about something in the middle between a) fatal failure and b) not bothering with anything else than just logging. Some kind of a WARN but more visible. I realized we have this metrics reporting functionality which kind of suits this purpose: after importing phase a first batch of metrics is being exported including the number of imported contexts. We could simply add the number of missing contexts so whenever importing phase is completed we could get to know, e.g. via prometheus/grafana alerting mechanism, that some concept was missing and inspect the situation without the interruption of ongoing workflow. If we assess the missing context is important and we should rerun the IIS importing phase then we could kill the mining job.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant