Prometheus for seldon models

[sasvaritoni]

I am using a Java model wrapped by the Seldon S2I wrapper, with seldon serving component of Kubeflow, basically followed this doc:
https://github.com/kubeflow/examples/blob/master/github_issue_summarization/03_serving_the_model.md

Now I would like to add Prometheus metrics to this model.
Is this supported out of the box by Seldon + Kubeflow?
Or is there some doc describing this scenario?

I found #1036 for TF serving, but could not find this mentioned for Seldon served models.

Thanks in advance!

[ukclivecox]

If you look at https://github.com/kubeflow/example-seldon you will see it installs via Helm the default Prometheus analytics and Grafana dashbaord:

kubectl -n kube-system create sa tiller
kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
helm init --service-account tiller
helm install seldon-core-analytics --name seldon-core-analytics --set grafana_prom_admin_password=password --set persistence.enabled=false --repo https://storage.googleapis.com/seldon-charts --namespace kubeflow

[sasvaritoni]

Great, thank you for pointing out.

So if I understand this correctly, from Kubeflow point of view this can be supported.
From the model / Seldon point of view, when we wrap e.g. a Java model with the Seldon S2I Java wrapper as a Spring Boot application, the requirement on the model app is that it should expose a Prometheus endpoint via the rest interface, so that it can be accessed by the Prometheus service when deployed with Kubeflow. Right?

[ukclivecox]

I think there are two things. Seldon Core's Operator exposes a set of default metrics via prometheus.
On the other hand, for custom metrics you are free do what ever you wish. We have not tested custom metrics with the S2I wrapping process so if for instance you use the python wrapper and REST then the wrapper itself uses Flask so if you needed to tie into that to expose the prometheus endpoint this might need so work. If however you are happy to expose the endpoint using your own HTTP server then this should be possible as long as there are no port conflicts.

Feel free to add further issues on the Seldon Core github repo.

[ukclivecox]

We have added further docs discussing the analytics available in Seldon Core.

[sasvaritoni]

This is really useful, thank you! I was able to get the Dashboard working with this doc.
I also plan to try adding custom metrics later.

One remark: in my cluster the app label was "app: grafana-prom-server" instead of "app: grafana-om-server". I don't know what it depends on.

[ukclivecox]

Thanks for the heads up. That was a typo which is fixed now. It should be as you say: grafana-prom-server