[WIP] Kubeflow operator and API #52
Conversation
proposals/kubeflow-operator.md
Outdated
| steps like uploading code or training model. This control plane then could expose API that can be leveraged by various programming languages, CLI, | ||
| plugins etc. | ||
|
|
||
| ## Goals |
There was a problem hiding this comment.
Any user case for this proposal?
proposals/kubeflow-operator.md
Outdated
| Central point to all this would be running http server + service that would provide endpoint to API. Spawning this service would be part of ksonnet apply. | ||
|
|
||
| * Storage backend will be required - whether it's PV-based or S3, service will need persistent file backend for things like code, logs or trained model checkpoints. | ||
| * User would specify backend framework on spawn (multi-backend is tbd). Depending on which backend is chosen, control-service would wrap given framework operator and make decision regarding tooling (like "if tensorflow then monitoring=tensorboard). |
There was a problem hiding this comment.
I have one question here:
- As you know, we defines different kinds of CRDs, then how to implement the logic about specifying the framework? If we implement a new kf-operator, could you add the API in this proposal?
There was a problem hiding this comment.
good question, I'll think about it, also since I failed at github, let's use new PR #62
|
I know this is still WIP but I think making progress on defining common APIs that translate across different frameworks is going to be very difficult. We already have an issue kubeflow/kubeflow#102 just trying to define a common API just for a predict method and that issue has basically gone nowhere. In particular, I don't see any consensus forming between the different frameworks (H20.AI, CMLE, Pipeline.AI, Seldon) about the API for a predict method. Getting those folks to agree on a common API just for Predict is already a major undertaking. This proposal is going way beyond that. For example, arriving at a common understanding of what a model is just as tricky. We already have a variety of formats; e.g.
I think its too early to try to define common APIs because we don't have solid examples illustrating the different use cases. Most of our examples are TensorFlow. We don't even have an XGBoost example yet. Its also not clear to me that the fact that we don't have common APIs is a pain point yet. For example, a common Predict API seems useful if you want to try different models without having to change your client. My suggestion would be to create an E2E example that can be solved with a DL framework and a non DL framework. For example, we could try predicting Zillow prices kubeflow/examples#16 with XGBoost and using a wide model in TF. I think that would help understand how much overlap there is between different frameworks. Also I think an XGBoost example would be hugely impactful. |
|
For model management kartib / model DB look pretty sweet. How much functionality does that give us? I'd love to see model DB incorporated into our E2E examples (kubeflow/examples#81) so that we can get first hand experience of the value add and identify missing features. |
|
Thanks everyone for feedback! Let me incorporate it and add some more thoughts I got over creating PoC for CLI and will publish new version of this. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There were multiple discussions regarding CLI, and one of solutions to
address common use cases would be to create full-fledged API endpoint.
This change is rough first attempt to describe API and architecture of
such service
This change is