Skip to content

Latest commit

 

History

History
63 lines (42 loc) · 4.47 KB

README.md

File metadata and controls

63 lines (42 loc) · 4.47 KB
Raw

Klever Model Registry

English | 中文

Klever Model Registry is a cloud-native model registry that uses Harbor as storage for training models. Users can use Klever Model Registry for model management, model extraction, model conversion and model serving.

Klever Model Registry is an open source project based on Apache 2.0 open source licenses and is a sub-project of Klever Cloud-Native Machine Learning Platform. Klever Model Registry is referred to as Klever in the following paragraphs.

Understanding Klever Model Registry

See the following topics and our API Docs to learn how to use Klever in your project:

Model Management

Users can upload and download their models by calling the API, and Klever will push the model into Harbor through 'ormb'.

ormb is another open-source project of Klever Cloud-Native Machine Learning Platform, which is an OCI(Open Container Initiative)-based registry that aims to distribute models and models' metadata by using the existed image registry. See the project ORMB for more information.

Users can upload the model to Harbor by specifying the project name, model name and the version of the model. To satisfy the ormb specification, the model package must have ormbfile.yaml, in which stores some information about the model, such as frame, format, etc. (We will support generating the ormbfile.yaml automatically in the near future, coming soon!) . Klever has acted as agent for all Harbor requests, project can be created through Klever if there is no Harbor project.

Model Extraction

When the model is pushed to Harbor, Klever will automatically create ModelJob to extract the model. ModelJob is a CRD defined by Klever. Klever will fill the blank of ModelJob.Spec.Extraction based on the format of the model. ModelJob will then generate a Job to execute model conversion. The current model extraction formats supported by Klever are:

  • SavedModel
  • ONNX
  • GraphDef
  • NetDef
  • Keras H5
  • CaffeModel
  • TorchScript
  • MXNetParams
  • PMML

The image of the Job who generated by ModelJob will extract the model and push the updated ormbfile.yaml to Harbor. See the detail code here: extract.

Model Conversion

The current model conversion formats supported by Klever are:

  • MXNetParams converted to ONNX
  • Keras H5 converted to SavedModel
  • CaffeModel converted to NetDef
  • NetDef converted to ONNX

Users can create ModelJob for model conversion by calling the API. The original format and target format of the model will be specified by ModelJob.Spec.Conversion Mmdnn.From and ModelJob.Spec.Conversion.Mmdnn.To. The image of the Job who generated by ModelJob will convert the model and push the updated ormbfile.yaml to Harbor. See the detail code here: convert.

Model Serving

Klever's model serving is based on Seldon-Core. Klever will create a Seldon Deployment when users deploy a model serving. The model will be downloaded in its Init Container via ormb-storage-initializer. If the model's format is PMML, the OpenScoring Image will be used to start the serving pod; If the model format is supported by Triton Server, the Triton Server Image will be used to start the serving pod, in which the image will automatically generate the config.pbtxt file required by Triton Server through the information in ormbfile.yaml.

Contributing to Klever

If you'd like to contribute to the Klever project, see the Klever Contributor's Guide.