MLServer

An open source inference server to serve your machine learning models.

⚠️ This is a Work in Progress.

Overview

MLServer aims to provide an easy way to start serving your machine learning models through a REST and gRPC interface, fully compliant with KFServing's V2 Dataplane spec.

You can read more about the goals of this project on the inital design document.

Examples

On the list below, you can find a few examples on how you can leverage mlserver to start serving your machine learning models.

• Serving a scikit-learn model
• Serving a xgboost model
• Serving a custom model
• Multi-Model Serving with multiple frameworks