Deploy python model without java router

[epa095]

We want to deploy a large number of models (modelling different things, not different versions of the same thing) using the python wrapper, and in many cases we just want the model, not any transformer or fancy AB routing. In our case we see that the Python model takes roughly 200Mb of memory, while the Java Spring application which is also deployed takes roughly 700 Mb. Is it possible to write a seldondeployment which does not deploy the java application for the cases when we just want to deploy a "naked" model? The relevant part of our seldondeployment is:

predictors: 
  - 
    annotations: 
      predictor_version: "v1"
    componentSpecs: 
      - spec: 
          containers: 
            - image: "..some-docker-image"
              name: "modelserver-{{auto-generated-name}}"
    graph: 
      children: []
      endpoint: 
        type: "REST"
      name: "modelserver-{{auto-generated-name}}"
      type: "MODEL"
    name: "modelserver-{{auto-generated-name}}"
    replicas: 1

[ukclivecox]

I think there are a couple of things here:

• Allowing direct access via the API for single model scenarios like yours where a service orchestrator is not required. This is in our roadmap. We need to conform the APIs internal and external to be the same so they can be used in this way and then allow an annotation to wire things up for the case where no service orchestrator is used.
• For the size of the running images, yes we want to ensure smaller image footprints. You can set the JAVA_OPTS for the engine to decrease the default JVM usage.

[ukclivecox]

Close for now. Please reopen if still relevant.