add ollama caching

[kaiehrhardt]

https://github.com/substratusai/kubeai/blob/main/api/v1/model_types.go#L24

[nstogner]

Hey Kai, as we discussed over chat, vLLM is typically the go-to for serving concurrent production traffic. Does that work for you, or is ollama caching still important for you?

[kaiehrhardt]

Hey Nick,

Ollama is just easier because you don't need an account with a token and have to join the corresponding models. From my point of view, this makes it easier to get started. So i still think it would make sense to support caching for ollama as well.

[nstogner]

That makes sense. We currently have 2 high priority features that we are focusing on: #132 and #266 ... We can probably fit this feature in after those.

[kaiehrhardt]

Sounds good. Thank you.