Feature request: add llama.cpp and openai as backends

[sa-]

This is an awesome project!

Ollama is based on llama.cpp and there are many openai compatible backends as long as the user can choose a different base url.

[ggozad]

Hey Samay!
Thank you :)
This has been proposed in the past, but I am reluctant to go ahead and add as a feature, primarily as it would make maintenance harder. Using the backend would be trivial, it's mostly maintaining compatible configurations options that can be tricky... My time is very limited till August and there are a number of updates that I would like to prioritize first.

I will leave this open though, and if someone makes a PR or sponsors it, I will be happy to help get things done.

[ggozad]

Our recent move to using the Chat API (thanks @yilmaz08 ) makes this a lot easier. I would prioritize open solutions to commercial ones, but if someone wants to put the effort I am happy to discuss and help.