Ollama chat endpoint support

[tquartus]

Hello, Andrew,

First off, thanks for your project! I wrote some code over a year ago for interacting with the OpenAI API via Emacs. OpenAI broke that several times with their changes. Recently, I started working with LLMs locally and decided to rewrite everything from scratch to support multiple APIs ... and discovered that, thankfully, someone had beaten me to that! :-)

I noticed that the existing package provided support only for the Ollama /generate endpoint, but not the /chat endpoint. I find the /generate endpoint simpler for one-off requests. But for chat conversations, with multiple interactions, I prefer the /chat endpoint. For that, I can generate a set of interactions from scratch, regenerate responses, or switch between different LLMs in the same conversation. So, I've introduced some changes to support the other endpoint. I believe it's backward compatible.

The main changes include:

1. A new :endpoint slot in the llm-ollama structure, which specifies the API endpoint to use. This slot defaults to generate, but can be set to chat to utilize the /chat endpoint.

2. Adjustments to the JSON encodings for making requests to the API and processing responses from the server. These changes accommodate the differences between the /generate and /chat endpoints. Specifically, separate helper functions have been created to handle the request data for each endpoint.

3. Modifications to the llm-ollama--chat-request function to call the appropriate helper function based on the specified endpoint.

4. Updates to the llm-ollama--url calls in the llm-chat and llm-chat-streaming methods to use the specified endpoint.

5. Updated documentation to reflect the new :endpoint slot in the llm-ollama structure.

Hopefully, support for both endpoints aligns with the package's goal of abstracting functionality to a higher level and concealing API variations.

I look forward to your feedback and the opportunity to contribute to the ongoing development of this package.

Best,
Thomas

[ahyatt]

Thanks for this change! Before I take a look, do you have FSF copyright assignment? This is needed to contribute substantial changes to this project, since it is part of GNU ELPA.

[tquartus]

In process. Emailed FSF just as I sent this in. Hopefully, won't take long. I'm a college professor, but our faculty handbook makes it clear they don't claim copyright over works by faculty. Hoping FSF will accept that without requiring additional signatures. Sorry for the delay. Will let you know when it's settled.

[ahyatt]

No problem, thank you for getting that process started! And without looking at your code, I'd suggest that it's OK to go to the chat endpoint by default. That is typically what we do for other providers.

[tquartus]

My FSF copyright assignment finally came through. Just wanted to let you know.

[ahyatt]

Thank you for these changes and for getting the copyright assignment!

[ahyatt]

You can just use "equal" here, string= is more about comparing without properties.

[ahyatt]

Do you think it makes a difference to use "generate", when we could just use "chat" always? If there's no appreciable difference, I'd prefer not having an option.

[ahyatt]

This is fine, but stylistically the rest of the code prefers to use (llm-ollama-endpoint provider), so can you change this and the other instance of this (assuming you keep the the endpoint slot?)

[tquartus]

Thanks! Based on your suggestions, I've removed support for the /generate endpoint and now only support the more versatile /chat endpoint. This simplifies the code and makes it more straightforward. As a result, the code involving string= and the :endpoint slot became unnecessary, so I've removed them as well, and also removed the added line from the README.org. Please let me know if there's anything else you'd like me to adjust.

[ahyatt]

Thank you for your change!