Consider adding an example for a dynamic link library

[Calandiel]

The "main" example is very ergonomic when it comes to integrating it as a CLI tool of a larger framework. However, it's either inconvenient when prompting in interactive mode (due to some ecosystems not having a good way of checking if the process is waiting on inputs) or slow (if the process is spawned for each prompt from scratch).
If there was an equivalent dynamic link library exposing a C compatible API, adoption of llama.cpp would be significantly easier.
I did some cursory reading of the codebase and I don't believe this would require a lot of effort to support so I think it may be worth looking into if @ggerganov isn't opposed to the idea.

Assuming this is considered, just to start the discussion on the subject of the exposed API, perhaps something along the lines of: void* ptr = model_load(model_name, ...), model_free(ptr), char* reply = model_prompt(ptr, "user provided prompt").

[ejones]

Hey, I think there is value in a shared library example, but wanted to point out that #1338 should start to address the inconveniences you mentioned. With the updates to sessions in that PR, main can be respawned to continue generation near-instantly, due to restoring the evaluation state.

[Calandiel]

That's fair but I think it's a parallel problem. I'm not trying to restart generation (interactive mode isn't explicitly desired), I'm trying to not interact with the disc when RAM will suffice ^-^
In other words, I meant this parsing of the sentence (note the square brackets):

However, it's either [[inconvenient when prompting in interactive mode (due to some ecosystems not having a good way of checking if the process is waiting on inputs)]] or [[slow (if the process is spawned for each prompt from scratch)]].

[ejones]

Ah, fair enough.