can i use verba via api ?

[michabbb]

instead of using the frontend ??? thanks!

[CarstenHoyer]

It looks like the fetching of context happens through /api/query - but after that the Manager look (ie querying Ollama) happens through a websocket. So I don't think it can be used as an API, but I think you could write a thin middlelayer to do it.

[danielhangan]

You can write custom route that follows the same pattern as websocket_generate_stream route.

async def generate_stream(payload: GeneratePayload):
    chunks, context = manager.retrieve_chunks([payload.query])

    retrieved_chunks_text = [chunk.text for chunk in chunks]

    full_text = ""
    async for chunk in manager.generate_stream_answer(
            [payload.query], retrieved_chunks_text, payload.conversation
        ):
        full_text += chunk["message"]
        if chunk["finish_reason"] == "stop":
            chunk["full_text"] = full_text
        yield f"data: {json.dumps(chunk)}\n\n"
    yield "data: [DONE]\n\n"

@router.post("/completion")
async def completion(payload: GeneratePayload):
    return StreamingResponse(generate_stream(payload), media_type="text/event-stream")

[thomashacker]

Great idea, we're working on an API for this! 🚀