Infer gui

A graphical user interface for local language models. It support different local backends:

• Llama.cpp server
• Koboldcpp
• Ollama

This interface uses no hidden magic: everything is explicit. What you see is what you prompt.

Features

• Template editor, model configuration, inference params editor
• Gbnf grammars, multimodal
• Multiple local and remote servers config

Install

With npm

Install:

npm i -g infergui

Run:

infergui

Run a local backend and open localhost:5183 in a browser

With Python

git clone https://github.com/synw/infergui
cd infergui/dist
python3 -m http.server

Run a local backend and open localhost:8000 in a browser

Screenshots

Templates editor

Gbnf grammar editor

Multimodal

Model picker (Ollama only)

Dev mode

Clone and install the dependencies:

npm i

To run in dev mode with watch autoreload:

npm run dev

Run a local backend and open localhost:5173 in a browser

To build and run:

npm run build
npm run local