Home

Proxy Configurations

You can use third party proxies such as OpenRouter, Groq, Fireworks or Together.ai. To use them, define the correlating host, path, API key, model, and if required the url scheme or port in the environment variables. The variables are prefixed as alternatives to OpenAI (openai_alternative_proxy_*), because Ayai expects the returned stream events and errors to mirror the shape of those returned by the OpenAI API.

Example Configurations

Groq

Key	Value
api_key	gsk_mre8eODUpfuh093Fjk9shg0u230fFFjkDopW3R47fFkdukufD8f9s
gpt_model	llama3-70b-8192
url_host	api.groq.com
url_path	/openai/v1/chat/completions
url_port	443
url_scheme	https

OpenRouter

Key	Value
api_key	sk-or-v1-16cb125d4d9501a6f5e01ef0f78f52343ffjasldjfS8e9e8fjcs34
gpt_model	nousresearch/nous-capybara-7b:free
url_host	openrouter.ai
url_path	/api/v1/chat/completions
url_port	443
url_scheme	https

Fireworks

Key	Value
api_key	2nYMLcw22TPl4ImiD2FM5mWSKdyfjk3jkflsYpwefPgxdfox
gpt_model	accounts/fireworks/models/gemma2-9b-it
url_host	api.fireworks.ai
url_path	/inference/v1/chat/completion
url_port	443
url_scheme	https

Together.ai

Key	Value
api_key	c3d7a9013cd83erf78d9fs0e8f9d8feef92394d49482e984ox405fod
gpt_model	Qwen/Qwen2-72B-Instruct
url_host	api.together.xyz
url_path	/v1/chat/completions
url_port	443
url_scheme	https