v1.46.8

What's Changed

• [Feat] Add proxy level prometheus metrics by @ishaan-jaff in #5789
• [ Proxy - User Management]: If user assigned to a team don't show Default Team by @ishaan-jaff in #5791
• [Feat] Add Error Handling for /key/list endpoint by @ishaan-jaff in #5787

Full Changelog: v1.46.7...v1.46.8

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	127.01951292939152	6.392058808295182	0.0	1912	0	87.00463700006367	2803.938766999977
Aggregated	Passed ✅	110.0	127.01951292939152	6.392058808295182	0.0	1912	0	87.00463700006367	2803.938766999977

v1.46.7

What's Changed

• feat(prometheus_api.py): support querying prometheus metrics for all-up + key-level spend on UI by @krrishdholakia in #5782
• [Fix-Bedrock] use Bedrock converse for "meta.llama3-8b-instruct-v1:0", "meta.llama3-70b-instruct-v1:0" by @ishaan-jaff in #5729
• [Feat] add Groq gemma2 9b pricing by @ishaan-jaff in #5788
• LiteLLM Minor Fixes & Improvements (09/18/2024) by @krrishdholakia in #5772
• [Feat] Add Azure gpt-35-turbo-0301 pricing by @ishaan-jaff in #5790
• test: replace gpt-3.5-turbo-0613 (deprecated model) by @krrishdholakia in #5794
• [Chore-Docs] fix curl on /get team info swagger by @ishaan-jaff in #5792

Full Changelog: v1.46.6...v1.46.7

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.7

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	150.0	168.9139116122553	6.325020266340649	0.0	1893	0	116.5782520000107	1552.0026590000384
Aggregated	Passed ✅	150.0	168.9139116122553	6.325020266340649	0.0	1893	0	116.5782520000107	1552.0026590000384

v1.46.6

What's Changed

• [Feat - GCS Bucket Logger] Use StandardLoggingPayload by @ishaan-jaff in #5771
• [Prometheus] track requested model by @ishaan-jaff in #5774
• [Feat-Proxy] Add Azure Assistants API - Create Assistant, Delete Assistant Support by @ishaan-jaff in #5777
• [Chore LiteLLM Proxy] enforce prometheus metrics as enterprise feature by @ishaan-jaff in #5769
• [Chore-Proxy] enforce jwt auth as enterprise feature by @ishaan-jaff in #5770

Full Changelog: v1.46.5...v1.46.6

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.6

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	150.0	170.55362281462155	6.40033081425939	0.0	1915	0	115.48961699998017	1217.0262289999982
Aggregated	Passed ✅	150.0	170.55362281462155	6.40033081425939	0.0	1915	0	115.48961699998017	1217.0262289999982

v1.46.5

What's Changed

• LiteLLM Minor Fixes & Improvements (09/17/2024) by @krrishdholakia in #5742
• Additional Fixes (09/17/2024) by @krrishdholakia in #5759

Full Changelog: v1.46.4...v1.46.5

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.5

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	120.0	144.0172449244393	6.411408280375216	0.0	1919	0	89.36769899992214	4350.86144600001
Aggregated	Passed ✅	120.0	144.0172449244393	6.411408280375216	0.0	1919	0	89.36769899992214	4350.86144600001

v1.46.4

What's Changed

• Bump next from 14.1.1 to 14.2.10 in /ui/litellm-dashboard by @dependabot in #5753
• [Fix] o1-mini causes pydantic warnings on reasoning_tokens by @ishaan-jaff in #5754
• [Feat-Proxy-DataDog] Log Redis, Postgres Failure events on DataDog by @ishaan-jaff in #5750
• [Fix] Router/ Proxy - Tag Based routing, raise correct error when no deployments found and tag filtering is on by @ishaan-jaff in #5745
• [Feat] Log Request metadata on gcs bucket logging by @ishaan-jaff in #5743

Full Changelog: v1.46.2...v1.46.4

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	140.0	162.22738145044985	6.304508753690671	0.0	1887	0	111.54934299997876	2507.1398680000243
Aggregated	Passed ✅	140.0	162.22738145044985	6.304508753690671	0.0	1887	0	111.54934299997876	2507.1398680000243

v1.46.2

What's Changed

• LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723) by @krrishdholakia in #5731
• [Fix-Proxy] deal with case when check view exists returns None by @ishaan-jaff in #5740
• Revert "[Fix-Proxy] deal with case when check view exists returns None " by @ishaan-jaff in #5741
• Litellm fix router testing by @krrishdholakia in #5748

Full Changelog: v1.46.1...v1.46.2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	125.37750355195692	6.398128484773343	0.0	1915	0	86.75882299996829	2125.433599999951
Aggregated	Passed ✅	110.0	125.37750355195692	6.398128484773343	0.0	1915	0	86.75882299996829	2125.433599999951

v1.46.1.dev2

What's Changed

• LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723) by @krrishdholakia in #5731
• [Fix-Proxy] deal with case when check view exists returns None by @ishaan-jaff in #5740
• Revert "[Fix-Proxy] deal with case when check view exists returns None " by @ishaan-jaff in #5741

Full Changelog: v1.46.1...v1.46.1.dev2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.1.dev2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	127.44153038969152	6.481679062555461	0.0	1940	0	86.10109800002874	2104.9686730000303
Aggregated	Passed ✅	110.0	127.44153038969152	6.481679062555461	0.0	1940	0	86.10109800002874	2104.9686730000303

v1.46.1.dev1

What's Changed

• LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723) by @krrishdholakia in #5731

Full Changelog: v1.46.1...v1.46.1.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.1.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	110.0	134.6393781724867	6.415203660604424	0.0	1919	0	89.29896299997608	3319.4228009999733
Aggregated	Passed ✅	110.0	134.6393781724867	6.415203660604424	0.0	1919	0	89.29896299997608	3319.4228009999733

v1.46.1

What's Changed

• Litellm stable dev by @krrishdholakia in #5711
• (models): Enable JSON Schema Support for Gemini 1.5 Flash Models by @F1bos in #5708
• Add unsupported o1 params by @Manouchehri in #5722
• Warning fix for Pydantic 2.0 (#5679) by @ishaan-jaff in #5707
• [Feat-Proxy] Add upperbound key duration param by @ishaan-jaff in #5727
• [Fix-Proxy] log exceptions from azure key vault on verbose_logger.exceptions by @ishaan-jaff in #5719
• [Fix-Proxy] Azure Key Management - Secret Manager by @ishaan-jaff in #5728
• [Feat-Proxy] Slack Alerting - allow using os.environ/ vars for alert to webhook url by @ishaan-jaff in #5726

Full Changelog: v1.46.0...v1.46.1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	150.0	170.7951071671865	6.417749524099106	0.0	1920	0	115.48751200001561	2641.8766410000103
Aggregated	Passed ✅	150.0	170.7951071671865	6.417749524099106	0.0	1920	0	115.48751200001561	2641.8766410000103

v1.46.0.dev1

What's Changed

• Litellm stable dev by @krrishdholakia in #5711
• (models): Enable JSON Schema Support for Gemini 1.5 Flash Models by @F1bos in #5708

Full Changelog: v1.46.0...v1.46.0.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.46.0.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	130.0	150.91571355091406	6.398594313787751	0.0	1915	0	102.77270100004898	1716.2696340000139
Aggregated	Passed ✅	130.0	150.91571355091406	6.398594313787751	0.0	1915	0	102.77270100004898	1716.2696340000139