Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

希望能量化成q4或者q6的形式，并尽快引入Ollama，这样个人也能用了，感谢！ #2

Closed
ricksuzade-maker opened this issue Jan 18, 2024 · 3 comments
Closed

希望能量化成q4或者q6的形式，并尽快引入Ollama，这样个人也能用了，感谢！ #2

ricksuzade-maker opened this issue Jan 18, 2024 · 3 comments

Comments

@ricksuzade-maker
Copy link

目前个人用户购买苹果机来运行大模型，比较可取
@AmintaCCCP
Copy link

+1 希望有gguf的

@jubgjf
Copy link
Collaborator

jubgjf commented Jan 26, 2024

llama.cpp现在已经支持mixtral架构，并且提供了hf->gguf格式转换的教程。chinese-mixtral-8x7b和原版mixtral-8x7b在架构方面没有区别，所以应该可以直接使用

@jubgjf
Copy link
Collaborator

jubgjf commented Feb 9, 2024

我们已发布基于Chinese-Mixtral-8x7B的chat模型：活字3.0，支持llama.cpp，AutoAWQ等量化框架，可以尝试使用

@jubgjf jubgjf closed this as completed Feb 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jubgjf @ricksuzade-maker @AmintaCCCP