You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
GPU support for GPT-J would basically require upstreaming it into llama.cpp. It's the one architecture that we still support using an old implementation from the original ggml repo.
cebtenzzre
changed the title
GPT-J model failing to fallback to CPU
[Feature] GPU support for GPT-J
Jun 13, 2024
Bug Report
When using Mac set to use metal, gpt-j model fails to fallback to CPU.
Steps to Reproduce
Expected Behavior
The gpt-j model has no GPU support should fallback to CPU.
Your Environment
https://huggingface.co/EleutherAI/gpt-j-6b
converted to gguf.Works in windows as expected with Vulkan.
The text was updated successfully, but these errors were encountered: