Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Application Not Using GPU Despite Installing LlamaSharp.Backend.Cuda12 #896

Open
ZCOREP opened this issue Aug 10, 2024 · 1 comment
Open
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers

Comments

@ZCOREP
Copy link

ZCOREP commented Aug 10, 2024

Description

Hello,

I recently downloaded LlamaSharp.Backend.Cuda12 with the expectation that my application would start utilizing my GPU. However, it continues to run on the CPU. I referred to the documentation here:
https://scisharp.github.io/LLamaSharp/0.15.0/FAQ/#why-gpu-is-not-used-when-i-have-installed-cuda

The documentation suggests adding the following line at the beginning of the code:

NativeLibraryConfig.Instance.WithLogs(LLamaLogLevel.Info);

Unfortunately, when I attempt to implement this, I encounter an error indicating that WithLogs method does not exist in NativeLibraryConfig.Instance. Could you please assist me with this issue? What steps should I take to resolve this?

Thank you for your help.

@martindevans
Copy link
Member

That looks like a documentation bug, I think it should be NativeLibraryConfig.All.WithLogs.

@martindevans martindevans added documentation Improvements or additions to documentation good first issue Good for newcomers labels Aug 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants