Make llama_samplers.cpp throw an error/exception on lack of choices during sampling #9952
SteelPh0enix
started this conversation in
Ideas
Replies: 1 comment
-
Open an issue with repro steps and this will get fixed. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I've recently noticed, while testing samplers options in llama.cpp, that current implementation aborts and crashes when sampling code (specifically,
llama_sampler_softmax_impl
in my case) encounters a situation where the length of tokens array is 0.IMO this should not be a critical error of this magnitude, and instead it should exit graciously, with either an exception or some other kind of error-propagation mechanism. However, i'm currently unfamiliar with llama.cpp codebase so i don't know what kind of approach would be preferred, hence i started this discussion.
Beta Was this translation helpful? Give feedback.
All reactions