Skip to content

v1.0.3 - Windows
Compare
Choose a tag to compare
@henk717 henk717 released this 23 Mar 20:44
· 5669 commits to concedo since this release
v1.0.3
1166fda

llamacpp-for-kobold-1.0.3

  • Applied the massive refactor from the parent repo. It was a huge pain but I managed to keep the old tokenizer untouched and retained full support for the original model formats.
  • Reduced default batch sizes greatly, as large batch sizes were causing bad output and high memory usage
  • Support dynamic context lengths sent from client.
  • TavernAI is working although I wouldn't recommend it, they spam the server with multiple requests of huge contexts so you're going to have a very painful time getting responses.

To use drag and drop a compatible quantized model for llamacpp on top of the exe.

Assets 3
Loading