GGML_ASSERT: /build/go-llama/llama.cpp/ggml-cuda.cu:6670: src0->type == GGML_TYPE_F16 9:27AM DBG GRPC(ggml-model-q4_0.gguf-127.0.0.1:33649): stderr SIGABRT: abort

[yourtiger]

LocalAI version:

Environment, CPU architecture, OS, and Version:

Linux iZwz98q2j698koh4a9ht5lZ 3.10.0-1160.92.1.el7.x86_64 #1 SMP Tue Jun 20 11:48:01 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
I download and use quay.io/go-skynet/local-ai:master-cublas-cuda12

Describe the bug

I Use PostMan

I Get Error In LocalAI

To Reproduce

Expected behavior

Logs

Additional context

I entered the container and ran it alone
CGO_ LDFLAGS="- cublas - cubart - L/usr/local/cuda/lib64/" LIBRARY_ PATH=$PWD C_ INCLUDE_ PATH=$PWD go run/ Examples - m "/models/gml model q4_0. gguf" - t 14- ngl 44- n 4096
It works good.

[yourtiger]

@mudler please help me。I almost succeeded

[yourtiger]

the new docker image fix it