Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

could not load model: rpc error: code = Unavailable desc = error reading from server: EOF #950

Closed
gitwittidbit opened this issue Aug 24, 2023 · 1 comment
Closed

could not load model: rpc error: code = Unavailable desc = error reading from server: EOF #950

gitwittidbit opened this issue Aug 24, 2023 · 1 comment
Assignees
@mudler
Labels
bug Something isn't working

Comments

@gitwittidbit
Copy link

LocalAI version:
Latest

Environment, CPU architecture, OS, and Version:
Linux deb11-local 5.10.0-25-amd64 #1 SMP Debian 5.10.191-1 (2023-08-16) x86_64 GNU/Linux
KVM hosted VM
32GB Ram NVIDIA RTX3090
Docker Version 20
NVidia Container Toolkit

Describe the bug
Running the lunademo curl request from the howtos
Output:
'{"error":{"code":500,"message":"could not load model: rpc error: code = Unavailable desc = error reading from server: EOF","type":""}}
'
Docker Logs: `@@@@@
Skipping rebuild
@@@@@
If you are experiencing issues with the pre-compiled builds, try setting REBUILD=true
If you are still experiencing issues with the build, try setting CMAKE_ARGS and disable the instructions set as needed:
CMAKE_ARGS="-DLLAMA_F16C=OFF -DLLAMA_AVX512=OFF -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF"
see the documentation at: https://localai.io/basics/build/index.html
Note: See also #288
@@@@@
CPU info:
model name : Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq vmx ssse3 cx16 pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cpuid_fault pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust smep erms xsaveopt arat umip md_clear arch_capabilities
CPU: AVX found OK
CPU: no AVX2 found
CPU: no AVX512 found
@@@@@
7:14AM INF Starting LocalAI using 8 threads, with models path: /models
7:14AM INF LocalAI version: 704323b (704323b)
7:14AM DBG Model: lunademo (config: {PredictionOptions:{Model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.2 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:lunademo F16:false Threads:0 Debug:false Roles:map[] Embeddings:false Backend:llama TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0}})
7:14AM DBG Extracting backend assets files to /tmp/localai/backend_data

┌───────────────────────────────────────────────────┐
│ Fiber v2.48.0 │
│ http://127.0.0.1:8080 │
│ (bound on host 0.0.0.0 and port 8080) │
│ │
│ Handlers ............ 59 Processes ........... 1 │
│ Prefork ....... Disabled PID ................ 14 │
└───────────────────────────────────────────────────┘

7:15AM DBG Request received:
7:15AM DBG Configuration read: &{PredictionOptions:{Model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.9 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:lunademo F16:false Threads:8 Debug:true Roles:map[] Embeddings:false Backend:llama TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0}}
7:15AM DBG Parameters: &{PredictionOptions:{Model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.9 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:lunademo F16:false Threads:8 Debug:true Roles:map[] Embeddings:false Backend:llama TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0}}
7:15AM DBG Prompt (before templating): How are you?
7:15AM DBG Template found, input modified to: How are you?

Response:

7:15AM DBG Prompt (after templating): How are you?

Response:

7:15AM DBG Loading model llama from wizardlm-13b-v1.2.ggmlv3.q4_0.bin
7:15AM DBG Loading model in memory from file: /models/wizardlm-13b-v1.2.ggmlv3.q4_0.bin
7:15AM DBG Loading GRPC Model llama: {backendString:llama model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin threads:8 assetDir:/tmp/localai/backend_data context:0xc00003c0b0 gRPCOptions:0xc0002c2a00 externalBackends:map[autogptq:/build/extra/grpc/autogptq/autogptq.py bark:/build/extra/grpc/bark/ttsbark.py diffusers:/build/extra/grpc/diffusers/backend_diffusers.py exllama:/build/extra/grpc/exllama/exllama.py huggingface-embeddings:/build/extra/grpc/huggingface/huggingface.py] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false}
7:15AM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/llama
7:15AM DBG GRPC Service for wizardlm-13b-v1.2.ggmlv3.q4_0.bin will be running at: '127.0.0.1:42467'
7:15AM DBG GRPC Service state dir: /tmp/go-processmanager1225292084
7:15AM DBG GRPC Service Started
rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:42467: connect: connection refused"
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr 2023/08/24 07:15:07 gRPC Server listening at 127.0.0.1:42467
7:15AM DBG GRPC Service Ready
7:15AM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:} sizeCache:0 unknownFields:[] Model:wizardlm-13b-v1.2.ggmlv3.q4_0.bin ContextSize:1024 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:8 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/wizardlm-13b-v1.2.ggmlv3.q4_0.bin Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 Tokenizer:}
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr SIGILL: illegal instruction
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr PC=0x87a09c m=4 sigcode=2
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr signal arrived during cgo execution
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr instruction bytes: 0xc4 0xe3 0x7d 0x39 0x8c 0x24 0x18 0x3 0x0 0x0 0x1 0x66 0x89 0x84 0x24 0x0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 19 [syscall]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.cgocall(0x85c6c0, 0xc0003496c0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/cgocall.go:157 +0x5c fp=0xc000349698 sp=0xc000349660 pc=0x424a9c
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/go-llama%2ecpp._Cfunc_load_model(0x7f71b8000b60, 0x400, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x200, ...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr _cgo_gotypes.go:266 +0x4d fp=0xc0003496c0 sp=0xc000349698 pc=0x852bad
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/go-llama%2ecpp.New({0xc000340090, 0x29}, {0xc00034c140, 0x7, 0x950400?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/go-llama/llama.go:28 +0x297 fp=0xc000349830 sp=0xc0003496c0 pc=0x8533b7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/LocalAI/pkg/backend/llm/llama.(*LLM).Load(0xc00006de90, 0xc00035a000)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/pkg/backend/llm/llama/llama.go:73 +0x8e7 fp=0xc000349960 sp=0xc000349830 pc=0x857ce7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/LocalAI/pkg/grpc.(*server).LoadModel(0xc00006df20, {0xc00035a000?, 0x51f086?}, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/pkg/grpc/server.go:50 +0xf7 fp=0xc000349a10 sp=0xc000349960 pc=0x85a6d7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/LocalAI/pkg/grpc/proto._Backend_LoadModel_Handler({0x9bd280?, 0xc00006df20}, {0xaa01c0, 0xc00030a3c0}, 0xc00034e070, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/pkg/grpc/proto/backend_grpc.pb.go:277 +0x170 fp=0xc000349a68 sp=0xc000349a10 pc=0x8463f0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).processUnaryRPC(0xc00019a1e0, {0xaa2df8, 0xc00033c000}, 0xc000142120, 0xc0001910e0, 0x1156150, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:1360 +0xe23 fp=0xc000349e48 sp=0xc000349a68 pc=0x82e283
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).handleStream(0xc00019a1e0, {0xaa2df8, 0xc00033c000}, 0xc000142120, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:1737 +0xa36 fp=0xc000349f68 sp=0xc000349e48 pc=0x8333d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).serveStreams.func1.1()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:982 +0x98 fp=0xc000349fe0 sp=0xc000349f68 pc=0x82bc58
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000349fe8 sp=0xc000349fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by google.golang.org/grpc.(*Server).serveStreams.func1
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:980 +0x18c
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 1 [IO wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc00018fb68 sp=0xc00018fb48 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.netpollblock(0xc00018fbf8?, 0x42412f?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc00018fba0 sp=0xc00018fb68 pc=0x451017
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.runtime_pollWait(0x7f71c179eef8, 0x72)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc00018fbc0 sp=0xc00018fba0 pc=0x482109
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*pollDesc).wait(0xc0000b0680?, 0x0?, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc00018fbe8 sp=0xc00018fbc0 pc=0x4f1a12
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*pollDesc).waitRead(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*FD).Accept(0xc0000b0680)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_unix.go:614 +0x2bd fp=0xc00018fc90 sp=0xc00018fbe8 pc=0x4f731d
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*netFD).accept(0xc0000b0680)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/fd_unix.go:172 +0x35 fp=0xc00018fd48 sp=0xc00018fc90 pc=0x65b655
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*TCPListener).accept(0xc0000126f0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/tcpsock_posix.go:148 +0x25 fp=0xc00018fd70 sp=0xc00018fd48 pc=0x673ec5
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*TCPListener).Accept(0xc0000126f0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/tcpsock.go:297 +0x3d fp=0xc00018fda0 sp=0xc00018fd70 pc=0x672fbd
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).Serve(0xc00019a1e0, {0xa9fa20?, 0xc0000126f0})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:844 +0x475 fp=0xc00018fee8 sp=0xc00018fda0 pc=0x82a875
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr github.com/go-skynet/LocalAI/pkg/grpc.StartServer({0x7fff590cfc4c?, 0xc000024190?}, {0xaa3960?, 0xc00006de90})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/pkg/grpc/server.go:178 +0x125 fp=0xc00018ff50 sp=0xc00018fee8 pc=0x85c185
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr main.main()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /build/cmd/grpc/llama/main.go:22 +0x85 fp=0xc00018ff80 sp=0xc00018ff50 pc=0x85c2e5
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.main()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:250 +0x207 fp=0xc00018ffe0 sp=0xc00018ff80 pc=0x4582a7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00018ffe8 sp=0xc00018ffe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 2 [force gc (idle)]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000056fb0 sp=0xc000056f90 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goparkunlock(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:387
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.forcegchelper()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:305 +0xb0 fp=0xc000056fe0 sp=0xc000056fb0 pc=0x458510
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000056fe8 sp=0xc000056fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by runtime.init.6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:293 +0x25
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 3 [GC sweep wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000057780 sp=0xc000057760 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goparkunlock(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:387
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.bgsweep(0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgcsweep.go:278 +0x8e fp=0xc0000577c8 sp=0xc000057780 pc=0x4448ce
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gcenable.func1()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgc.go:178 +0x26 fp=0xc0000577e0 sp=0xc0000577c8 pc=0x439b86
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000577e8 sp=0xc0000577e0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by runtime.gcenable
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgc.go:178 +0x6b
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 4 [GC scavenge wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0xc000080000?, 0xa98988?, 0x1?, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000057f70 sp=0xc000057f50 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goparkunlock(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:387
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.(*scavengerState).park(0x11a2fe0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc000057fa0 sp=0xc000057f70 pc=0x4427f3
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.bgscavenge(0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgcscavenge.go:628 +0x45 fp=0xc000057fc8 sp=0xc000057fa0 pc=0x442dc5
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gcenable.func2()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgc.go:179 +0x26 fp=0xc000057fe0 sp=0xc000057fc8 pc=0x439b26
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000057fe8 sp=0xc000057fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by runtime.gcenable
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mgc.go:179 +0xaa
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 5 [finalizer wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x1a0?, 0x11a3500?, 0x60?, 0x78?, 0xc000056770?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000056628 sp=0xc000056608 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.runfinq()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000567e0 sp=0xc000056628 pc=0x438bc7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000567e8 sp=0xc0000567e0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by runtime.createfing
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/mfinal.go:163 +0x45
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 35 [select]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0xc000357f00?, 0x2?, 0xc3?, 0x51?, 0xc000357ed4?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000357d60 sp=0xc000357d40 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.selectgo(0xc000357f00, 0xc000357ed0, 0x67e3e9?, 0x0, 0xc000322000?, 0x1)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/select.go:327 +0x7be fp=0xc000357ea0 sp=0xc000357d60 pc=0x4682be
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0003040a0, 0x1)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/controlbuf.go:418 +0x115 fp=0xc000357f30 sp=0xc000357ea0 pc=0x79fa35
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc00034e000)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/controlbuf.go:552 +0x91 fp=0xc000357f90 sp=0xc000357f30 pc=0x7a01b1
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func2()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:341 +0xda fp=0xc000357fe0 sp=0xc000357f90 pc=0x7b7b9a
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000357fe8 sp=0xc000357fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:338 +0x1bb3
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 36 [select]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0xc000052f70?, 0x4?, 0x10?, 0x0?, 0xc000052ec0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000052d08 sp=0xc000052ce8 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.selectgo(0xc000052f70, 0xc000052eb8, 0x0?, 0x0, 0x0?, 0x1)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/select.go:327 +0x7be fp=0xc000052e48 sp=0xc000052d08 pc=0x4682be
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc00033c000)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:1155 +0x233 fp=0xc000052fc8 sp=0xc000052e48 pc=0x7bf273
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func4()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:344 +0x26 fp=0xc000052fe0 sp=0xc000052fc8 pc=0x7b7a86
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000052fe8 sp=0xc000052fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:344 +0x1bf8
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr goroutine 37 [IO wait]:
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.gopark(0x447a60?, 0xb?, 0x0?, 0x0?, 0x6?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/proc.go:381 +0xd6 fp=0xc000314aa0 sp=0xc000314a80 pc=0x4586d6
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.netpollblock(0x4d5da5?, 0x42412f?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc000314ad8 sp=0xc000314aa0 pc=0x451017
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.runtime_pollWait(0x7f71c179ee08, 0x72)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/netpoll.go:306 +0x89 fp=0xc000314af8 sp=0xc000314ad8 pc=0x482109
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*pollDesc).wait(0xc000308000?, 0xc00031a000?, 0x0)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000314b20 sp=0xc000314af8 pc=0x4f1a12
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*pollDesc).waitRead(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr internal/poll.(*FD).Read(0xc000308000, {0xc00031a000, 0x8000, 0x8000})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc000314bb8 sp=0xc000314b20 pc=0x4f2df9
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*netFD).Read(0xc000308000, {0xc00031a000?, 0x1060100000000?, 0x8?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/fd_posix.go:55 +0x29 fp=0xc000314c00 sp=0xc000314bb8 pc=0x6594c9
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*conn).Read(0xc00030c000, {0xc00031a000?, 0x60?, 0x0?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/net/net.go:183 +0x45 fp=0xc000314c48 sp=0xc000314c00 pc=0x66b005
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr net.(*TCPConn).Read(0x800010601?, {0xc00031a000?, 0x0?, 0xc000314ca8?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr :1 +0x29 fp=0xc000314c78 sp=0xc000314c48 pc=0x67e0e9
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr bufio.(*Reader).Read(0xc000318000, {0xc000332040, 0x9, 0x0?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/bufio/bufio.go:237 +0x1bb fp=0xc000314cb0 sp=0xc000314c78 pc=0x5d2dfb
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr io.ReadAtLeast({0xa9c3e0, 0xc000318000}, {0xc000332040, 0x9, 0x9}, 0x9)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/io/io.go:332 +0x9a fp=0xc000314cf8 sp=0xc000314cb0 pc=0x4cf89a
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr io.ReadFull(...)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/io/io.go:351
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr golang.org/x/net/http2.readFrameHeader({0xc000332040?, 0x9?, 0xc000136048?}, {0xa9c3e0?, 0xc000318000?})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:237 +0x6e fp=0xc000314d48 sp=0xc000314cf8 pc=0x78b20e
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr golang.org/x/net/http2.(*Framer).ReadFrame(0xc000332000)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:498 +0x95 fp=0xc000314df8 sp=0xc000314d48 pc=0x78ba55
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc00033c000, 0x0?, 0x0?)
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:642 +0x167 fp=0xc000314f10 sp=0xc000314df8 pc=0x7baec7
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).serveStreams(0xc00019a1e0, {0xaa2df8?, 0xc00033c000})
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:969 +0x162 fp=0xc000314f80 sp=0xc000314f10 pc=0x82b9a2
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr google.golang.org/grpc.(*Server).handleRawConn.func1()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:912 +0x46 fp=0xc000314fe0 sp=0xc000314f80 pc=0x82b246
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr runtime.goexit()
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000314fe8 sp=0xc000314fe0 pc=0x487561
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr created by google.golang.org/grpc.(*Server).handleRawConn
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr /go/pkg/mod/google.golang.org/[email protected]/server.go:911 +0x185
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rax 0x0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rbx 0xac06e0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rcx 0x7f71c30121a0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rdx 0x7f7215b1e6d8
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rdi 0x7f7215b1e6c8
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rsi 0x7f7215b16e38
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rbp 0x7f71c30122c0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rsp 0x7f71c3011f40
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r8 0x0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r9 0x7f71b8000080
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r10 0xfffffffffffffa9e
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r11 0x7f7215a21990
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r12 0x1
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r13 0x7f71c3012060
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r14 0x7f71c3011ff0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr r15 0x7f71c3012160
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rip 0x87a09c
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr rflags 0x10246
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr cs 0x33
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr fs 0x0
7:15AM DBG GRPC(wizardlm-13b-v1.2.ggmlv3.q4_0.bin-127.0.0.1:42467): stderr gs 0x0
[172.18.0.1]:43586 500 - POST /v1/chat/completions
`

To Reproduce
See before

Expected behavior
Response from the model

Logs
See before

Additional context
I noticed in the logs that it says "SchedulerType: CUDA:false" Not sure whether this means anything.
@gitwittidbit gitwittidbit added the bug Something isn't working label Aug 24, 2023
@mudler
Copy link
Owner

mudler commented Aug 25, 2023

Duplicate of #288

@mudler mudler marked this as a duplicate of #288 Aug 25, 2023
@mudler mudler closed this as completed Aug 25, 2023
@justaCasualCoder justaCasualCoder mentioned this issue Dec 18, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mudler mudler

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mudler @gitwittidbit