not working with AnythingLLM #1459

35develr · 2023-12-19T16:59:43Z

35develr
Dec 19, 2023

Hello togeter,

i run LocalAi with Mistral7B and successfully attached AnythingLLM, which is responding to chat with Mistra7B.
But when i try to upload a file i get an Error inside LocalAi.

The yaml for the embedding

name: text-embedding-ada-002
backend: sentencetransformers
embeddings: true
parameters:
model: mistral-7b-openorca.Q6_K.gguf

[127.0.0.1]:44650 200 - GET /readyz

[127.0.0.1]:49148 200 - GET /readyz

5:00PM DBG Request received:

5:00PM DBG Parameter Config: &{PredictionOptions:{Model:mistral-7b-openorca.Q6_K.gguf Language: N:0 TopP:0 TopK:0 Temperature:0 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:text-embedding-ada-002 F16:false Threads:4 Debug:true Roles:map[] Embeddings:true Backend:sentencetransformers TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} PromptStrings:[] InputStrings:[Dies ist ein Text.] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:0 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}}

5:00PM DBG Loading model sentencetransformers from mistral-7b-openorca.Q6_K.gguf

5:00PM DBG Model already loaded in memory: mistral-7b-openorca.Q6_K.gguf

[172.23.0.1]:33428 500 - POST /v1/embeddings

HERE IS SEE A 500 ERROR AT POST DATA

[127.0.0.1]:44682 200 - GET /readyz`

How can i fix this?