Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Out of date huggingface-embeddings backend #1783

Closed
TwinFinz opened this issue Mar 1, 2024 · 7 comments
Closed

Out of date huggingface-embeddings backend #1783

TwinFinz opened this issue Mar 1, 2024 · 7 comments
Labels
dependencies enhancement New feature or request roadmap up for grabs Tickets that no-one is currently working on

Comments

@TwinFinz
Copy link
Contributor

TwinFinz commented Mar 1, 2024

LocalAI version:
LocalAI version: c8be839 (c8be839)

Environment, CPU architecture, OS, and Version:
Linux LocalAi 6.5.11-8-pve #1 SMP PREEMPT_DYNAMIC PMX 6.5.11-8 (2024-01-30T12:27Z) x86_64 x86_64 x86_64 GNU/Linux

Describe the bug
Out of date huggingface-embeddings backend cause model to not load/function.

To Reproduce
Use the following yaml to attempt to use "nomic-embed" model.

yaml

name: nomic-embed
backend: huggingface-embeddings
embeddings: true
parameters:
  model: nomic-ai/nomic-embed-text-v1

Expected behavior
Generate and return of embeddings.

Logs
7:52AM DBG Loading Model nomic-ai/nomic-embed-text-v1 with gRPC (file: /build/models/nomic-ai/nomic-embed-text-v1) (backend: huggingface-embeddings): {backendString:huggingface-embeddings model:nomic-ai/nomic-embed-text-v1 threads:4 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc000338000 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh petals:/build/backend/python/petals/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:true parallelRequests:true}
7:52AM DBG Loading external backend: /build/backend/python/sentencetransformers/run.sh
7:52AM DBG Loading GRPC Process: /build/backend/python/sentencetransformers/run.sh
7:52AM DBG GRPC Service for nomic-ai/nomic-embed-text-v1 will be running at: '127.0.0.1:44505'
7:52AM DBG GRPC Service state dir: /tmp/go-processmanager1812659671
7:52AM DBG GRPC Service Started
7:52AM DBG GRPC(nomic-ai/nomic-embed-text-v1-127.0.0.1:44505): stderr Server started. Listening on: 127.0.0.1:44505
7:52AM DBG GRPC Service Ready
7:52AM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:} sizeCache:0 unknownFields:[] Model:nomic-ai/nomic-embed-text-v1 ContextSize:0 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:true NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:4 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/build/models/nomic-ai/nomic-embed-text-v1 Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type:}
7:52AM DBG [WatchDog] Watchdog checks for idle connections
7:52AM DBG GRPC(nomic-ai/nomic-embed-text-v1-127.0.0.1:44505): stderr You try to use a model that was created with version 2.4.0.dev0, however, your version is 2.2.2. This might cause unexpected behavior or errors. In that case, try to update to the latest version.
7:52AM DBG GRPC(nomic-ai/nomic-embed-text-v1-127.0.0.1:44505): stderr
7:52AM DBG GRPC(nomic-ai/nomic-embed-text-v1-127.0.0.1:44505): stderr
7:52AM DBG GRPC(nomic-ai/nomic-embed-text-v1-127.0.0.1:44505): stderr
[192.168.0.106]:7512 500 - POST /embeddings

Additional context
Looks like something just needs to be updated. Should be pretty simple.
@TwinFinz TwinFinz added bug Something isn't working unconfirmed labels Mar 1, 2024
@mudler mudler added enhancement New feature or request up for grabs Tickets that no-one is currently working on roadmap dependencies and removed bug Something isn't working unconfirmed labels Mar 1, 2024
@mudler
Copy link
Owner

mudler commented Mar 1, 2024 •
edited
Loading

would need to be updated here:

@TwinFinz want to take a stab at it in a PR? should be relatively simple

@TwinFinz
Copy link
Contributor Author

TwinFinz commented Mar 1, 2024 via email

@TwinFinz
Copy link
Contributor Author

TwinFinz commented Mar 1, 2024 via email

@TwinFinz
Copy link
Contributor Author

TwinFinz commented Mar 1, 2024 via email

@mudler mudler mentioned this issue Mar 2, 2024
Closed
@TwinFinz TwinFinz mentioned this issue Mar 5, 2024
Merged
@mudler
Copy link
Owner

mudler commented Mar 25, 2024

This is fixed I guess, can we close this @TwinFinz ?

@TwinFinz
Copy link
Contributor Author

TwinFinz commented Mar 25, 2024 via email

@mudler
Copy link
Owner

mudler commented Mar 26, 2024

sounds good, thanks @TwinFinz ! it is also passing tests, so we should be safe too!

@mudler mudler closed this as completed Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
dependencies enhancement New feature or request roadmap up for grabs Tickets that no-one is currently working on
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update backend Requirements/Libraries to latest versions.
2 participants
@mudler @TwinFinz