[User] Missing embedding server endpoint

[yunghoy]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

One of two should be available: (1) Embedding endpoint or (2) Readme Doc to use embedding method in the latest changes.

Current Behavior

The changelogs only shows that embedding mode is removed but does not provide how to use embedding mode in the new code.

Environment and Context

Embedding is a one of critical feature in Llama.cpp. People are using the embedding for Langchain, QA or expert system. How critical it is? The existence of embedding endpoint makes that people can use llama.cpp as a backend service instead of OpenAI or llama-cpp-python (hard to scale out with this amateur approach) in real production. However, the new changelog does not explain how to use the embedding but only state that embedding mode is removed.

Failure Information (for bugs)

No further information about embedding in llama.cpp endpoint is the bug.

Steps to Reproduce

Failure Logs

[SlyEcho]

The way the embedding feature worked made the whole program more complex, it was basically two different servers.

I have to ask you also how you were using the feature, specifically in regards to the points raised in this issue #899?

[yunghoy]

The way the embedding feature worked made the whole program more complex, it was basically two different servers.

I have to ask you also how you were using the feature, specifically in regards to the points raised in this issue #899?

Having or not having two different servers is not the issue. Removing the whole embedding feature is the issue. Never heard removing the code since those have problems although the feature is used. This embedding feature is the feature that leverage Llama.cpp to production level tool. I would say the decision is huge failure if you just removed embeddings, once again, server endpoint, because it makes the architecture complex.

The link you refered is using embedding exe file but not the server endpoint. Never used the embedding file. I'm using embedding endpoint but it's working. Langchain returns QA results properly.