-
Notifications
You must be signed in to change notification settings - Fork 192
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update Kubernetes manifest files for deploying ChatQnA
Update Kubernetes manifest files for deploying ChatQnA without GMC. Signed-off-by: Lianhao Lu <[email protected]>
- Loading branch information
Showing
25 changed files
with
2,282 additions
and
901 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
<h1 align="center" id="title">Deploy ChatQnA in Kubernetes Cluster</h1> | ||
|
||
> [NOTE] | ||
> The following values must be set before you can deploy: | ||
> HUGGINGFACEHUB_API_TOKEN | ||
> You can also customize the "MODEL_ID" if needed. | ||
> You need to make sure you have created the directory `/mnt/opea-models` to save the cached model on the node where the ChatQnA workload is running. Otherwise, you need to modify the `chatqna.yaml` file to change the `model-volume` to a directory that exists on the node. | ||
## Deploy On Xeon | ||
|
||
``` | ||
cd GenAIExamples/ChatQnA/kubernetes/manifests/xeon | ||
export HUGGINGFACEHUB_API_TOKEN="YourOwnToken" | ||
sed -i "s/insert-your-huggingface-token-here/${HUGGINGFACEHUB_API_TOKEN}/g" chatqna.yaml | ||
kubectl apply -f chatqna.yaml | ||
``` | ||
|
||
## Deploy On Gaudi | ||
|
||
``` | ||
cd GenAIExamples/ChatQnA/kubernetes/manifests/gaudi | ||
export HUGGINGFACEHUB_API_TOKEN="YourOwnToken" | ||
sed -i "s/insert-your-huggingface-token-here/${HUGGINGFACEHUB_API_TOKEN}/g" chatqna.yaml | ||
kubectl apply -f chatqna.yaml | ||
``` | ||
|
||
## Verify Services | ||
|
||
To verify the installation, run the command `kubectl get pod` to make sure all pods are running. | ||
|
||
Then run the command `kubectl port-forward svc/chatqna 8888:8888` to expose the ChatQnA service for access. | ||
|
||
Open another terminal and run the following command to verify the service if working: | ||
|
||
```console | ||
curl http://localhost:8888/v1/chatqna \ | ||
-H 'Content-Type: application/json' \ | ||
-d '{"messages": "What is the revenue of Nike in 2023?"}' | ||
``` |
45 changes: 0 additions & 45 deletions
45
ChatQnA/kubernetes/manifests/chaqna-xeon-backend-server.yaml
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.