From 6f7a24e3ad2efb99facfbfe90b53db4b23b2612a Mon Sep 17 00:00:00 2001 From: Steve Zhang Date: Fri, 6 Sep 2024 00:35:58 +0800 Subject: [PATCH] Update the GMC README based on current changes. (#396) * update the GMC README based on current changes. Signed-off-by: zhlsunshine * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change the doc based on comments. Signed-off-by: zhlsunshine --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- microservices-connector/README.md | 103 +++++++++++------------------- 1 file changed, 38 insertions(+), 65 deletions(-) diff --git a/microservices-connector/README.md b/microservices-connector/README.md index e84de975..bce3c373 100644 --- a/microservices-connector/README.md +++ b/microservices-connector/README.md @@ -36,83 +36,64 @@ There are `2` components in this repo: - 1. `manager`: controller manager to handle GMC CRD - 2. `router`: route the traffic among the microservices defined in GMC -#### How to build these binaries? +### GMC build + +#### Binaries building ```sh make build ``` -#### How to build docker images for these 2 components? +#### Docker images building ```sh make docker.build ``` -#### How to delete these components' binaries? +#### Binaries deleting ```sh make clean ``` -### To Deploy on the cluster +### GMC Deployment on K8s cluster -**Build and push your image to the location specified by `CTR_IMG`:** +**GMC Images NOTE:** This image ought to be published in [OPEA docker hub](https://hub.docker.com/u/opea), including [gmcmanager](https://hub.docker.com/r/opea/gmcmanager) and [gmcrouter](https://hub.docker.com/r/opea/gmcrouter). Make sure you have the proper permission to the registry and use the latest images. -```sh -make docker.build docker.push CTR_IMG=/gmcmanager:tag -``` +There are 2 methods for deploying GMC on K8s cluster: -**NOTE:** This image ought to be published in the personal registry you specified. -And it is required to have access to pull the image from the working environment. -Make sure you have the proper permission to the registry if the above commands don’t work. +- Deploy via native kubectl +- Deploy via helm chart -**Install GMC CRD** +#### Deploy using native kubectl -```sh -kubectl apply -f config/crd/bases/gmc.opea.io_gmconnectors.yaml -``` +There are 3 steps for deploying GMC on K8s cluster as below: -**Copy GMC router manifest** +- Install GMC CRD +- Prepare GenAI Components and GMC Router manifests +- Deploy GMC Manager -```sh -cp $(pwd)/config/gmcrouter/gmc-router.yaml -p $(pwd)/config/manifests/ -``` +**Deploy GMC NOTE:** -**NOTE:** -before apply the manifests, please replace your own huggingface tokens in the manifests +- Before installting the manifests, please replace your own huggingface tokens +- `MOUNT_DIR` is the `hostPath` to save the models on you cluster hosts, so please make sure it is ready on every node of the k8s nodes and the microservices have enough permission to access it. +- The `SYSTEM_NAMESPACE` should keep the same with the namespace defined in `gmc-manager.yaml` and `gmc-manager-rbac.yaml` +- The configmap name `gmcyaml` is defined in gmcmanager deployment Spec. Please modify accordingly if you want + use a different name for the configmap ```sh +# Install GMC CRD +kubectl apply -f config/crd/bases/gmc.opea.io_gmconnectors.yaml +# Prepare GenAI Components and GMC Router manifests +cp $(pwd)/config/gmcrouter/gmc-router.yaml -p $(pwd)/config/manifests/ +export YOUR_HF_TOKEN= +export MOUNT_DIR= find . -name '*.yaml' -type f -exec sed -i "s#insert-your-huggingface-token-here#$YOURTOKEN#g" {} \; -``` - -if you have pre-defined directory to save the models on you cluster hosts, please set the path to the manifests - -```sh find . -name '*.yaml' -type f -exec sed -i "s#path: /mnt/opea-models#path: $MOUNT_DIR#g" {} \; -``` - -**Create Namespace for gmcmanager deployment** -\*\*Create Namespace for gmcmanager deployment - -```sh +# Deploy GMC Manager export SYSTEM_NAMESPACE=system kubectl create namespace $SYSTEM_NAMESPACE -``` - -**NOTE:** The `SYSTEM_NAMESPACE` should keep the same with the namespace defined in gmc-manager.yaml and gmc-manager-rbac.yaml. - -**Create ConfigMap for GMC to hold GenAI Components and GMC Router manifests** - -```sh kubectl create configmap gmcyaml -n $SYSTEM_NAMESPACE --from-file $(pwd)/config/manifests -``` - -**NOTE:** The configmap name `gmcyaml' is defined in gmcmanager deployment Spec. Please modify accordingly if you want -use a different name for the configmap. - -**Install GMC manager** - -```sh kubectl apply -f $(pwd)/config/rbac/gmc-manager-rbac.yaml kubectl apply -f $(pwd)/config/manager/gmc-manager.yaml ``` @@ -125,33 +106,25 @@ NAME READY STATUS RESTARTS AGE gmc-controller-78f9c748cb-ltcdv 1/1 Running 0 3m ``` -### Next Step - -Please refer to this [usage_guide](./usage_guide.md) for sample use cases. - -### To Uninstall - -**Delete the instances (CRs) from the cluster:** +**To Uninstall via kubectl** ```sh +# Delete the instances (CRs) from the cluster kubectl delete -k config/samples/ -``` - -**Delete the APIs(CRDs) from the cluster:** - -```sh +# Delete the APIs(CRDs) from the cluster make uninstall +# UnDeploy the controller from the cluster +make undeploy ``` -**UnDeploy the controller from the cluster:** +#### Deploy via helm chart -```sh -make undeploy -``` +Please refer to [helm chart README](./helm/README.md) for deploying GMC using helm chart. -### Deploy using helm chart +### Next Step -Please refer to the [helm chart README](./helm/README.md) for deploying GMC using helm chart. +Please refer to [usage_guide](./usage_guide.md) for sample use cases. +Please refer to [advanced_use_cases](./config/samples/ChatQnA/use_cases.md) for more use cases based on ChatQnA example. ### Troubleshooting guide