diff --git a/CodeGen/README.md b/CodeGen/README.md index cbf94bbb8..fcf0f3e33 100644 --- a/CodeGen/README.md +++ b/CodeGen/README.md @@ -43,6 +43,8 @@ By default, the LLM model is set to a default value as listed below: [meta-llama/CodeLlama-7b-hf](https://huggingface.co/meta-llama/CodeLlama-7b-hf) is a gated model that requires submitting an access request through Hugging Face. You can replace it with another model. Change the `LLM_MODEL_ID` below for your needs, such as: [Qwen/CodeQwen1.5-7B-Chat](https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat), [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) +If you choose to use `meta-llama/CodeLlama-7b-hf` as LLM model, you will need to visit [here](https://huggingface.co/meta-llama/CodeLlama-7b-hf), click the `Expand to review and access` button to ask for model access. + ### Setup Environment Variable To set up environment variables for deploying ChatQnA services, follow these steps: @@ -136,6 +138,9 @@ Two ways of consuming CodeGen Service: -H 'Content-Type: application/json' ``` -2. (Docker only) If all microservices work well, check the port ${host_ip}:7778, the port may be allocated by other users, you can modify the `compose.yaml`. +2. If you get errors like "aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host xx.xx.xx.xx:8028", check the `tgi service` first. If there is "Cannot access gated repo for url + https://huggingface.co/meta-llama/CodeLlama-7b-hf/resolve/main/config.json." error of `tgi service`, Then you need to ask for model access first. Follow the instruction in the [Required Models](#required-models) section for more information. + +3. (Docker only) If all microservices work well, check the port ${host_ip}:7778, the port may be allocated by other users, you can modify the `compose.yaml`. -3. (Docker only) If you get errors like "The container name is in use", change container name in `compose.yaml`. +4. (Docker only) If you get errors like "The container name is in use", change container name in `compose.yaml`. diff --git a/Translation/docker_compose/intel/cpu/xeon/README.md b/Translation/docker_compose/intel/cpu/xeon/README.md index 306f8e35d..651b20950 100644 --- a/Translation/docker_compose/intel/cpu/xeon/README.md +++ b/Translation/docker_compose/intel/cpu/xeon/README.md @@ -10,9 +10,24 @@ For detailed information about these instance types, you can refer to this [link After launching your instance, you can connect to it using SSH (for Linux instances) or Remote Desktop Protocol (RDP) (for Windows instances). From there, you'll have full access to your Xeon server, allowing you to install, configure, and manage your applications as needed. -## 🚀 Build Docker Images +## 🚀 Prepare Docker Images -First of all, you need to build Docker Images locally and install the python package of it. +For Docker Images, you have two options to prepare them. + +1. Pull the docker images from docker hub. + + - More stable to use. + - Will be automatically downloaded when using docker compose command. + +2. Build the docker images from source. + + - Contain the latest new features. + + - Need to be manually build. + +If you choose to pull docker images form docker hub, skip this section and go to [Start Microservices](#start-microservices) part directly. + +Follow the instructions below to build the docker images from source. ### 1. Build LLM Image @@ -45,7 +60,7 @@ docker build -t opea/translation-ui:latest --build-arg https_proxy=$https_proxy ```bash cd GenAIComps -docker build -t opea/translation-nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/nginx/Dockerfile . +docker build -t opea/nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/nginx/Dockerfile . ``` Then run the command `docker images`, you will have the following Docker Images: @@ -53,7 +68,7 @@ Then run the command `docker images`, you will have the following Docker Images: 1. `opea/llm-tgi:latest` 2. `opea/translation:latest` 3. `opea/translation-ui:latest` -4. `opea/translation-nginx:latest` +4. `opea/nginx:latest` ## 🚀 Start Microservices @@ -101,6 +116,15 @@ Change the `LLM_MODEL_ID` below for your needs. docker compose up -d ``` +> Note: The docker images will be automatically downloaded from `docker hub`: + +```bash +docker pull opea/llm-tgi:latest +docker pull opea/translation:latest +docker pull opea/translation-ui:latest +docker pull opea/nginx:latest +``` + ### Validate Microservices 1. TGI Service diff --git a/Translation/docker_compose/intel/cpu/xeon/compose.yaml b/Translation/docker_compose/intel/cpu/xeon/compose.yaml index e8eafca4f..108a5086d 100644 --- a/Translation/docker_compose/intel/cpu/xeon/compose.yaml +++ b/Translation/docker_compose/intel/cpu/xeon/compose.yaml @@ -66,7 +66,7 @@ services: ipc: host restart: always translation-xeon-nginx-server: - image: ${REGISTRY:-opea}/translation-nginx:${TAG:-latest} + image: ${REGISTRY:-opea}/nginx:${TAG:-latest} container_name: translation-xeon-nginx-server depends_on: - translation-xeon-backend-server diff --git a/Translation/docker_compose/intel/hpu/gaudi/README.md b/Translation/docker_compose/intel/hpu/gaudi/README.md index 9f234496c..a9e807a12 100644 --- a/Translation/docker_compose/intel/hpu/gaudi/README.md +++ b/Translation/docker_compose/intel/hpu/gaudi/README.md @@ -2,9 +2,24 @@ This document outlines the deployment process for a Translation application utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on Intel Gaudi server. The steps include Docker image creation, container deployment via Docker Compose, and service execution to integrate microservices such as We will publish the Docker images to Docker Hub, it will simplify the deployment process for this service. -## 🚀 Build Docker Images +## 🚀 Prepare Docker Images -First of all, you need to build Docker Images locally. This step can be ignored after the Docker images published to Docker hub. +For Docker Images, you have two options to prepare them. + +1. Pull the docker images from docker hub. + + - More stable to use. + - Will be automatically downloaded when using docker compose command. + +2. Build the docker images from source. + + - Contain the latest new features. + + - Need to be manually build. + +If you choose to pull docker images form docker hub, skip to [Start Microservices](#start-microservices) part directly. + +Follow the instructions below to build the docker images from source. ### 1. Build LLM Image @@ -37,7 +52,7 @@ docker build -t opea/translation-ui:latest --build-arg https_proxy=$https_proxy ```bash cd GenAIComps -docker build -t opea/translation-nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/nginx/Dockerfile . +docker build -t opea/nginx:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/nginx/Dockerfile . ``` Then run the command `docker images`, you will have the following four Docker Images: @@ -45,7 +60,7 @@ Then run the command `docker images`, you will have the following four Docker Im 1. `opea/llm-tgi:latest` 2. `opea/translation:latest` 3. `opea/translation-ui:latest` -4. `opea/translation-nginx:latest` +4. `opea/nginx:latest` ## 🚀 Start Microservices @@ -93,6 +108,15 @@ Change the `LLM_MODEL_ID` below for your needs. docker compose up -d ``` +> Note: The docker images will be automatically downloaded from `docker hub`: + +```bash +docker pull opea/llm-tgi:latest +docker pull opea/translation:latest +docker pull opea/translation-ui:latest +docker pull opea/nginx:latest +``` + ### Validate Microservices 1. TGI Service diff --git a/Translation/docker_compose/intel/hpu/gaudi/compose.yaml b/Translation/docker_compose/intel/hpu/gaudi/compose.yaml index 6eefd6492..3d8b0ab47 100644 --- a/Translation/docker_compose/intel/hpu/gaudi/compose.yaml +++ b/Translation/docker_compose/intel/hpu/gaudi/compose.yaml @@ -67,7 +67,7 @@ services: ipc: host restart: always translation-gaudi-nginx-server: - image: ${REGISTRY:-opea}/translation-nginx:${TAG:-latest} + image: ${REGISTRY:-opea}/nginx:${TAG:-latest} container_name: translation-gaudi-nginx-server depends_on: - translation-gaudi-backend-server diff --git a/Translation/docker_image_build/build.yaml b/Translation/docker_image_build/build.yaml index a1562060b..2230a8657 100644 --- a/Translation/docker_image_build/build.yaml +++ b/Translation/docker_image_build/build.yaml @@ -28,4 +28,4 @@ services: context: GenAIComps dockerfile: comps/nginx/Dockerfile extends: translation - image: ${REGISTRY:-opea}/translation-nginx:${TAG:-latest} + image: ${REGISTRY:-opea}/nginx:${TAG:-latest}