-
Notifications
You must be signed in to change notification settings - Fork 48
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add docsum example on both xeon and gaudi node (#105)
* update codegen example and add docsum example. Signed-off-by: zhlsunshine <[email protected]> * add example of docsum on both xeon and gaudi. Signed-off-by: zhlsunshine <[email protected]> * add e2d test for the example of docsum. Signed-off-by: zhlsunshine <[email protected]> * format the e2e test script. Signed-off-by: zhlsunshine <[email protected]> * go on fixing the e2e script for gaudi. Signed-off-by: zhlsunshine <[email protected]> * fix the e2e error. Signed-off-by: zhlsunshine <[email protected]> * fix e2e test by changing the image format. Signed-off-by: zhlsunshine <[email protected]> * disable the docsum validate first. Signed-off-by: zhlsunshine <[email protected]> * enable xeon and gaudi docsum validate. Signed-off-by: zhlsunshine <[email protected]> * disable docsum example e2e feature. Signed-off-by: zhlsunshine <[email protected]> * enable the docsum e2e test validation. Signed-off-by: zhlsunshine <[email protected]> * update the tgi service yaml template. Signed-off-by: zhlsunshine <[email protected]> * fix e2e error for docsum. Signed-off-by: zhlsunshine <[email protected]> * need to consider the order of yaml file applying. Signed-off-by: zhlsunshine <[email protected]> * revert the tgi service yaml file. Signed-off-by: zhlsunshine <[email protected]> * disable the e2e test. Signed-off-by: zhlsunshine <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Loading branch information
1 parent
95890d2
commit c888170
Showing
9 changed files
with
107 additions
and
63 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
# Copyright (C) 2024 Intel Corporation | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
apiVersion: gmc.opea.io/v1alpha3 | ||
kind: GMConnector | ||
metadata: | ||
labels: | ||
app.kubernetes.io/name: gmconnector | ||
app.kubernetes.io/managed-by: kustomize | ||
gmc/platform: gaudi | ||
name: docsum | ||
namespace: docsum-gaudi | ||
spec: | ||
routerConfig: | ||
name: router | ||
serviceName: router-service | ||
nodes: | ||
root: | ||
routerType: Sequence | ||
steps: | ||
- name: DocSumGaudi | ||
data: $response | ||
internalService: | ||
serviceName: docsum-llm-uservice | ||
config: | ||
endpoint: /v1/chat/docsum | ||
HUGGING_FACE_HUB_TOKEN: <HUGGING_FACE_HUB_TOKEN> | ||
HF_TOKEN: <HF_TOKEN> | ||
PORT: "9009" | ||
- name: TgiGaudi | ||
internalService: | ||
serviceName: tgi-gaudi-svc | ||
config: | ||
LANGCHAIN_TRACING_V2: "false" | ||
LANGCHAIN_PROJECT: "opea-llm-service" | ||
endpoint: /generate | ||
isDownstreamService: true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
# Copyright (C) 2024 Intel Corporation | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
apiVersion: gmc.opea.io/v1alpha3 | ||
kind: GMConnector | ||
metadata: | ||
labels: | ||
app.kubernetes.io/name: gmconnector | ||
app.kubernetes.io/managed-by: kustomize | ||
gmc/platform: xeon | ||
name: docsum | ||
namespace: docsum | ||
spec: | ||
routerConfig: | ||
name: router | ||
serviceName: router-service | ||
nodes: | ||
root: | ||
routerType: Sequence | ||
steps: | ||
- name: DocSum | ||
data: $response | ||
internalService: | ||
serviceName: docsum-llm-uservice | ||
config: | ||
endpoint: /v1/chat/docsum | ||
HUGGING_FACE_HUB_TOKEN: <HUGGING_FACE_HUB_TOKEN> | ||
HF_TOKEN: <HF_TOKEN> | ||
PORT: "9009" | ||
- name: Tgi | ||
internalService: | ||
serviceName: tgi-svc | ||
config: | ||
LANGCHAIN_TRACING_V2: "false" | ||
LANGCHAIN_PROJECT: "opea-llm-service" | ||
endpoint: /generate | ||
isDownstreamService: true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters