Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

optimize rerank with backend ref #579

Merged
merged 44 commits into from
Sep 10, 2024
Merged

optimize rerank with backend ref #579

merged 44 commits into from
Sep 10, 2024

Conversation

a32543254
Copy link
Collaborator

@a32543254 a32543254 commented Aug 29, 2024 •
edited
Loading

Description

Optimize the rerank with backend ref module
image

<style> </style>
model type input shape Mosec (s) NS (s) ratio
bge-reranker-large [2, 14] 0.0537 0.0359 1.5x
bge-reranker-large [24, 512] 2.610966 1.184117 2.2x

Issues

List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

Describe the tests that you ran to verify your changes.

@lvliang-intel
Copy link
Collaborator

@a32543254,

Please fix pre-commit issue and also add a test case.
image

@lvliang-intel
Copy link
Collaborator

@a32543254，
Please fix the Docker file issue.
image

@a32543254
Copy link
Collaborator Author

@a32543254， Please fix the Docker file issue. image

Thanks, I'm co-working with Suyue on this ut test now, will be fixed soon.

a32543254 and others added 26 commits September 2, 2024 23:33
@a32543254
add rerank with neural speed
0837cc4 
Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
add the code
7a2a90b 
Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
add the code
4bf3b59 
Signed-off-by: Dong, Bo1 <[email protected]>
@pre-commit-ci @a32543254
[pre-commit.ci] auto fixes from pre-commit.com hooks
aa28c54 
for more information, see https://pre-commit.ci

Signed-off-by: Dong, Bo1 <[email protected]>
@Spycsh @pre-commit-ci @a32543254
fix mismatched response format w/wo streaming guardrails (opea-projec…
d115a93 
…t#568)

* fix mismatched response format w/wo streaming  guardrails

* fix & debug

* fix & rm debug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <[email protected]>
@Spycsh @pre-commit-ci @a32543254
Fix guardrails out handle logics for space linebreak and quote (opea-…
b2e918e 
…project#571)

* fix mismatched response format w/wo streaming  guardrails

* fix & debug

* fix & rm debug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* debug

* debug

* debug

* fix pre-space and linebreak

* fix pre-space and linebreak

* fix single/double quote

* fix single/double quote

* remove debug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <[email protected]>
@BaoHuiling @pre-commit-ci @a32543254
BUG FIX: LVM security fix (opea-project#572)
29a5234 
* add url validator

Signed-off-by: BaoHuiling <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add validation for video_url

Signed-off-by: BaoHuiling <[email protected]>

---------

Signed-off-by: BaoHuiling <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <[email protected]>
@ZePan110 @a32543254
Modify output messages. (opea-project#569)
61ec380 
* Reduced output.

Signed-off-by: zepan <[email protected]>

* Output the location where the modified Dockerfile file is referenced.

Signed-off-by: zepan <[email protected]>

* for test

Signed-off-by: zepan <[email protected]>

* Restore test file.

Signed-off-by: zepan <[email protected]>

---------

Signed-off-by: zepan <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@lkk12014402 @pre-commit-ci @a32543254
refine logging code. (opea-project#559)
4e3e6d4 
* add ut and refine logging code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update microservice port.

---------

Co-authored-by: root <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <[email protected]>
@sharanshirodkar7 @pre-commit-ci
@lvliang-intel @a32543254
adding lancedb to langchain vectorstores (opea-project#291)
b0be968 
* adding lancedb to langchain vectorstores

Signed-off-by: sharanshirodkar7 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: sharanshirodkar7 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: lvliang-intel <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@letonghan @pre-commit-ci @a32543254
Refine Dataprep Milvus MS (opea-project#570)
6f92766 
Signed-off-by: letonghan <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
final version
cff5686 
Signed-off-by: Dong, Bo1 <[email protected]>
@pre-commit-ci @a32543254
[pre-commit.ci] auto fixes from pre-commit.com hooks
1e00804 
for more information, see https://pre-commit.ci

Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
update the readme
45c3940 
Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
add the sign
d20d16f 
Signed-off-by: Dong, Bo1 <[email protected]>
@pre-commit-ci @a32543254
[pre-commit.ci] auto fixes from pre-commit.com hooks
90dae42 
for more information, see https://pre-commit.ci

Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
fix error for pre ci
eed64ed 
Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
add the ut
fd72d52 
Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
update docker file
7f5228d 
Signed-off-by: Dong, Bo1 <[email protected]>
@chensuyue @pre-commit-ci @a32543254
update CI test log achieve (opea-project#577)
1845be6 
Signed-off-by: chensuyue <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <[email protected]>
@tileintel @pre-commit-ci @a32543254
Multimodal dataprep (opea-project#575)
a617036 
* multimodal embedding for MM RAG for videos

Signed-off-by: Tiep Le <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* develop data prep first commit

Signed-off-by: Tiep Le <[email protected]>

* develop dataprep microservice for multimodal data

Signed-off-by: Tiep Le <[email protected]>

* multimodal langchain for dataprep

Signed-off-by: Tiep Le <[email protected]>

* update README

Signed-off-by: Tiep Le <[email protected]>

* update README

Signed-off-by: Tiep Le <[email protected]>

* update README

Signed-off-by: Tiep Le <[email protected]>

* update README

Signed-off-by: Tiep Le <[email protected]>

* cosmetic

Signed-off-by: Tiep Le <[email protected]>

* test for multimodal dataprep

Signed-off-by: Tiep Le <[email protected]>

* update test

Signed-off-by: Tiep Le <[email protected]>

* update test

Signed-off-by: Tiep Le <[email protected]>

* update test

Signed-off-by: Tiep Le <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* cosmetic update

Signed-off-by: Tiep Le <[email protected]>

* remove langsmith

Signed-off-by: Tiep Le <[email protected]>

* update API to remove /dataprep from API names and remove langsmith

Signed-off-by: Tiep Le <[email protected]>

* update test

Signed-off-by: Tiep Le <[email protected]>

* update the error message per PR reviewer

Signed-off-by: Tiep Le <[email protected]>

---------

Signed-off-by: Tiep Le <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <[email protected]>
@berkecanrizai @pre-commit-ci
@Spycsh @a32543254
add: Pathway vector store and retriever as LangChain component (opea-…
ff66f8b 
…project#342)

* nb

Signed-off-by: Berke <[email protected]>

* init changes

Signed-off-by: Berke <[email protected]>

* docker

Signed-off-by: Berke <[email protected]>

* example data

Signed-off-by: Berke <[email protected]>

* docs(readme): update, add commands

Signed-off-by: Berke <[email protected]>

* fix: formatting, data sources

Signed-off-by: Berke <[email protected]>

* docs(readme): update instructions, add comments

Signed-off-by: Berke <[email protected]>

* fix: rm unused parts

Signed-off-by: Berke <[email protected]>

* fix: image name, compose env vars

Signed-off-by: Berke <[email protected]>

* fix: rm unused part

Signed-off-by: Berke <[email protected]>

* fix: logging name

Signed-off-by: Berke <[email protected]>

* fix: env var

Signed-off-by: Berke <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Berke <[email protected]>

* fix: rename pw docker

Signed-off-by: Berke <[email protected]>

* docs(readme): update input sources

Signed-off-by: Berke <[email protected]>

* nb

Signed-off-by: Berke <[email protected]>

* init changes

Signed-off-by: Berke <[email protected]>

* fix: formatting, data sources

Signed-off-by: Berke <[email protected]>

* docs(readme): update instructions, add comments

Signed-off-by: Berke <[email protected]>

* fix: rm unused part

Signed-off-by: Berke <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Berke <[email protected]>

* fix: rename pw docker

Signed-off-by: Berke <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Berke <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat: mv vector store, naming, clarify instructions, improve ingestion components

Signed-off-by: Berke <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tests: add pw retriever test
fix: update docker to include libmagic

Signed-off-by: Berke <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* implement suggestions from review, entrypoint, reqs, comments, https_proxy.

Signed-off-by: Berke <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: update docker tags in test and readme

Signed-off-by: Berke <[email protected]>

* tests: add separate pathway vectorstore test

Signed-off-by: Berke <[email protected]>

---------

Signed-off-by: Berke <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sihan Chen <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@BaoHuiling @pre-commit-ci
@chensuyue @a32543254
Add local Rerank microservice for VideoRAGQnA (opea-project#496)
d4d0fc7 
* initial commit

Signed-off-by: BaoHuiling <[email protected]>

* save

Signed-off-by: BaoHuiling <[email protected]>

* add readme, test script, fix bug

Signed-off-by: BaoHuiling <[email protected]>

* update video URL

Signed-off-by: BaoHuiling <[email protected]>

* use default

Signed-off-by: BaoHuiling <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update core dependency

Signed-off-by: BaoHuiling <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* use p 5000

Signed-off-by: BaoHuiling <[email protected]>

* use 5037

Signed-off-by: BaoHuiling <[email protected]>

* update ctnr name

Signed-off-by: BaoHuiling <[email protected]>

* remove langsmith

Signed-off-by: BaoHuiling <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add rerank algo desc in readme

Signed-off-by: BaoHuiling <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: BaoHuiling <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: chen, suyue <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@ZePan110 @a32543254
Add Scan Container. (opea-project#560)
0687fd3 
Signed-off-by: zepan <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@BaoHuiling @a32543254
fix SearchedMultimodalDoc in docarray (opea-project#583)
6860e0c 
Signed-off-by: BaoHuiling <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@ZePan110 @a32543254
update image build yaml (opea-project#529)
2ea22d0 
Signed-off-by: chensuyue <[email protected]>
Signed-off-by: zepan <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
Liangyx2 and others added 6 commits September 2, 2024 23:33
@Liangyx2 @pre-commit-ci @a32543254
add microservice for intent detection (opea-project#131)
5ada66f 
* add microservice for intent detection

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update license copyright

Signed-off-by: Liangyx2 <[email protected]>

* add ut

Signed-off-by: Liangyx2 <[email protected]>

* refine

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update folder

Signed-off-by: Liangyx2 <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix test

Signed-off-by: Liangyx2 <[email protected]>

---------

Signed-off-by: Liangyx2 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Dong, Bo1 <[email protected]>
@ZePan110 @a32543254
Make the scanning method optional. (opea-project#580)
d868291 
Signed-off-by: zepan <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@kevinintel @a32543254
add code owners (opea-project#586)
c8c5cf7 
Signed-off-by: Dong, Bo1 <[email protected]>
@letonghan @a32543254
remove revision for tei (opea-project#584)
f7a469c 
Signed-off-by: letonghan <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@ZePan110 @a32543254
Bug fix (opea-project#591)
5ceb570 
* Check if the document exists.

Signed-off-by: zepan <[email protected]>

* Add flag output.

Signed-off-by: zepan <[email protected]>

* Modify nginx readme.

Signed-off-by: zepan <[email protected]>

* Modify document detection logic

Signed-off-by: zepan <[email protected]>

---------

Signed-off-by: zepan <[email protected]>
Signed-off-by: Dong, Bo1 <[email protected]>
@a32543254
fix ut issue
98fd44c 
Signed-off-by: Dong, Bo1 <[email protected]>
a32543254 and others added 10 commits September 5, 2024 02:01
@lvliang-intel lvliang-intel merged commit d76751a into opea-project:main Sep 10, 2024
9 of 10 checks passed
lkk12014402 pushed a commit that referenced this pull request Sep 19, 2024
@chensuyue
add secrets for test (#579)
3c9e2aa 
Signed-off-by: chensuyue <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lvliang-intel lvliang-intel lvliang-intel approved these changes

@Spycsh Spycsh Spycsh approved these changes

@XuhuiRen XuhuiRen Awaiting requested review from XuhuiRen

@XinyuYe-Intel XinyuYe-Intel Awaiting requested review from XinyuYe-Intel

@letonghan letonghan Awaiting requested review from letonghan

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.