Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: hybrid search support choice for normalization #37477

Open
1 task done
yiwen92 opened this issue Nov 6, 2024 · 1 comment
Open
1 task done

[Feature]: hybrid search support choice for normalization #37477

yiwen92 opened this issue Nov 6, 2024 · 1 comment
Assignees
Labels
kind/feature Issues related to feature request from users

Comments

@yiwen92
Copy link

yiwen92 commented Nov 6, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Is your feature request related to a problem? Please describe.

Now in milvus, weighted ranker need to do normolization first, in order to combine different metric types from multi-recall ways.
However, in some embedding models like bge-m3, mgte. They do not need this normolization and they can plus the distance directly.
img_v3_02gc_27622dac-29c6-4253-a339-80e02dea3cbg

Describe the solution you'd like.

Add a param in rerank to control whether need do normalization or not.

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

@yiwen92 yiwen92 added the kind/feature Issues related to feature request from users label Nov 6, 2024
@xiaofan-luan
Copy link
Collaborator

Is there an existing issue for this?

  • I have searched the existing issues

Is your feature request related to a problem? Please describe.

Now in milvus, weighted ranker need to do normolization first, in order to combine different metric types from multi-recall ways. However, in some embedding models like bge-m3, mgte. They do not need this normolization and they can plus the distance directly. img_v3_02gc_27622dac-29c6-4253-a339-80e02dea3cbg

Describe the solution you'd like.

Add a param in rerank to control whether need do normalization or not.

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

this means their vector distance is already normalized.

No matter what kind of normalization you do the result won't change.

There is not need to add extra complexity to the api

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Issues related to feature request from users
Projects
None yet
Development

No branches or pull requests

2 participants