Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Search iterator using HNSW failed for ef(1024) should be larger than k(10000) #27284

Closed
1 task done
NicoYuan1986 opened this issue Sep 21, 2023 · 1 comment
Closed
1 task done
Assignees
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Milestone

Comments

@NicoYuan1986
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version:2.3.0
- Deployment mode(standalone or cluster):standalone
- MQ type(rocksmq, pulsar or kafka):    rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2):2.4.0.dev124
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Search iterator using HNSW failed for ef(1024) should be larger than k(10000).

>>> batch_size = 1000
>>> search_params = {"metric_type": "COSINE", "params": {"ef": 1024}}
>>> search_iterator = collection.search_iterator(vectors[:1], "float_vector", search_params, batch_size=batch_size)
RPC error: [search], <MilvusException: (code=1, message=attempt #0: fail to Search, QueryNode ID=2, reason=worker(2) query failed: UnexpectedError: Assert "failed to search: out of range in json: ef(1024) should be larger than k(10000)" at /go/src/github.com/milvus-io/milvus/internal/core/src/index/VectorMemIndex.cpp:319: channel=by-dev-rootcoord-dml_0_443752784731590555v0: fail to search/query shard leader: attempt #1: no available shard delegator found: service unavailable: fail to search on all shard leaders)>, <Time:{'RPC start': '2023-08-23 11:11:06.121272', 'RPC error': '2023-08-23 11:11:06.728015'}>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/nico/lib/python3.10/site-packages/pymilvus/orm/collection.py", line 813, in search_iterator
    return SearchIterator(
  File "/root/nico/lib/python3.10/site-packages/pymilvus/orm/iterator.py", line 308, in __init__
    self.__init_search_iterator()
  File "/root/nico/lib/python3.10/site-packages/pymilvus/orm/iterator.py", line 311, in __init_search_iterator
    init_page = self.__execute_next_search(self._param, self._expr)
  File "/root/nico/lib/python3.10/site-packages/pymilvus/orm/iterator.py", line 469, in __execute_next_search
    res = self._conn.search(
  File "/root/nico/lib/python3.10/site-packages/pymilvus/decorators.py", line 127, in handler
    raise e from e
  File "/root/nico/lib/python3.10/site-packages/pymilvus/decorators.py", line 123, in handler
    return func(*args, **kwargs)
  File "/root/nico/lib/python3.10/site-packages/pymilvus/decorators.py", line 162, in handler
    return func(self, *args, **kwargs)
  File "/root/nico/lib/python3.10/site-packages/pymilvus/decorators.py", line 102, in handler
    raise e from e
  File "/root/nico/lib/python3.10/site-packages/pymilvus/decorators.py", line 68, in handler
    return func(*args, **kwargs)
  File "/root/nico/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 774, in search
    return self._execute_search_requests(
  File "/root/nico/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 735, in _execute_search_requests
    raise pre_err from pre_err
  File "/root/nico/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 726, in _execute_search_requests
    raise MilvusException(response.status.error_code, response.status.reason)
pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=attempt #0: fail to Search, QueryNode ID=2, reason=worker(2) query failed: UnexpectedError: Assert "failed to search: out of range in json: ef(1024) should be larger than k(10000)" at /go/src/github.com/milvus-io/milvus/internal/core/src/index/VectorMemIndex.cpp:319: channel=by-dev-rootcoord-dml_0_443752784731590555v0: fail to search/query shard leader: attempt #1: no available shard delegator found: service unavailable: fail to search on all shard leaders)>

Expected Behavior

search successfully

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

@NicoYuan1986
Copy link
Contributor Author

NicoYuan1986 commented Oct 16, 2023

Verify fixed.
pymilvus: 2.3.1.post1.dev6
milvus: master-20231015-ff19cb27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

2 participants