It takes very long time to show index info.

[congguosn]

Describe the bug:
I have a collection with 10 fields, and each field has its individual index. When I am trying to interact with milvus with attu, the page became very sluggish. I found the time mostly spent on obtaining the index info which is called in below code snippet.

const indexRes = await this.getIndexInfo(...)

Steps to reproduce:

1. create a collection with 10 fields.
2. create index for each of the field.
3. login to attu, it takes long time to list the collections in the collection page. (For getAllCollections will try to obtain index info.)

Attu version:
v2.3.2

[shanghaikid]

Maybe we need to cache the index info. will fix this in the next release.

[shanghaikid]

On my testing on a standalone milvus deployed with docker, create 10 indexes on a collection, it is not slow to get the index info. Can you give me more information about your collection and data size?

[congguosn]

Yes, below is the detailed info.

1. 1 collection x 1 partiton x 1 shard
2. entities number: 70 million
3. vector dim: 1600
4. 8 scalar index, most of them are varchar type except 1 of them is INT32
5. milvus version: 2.2.14
6. attu version: 2.3.2

[congguosn]

indexcoord.log
FYI

[shanghaikid]

don't know why.. I've test a 5 indexes collection with 60m data on Milvus v2.2.15, everything looks good. it will be a little bit slow, but it's not like a huge performance impact if you have big data.

@yanliang567 do you have any idea on this? does it relate to the specific milvus version?

[yanliang567]

@congguosn could you please retry with milvus sdk, e.g. pymilvus to check the latency is on millvus side or on web side.

[yhmo]

This is not Attu bug, it is an issue of milvus. I just talked with Zhangcai.
DescribeIndex() interface does the following things:

1. get index parameters from etcd
2. for each index, check its state, whether the index is finished or not
3. calculate the indexed row count for each index

To calculate indexed row count, the indexcorrd iterates each segment in the collection. With more segments, it takes more time to calculate.

From the log, I saw each index takes 0.5 seconds to calculate the indexed row count. 10 indexes cost 5 seconds. Currently, this behavior is expected.

[shanghaikid]

This is not Attu bug, it is an issue of milvus. I just talked with Zhangcai. DescribeIndex() interface does the following things:

1. get index parameters from etcd
2. for each index, check its state, whether the index is finished or not
3. calculate the indexed row count for each index

To calculate indexed row count, the indexcorrd iterates each segment in the collection. With more segments, it takes more time to calculate.

From the log, I saw each index takes 0.5 seconds to calculate the indexed row count. 10 indexes cost 5 seconds. Currently, this behavior is expected.

Is there any improvement about the newer version, for example: 2.3.x ?

[congguosn]

I see, but seems FE is calling api/v1/collections for nearly every page on attu, which will try to get Index Info then. That maybe unnecessary in my opinion.

FE -> api/v1/collections -> getAllCollections -> getIndexInfo (long time call)

[shanghaikid]

I see, but seems FE is calling api/v1/collections for nearly every page on attu, which will try to get Index Info then. That maybe unnecessary in my opinion.

FE -> api/v1/collections -> getAllCollections -> getIndexInfo (long time call)

yeah, we can optimize this.

[shanghaikid]

milvus-io/milvus#29313

[shanghaikid]

https://github.com/zilliztech/attu/releases/tag/v2.3.5 is just released, it should improve the performance for the case.