Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vector Search #9032

Closed
breezewish opened this issue May 11, 2024 · 0 comments · Fixed by #9486
Closed

Vector Search #9032

breezewish opened this issue May 11, 2024 · 0 comments · Fixed by #9486
Labels
type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@breezewish
Copy link
Member

breezewish commented May 11, 2024 •
edited
Loading

Enhancement

Support Vector Search.

TiDB tracking issue: pingcap/tidb#54245
@breezewish breezewish added the type/enhancement The issue or PR belongs to an enhancement. label May 11, 2024
@breezewish breezewish mentioned this issue May 11, 2024
Merged
12 tasks
@JaySon-Huang JaySon-Huang added type/feature-request Categorizes issue or PR as related to a new feature. and removed type/enhancement The issue or PR belongs to an enhancement. labels May 11, 2024
ti-chi-bot bot pushed a commit that referenced this issue May 11, 2024
@breezewish
metrics: Add Vector Search panel (#9033)
2378444 
ref #9032
@JaySon-Huang JaySon-Huang mentioned this issue May 16, 2024
Closed
@breezewish breezewish mentioned this issue Jun 26, 2024
Open
This was referenced Jul 29, 2024
Merged
Merged
ti-chi-bot bot added a commit that referenced this issue Jul 31, 2024
@Lloyd-Pottiger @JaySon-Huang @ti-chi-bot
compute: Add Vector data type and functions (#9262)
0856586 
ref #9032

Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: JaySon-Huang <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@JaySon-Huang JaySon-Huang mentioned this issue Aug 1, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 1, 2024
@JaySon-Huang
coprocessor: Refactor encoding TiDBColumn (#9279)
57c542e 
ref #9032
ti-chi-bot bot added a commit that referenced this issue Aug 12, 2024
@Lloyd-Pottiger @JaySon-Huang @ti-chi-bot
storage: Support vector index and ANN hint (#9261)
32d911b 
ref #9032

Co-authored-by: JaySon-Huang <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@Lloyd-Pottiger Lloyd-Pottiger mentioned this issue Aug 12, 2024
Merged
12 tasks
@JaySon-Huang JaySon-Huang mentioned this issue Aug 21, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 21, 2024
@JaySon-Huang @Lloyd-Pottiger
*: Vector Data types and Functions (#9341)
0d0463a 
ref #9032

*: Vector Data types and Functions

Support parsing vector data type written by TiDB
Support basic functions for vector data type: CastVectorAsText, VecDims, VecL1Distance, VecL2Distance, VecL2Norm, VecCosineDistance, VecNegativeInnerProduct

Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: Lloyd-Pottiger <[email protected]>
Co-authored-by: JaySon-Huang <[email protected]>
ti-chi-bot bot pushed a commit that referenced this issue Aug 26, 2024
@Lloyd-Pottiger @JaySon-Huang
storage: Use mmap to view vector index (#9313)
d6abc70 
ref #9032

storage: Use mmap to view vector index

Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: JaySon-Huang <[email protected]>
This was referenced Aug 26, 2024
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Aug 26, 2024
@Lloyd-Pottiger @breezewish
storage: Add vector search metrics (#9349)
3f15689 
ref #9032

storage: Add vector search metrics

Signed-off-by: Wish <[email protected]>

Co-authored-by: Wenxuan <[email protected]>
ti-chi-bot bot pushed a commit that referenced this issue Aug 26, 2024
@EricZequan @JaySon-Huang
libcommon: Provide RSS details metrics (#9218)
bfb8c5d 
ref #9032

Signed-off-by: “EricZequan” <[email protected]>

Co-authored-by: JaySon <[email protected]>
This was referenced Aug 26, 2024
Merged
Merged
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Aug 27, 2024
@Lloyd-Pottiger @breezewish @JaySon-Huang
storage: Add keyspace_id to DT system tables (#9371)
d113404 
ref #9032

Signed-off-by: Wish <[email protected]>

Co-authored-by: Wenxuan <[email protected]>
Co-authored-by: JaySon <[email protected]>
ti-chi-bot bot added a commit that referenced this issue Aug 27, 2024
@Lloyd-Pottiger @breezewish
@JaySon-Huang @ti-chi-bot
storage: Fix TiFlash system tables cannot query in Keyspace Mode (#9370)
d94b18a 
ref #9032

Signed-off-by: Wish <[email protected]>

Co-authored-by: Wenxuan <[email protected]>
Co-authored-by: JaySon <[email protected]>
Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot pushed a commit that referenced this issue Aug 27, 2024
@Lloyd-Pottiger
*: use SimSIMD for vectors (#9372)
5f08ae6 
ref #9032

*: use SimSIMD for vectors

Signed-off-by: Lloyd-Pottiger <[email protected]>
@Lloyd-Pottiger Lloyd-Pottiger mentioned this issue Aug 28, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Aug 28, 2024
@Lloyd-Pottiger
storage: Add system.dt_local_indexes (#9379)
3e3f23c 
ref #9032

storage: Add system.dt_local_indexes

Signed-off-by: Lloyd-Pottiger <[email protected]>
@JaySon-Huang JaySon-Huang mentioned this issue Sep 29, 2024
Merged
12 tasks
ti-chi-bot bot added a commit that referenced this issue Sep 29, 2024
@JaySon-Huang @ti-chi-bot
vector: Fix join by array meet error "no supertype for types Array(Fl…
9f7fdbc 
…oat32), Nullable(Array(Float32))" (#9490)

ref #9032

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@JaySon-Huang JaySon-Huang mentioned this issue Sep 29, 2024
Merged
12 tasks
@Lloyd-Pottiger Lloyd-Pottiger mentioned this issue Sep 30, 2024
Closed
12 tasks
@ti-chi-bot ti-chi-bot bot closed this as completed in 2a198f1 Sep 30, 2024
@JaySon-Huang JaySon-Huang mentioned this issue Sep 30, 2024
Merged
12 tasks
@JaySon-Huang JaySon-Huang mentioned this issue Sep 30, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 30, 2024
@JaySon-Huang @Lloyd-Pottiger
tests: fix build on m1 macbook && re-enable clang-tidy checks (#9500)
23f24ac 
ref #9032

Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: Lloyd-Pottiger <[email protected]>
@breezewish breezewish mentioned this issue Oct 8, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 8, 2024
@breezewish @JaySon-Huang
storage: Fix vector index not used (#9503)
02784a4 
ref #9032

Fix ANNIndexInfo is not utilized when TiDB specifies it

Signed-off-by: Wish <[email protected]>

Co-authored-by: JaySon-Huang <[email protected]>
This was referenced Oct 9, 2024
Merged
Closed
ti-chi-bot bot pushed a commit that referenced this issue Oct 10, 2024
@JaySon-Huang
Storage: Abort adding vector index if encryption enabled (#9504)
cea728d 
ref #9032, close #9505

Storage: Abort adding vector index if encryption enabled
@JaySon-Huang JaySon-Huang mentioned this issue Oct 11, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 11, 2024
@JaySon-Huang
tests: Fix asan check failures (#9520)
acfba93 
ref #9032
ti-chi-bot bot pushed a commit that referenced this issue Oct 12, 2024
@JaySon-Huang
tests: Enable vector search fullstack tests (#9501)
76a4cd5 
ref #9032
@JaySon-Huang JaySon-Huang mentioned this issue Oct 14, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 14, 2024
@JaySon-Huang
metrics: Add Thread CPU panel for building vector index (#9525)
463fce3 
ref #9032
@Lloyd-Pottiger Lloyd-Pottiger mentioned this issue Oct 16, 2024
Merged
12 tasks
ti-chi-bot bot added a commit that referenced this issue Oct 18, 2024
@Lloyd-Pottiger @ti-chi-bot
Storages: support building vector index for ColumnFileTiny (Part 1) (#…
9eb0c80 
…9534)

ref #9032

Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
This was referenced Oct 21, 2024
Merged
Merged
Merged
ti-chi-bot bot added a commit that referenced this issue Oct 25, 2024
@Lloyd-Pottiger @ti-chi-bot
Storages: support building vector index for ColumnFileTiny (Part 2) (#…
a1a3b9b 
…9546)

ref #9032

Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot added a commit that referenced this issue Oct 28, 2024
@Lloyd-Pottiger @ti-chi-bot
Storages: refactor DMFile vector search (#9549)
0853523 
ref #9032

Storages: refactor DMFile vector search

Signed-off-by: Lloyd-Pottiger <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot pushed a commit that referenced this issue Oct 31, 2024
@Lloyd-Pottiger
Storages: support building vector index for ColumnFileTiny (Part 3) (#…
993d8c7 
…9547)

ref #9032

Signed-off-by: Lloyd-Pottiger <[email protected]>
yibin87 pushed a commit to yibin87/tiflash that referenced this issue Oct 31, 2024
@Lloyd-Pottiger @yibin87
Storages: support building vector index for ColumnFileTiny (Part 3) (p…
6918b9d 
…ingcap#9547)

ref pingcap#9032

Signed-off-by: Lloyd-Pottiger <[email protected]>
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this issue Nov 1, 2024
@JaySon-Huang
metrics: Add Thread CPU panel for building vector index (pingcap#9525)
c29155b 
ref pingcap#9032
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

*: support vector search pingcap/tiflash
2 participants
@breezewish @JaySon-Huang