Storage: Add error message when fail to build local index

[Lloyd-Pottiger]

What problem does this PR solve?

Issue Number: ref #9032

Problem Summary:

What is changed and how it works?

• Add 2 columns to system.dt_local_indexes
  • column_name is removed from the system.dt_local_indexes, let the tidb fill the index_name
  • error_message when it contains non-empty string, it means the index can not be built on tiflash
• An std::unordered_map<IndexID, String> fatal_error_when_building_local_index is added to each Segment:
  • The error message will be set when:
    • When the task can not be pushed to the LocalIndexScheduler, for example, need to large memory to be built
    • When the task throws any exception in DeltaMergeStore::segmentEnsureStableIndex
  • The error message will be clear when:
    • New local tasks is pushed to the LocalIndexScheduler with the same index_id
    • The index is dropped from tidb
  • generateLocalIndexInfos now will also return the index_ids that is dropped, so that we can cleanup the error message of the dropped indexes

Storage: Add error message when fail to build local index

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

create table x(e vector(3) not null);
set sql_mode='';
alter table x set tiflash replica 1;
insert into x values(); -- now this will insert a vector with dimension=0

# create vector index on tidb, fail to build index
ALTER TABLE x ADD VECTOR INDEX idx_e_l2 ((VEC_L2_DISTANCE(e))) USING HNSW;

# check the error message is returned to `information_schema.tiflash_indexes`
select * from information_schema.tiflash_indexes where tidb_table = 'x';

or:

# starts a tifalsh server with tcp_port enabled
> /path/to/tiflash server --config-file=... --tcp_port=9099

# connect to tidb and create the table and vector index
CREATE TABLE `t` (  `v` vector(5) DEFAULT NULL,  `v2` vector(3) DEFAULT NULL);
INSERT INTO t VALUES ('[8.7, 5.7, 7.7, 9.8, 1.5]','[1, 2, 3]')
alter table t set tiflash replica 1;

# starts a tiflash client to enable the failpoint of building local index
>  /path/to/tiflash client --host=127.0.0.1 --port=9099
TiFlash :) DBGInvoke enable_fail_point(exception_build_local_index_for_file)

# create vector index on tidb
ALTER TABLE t ADD VECTOR INDEX idx_v2_l2 ((VEC_L2_DISTANCE(v2))) USING HNSW;
# check the error message is returned to `information_schema.tiflash_indexes`
select * from information_schema.tiflash_indexes where tidb_table = 't';

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

[JaySon-Huang]

LGTM

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JaySon-Huang, JinheLin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [JaySon-Huang,JinheLin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-09-27 09:10:40.441604257 +0000 UTC m=+395.861817265: ☑️ agreed by JaySon-Huang.
• 2024-09-27 09:14:16.610229676 +0000 UTC m=+612.030442683: ☑️ agreed by JinheLin.