Remove "tikv" from isolation-read.engines in config file will make tidb unable to execute some DDL jobs as ddl worker

[time-and-fate]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

tidb config file:

[isolation-read]
engines = ["tiflash", "tidb"]

use test
create table t(a int);
alter table t modify column a int not null;

2. What did you expect to see? (Required)

The DDL runs successfully.

3. What did you see instead (Required)

ERROR 1815 (HY000): Internal : No access path for table 't' is found with 'tidb_isolation_read_engines' = '', valid values can be 'tikv'.

In another case, if there are two tidbs, one tidb (tidb1) with config file above is the ddl owner, and the other (tidb2) uses the default config. If we send the DDL to tidb2, it will block for a long time and return error, and there will be error log the same as above in tidb1.

4. What is your TiDB version? (Required)

v5.0.0 - v7.2.0

[time-and-fate]

Some details in the current implementation:

1. For a new session, the IsolationReadEngines are set according to the config file:
   

   tidb/sessionctx/variable/session.go

   Lines 2027 to 2036 in 7df405d

   	for _, engine := range config.GetGlobalConfig().IsolationRead.Engines {
   	switch engine {
   	case kv.TiFlash.Name():
   	vars.IsolationReadEngines[kv.TiFlash] = struct{}{}
   	case kv.TiKV.Name():
   	vars.IsolationReadEngines[kv.TiKV] = struct{}{}
   	case kv.TiDB.Name():
   	vars.IsolationReadEngines[kv.TiDB] = struct{}{}
   	}
   	}

2. The logic of "isolation read engines" has a special check for mysql database. Tables in mysql will not be affected by the setting:
   

   tidb/planner/core/planbuilder.go

   Lines 1504 to 1506 in 397062f

   	if dbName.L == mysql.SystemDB {
   	return paths, nil
   	}

   
   So, many other internal sessions are not affected by this issue, because they only need to read from the mysql database. And this also means "isolation read engines" are ineffective on mysql database.

3. For some internal sessions (including this case), they are set not to load the newest global variables:
   

   tidb/session/session.go

   Line 1340 in af04707

   se.sessionVars.CommonGlobalLoaded = true

   
   This doesn't cause this bug itself, but prevents us from workaround by updating the global variables.

4. For some DDL jobs, the DDL worker needs to read from the specified table of the DDL (like table t in this case):
   doModifyColumn() -> modifyColsFromNull2NotNull() -> checkForNullValue()

[time-and-fate]

I think the internal session should not be affected by the setting of "isolation read engines" to make sure all internal SQLs can succeed as expected.

[bb7133]

IMO for now the current behavior is expected: if "tikv" is removed from `isolation-read.engines' TiDB does not guarantee the interactions with TiKV(like executing DDL) work fine.

If we think this is a problem, it can be treated as a further enhancements in the future.