create local storage met inconsistent executor ID of local file error and panic

[amyangfei]

What did you do?

run engine integration test
ref: https://ci2.pingcap.net/blue/organizations/jenkins/engine_ghpr_integration_test/detail/engine_ghpr_integration_test/1120/pipeline

detail logs:

log-dm_case_sensitive-dm_full_mode.tar.gz

What did you expect to see?

No response

What did you see instead?

[2022/10/10 02:38:11.406 +00:00] [PANIC] [file_manager.go:308] ["inconsistent executor ID of local file"] [fileScope="{"Executor":"server-executor-2-0a5789b4","WorkerID":"4fc9df31-d6d5-4704-a2db-c5d110932e5e","Name":"31788357-36c4-49a9-907f-39ac23856517/mysql-01"}"] [creator=""server-executor-2-0a5789b4""] [currentExecutor=server-executor-0-7ae07045] [stack="github.com/pingcap/tiflow/engine/pkg/externalresource/internal/local.(*FileManager).validateIdent
        github.com/pingcap/tiflow/engine/pkg/externalresource/internal/local/file_manager.go:308
github.com/pingcap/tiflow/engine/pkg/externalresource/internal/local.(*FileManager).GetPersistedResource
        github.com/pingcap/tiflow/engine/pkg/externalresource/internal/local/file_manager.go:84
github.com/pingcap/tiflow/engine/pkg/externalresource/broker.(*DefaultBroker).getPersistResource
        github.com/pingcap/tiflow/engine/pkg/externalresource/broker/broker.go:247
github.com/pingcap/tiflow/engine/pkg/externalresource/broker.(*DefaultBroker).OpenStorage
        github.com/pingcap/tiflow/engine/pkg/externalresource/broker/broker.go:106
github.com/pingcap/tiflow/engine/framework.(*DefaultBaseWorker).OpenStorage
        github.com/pingcap/tiflow/engine/framework/worker.go:514
github.com/pingcap/tiflow/engine/executor/dm.(*dmWorker).setupStorage
        github.com/pingcap/tiflow/engine/executor/dm/worker.go:195
github.com/pingcap/tiflow/engine/executor/dm.(*dmWorker).InitImpl
        github.com/pingcap/tiflow/engine/executor/dm/worker.go:139
github.com/pingcap/tiflow/engine/framework.(*DefaultBaseWorker).Init
        github.com/pingcap/tiflow/engine/framework/worker.go:257
github.com/pingcap/tiflow/engine/framework/internal/eventloop.(*Runner[...]).doRun
        github.com/pingcap/tiflow/engine/framework/internal/eventloop/runner.go:85
github.com/pingcap/tiflow/engine/framework/internal/eventloop.(*Runner[...]).Run
        github.com/pingcap/tiflow/engine/framework/internal/eventloop/runner.go:60
github.com/pingcap/tiflow/engine/executor/worker.(*TaskRunner).launchTask.func1
        github.com/pingcap/tiflow/engine/executor/worker/task_runner.go:223"]

Versions of the cluster

Dataflow Engine version (run tiflow version):

master@0f2a253