[Bug] the messages in the delay queue are replayed when I upgraded broker from 4.9.1 to 5.1.1

[gaoyf]

Before Creating the Bug Report

• I found a bug, not just asking a question, which should be created in GitHub Discussions.

• I have searched the GitHub Issues and GitHub Discussions of this repository and believe that this is not a duplicate.

• I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.

Runtime platform environment

OS: CentOS 6.9

RocketMQ version

branch: (develop|tag 5.1.1) version: 5.1.1

JDK Version

JDK: 1.8.0_202

Describe the Bug

When I upgrade the broker from 4.9.1 to 5.1.1, when the broker starts to load data for more than 10 seconds, then the persist timing task in ScheduleMessageService will be executed, but the load method has not been executed yet, so the existing deleyOffset.json overwrites, eventually causing the schedule message to be replayed.

Steps to Reproduce

As long as you find a broker with a scheduled message, and the commitlog reaches hundreds of G, it can be reproduced after starting with 5.x for more than 10 seconds. You can refer to the case I wrote for details.

What Did You Expect to See?

the messages in the delay queue are not replayed when I upgraded broker.

What Did You See Instead?

the messages in the delay queue are replayed when I upgraded broker.

Additional Context

No response

[gaoyf]

PR is #6882