[BUG] org.opensearch.cache.store.disk.EhCacheDiskCacheTests.classMethod is flaky

[reta]

Describe the bug

The test case org.opensearch.cache.store.disk.EhCacheDiskCacheTests.classMethod is flaky:

com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=81, name=MappedByteBufferSource Async Flush Thread, state=WAITING, group=TGRP-EhCacheDiskCacheTests]
        at java.base/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base/java.util.concurrent.locks.LockSupport.park(LockSupport.java:371)
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(AbstractQueuedSynchronizer.java:519)
        at java.base/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3780)
        at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3725)
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1707)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1070)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

Related component

Storage:Remote

To Reproduce

./gradlew ':plugins:cache-ehcache:test' --tests "org.opensearch.cache.store.disk.EhCacheDiskCacheTests" -Dtests.seed=4D5D98F56511FEC6 

Expected behavior

The test must always pass

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
Standard

Host/Environment (please complete the following information):

• CI

Additional context

• https://build.ci.opensearch.org/job/gradle-check/35254/testReport/junit/org.opensearch.cache.store.disk/EhCacheDiskCacheTests/classMethod/

[reta]

@sgup432 @peteralfonsi please take a look, may be caused by #12734 or #12709

[andrross]

[Triage - attendees 1 2 3 4 5 6]

Fixed by #12764