[BUG] CSI failed to recover FUSE mount point for AlluxioRuntime

[TrafalgarZZZ]

What is your environment(Kubernetes version, Fluid version, etc.)

Describe the bug
FUSE Recovery failed when using AlluxioRuntime.

What you expect to happen:
Alluxio FUSE should be successfully recovered after deleting the FUSE pod.

How to reproduce it

Simply run the e2e script in #2477 can reproduce this bug.

Additional Information

[TrafalgarZZZ]

The bug is probably caused by an incorrect order among Pod Readiness, FUSE mount point readiness and CSI Plugin's recover() func logic.

Currently, CSI plugin recovers broken mount points only if it detects any FUSE container restarts and become ready. However, it is possible that the execution order goes like:

1. Alluxio FUSE Pod restarts
2. Alluxio FUSE Pod ready
3. CSI detects container restart and recovers broken mount points (At this time, Alluxio FUSE mount point is not ready, so nothing happened)
4. Alluxio FUSE mount point ready
5. CSI Plugin won't retry Step 1 to 4 because it detects no container restarts

[cheyang]

The bug is probably caused by an incorrect order among Pod Readiness, FUSE mount point readiness and CSI Plugin's recover() func logic.

Currently, CSI plugin recovers broken mount points only if it detects any FUSE container restarts and become ready. However, it is possible that the execution order goes like:

1. Alluxio FUSE Pod restarts
2. Alluxio FUSE Pod ready
3. CSI detects container restart and recovers broken mount points (At this time, Alluxio FUSE mount point is not ready, so nothing happened)
4. Alluxio FUSE mount point ready
5. CSI Plugin won't retry Step 1 to 4 because it detects no container restarts

Can scanning /proc/self/mountinfo solve this issue?