ROMIO: incorrect flattened file offset-length pairs when using MPICH 4.0.3 and prior

[wkliao]

When using MPICH 4.0.3 and prior, I encounter an incorrect I/O result.
It happens when using MPI_Type_indexed() or MPI_Type_create_hindexed()
to concatenate multiple datatype created from MPI_Type_create_subarray().
The concatenated datatype is then used to define a fileview, when opening
a file. Adding printf statement inside of ROMIO reveals incorrect flattened
file offset-length pairs.

This issue does not happen when using MPICH 4.1 and later. As pointed out
in issue #7163, it is because MPIX_Type_iov functions are used to flatten
the fileview datatype since version 4.1 and MPIX_Type_iov functions
appear to be able to produce correct offset-length pairs.

I encountered this problem when running jobs on Perlmutter at NERSC,
whose MPI makes use of a ROMIO from MPICH earlier than 4.1. I believe
there are other MPI vendors that also use an older version of ROMIO.

Smaller reproducers are available in

• indexed_fsize.c
• hindexed_fsize.c

The difference between the two programs are the call to
MPI_Type_indexed() and MPI_Type_create_hindexed().
If they are replaced by MPI_Type_create_struct(), as
in struct_fsize.c, then the generated offset-length pairs
are correct and hence the test program passed.