Reproducible filesystem corruption with too-low RAM buffer size

[patrakov]

When given "-m something_too_low", fsremap corrupts data. The precise value of "too low" is dependent on the data to be remapped, I have seen a case when 64 MB is not enough.

Here is a test case that relies only on downloadable files.

wget https://downloads.lede-project.org/releases/17.01.2/targets/x86/64/lede-17.01.2-x86-64-rootfs-ext4.img.gz
zcat lede-17.01.2-x86-64-rootfs-ext4.img.gz > lede-17.01.2-x86-64-rootfs-ext4.img
losetup /dev/loop7 lede-17.01.2-x86-64-rootfs-ext4.img
fstransform --opts-fsremap="-m 262144" /dev/loop7 xfs
xfs_repair -n /dev/loop7
# lots of corruption found

[cosmos72]

Thanks, I will look into it

[patrakov]

Any news?

[cosmos72]

not yet, sorry.
Hopefully the winter holidays will give me enough spare time...

[cosmos72]

Bug confirmed. Working on it.

[patrakov]

Still working?

[cosmos72]

Hello,
sorry for the delay.

I reproduced the bug, but in addition to the (very) little available time, I am having some difficulty in finding where the behaviour of the low-RAM case starts diverging from the normal case (ponder)

[giomasce]

It might be sensible to note here too that I added a patch in Debian to display a warning specifically for this bug: https://salsa.debian.org/debian/fstransform/blob/master/debian/patches/0001-Output-explicit-warning-about-data-corruption-with-l.patch

Of course a proper fix would be better, but at least users are warned.

[cosmos72]

Makes sense.
Thanks for reminding me of this issue, I restarted working on it.

[cosmos72]

After two days rewriting a lot of code (in another branch), I finally found the cause:

the loop copying continuous segments larger than RAM buffer was bugged, it incremented only one iteration variable instead of all three.

Fixed in 1a93573