casenorm/mixed_create_failure broken on FreeBSD (false positive heretofor) – silent data loss (no link to O_CREATed inode) + inode leak

[nabijaczleweli]

System information

Type	Version/Name
Distribution	FreeBSD freebsd 13.0-RELEASE FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr 9 04:24:09 UTC 2021 [email protected]:/usr/obj/usr/src/amd64.a md64/sys/GENERIC amd64
OpenZFS Version	HEAD (1282274 + userspace delta)

Describe the problem you're observing

On HEAD, as reference, see http://build.zfsonlinux.org/builders/FreeBSD%20main%20amd64%20%28TEST%29/builds/3262/steps/shell_4/logs/log:

20:50:08.70 NOTE: cmd: touch /mnt/testdir/file/ABCDEFGHIJKL ret: 0 out=
20:50:08.70 NOTE: Test rename \"sample_name\" rename
20:50:08.71 /usr/local/share/zfs/zfs-tests/tests/functional/casenorm/mixed_create_failure.ksh[133]: test_ops[125]: log_err: not found [No such file or directory]
20:50:08.71 NOTE: Testing create of dir
20:50:08.71 NOTE: The op : mkdir

When I fixed that, I got the loop also running to completion and

04:52:12.63 NOTE: cmd: touch /mnt/testroot/testdir/file/ABCDEFGHIJKL ret: 0 out=
04:52:12.63 NOTE: Test rename "sample_name" rename
04:52:12.64 SUCCESS: touch /mnt/testroot/testdir/file/tmp_obj
04:52:12.65 touch /mnt/testroot/testdir/file/tmp_obj failed: usage: mv [-f | -i | -n] [-hv] source target
04:52:12.65        mv [-f | -i | -n] [-v] source ... directory

(coming from the empty $save_name – i.e. the touch never failed).

Additional testing outside the ZTS confirms this – this test doesn't fail on FreeBSD, but!!!!!! The file doesn't get created, either!:

% ls -l /testpool/fs/ABCDEFGHIJKL
ls: /testpool/fs/ABCDEFGHIJKL: No such file or directory
% echo dupa > /testpool/fs/ABCDEFGHIJKL
% ls -l /testpool/fs/ABCDEFGHIJKL
ls: /testpool/fs/ABCDEFGHIJKL: No such file or directory

% ktrace touch /testpool/fs/ABCDEFGHIJKL
% kdump | tail -14
 88833 touch    CALL  mprotect(0x203000,0x1000,0x1<PROT_READ>)
 88833 touch    RET   mprotect 0
 88833 touch    CALL  fstatat(AT_FDCWD,0x7fffffffee0e,0x7fffffffe950,0)
 88833 touch    NAMI  "/testpool/fs/ABCDEFGHIJKL"
 88833 touch    RET   fstatat -1 errno 2 No such file or directory
 88833 touch    CALL  openat(AT_FDCWD,0x7fffffffee0e,0x201<O_WRONLY|O_CREAT>,0666<S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH>)
 88833 touch    NAMI  "/testpool/fs/ABCDEFGHIJKL"
 88833 touch    RET   openat 3
 88833 touch    CALL  fstat(0x3,0x7fffffffe950)
 88833 touch    STRU  struct stat {dev=11961901315426196474, ino=14881, mode=0100644, nlink=1, uid=1001, gid=0, rdev=18446744073709551615, atime=1647269330.748562000, mtime=1647269330.748562000, ctime=1647269330.748562000, birthtime=1647269330.748562000, size=0, blksize=131072, blocks=1, flags=0x800 }
 88833 touch    RET   fstat 0
 88833 touch    CALL  close(0x3)
 88833 touch    RET   close 0
 88833 touch    CALL  exit(0)
% ls -l /testpool/fs/ABCDEFGHIJKL
ls: /testpool/fs/ABCDEFGHIJKL: No such file or directory

Whichever error prevented the creation of the link (since inodes do get allocated) is not bubbled down to userspace.

For comparison, under Linux (where this works), the strace looks like this:

close(3)                                = 0
openat(AT_FDCWD, "/testpool/fs/abCDEFGHijKL", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = -1 ENOSPC (No space left on device)
utimensat(AT_FDCWD, "/testpool/fs/abCDEFGHijKL", NULL, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3

[nabijaczleweli]

Haha, it also leaks the inode:

$ df -i /testpool/fs/
Filesystem  1K-blocks Used    Avail Capacity iused    ifree %iused  Mounted on
testpool/fs  40140633 2447 40138186     0%   32442 80276373    0%   /testpool/fs
$ while :; do echo > /testpool/fs/ABCDEFGHIJKL; done
^C
$ cat /testpool/fs/ABCDEFGHIJKL
cat: /testpool/fs/ABCDEFGHIJKL: No such file or directory
$ df -i /testpool/fs/
Filesystem  1K-blocks Used    Avail Capacity iused    ifree %iused  Mounted on
testpool/fs  40140632 3605 40137027     0%   50116 80274054    0%   /testpool/fs

And a scrub hasn't fixed it :)