zdb: include cloned blocks in block statistics

[robn]

Motivation and Context

This gives zdb -b support for clone blocks.

Previously, it didn't know what clones were, so would count their space allocation multiple times and then report leaked space (or, in debug, would assert trying to claim blocks a second time).

This commit fixes those bugs, and reports the number of clones and the space "used" (saved) by them.

Sponsored-By: OpenDrives Inc.
Sponsored-By: Klara Inc.

Description

If the BRT is active, we set up our own tree to track the refcounts for cloned blocks.

The first time we see a block, we look it up in the real BRT. If its there (ie has clones), we record the refcount in our own tree, then count the block as normal.

The second and later time we see the block, we look it up in our own tree. We count the clone, decrement the refcount, and return (so as not to claim the block a second time).

Later, when finalising the stats, we reduce the total allocation on the pool by the total of all cloned blocks (much like we do for dedup blocks). This stops the size being reported as "leaked".

When the last clone is seen, we remove our memory of the block entirely, to try to keep memory usage down.

This adds a function to the BRT proper, brt_entry_get_refcount, to lookup the entry and return its refcount.

It's worth noting that this could be easily adjusted to check for the BRT refcount being incorrect, however that's not a thing that can happen at the moment (pretty sure) and not quite why I was here, so I didn't bother.

How Has This Been Tested?

By hand:

# zpool create tank loop0 loop1 loop2 loop3
# dd if=/dev/random of=/tank/file bs=4K count=1
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000226594 s, 18.1 MB/s
# zpool sync
# clonefile -c /tank/file /tank/clone
using FICLONE
file offsets: src=0/4096; dst=0/4096
# clonefile -c /tank/file /tank/clone2
using FICLONE
file offsets: src=0/4096; dst=0/4096
# clonefile -c /tank/file /tank/clone3
using FICLONE
file offsets: src=0/4096; dst=0/4096
# clonefile -c /tank/file /tank/clone4
using FICLONE
file offsets: src=0/4096; dst=0/4096
# clonefile -c /tank/file /tank/clone5
using FICLONE
file offsets: src=0/4096; dst=0/4096
# clonefile -c /tank/file /tank/clone6
using FICLONE
file offsets: src=0/4096; dst=0/4096
# zpool sync
# zdb -b tank

Traversing all blocks to verify nothing leaked ...

loading concrete vdev 3, metaslab 4 of 5 ...

	No leaks (block sum matches space maps exactly)

	bp count:                    87
	ganged count:                 0
	bp logical:             2824704      avg:  32467
	bp physical:              79360      avg:    912     compression:  35.59
	bp allocated:            167424      avg:   1924     compression:  16.87
	bp deduped:                   0    ref>1:      0   deduplication:   1.00
	bp cloned:                24576    count:      6
	Normal class:            116736     used:  0.03%
	Embedded log class              0     used:  -nan%

	additional, non-pointer bps of type 0:         36

I don't see any existing tests of zdb -b, so I haven't added anything for this. Some tests use zdb -b but I'm not expecting this to affect their behaviour.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[behlendorf]

The approach here makes sense to me. When looking the the two zloop failures we appear to have two problems though:

1. ztest appears not to have been extended to exercise the BRT code. That's a shame since I'd have expected it to report this leak right away and we would have caught this. We should add at least one test which calls zfs_clone_range().

2. ztest itself appears to have failed since the instances ran out of free space. That wasn't caused by this PR but we should determine a reasonable way to avoid that prevent these kind of false positive failures.

[oromenahar]

looks good for me, tanks @robn

[robn]

@behlendorf I agree on both of the ztest things. Unless you feel very strongly about it though, I'd rather look at them in another PR, mostly because I don't have the time budget to definitely get it done before 2.2. If not, then I may be able to make a little time for it on the weekend, I'll see what I can do.