tatt: poor use-flag randomization logic with "--use-combos x"

[juippis]

Hey,

I often like to compile packages with 6-12 use flag combinations to test these widely. But pkgdev tatt has a very poor randomization logic. Try Firefox or Thunderbird for example, they have tons of use flags available (not including l10n_), but pkgdev tatt only seems to select a handful of use flags from these packages and it proceeds to toggle these selected ones. Not to mention, it doesn't consider the profile-defaults at all. So it may toggle use flags that are profile/ebuild enabled/disabled, without actually changing the outcome.

A great example:

pkgdev tatt --emerge-opts="--autounmask=y --verbose" --use-combos 6 --ignore-prefixes="llvm_slot,pgo,elibc,debug,selinux" --extra-env-file test.conf -p =www-client/firefox-128.3.0 -j tatt-firefox-128.3.0

Results in:

    TUSE="-jack -system-webp -pulseaudio -system-png " tatt_test_pkg '=www-client/firefox-128.3.0' || test_ret=1
    TUSE="-system-png -system-webp -jack -pulseaudio telemetry " tatt_test_pkg '=www-client/firefox-128.3.0' || test_ret=1
    TUSE="-jack -pulseaudio -system-png " tatt_test_pkg '=www-client/firefox-128.3.0' || test_ret=1
    TUSE="-jack telemetry -pulseaudio -system-png " tatt_test_pkg '=www-client/firefox-128.3.0' || test_ret=1
    TUSE="-jack -system-webp -pulseaudio " tatt_test_pkg '=www-client/firefox-128.3.0' || test_ret=1
    TUSE="-jack -system-webp telemetry -pulseaudio " tatt_test_pkg '=www-client/firefox-128.3.0' || test_ret=1

As you can see, while Firefox has multiple more use flags available, pkgdev tatt only selected -system-png -system-webp -jack -pulseaudio telemetry to do 6 "different" runs from.
-jack -system-png -system-webp -pulseaudio +telemetry are the profile/ebuild defaults (in default/linux/amd64/23.0), so essentially all of the runs above are identical and no use flag changes will happen between them.

It should try to randomize more use flag combinations and utilize all available use flags:
X clang dbus gmp-autoupdate jumbo-build system-av1 system-harfbuzz system-icu system-jpeg system-libevent system-libvpx telemetry -eme-free -gnome-shell -hardened -hwaccel -jack -libproxy -lto -openh264 -pgo -pulseaudio -sndio -system-png -system-webp -wayland -wifi

For example, compare to pkg-testing-tool:
USE="dbus" pkg-testing-tool --append-emerge '--autounmask=y --oneshot' --extra-env-file 'test.conf' --test-feature-scope never --append-required-use '!pgo' --max-use-combinations 6 -p '=www-client/firefox-128.3.0'

=www-client/firefox-128.3.0    USE: -X -clang dbus -eme-free gmp-autoupdate -gnome-shell -hardened -hwaccel jack jumbo-build -libproxy llvm_slot_17 -llvm_slot_18 lto -openh264 -pgo -pulseaudio -sndio -system-av1 system-harfbuzz system-icu -system-jpeg -system-libevent system-libvpx -system-png -system-webp -telemetry wayland wifi
=www-client/firefox-128.3.0    USE: -X -clang dbus eme-free -gmp-autoupdate gnome-shell -hardened hwaccel -jack jumbo-build -libproxy llvm_slot_17 -llvm_slot_18 -lto openh264 -pgo pulseaudio -sndio -system-av1 -system-harfbuzz -system-icu -system-jpeg system-libevent -system-libvpx -system-png system-webp -telemetry wayland -wifi
=www-client/firefox-128.3.0    USE: X clang dbus eme-free gmp-autoupdate -gnome-shell -hardened hwaccel -jack jumbo-build -libproxy -llvm_slot_17 llvm_slot_18 -lto openh264 -pgo pulseaudio -sndio system-av1 system-harfbuzz -system-icu -system-jpeg system-libevent -system-libvpx -system-png system-webp -telemetry -wayland wifi
=www-client/firefox-128.3.0    USE: X -clang dbus eme-free -gmp-autoupdate -gnome-shell hardened hwaccel jack jumbo-build libproxy -llvm_slot_17 llvm_slot_18 lto -openh264 -pgo pulseaudio sndio system-av1 -system-harfbuzz system-icu system-jpeg -system-libevent -system-libvpx system-png system-webp telemetry -wayland -wifi
=www-client/firefox-128.3.0    USE: -X -clang dbus eme-free gmp-autoupdate gnome-shell hardened -hwaccel -jack -jumbo-build libproxy llvm_slot_17 -llvm_slot_18 lto -openh264 -pgo -pulseaudio -sndio system-av1 system-harfbuzz system-icu -system-jpeg system-libevent system-libvpx -system-png -system-webp -telemetry wayland wifi
=www-client/firefox-128.3.0    USE: X clang dbus -eme-free -gmp-autoupdate -gnome-shell hardened -hwaccel -jack -jumbo-build libproxy llvm_slot_17 -llvm_slot_18 -lto -openh264 -pgo -pulseaudio sndio -system-av1 system-harfbuzz -system-icu system-jpeg -system-libevent system-libvpx -system-png -system-webp telemetry -wayland -wifi

Every use flag gets toggled and tested at some point.

I know it's a hard problem to solve, but I hope I was able to write clearly what I find problematic about it :)