Test checks affect results; Local execution feeling safe; Image Mode support; Subresults in tmt-report-result; Discover adjust-tests

See the release notes for the list of interesting changes.

What's Changed

• Add links to the quay.io repo with tmt images by @psss in #3256
• Fix the copr repo name in the container files by @psss in #3259
• Disable package manager tests of dnf5 on Fedora 39 by @happz in #3262
• Test result key should be implemented as enum, not a string by @happz in #3253
• Increase linkcheck retries from 1 to 3 by @martinhoyer in #3264
• Remove skip from the ResultInterpret enumeration by @psss in #3269
• Remove data-path field from tmt-report-result output by @seberm in #3270
• Replace open+read/write with pathlib's read/write_text/bytes by @happz in #3263
• Replace get() calls with direct data access in discover/shell plugin by @happz in #3249
• Add caching of pip, hatch and hatch envs in doc-tests by @martinhoyer in #3241
• Isolate "effective config path" code in a function by @happz in #3250
• Multiplication was used incorrectly in examples by @lukaszachy in #3278
• Add a new Test Runner section to the guide by @psss in #3201
• Translate beaker.pool hardware requirement properly for mrack by @skycastlelily in #3271
• Improve message provided when plugin IDs collide by @happz in #3288
• Add an extra escape wrapper for illegal XML chars in junit report plugin by @seberm in #3289
• New tests for adjusting duration of a test by @pellecchialuigi in #3281
• The --feeling-safe option required for local provision by @psss in #3282
• Improve CLI rendering of plugin docs by @happz in #3301
• Override test metada from plan using adjust-tests by @lukaszachy in #2865
• Include error and traceback in log file by @happz in #3247
• Suggest using direnv when working on tmt tests by @psss in #3285
• Bump pre-commit checks by @happz in #3303
• Pass {USER, COREOS}_DATA separately in testcloud by @frantisekz in #3266
• Support Fedora Image Mode by @thrix in #3229
• Call the tmt-file-submit script directly by @psss in #3310
• Wait for guest connection before test restart by @happz in #3291
• Support multiple links in the tmt link command by @psss in #3309
• Avoid using raw step data when injecting phases by @happz in #3248
• Add Debian 12.7 among container test images by @happz in #3306
• Translate cpu.stepping hardware requirement for mrack by @skycastlelily in #3295
• Mention the Test Runner section in release notes by @psss in #3312
• Save the subresults for tmt-report-result by @seberm in #3200
• Translate cpu.frequency hardware requirement by @skycastlelily in #3296
• Implement check result key, checks affect test result by default by @martinhoyer in #3239
• Release 1.38.0 by @thrix in #3318

New Contributors

• @pellecchialuigi made their first contribution in #3281

Full Changelog: 1.37.0...1.38.0

New `tmt link` command; `tmt try` options `--epel` and `--install`; reporting improvements

See the release notes for the list of interesting changes.

What's Changed

• Explicitly state polarion report authentication limitation by @KwisatzHaderach in #3188
• Hide internal fields of provision plugins by @happz in #3181
• Drop no longer needed apt update from package manager tests by @happz in #3189
• Fix quay.io builds by @martinhoyer in #3194
• Move structured field into its own module under tmt.utils by @happz in #3192
• Pre commit bump by @martinhoyer in #3197
• Add make target for fetching base images of custom test images we build by @happz in #3190
• Use consistent version names in the release notes by @psss in #3205
• Move and rename SSH master socket path to avoid path length limit by @happz in #3196
• Update label names used for configuring packit jobs by @psss in #3207
• Remove support for junit-xml, use Jinja2 instead by @seberm in #3150
• Fix command detection in the out-of-session test by @psss in #3216
• Fix badly added info about changes in junit report plugin by @seberm in #3218
• Document steps for releasing hotfix changes by @psss in #3217
• Drop no longer needed is_selinux_supported() by @happz in #3191
• Fix position of hardware filter in mrack request by @happz in #3211
• Move ReST rendering for CLI help into tmt.utils submodule by @happz in #3222
• Improve template url generations by @LecrisUT in #3108
• Add sphinx-builder linkcheck by @LecrisUT in #2489
• Reportportal should note guest hostname for investigation by @happz in #3209
• Check both stdout and stderr in package manager tests by @psss in #3227
• Add missing where, name and order keywords into schema by @bajertom in #3225
• Log the information about used beakerlib libraries by @seberm in #3208
• Document the known issue label on the contribute page by @psss in #3220
• Fix missing f-string in the tmt lint check by @psss in #3230
• Do not traceback upon an empty docstring by @psss in #3233
• Add epel prepare plugin using feature into tmt try by @falconizmi in #2985
• Use Jinja2 and polarion JUnit flavor to generate XUnit in polarion report plugin by @seberm in #3166
• Quote values in the tmt-report-results.yaml file by @seberm in #3231
• Drop bootstrap logger requirement from module importer by @happz in #3240
• Forbid use of os.path.join, point user to tmt._compat.Path by @happz in #3210
• Add the link command and linking new tests to jira by @tkoscieln in #2922
• Store the original test, check and subresult outcome in results by @happz in #3147
• Add install prepare plugin using install into tmt try by @falconizmi in #3071
• Add the iommu specification for hardware by @qiankehan in #3100
• Forbid use of os.path, point user to tmt._compat.Path by @happz in #3213
• Add support for kickstart to mrack plugin by @skycastlelily in #3064
• Rewrite reportportal test results reporting by @kkaarreell in #3223
• Release 1.37.0 by @psss in #3255

New Contributors

• @tkoscieln made their first contribution in #2922
• @qiankehan made their first contribution in #3100

Full Changelog: 1.36.0...1.37.0

Hotfix for the ssh master socket

What's Changed

• Move and rename SSH master socket path to avoid path length limit by @happz in #3196
• Release 1.36.1 by @psss in #3206

Custom test results validation; API deprecation warnings; Beaker job ownership; Subresults preparation

See the release notes for the list of interesting changes.

What's Changed

• Add primitives for loading and saving step results [Step results #3] by @happz in #2978
• Correct the description for where under finish by @mcasquer in #3136
• Rename the result phase to subresult by @seberm in #3106
• Use the default branch for polarion tests by @psss in #3145
• pre-commit hooks update; mypy, ruff fixes by @martinhoyer in #3142
• Use concrete step data class when injecting requires by @happz in #3144
• Introduce submodules utils.git and utils.templates by @happz in #2983
• Fix if typo in report spec description by @mcasquer in #3146
• Fix getting excluded variables from environment variable by @psss in #3148
• Prepare and finish steps save results of their phases [Step results #4] by @happz in #2979
• Fix docstrings of regex template filters by @happz in #3011
• Support v0.0.70 and v0.0.72 Artemis API by @happz in #3149
• Do not enable epel on EL 10 by @spetrosi in #3128
• Show subresults and their checks in report display plugin by @seberm in #3107
• Remove redundant parsing user_data code by @skycastlelily in #3119
• reportportal: upload all test case results by @kkaarreell in #3135
• Add (custom) result file validation, schema-based by @happz in #3153
• Fix handling of empty set of allowed linters by @happz in #3158
• Add warnings.deprecated by @LecrisUT in #3062
• Improve the design of the checks and phases in HTML report plugin by @seberm in #3095
• Drop no longer valid limitations of multihost from docs by @happz in #3169
• Fix the operator to be not equal in string values by @mcasquer in #3170
• Fix thundebolt typo by @mcasquer in #3171
• Add Fedora 41 among images we test against by @happz in #3172
• Extend xfail to F41 for pint and nitrate issues by @martinhoyer in #3173
• Move timestamp/duration formatting to utils [Step results #5] by @happz in #3019
• Changing Clean.guests() to accept run IDs by @skycastlelily in #3163
• Move SSH master sockets under workdir by @happz in #3165
• Support different job owner when creating Beaker jobs by @happz in #3179
• Fix running of Ansible playbooks shipped with tmt by @falconizmi in #3126
• Use TMT_REPORT_ARTIFACTS_URL envvar again by @kkaarreell in #3184
• Release 1.36.0 by @martinhoyer in #3187

New Contributors

• @spetrosi made their first contribution in #3128
• @falconizmi made their first contribution in #3126

Full Changelog: 1.35.0...1.36.0

Separate restraint results, improved linting, cleaning & pruning

See the release notes for the list of interesting changes.

What's Changed

• Bump pre-commit checks by @happz in #3023
• Fix inaccuracy of --failed-only release note and docs by @happz in #3030
• Add more tag separator options to polarion import by @KwisatzHaderach in #3010
• Drop pint higher-bound by @LecrisUT in #2741
• Fix our custom Fedora CoreOS test image to avoid dnf/dnf5 conflicts by @happz in #3014
• Workaround for pip test installation on py3.13 by @martinhoyer in #3043
• Ruff rules update by @martinhoyer in #2842
• Use ansible playbook from worktree by @mkluson in #2989
• Add favicon to tmt docs by @happz in #3021
• Add missing test case for parsing device HW requirement by @happz in #3012
• Remove prepare/install fields leaking into prepare step data by @happz in #2986
• Add a linter for Jinja2 templates by @happz in #3048
• Use vault.centos for centos-7 image repos by @lukaszachy in #3070
• Support location.lab-controller hw requirement for mrack by @skycastlelily in #2927
• Add template helper for raising errors by @happz in #3009
• Add descriptions to well-known process exit codes by @happz in #2955
• Allow pre-commit to run in subpath by @LecrisUT in #2007
• Do not install epel-next-release on RHEL-8-like guests by @happz in #3081
• Fix normalization of playbook/playbooks keys of ansible plugin by @happz in #3053
• Fix normalization of regular expression lists by @happz in #3083
• Support system.numanodes hardware requirement for mrack by @skycastlelily in #2925
• Update clean part of the example doc by @skycastlelily in #2883
• Parse the cpu.hyper-threading hardware requirement by @skycastlelily in #2949
• Fix handling of installation of packages from subdirectory by @happz in #3028
• Turn tmt/utils.py into a subpackage by @happz in #2982
• Pin the Sphinx version and update the docs theme by @psss in #3087
• Switch to packit-handled rpm changelog and specfile cleanup by @martinhoyer in #2880
• Mention hard reboot limited support in documentation by @happz in #3068
• Polish docs of the avc test check by @happz in #3029
• Add option to skip output log in junit and polarion reports by @KwisatzHaderach in #3003
• Skip testing debuginfo install on coreos images by @psss in #3091
• Let plugin go() method return a value [Step results #1] by @happz in #2963
• Cleanup deprecated stuff from EPEL8 by @LecrisUT in #3060
• Mention the include-output-log key in release notes by @psss in #3093
• Update HW requirement implementation links to match reality by @skycastlelily in #3066
• Improve initialization message by @abitrolly in #3039
• Polish docs Makefile by @happz in #3045
• Fix collection of know keys for test, plan and story T001 linter by @happz in #3084
• Add documention best practices, adjust release notes by @happz in #3007
• The --quiet option shall silence warnings by @happz in #2909
• Make mypy happy about the error.schema content by @psss in #3098
• Save results before raising exception by @pholica in #2996
• Coerce dimensionless values of SizeConstraint to values with dimensions by @skycastlelily in #3076
• Parse the disk.sector-size hardware requirement by @skycastlelily in #2972
• Add HW requirement support matrix to docs by @happz in #3025
• Propagate result phase into the report plugins by @seberm in #3094
• Minor grammatical update to README by @subpop in #3099
• Allow additional, local CSS file for documentation by @happz in #3026
• Support cpu.hyper-threading hw requirement for mrack by @skycastlelily in #2953
• Fix links to plugins to start with / like other links by @happz in #3104
• Fix a repeating typo in user guide by @mcasquer in #3111
• Change Polarion field which stores tmt id by @KwisatzHaderach in #3080
• Fix loose ends in documentation after renaming plugin hrefs by @happz in #3115
• Support tpm.version hardware requirement for mrack by @skycastlelily in #3022
• Add _compat package by @LecrisUT in #3059
• Use pyproject.toml in the plugin examples by @LecrisUT in #2812
• Support disk.sector-size hardware requirement for mrack by @skycastlelily in #2973
• Allow tmt try IP@connect by @lukaszachy in #2991
• Define Beaker pool HW requirement by @happz in #3058
• Add favicon and logo to rtd by @martinhoyer in #3072
• Avoid conflict between theme CSS and custom CSS files by @happz in #3121
• Use checkmark and cross for hardware support matrix by @LecrisUT in #3122
• Create run.yaml for tmt try by @skycastlelily in #2889
• Move "common keys" section of plugin docs to top of the page by @happz in #3047
• Add the cpu.vendor hardware requirement for mrack by @qcheng-redhat in #2910
• Introduce "phase result" for prepare/finish plugins to return [Step results #2] by @happz in #2977
• Support beaker.pool hardware requirement for mrack by @skycastlelily in #3074
• Retry on error 504 from ReportPortal by @kkaarreell in #3092
• Require --feeling-safe for local provisioning by @martinhoyer in #3103
• Accept --id option multiple times for clean and status by @skycastlelily in #2891
• Add support for cpu.processors HW requirement to virtual plugin by @happz in #3016
• Extend coverage of prepare/finish ansible plugin with more images by @happz in #3110
• base: Honor .gitignore for rsync by @cgwalters in #3037
• Added lint check for duplicate test ids by @engelmi in #3017
• Use per-guest SSH master sockets to prevent re-use by @happz in #3127
• Move abort detection into test invocation by @happz in #3040
• Disable to print color if --no-color is specified by @idorax in #2816
• Improve version comparison in --feeling-safe check by @happz in #3133
• Switch testcloud plugin to the Workarounds API by @frantisekz in #3036
• Addition of test coverage for extended features of reportportal plugin by @4N0body5 in #2929
• Fix a bug when prune is used along with path by @KwisatzHaderach in #3132
• Packit configuration maintenance by @LecrisUT in #3024
• Implement separate results for rstrnt-report-result by @pfdaly in #2689
• Implement --last, --id and --skip for tmt clean by @skycastlelily in #2806
• Release 1.35.0 by @martinhoyer in #3141

New Contributors

• @mkluson made their first contribution in #2989
• @abitrolly made their first contribution in #3039
• @pholica made their first contribution in #2996
• @seberm made their first contribution in #3094
• @subpop made their first contribution in #3099
• @mcasquer made their first contribution in #3111
• @cgwalters made their first contribution in #3037
• @engelmi made their first contribution in #3017

Full Changelog:...

Duration multiplication; `--failed-only` option; Multiple disks; Reportportal descriptions

See the release notes for the list of interesting changes.

• Run the no-commit-to-branch check only locally by @psss in #2913
• Update interaction with Fedora rawhide by @happz in #2931
• Drop duplicate list of supported HW requirements in beaker plugin by @happz in #2902
• Update distros in the mrack provisioning config by @psss in #2932
• Enable pydocstyle checks in ruff by @happz in #2906
• Enforce "one blank line after class docstring" rule by @happz in #2934
• Support cpu.model-name hardware requirement for mrack by @skycastlelily in #2921
• Pass correct spec to _parse_system and _parse_location by @skycastlelily in #2924
• Cover tmt.steps.prepare.distgit with pyright by @happz in #2944
• Fix pre-commit typing python version by @LecrisUT in #2959
• Ignore report files created by pytest-html by @happz in #2962
• Cover tmt.steps.prepare.shell with pyright checks by @happz in #2943
• Fix some invalid links in documentation by @happz in #2940
• Add konflux as possible initiator by @lukaszachy in #2947
• Clarify the trigger context dimension values by @psss in #2961
• Cover tmt.trying with pyright check by @happz in #2942
• Fix validation of fmf id URL, it must be cloneable first by @happz in #2957
• Allow multiplication in duration input value by @lukaszachy in #2845
• Tweak docs to include how to see details about test failures by @mvo5 in #2541
• Polish test-check/dmesg documentation by @happz in #2897
• Display locals in traceback when requested by @happz in #2900
• Fix path to watchdog log by @happz in #2901
• Extend results schema to allow test phases by @happz in #2933
• Fix "skip missing" behavior when installing debuginfo packages with dnf by @happz in #2956
• Fix multiple rules adjust example in duration spec by @psss in #2971
• Require python3.9 in 'develop' target by @lukaszachy in #2993
• Document custom container images built for tmt tests by @happz in #2930
• Split provision/virtual plans to subgroups per step by @happz in #2938
• Add support for basic rerun of tests by @KwisatzHaderach in #2521
• Make test shell wrapper filename even more unique by @happz in #2998
• Simplify parsing of HW requirements and add missing ones by @happz in #2928
• Improve rendering of story links in our docs by @happz in #3001
• Add support for multiple disks to testcloud plugin by @happz in #2767
• Copy launch description to suite description by @kkaarreell in #2992
• Keep patched sources directly in TMT_SOURCE_DIR by @lukaszachy in #2999
• Fix normalization of hardware key/option by @happz in #3015

Full Changelog: 1.33.0...1.34.0

1.33.0 SSH options, test restart counter, hardware specifications improvements, pre-commit fixes, rpm-ostree improvements

What's Changed

• Update logger-name for clean by @skycastlelily in #2853
• Document custom Jinja template filters by @happz in #2769
• Fix incorrect translation of cpu.processors to Beaker by @skycastlelily in #2851
• Add the cpu.vendor hardware requirement by @qcheng-redhat in #2875
• Allow pip install from git-archive by @LecrisUT in #2879
• Add support cpu.cores hardware requirement for mrack by @skycastlelily in #2728
• Polish prepare/ansible documentation by @happz in #2824
• Re-use "become" image to fix sudo-related trouble in /tests/provision/user by @happz in #2892
• Fix AVC denial detection on Fedora rawhide by @happz in #2881
• Accept SSH options from environment variable by @happz in #2783
• Add a new section Review to contributor docs by @psss in #2886
• Add support for virtualization.hypervisor hardware requirement by @skycastlelily in #2733
• Make junit_xml more visible to linters by @happz in #2782
• Allow custom failure patterns in dmesg test check by @happz in #2786
• Fix Alpine Linux support by @thrix in #2828
• Fix inability of rpm-ostree to install multiple packages by @happz in #2887
• Log hash of the beakerlib library repo by @lukaszachy in #2843
• Allow duration in the execute schema by @lukaszachy in #2898
• Slightly improve rendring of ReST for command line by @happz in #2823
• Fix copy/paste issue for combine_hw_disk_size() by @mvo5 in #2899
• Fix error message if no supported package manager found by @thrix in #2829
• Add test restart counter, similar to TMT_REBOOT_COUNT by @happz in #2787
• Add codespell pre-commit and fix revealed errors by @LecrisUT in #2819
• Test restart on crash by @happz in #2870
• Define location.lab-controller hardware requirement by @happz in #2794
• Parse the system hardware requirement by @happz in #2795
• Improve fail message when library doesn't exist by @lukaszachy in #2839
• Install dependencies of upgrade tasks by @martinhoyer in #2866
• Simplify the documentation building by @LecrisUT in #2483
• Fix installation of packages from files by @happz in #2914

New Contributors

• @qcheng-redhat made their first contribution in #2875
• @mvo5 made their first contribution in #2899

Full Changelog: 1.32.2...1.33.0

Package manager discovery order

See the release notes for the list of interesting changes.

• Refresh Beaker API client in mrack to use up-to-date Kerberos ticket by @happz in #2775
• Correctly show the --provision-timeout default by @psss in #2817
• Remove the .travis.yml config file by @LecrisUT in #2811
• Clarify the --last symlink creation race a bit by @psss in #2818
• Removing c9s workaround for pytest-container by @martinhoyer in #2836
• Update pre-commit checks and add a couple of new ones by @happz in #2835
• Add unit test for package manager installing a file system path by @happz in #2830
• Refactor package manager discovery to allow priorities by @happz in #2861
• Removing /var/tmp workdir from rpm packaging by @martinhoyer in #2876

Full Changelog: 1.32.1...1.32.2

1.32.1

• Update pytest on c9s when using site-packages
• Fixing spec file version in order to be able to update Fedora packages.

1.32.0 Package managers; Hardware specifications improvements; Watchdog test check..

See the release notes for the list of interesting changes.

• Fix dmesg check test for recent Fedoras by @happz in #2674
• Support URL as post-install-script for artemis plugin by @skycastlelily in #2628
• Enable the full test execution using a label by @psss in #2683
• Drop two no longer used methods from prepare step by @happz in #2672
• Underline try menu keys by @lukaszachy in #2679
• Allow custom boot/connect timeouts for testcloud VMs by @happz in #2677
• Cover tmt.templates with pyright by @happz in #2644
• Make checks smarter about environments in which they should not run by @happz in #2686
• Introduce "topology" addresses for guests by @happz in #2670
• Test the become key under virtual provision by @carlosrodfern in #2659
• Collect test/test check results in test invocation by @happz in #2608
• Check for files in rpm-ostree install script by @KwisatzHaderach in #2692
• Properly detect if test is enabled somewhere by @lukaszachy in #2662
• Use the PROVISION_HOW variable in provision plans by @psss in #2688
• Remove mention about the full test suite from docs by @psss in #2701
• Add essential requirements to checks by @happz in #2684
• Enable additional_coverage tests for pull requests by @psss in #2700
• Convert environment handling into a dict-like class by @happz in #2612
• "Show default" flag was dropped from CLI option initialization by @happz in #2710
• Bump supported Artemis API versions to 0.0.69 by @happz in #2711
• Add Environment and FmfContext among loggable types by @happz in #2709
• Mark test check tests with more fitting provision tags by @happz in #2707
• Move deferred user stories into a separate section by @psss in #2715
• Add support for disk.model-name hardware requirement by @skycastlelily in #2708
• Extend the duration for time-demanding tests by @psss in #2718
• Set acl permissions on workdir root to ensure multi-user access when umask is 0027 by @carlosrodfern in #2510
• Use /usr/sbin/sestatus instead of /usr/bin/sestatus by @guoguojenna in #2720
• Log the beaker job whiteboard by @rh-mcermak in #2719
• Disable password authentication when using keys by @frantisekz in #2695
• Document how to enable verbose/debug logging by @psss in #2727
• Pass domain_configuration to testcloud's wake() by @frantisekz in #2724
• Include context in the test metadata.yaml file by @LecrisUT in #2716
• Support disk.model-name hw requirement for mrack by @skycastlelily in #2705
• Remove now unnecessary NON_KVM_ADDITIONAL_WAIT from testcloud by @frantisekz in #2699
• Add support for disk.driver hardware requirement by @skycastlelily in #2667
• Disable IPv6 for /plans/provision/virtual in CI by @happz in #2735
• Set default formatting of Pint units to use symbols, not names by @happz in #2736
• Add support for emulated TPM into virtual provision plugin by @happz in #2596
• Bootstrap unit tests for Beaker XML transformation of HW requirements by @happz in #2734
• Cover tmt.hardware with pyright check by @happz in #2742
• Support disk.driver hw requirement for mrack by @skycastlelily in #2668
• Implement the new test check watchdog by @happz in #2412
• Cover report/junit with pyright by @happz in #2725
• Include a simple config for the polarion plugin by @psss in #2750
• Move /tests/run/shell to /plans/provision/local by @psss in #2753
• Allow commands whose output is not logged unless they fail by @happz in #2751
• Extract "package manager" functionality into plugins by @happz in #2557
• Run /plans/install/docs in core packit jobs by @psss in #2747
• Finish unit testing of HW transformations in mrack plugin by @happz in #2748
• Debug output for discover -h fmf --modified-only by @lukaszachy in #2760
• Select no tests if modified-only returns nothing by @lukaszachy in #2761
• Add new environment variable for test iteration identifier by @therazix in #2726
• Fix typos in the tmt try help message examples by @psss in #2778
• Update pre-commit checks by @happz in #2776
• Increase the default test duration a bit by @psss in #2777
• Clarify that plan context is not applied to plan by @psss in #2779
• Document test tags and mark beakerlib integration by @lukaszachy in #2731
• Add support for linking artifacts from the report by @bajertom in #2732
• Introduce ansible-lint to format ansible files by @idorax in #2666
• Add support for zcrypt to Beaker plugin by @happz in #2697
• Prepare dist-git sources using rpmbuild -bp by @lukaszachy in #2641
• Use TMT_REPORT_ARTIFACTS_URL in Polarion report plugin by @KwisatzHaderach in #2785
• Allow keep-git-metadata to be used without url by @lukaszachy in #2698
• Update the reportportal plugin by @4N0body5 in #2702
• Support Alpine Linux apk package manager by @thrix in #2788

New Contributors

• @skycastlelily made their first contribution in #2628
• @rh-mcermak made their first contribution in #2719
• @LecrisUT made their first contribution in #2716

Full Changelog: 1.31.0...1.32.0