v5.3.7

v5.3.7

This release contains 3 new tests from our new update v1.6 Best Practices documentation. It also includes fixes and improvements surrounding support for cluster-wide operator detection for our existing tests.

New Tests

• operator: Add no hugepages test by @sebrandon1 in #2505
• operator: add olm.skipRange annotation test by @sebrandon1 in #2508
• networking: Add MTU value test for SRIOV NAD's by @sebrandon1 in #2514

Test Enhancements

• Cluster-wide operators support improvements by @greyerof in #2479

CI Updates

• Remove ref for operator branch by @sebrandon1 in #2517
• Adjust typos linter; one typo by @sebrandon1 in #2504
• Manually re-run CATALOG.md build by @sebrandon1 in #2509
• Fix typo by @sebrandon1 in #2512

Operator Script Updates

• Updating the default list of telco operators by @edcdavid in #2523
• Authenticate results spreadsheet creation using a google service account by @shirmoran in #2497
• Add log messages to results spreadsheet app by @shirmoran in #2503

Dependency Updates

• Update openshift libraries manually by @sebrandon1 in #2506
• Bump actions/upload-artifact from 4.4.1 to 4.4.3 by @dependabot in #2507
• Bump helm.sh/helm/v3 from 3.16.1 to 3.16.2 by @dependabot in #2510
• Bump google.golang.org/api from 0.199.0 to 0.200.0 by @dependabot in #2511
• Bump ubi9/python-39 from 1-197.1726664308 to 1-197.1726696853 in /.github/actions/documentation by @dependabot in #2519
• Bump google.golang.org/api from 0.200.0 to 0.201.0 by @dependabot in #2520
• Bump github.com/redhat-best-practices-for-k8s/privileged-daemonset from 1.0.37 to 1.0.38 by @dependabot in #2521
• Bump github.com/redhat-best-practices-for-k8s/oct from 0.0.25 to 0.0.26 by @dependabot in #2522
• Bump github/codeql-action from 3.26.12 to 3.26.13 by @dependabot in #2516
• Bump github.com/k8snetworkplumbingwg/network-attachment-definition-client from 1.7.4 to 1.7.5 by @dependabot in #2518

Full Changelog: v5.3.6...v5.3.7

v5.3.6

v5.3.6

v5.3.6 contains a small update to bring in 4.16 and 4.17 OCP versions into the compatibility test matrix.

For more information about OpenShift Lifecycle dates: https://access.redhat.com/support/policy/updates/openshift

Improvements

• compatibility: Add 4.16 and 4.17 records to matrix by @sebrandon1 in #2485
• Switch comparisons from reflect.DeepEqual to slices/maps libs by @sebrandon1 in #2484
• Assert equal vs Assert true for comparisons by @sebrandon1 in #2486

Dependency Updates

• Bump github.com/redhat-best-practices-for-k8s/privileged-daemonset from 1.0.35 to 1.0.36 by @dependabot in #2468
• Bump github.com/redhat-best-practices-for-k8s/oct from 0.0.23 to 0.0.24 by @dependabot in #2469
• Bump google.golang.org/api from 0.198.0 to 0.199.0 by @dependabot in #2472
• Bump actions/checkout from 4.1.7 to 4.2.0 by @dependabot in #2471
• Bump docker/build-push-action from 6.7.0 to 6.8.0 by @dependabot in #2474
• Bump docker/build-push-action from 6.8.0 to 6.9.0 by @dependabot in #2477
• Bump github/codeql-action from 3.26.9 to 3.26.10 by @dependabot in #2478
• Bump github.com/k8snetworkplumbingwg/network-attachment-definition-client from 1.7.3 to 1.7.4 by @dependabot in #2481
• Update Go to v1.23.2 by @sebrandon1 in #2482
• Bump golangci/golangci-lint-action from 6.1.0 to 6.1.1 by @dependabot in #2487
• Bump docker/setup-buildx-action from 3.6.1 to 3.7.0 by @dependabot in #2488
• Bump github.com/redhat-best-practices-for-k8s/oct from 0.0.24 to 0.0.25 by @dependabot in #2490
• Bump github.com/redhat-best-practices-for-k8s/certsuite-claim from 1.0.49 to 1.0.50 by @dependabot in #2489
• Bump github.com/redhat-best-practices-for-k8s/privileged-daemonset from 1.0.36 to 1.0.37 by @dependabot in #2491
• Bump docker/setup-buildx-action from 3.7.0 to 3.7.1 by @dependabot in #2493
• Bump github/codeql-action from 3.26.10 to 3.26.11 by @dependabot in #2494
• Bump golang.org/x/term from 0.24.0 to 0.25.0 by @dependabot in #2495
• Bump actions/checkout from 4.2.0 to 4.2.1 by @dependabot in #2500
• Bump github/codeql-action from 3.26.11 to 3.26.12 by @dependabot in #2501
• Bump actions/upload-artifact from 4.4.0 to 4.4.1 by @dependabot in #2502
• Update probe to v0.0.10 by @sebrandon1 in #2498
• Update preflight to v1.10.1 by @sebrandon1 in #2499

Full Changelog: v5.3.5...v5.3.6

v5.3.5

v5.3.5

There was an improvement for the SCC categories test and now you are also able to see the subscriptions, operator groups, and the install plans in the claim file.

Improvements

• Save all the subscriptions, operator groups, install plans configured in the cluster to the claim file by @aabughosh in #2453
• Fixes for certsuite claim compare UTs. by @greyerof in #2455
• Remove cnf by @bnshr in #2452
• Add Results spreadsheet constants and public functions by @shirmoran in #2466
• Fix for the SCC categories check. by @greyerof in #2460

Dependency Updates

• Bump github.com/redhat-best-practices-for-k8s/certsuite-claim from 1.0.48 to 1.0.49 by @dependabot in #2449
• Update probe image to v0.0.9 by @sebrandon1 in #2448
• Update operator-sdk to v1.37.0 by @sebrandon1 in #2450
• Bump peter-evans/create-pull-request from 7.0.3 to 7.0.4 by @dependabot in #2454
• Bump peter-evans/create-pull-request from 7.0.4 to 7.0.5 by @dependabot in #2456
• Bump github/codeql-action from 3.26.7 to 3.26.8 by @dependabot in #2457
• Bump google.golang.org/api from 0.197.0 to 0.198.0 by @dependabot in #2458
• Bump github.com/mittwald/go-helm-client from 0.12.13 to 0.12.14 by @dependabot in #2465
• Bump ubi9/python-39 from 1-197.1725907694 to 1-197.1726664308 in /.github/actions/documentation by @dependabot in #2462
• Bump ubi9/ubi from 9.4-1214.1725849297 to 9.4-1214.1726694543 by @dependabot in #2464
• Bump ubi9/ubi-minimal from 9.4-1227.1725849298 to 9.4-1227.1726694542 by @dependabot in #2463
• Bump github/codeql-action from 3.26.8 to 3.26.9 by @dependabot in #2467

Full Changelog: v5.3.4...v5.3.5

v5.3.4

v5.3.4

This is a patch update to bump a number of dependencies. No functional changes related to the tests themselves.

Improvements

• Remove logrus as direct dependency by @sebrandon1 in #2417
• Fix flaky test TestEvaluateAPICompliance by @sebrandon1 in #2430
• Brute force TestEvaluateAPICompliance unit test fix by @sebrandon1 in #2438

CI Updates

• Remove collector's sanity check from pre-main workflow by @shirmoran in #2420
• Add docker image cleanup to self-hosted YAMLs by @sebrandon1 in #2421
• Restrict OCP self-hosted PR runners by @sebrandon1 in #2437

Operator Scripting Updates

• Add upload results-spreadsheet certsuite sub-command by @shirmoran in #2389
• Make necessary results spreadsheet functions public by @shirmoran in #2442
• Add public constants for results spreadsheet by @shirmoran in #2447
• Add Setter and Getter functions to results spreadsheet google creds by @shirmoran in #2443

Dependency Updates

• Bump ubi9/ubi-minimal from 9.4-1227 to 9.4-1227.1725849298 by @dependabot in #2410
• Bump ubi9/ubi from 9.4-1214 to 9.4-1214.1725849297 by @dependabot in #2411
• Bump ubi9/python-39 from 1-197 to 1-197.1725907694 in /.github/actions/documentation by @dependabot in #2412
• Bump github.com/redhat-best-practices-for-k8s/oct from 0.0.21 to 0.0.22 by @dependabot in #2413
• Bump github.com/redhat-best-practices-for-k8s/certsuite-claim from 1.0.47 to 1.0.48 by @dependabot in #2414
• Bump github.com/redhat-best-practices-for-k8s/privileged-daemonset from 1.0.33 to 1.0.34 by @dependabot in #2415
• Update probe image to v0.0.8 by @sebrandon1 in #2416
• Bump golang.org/x/oauth2 from 0.22.0 to 0.23.0 by @dependabot in #2423
• Bump peter-evans/create-pull-request from 7.0.1 to 7.0.2 by @dependabot in #2422
• Bump k8s.io/apiextensions-apiserver from 0.31.0 to 0.31.1 by @dependabot in #2425
• Bump google.golang.org/api from 0.193.0 to 0.197.0 by @dependabot in #2426
• Bump helm.sh/helm/v3 from 3.15.4 to 3.16.1 by @dependabot in #2428
• Update yaml from v2 to v3 by @sebrandon1 in #2418
• Bump github.com/redhat-best-practices-for-k8s/privileged-daemonset from 1.0.34 to 1.0.35 by @dependabot in #2431
• Bump github.com/redhat-best-practices-for-k8s/oct from 0.0.22 to 0.0.23 by @dependabot in #2432
• Bump k8s.io/kubectl from 0.31.0 to 0.31.1 by @dependabot in #2433
• Bump github/codeql-action from 3.26.6 to 3.26.7 by @dependabot in #2436
• Bump github.com/k8snetworkplumbingwg/network-attachment-definition-client from 1.7.1 to 1.7.3 by @dependabot in #2434
• Bump peter-evans/create-pull-request from 7.0.2 to 7.0.3 by @dependabot in #2445

Full Changelog: v5.3.3...v5.3.4

v5.3.3

v5.3.3

In v5.3.3 we have added a few notable items:

• Fix for the operator-install-source test to correctly search through all available subscriptions for a valid subscription for cluster-wide operators.
• The observability suite has a new test that checks for future API version incompatibilities in workloads.

Improvements

• refactor : Renaming certsuite command tool by @bnshr in #2379
• refactor : Changes made for the change of config file name by @bnshr in #2400
• Fix some typos by @sebrandon1 in #2398
• Rename references from 'debug' to 'probe' by @sebrandon1 in #2397
• Add check to ensure workload compliance with the next k8s version by @tkrishtop in #2303
• Look through all subscriptions for install source by @sebrandon1 in #2407

Dependency Updates

• Bump ubi9/ubi from 9.4-1181 to 9.4-1181.1724035907 by @dependabot in #2382
• Bump github.com/Masterminds/semver/v3 from 3.2.1 to 3.3.0 by @dependabot in #2383
• Bump github/codeql-action from 3.26.5 to 3.26.6 by @dependabot in #2384
• Bump actions/upload-artifact from 4.3.6 to 4.4.0 by @dependabot in #2391
• Bump peter-evans/create-pull-request from 6.1.0 to 7.0.0 by @dependabot in #2392
• Bump ubi9/python-39 from 1-192.1724040313 to 1-197 in /.github/actions/documentation by @dependabot in #2396
• Bump ubi9/ubi from 9.4-1181.1724035907 to 9.4-1214 by @dependabot in #2393
• Bump ubi9/ubi-minimal from 9.4-1194 to 9.4-1227 by @dependabot in #2394
• Bump golang.org/x/term from 0.23.0 to 0.24.0 by @dependabot in #2395
• Bump github.com/redhat-best-practices-for-k8s/oct from 0.0.20 to 0.0.21 by @dependabot in #2404
• Bump peter-evans/create-pull-request from 7.0.0 to 7.0.1 by @dependabot in #2403
• Update Go to v1.23.1 by @sebrandon1 in #2402

CI Updates

• Update golangci settings by @sebrandon1 in #2386
• Avoid failing workflow when collectors sanity check fails by @shirmoran in #2390
• Temporarily disable collector sanity check by @sebrandon1 in #2399
• Remove bad ref to QE repo in ARM runs by @sebrandon1 in #2401
• Adjust docker to use /mnt sdb for storage in QE hosted by @sebrandon1 in #2406

New Contributors

• @tkrishtop made their first contribution in #2303

Full Changelog: v5.3.2...v5.3.3

v5.3.2

v5.3.2

A quick update to adjust compliant/non-compliant object logic in the operator test suite.

What's Changed

• Update probe to v0.0.7 by @sebrandon1 in #2378
• Fix OperatorAutomountTokens reversed result by @edcdavid in #2381

Full Changelog: v5.3.1...v5.3.2

v5.3.1

v5.3.1

This new version of the certsuite includes some more logging and logic improvements around gathering service accounts and specifically logging around the automount service account token tests in accesscontrol and operator suites.

Fixes and Improvements

• doc : Fix on topics by @bnshr in #2356
• Gather service accounts in claim file by @edcdavid in #2368

CI Updates

• Add ARM64 based QE runner by @sebrandon1 in #2363

Dependency Updates

• Update Go to v1.22.6 by @sebrandon1 in #2360
• Update openshift libraries manually by @sebrandon1 in #2361
• Update GolangCI-lint to v1.60.1 by @sebrandon1 in #2362
• Bump ubi9/python-39 from 1-192.1723128185 to 1-192.1724040313 in /.github/actions/documentation by @dependabot in #2365
• Bump github/codeql-action from 3.26.3 to 3.26.4 by @dependabot in #2366
• Update GolangCI-lint to v1.60.2; various fixes by @sebrandon1 in #2367
• Update GolangCI-lint to v1.60.3 by @sebrandon1 in #2372
• Bump github.com/redhat-best-practices-for-k8s/oct from 0.0.19 to 0.0.20 by @dependabot in #2375
• Bump github.com/redhat-best-practices-for-k8s/certsuite-claim from 1.0.45 to 1.0.46 by @dependabot in #2376
• Bump github.com/redhat-best-practices-for-k8s/privileged-daemonset from 1.0.31 to 1.0.33 by @dependabot in #2377
• Bump github/codeql-action from 3.26.4 to 3.26.5 by @dependabot in #2374
• Bump github.com/mittwald/go-helm-client from 0.12.12 to 0.12.13 by @dependabot in #2373

Full Changelog: v5.3.0...v5.3.1

v5.3.0

v5.3.0

We have completed an organization rename:

• test-network-function --> redhat-best-practices-for-k8s

This has affected changes across all of our repositories to change all of our go modules, etc. We have also changed all of our image repositories on quay as well:

https://quay.io/organization/redhat-best-practices-for-k8s

Improvements

• Add option to sanitize claim file result output by @sebrandon1 in #2299
• Remove unused func in claimhelper by @sebrandon1 in #2306
• Revert "Remove unused func in claimhelper" by @sebrandon1 in #2309
• Add claimhelper pkg unit tests by @sebrandon1 in #2307
• Various CI Updates by @sebrandon1 in #2308
• Enable observability suite for PR OCP testing by @sebrandon1 in #2311
• Rename project to 'certsuite' by @sebrandon1 in #2321
• Fix unstable push; add legacy image to tnf image yaml by @sebrandon1 in #2324
• Fix DEBUG_ variables by @sebrandon1 in #2326
• Fix for the false positive of access-control's capabilities tcs. by @greyerof in #2352
• Fix badges after repo rename by @rdavid in #2336
• GetOwnerReferences does not return error by @rdavid in #2333
• Updated URL for github docs. by @greyerof in #2334
• cli: update Certsuite banner to v5.3 by @jmontesi in #2335

Dependency Updates

• Update operator-sdk to v1.36.0 by @sebrandon1 in #2304
• Update operator-sdk to v1.36.1 by @sebrandon1 in #2330
• Bump github.com/test-network-function/oct from 0.0.16 to 0.0.17 by @dependabot in #2305
• Bump ubi9/python-39 from 1-192 to 1-192.1722518946 in /.github/actions/documentation by @dependabot in #2314
• Bump actions/upload-artifact from 4.3.4 to 4.3.5 by @dependabot in #2312
• Bump github.com/k8snetworkplumbingwg/network-attachment-definition-client from 1.7.0 to 1.7.1 by @dependabot in #2313
• Bump golang.org/x/term from 0.22.0 to 0.23.0 by @dependabot in #2317
• Bump github.com/mittwald/go-helm-client from 0.12.10 to 0.12.11 by @dependabot in #2316
• Bump github/codeql-action from 3.25.15 to 3.26.0 by @dependabot in #2318
• Bump actions/upload-artifact from 4.3.5 to 4.3.6 by @dependabot in #2319
• Bump actions/upload-artifact from 4.3.5 to 4.3.6 by @dependabot in #2327
• Bump github.com/mittwald/go-helm-client from 0.12.11 to 0.12.12 by @dependabot in #2328
• Bump github.com/docker/docker from 26.1.4+incompatible to 26.1.5+incompatible by @dependabot in #2329
• Update debug image to v0.0.4 by @sebrandon1 in #2315
• Bump docker/build-push-action from 6.5.0 to 6.6.1 by @dependabot in #2323
• Bump github/codeql-action from 3.26.2 to 3.26.3 by @dependabot in #2358
• Bump ubi9/python-39 from 1-192.1722518946 to 1-192.1723128185 in /.github/actions/documentation by @dependabot in #2359
• Bump github.com/redhat-best-practices-for-k8s/oct from 0.0.18 to 0.0.19 by @dependabot in #2349
• Bump github.com/operator-framework/api from 0.26.0 to 0.27.0 by @dependabot in #2353
• Bump docker/build-push-action from 6.6.1 to 6.7.0 by @dependabot in #2343
• Bump helm.sh/helm/v3 from 3.15.3 to 3.15.4 by @dependabot in #2345
• Bump github/codeql-action from 3.26.0 to 3.26.2 by @dependabot in #2346
• Update k8s deps + controller runtime by @sebrandon1 in #2350
• Update certsuite-probe to v0.0.6 by @sebrandon1 in #2337
• Update operator-sdk to v1.36.1 by @sebrandon1 in #2330

Full Changelog: v5.2.3...v5.3.0

v5.2.3

v5.2.3

The v5.2.3 release of the test suite has a couple of notable changes such as a fix for the PDB test case and marking the read-only filesystem operator test as an 'Optional' test case.

This is also the first release that is using the new k8s-best-practices-debug image for the debug pod that spawns as part of the test suite.

Test Case Changes

• Mark read-only filesystem test as Optional by @sebrandon1 in #2245
• tests/observability: fix observability-pod-disruption-budged test case by @jmontesi in #2285

Improvements

• Add scaling helper unit tests by @sebrandon1 in #2249
• cmd/certsuite: update default value for log file in "check results" by @jmontesi in #2251
• Add 'new' image tag to push during CI runs by @sebrandon1 in #2254
• cmd/certsuite: refactor main to allow unit testing subcommands by @jmontesi in #2259
• cmd/certsuite: move the "list" flag from "run" to "info" by @jmontesi in #2264
• internal/datautil: add new package for basic data handling by @jmontesi in #2279
• Remove RELEASE_LEVEL by @sebrandon1 in #2283
• Change debug-partner to k8s-best-practices-debug by @sebrandon1 in #2282
• use ./certsuite instead of ./tnf (no longer exists) by @edcdavid in #2289
• Remove result cast; -claim update by @sebrandon1 in #2296

Documentation Updates

• refactor : Update cnf doc link to k8s doc link by @bnshr in #2257
• Fix broken link for tnf_config.yml by @vikasmulaje in #2280
• Add data collection doc by @shirmoran in #2281

Dependency Updates

• Bump github.com/test-network-function/oct from 0.0.14 to 0.0.15 by @dependabot in #2246
• Bump github/codeql-action from 3.25.11 to 3.25.12 by @dependabot in #2248
• Bump github.com/test-network-function/privileged-daemonset from 1.0.27 to 1.0.28 by @dependabot in #2247
• Bump docker/build-push-action from 6.3.0 to 6.4.0 by @dependabot in #2255
• Bump github.com/test-network-function/test-network-function-claim from 1.0.41 to 1.0.42 by @dependabot in #2256
• Update preflight to v1.10.0 by @sebrandon1 in #2260
• Missed version update for golangci by @sebrandon1 in #2261
• Bump docker/build-push-action from 6.4.0 to 6.4.1 by @dependabot in #2262
• Bump k8s.io/client-go from 0.30.2 to 0.30.3 by @dependabot in #2267
• Bump k8s.io/apiextensions-apiserver from 0.30.2 to 0.30.3 by @dependabot in #2268
• Bump k8s.io/kubectl from 0.30.2 to 0.30.3 by @dependabot in #2266
• Bump github/codeql-action from 3.25.12 to 3.25.13 by @dependabot in #2270
• Bump docker/setup-buildx-action from 3.4.0 to 3.5.0 by @dependabot in #2271
• Bump docker/login-action from 3.2.0 to 3.3.0 by @dependabot in #2273
• Bump docker/setup-qemu-action from 3.1.0 to 3.2.0 by @dependabot in #2272
• Bump docker/build-push-action from 6.4.1 to 6.5.0 by @dependabot in #2274
• Bump github.com/test-network-function/oct from 0.0.15 to 0.0.16 by @dependabot in #2275
• Bump ubi9/ubi from 9.4-1123.1719560047 to 9.4-1181 by @dependabot in #2288
• Bump ubi9/ubi-minimal from 9.4-1134 to 9.4-1194 by @dependabot in #2287
• Bump ubi9/python-39 from 1-186.1720018722 to 1-192 in /.github/actions/documentation by @dependabot in #2286
• Bump docker/setup-buildx-action from 3.5.0 to 3.6.1 by @dependabot in #2297
• Bump github.com/test-network-function/privileged-daemonset from 1.0.28 to 1.0.29 by @dependabot in #2298
• Bump github.com/docker/docker from 25.0.5+incompatible to 26.1.4+incompatible by @dependabot in #2300
• Bump golangci/golangci-lint-action from 6.0.1 to 6.1.0 by @dependabot in #2301
• Bump github/codeql-action from 3.25.14 to 3.25.15 by @dependabot in #2293
• Bump ossf/scorecard-action from 2.3.3 to 2.4.0 by @dependabot in #2292
• Bump github.com/test-network-function/test-network-function-claim from 1.0.42 to 1.0.43 by @dependabot in #2294
• Bump github/codeql-action from 3.25.13 to 3.25.14 by @dependabot in #2290

New Contributors

• @vikasmulaje made their first contribution in #2280

Full Changelog: v5.2.2...v5.2.3

v5.2.2

v5.2.2

This was a quick turnaround release where we did some notable changes:

• Changed the access-control-namespace test from Mandatory to Optional for all scenarios.
• Released a new version of the parser with v0.4.7. Found here.

Note: There were no changes in the -partner repository, so the partner version.json did not need an update at this point.

Improvements

• Bump parser version to v0.4.7 by @edcdavid in #2243
• Adjust 'access-control-namespace' test to be Optional by @sebrandon1 in #2242
• Bump parser version to v0.4.6 by @edcdavid in #2238
• Repository folder structure reorganization by @jmontesi in #2237

Dependency Updates

• Bump actions/setup-go from 5.0.1 to 5.0.2 by @dependabot in #2239
• Bump helm.sh/helm/v3 from 3.15.2 to 3.15.3 by @dependabot in #2241

Full Changelog: v5.2.1...v5.2.2