Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor e2e test setup #264

Merged
merged 40 commits into from
Apr 10, 2023
Merged

Refactor e2e test setup #264

merged 40 commits into from
Apr 10, 2023

Commits on Apr 8, 2023

  1. Refactor e2e test setup 

    We recently made a change to ensure we clean up resources properly after
we perform e2e tests. Specifically to avoid race conditions that leaves
the compliance operator resources in a bricked state, even when the
tests pass.

This commit uses a similar pattern for test setup by leveraging the
`setUp` method before calling `m.Run()` to make sure the cluster is
ready before we run the actual tests.

One advantage is that we're simplifying the logic to a single layer
before the tests run.

Further changes will make it easier to decouple the framework, context,
and e2eutils into a single set of utilities we can share across
repositories and test suites.
    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    adc9669 View commit details
    Browse the repository at this point in the history

  2. Add parallel test module 

    Now that the framework is mostly decoupled from the tests, we can reuse
it for separate test modules for parallel and serial tests.

The benefit of this is that we can flatten the tests. Which aid
readability, reduces the ammount of variables we need to pass into tests
regardless if they're used, allows us to use native go tools for test
discovery, and gives use the opportunity to remove the custom test
runner implementation we have to handle parallel and serial tests.
    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    31538f0 View commit details
    Browse the repository at this point in the history

  3. Remove dead code from e2e refactor

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    dea1379 View commit details
    Browse the repository at this point in the history

  4. Port TestInvalidBundleWithUnexistentRef to parallel tests 

    Move TestInvalidBundleWithUnexistentRef to the new parallel test module,
which provides a flat structure and doesn't rely on the custom test
runner from helpers.go.
    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    e193df0 View commit details
    Browse the repository at this point in the history

  5. Port TestInvalidBundleWithNoTag to parallel tests

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    a2fa7b8 View commit details
    Browse the repository at this point in the history

  6. Port TestParsingErrorRestartsParserInitContainer to parallel tests

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    6fdc3f3 View commit details
    Browse the repository at this point in the history

  7. Port TestRulesAreClassifiedAppropriately to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    5849c68 View commit details
    Browse the repository at this point in the history

  8. Port TestSingleScanSucceeds to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    00f91c4 View commit details
    Browse the repository at this point in the history

  9. Port TestScanProducesRemediations to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    a404552 View commit details
    Browse the repository at this point in the history

  10. Port TestSingleScanWithStorageSucceeds to parallel tests

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    78b3a73 View commit details
    Browse the repository at this point in the history

  11. Port TestScanWithUnexistentResourceFails to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    419997e View commit details
    Browse the repository at this point in the history

  12. Port TestScanStorageOutOfLimitRangeFails to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    31aa738 View commit details
    Browse the repository at this point in the history

  13. Port TestSingleTailoredScanSucceeds to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    5db30ae View commit details
    Browse the repository at this point in the history

  14. Port TestSingleTailoredPlatformScanSucceeds to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    9147a45 View commit details
    Browse the repository at this point in the history

  15. Port TestScanWithNodeSelectorFiltersCorrectly to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    9c45a2b View commit details
    Browse the repository at this point in the history

  16. Port TestScanWithNodeSelectorNoMatches to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    752b2b2 View commit details
    Browse the repository at this point in the history

  17. Port TestScanWithInvalidScanTypeFails to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    6393fee View commit details
    Browse the repository at this point in the history

  18. Port TestScanWithInvalidContentFails to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    f024ba2 View commit details
    Browse the repository at this point in the history

  19. Port TestScanWithInvalidProfileFails to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    5af0571 View commit details
    Browse the repository at this point in the history

  20. Port TestMalformedTailoredScanFails to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    eed6a45 View commit details
    Browse the repository at this point in the history

  21. Port TestScanWithEmptyTailoringCMNameFails to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    b282e8a View commit details
    Browse the repository at this point in the history

  22. Port TestScanWithMissingTailoringCMFailsAndRecovers to parallel test … 

    …module
    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    ebb1cf1 View commit details
    Browse the repository at this point in the history

  23. Port TestMissingPodInRunningState to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    8ea874a View commit details
    Browse the repository at this point in the history

  24. Port TestApplyGenericRemediation to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    5f44887 View commit details
    Browse the repository at this point in the history

  25. Port TestPatchGenericRemediation to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    e682288 View commit details
    Browse the repository at this point in the history

  26. Port TestGenericRemediationFailsWithUnkownType to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    e728b64 View commit details
    Browse the repository at this point in the history

  27. Port TestSuiteWithInvalidScheduleShowsError to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    93d28fa View commit details
    Browse the repository at this point in the history

  28. Port TestScheduledSuite to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    db5aa70 View commit details
    Browse the repository at this point in the history

  29. Port TestScheduledSuitePriorityClass to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    5c688b7 View commit details
    Browse the repository at this point in the history

  30. Port TestScheduledSuiteInvalidPriorityClass to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    5d78ea5 View commit details
    Browse the repository at this point in the history

  31. Port TestScheduledSuiteUpdate to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    354b3a9 View commit details
    Browse the repository at this point in the history

  32. Port TestSuiteWithContentThatDoesNotMatch to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    35f75c5 View commit details
    Browse the repository at this point in the history

  33. Port TestScanSettingBinding to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    0c9e3d0 View commit details
    Browse the repository at this point in the history

  34. Port TestScanSettingBindingTailoringAndNonDefaultRole to parallel tes… 

    …t module
    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    33d2888 View commit details
    Browse the repository at this point in the history

  35. Port TestScanSettingBindingUsesDefaultScanSetting to parallel test mo… 

    …dule
    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    fc8ca45 View commit details
    Browse the repository at this point in the history

  36. Port TestScanSettingBindingWatchesTailoredProfile to parallel test mo… 

    …dule
    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    126b8d3 View commit details
    Browse the repository at this point in the history

  37. Port TestManualRulesTailoredProfile to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    75e8a15 View commit details
    Browse the repository at this point in the history

  38. Port TestCheckDefaultKubeletConfig to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    ac3968c View commit details
    Browse the repository at this point in the history

  39. Port TestHideRule to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    2374f73 View commit details
    Browse the repository at this point in the history

  40. Port TestScheduledSuiteTimeoutFail to parallel test module

    @rhmdnd
    rhmdnd committed Apr 8, 2023
    Configuration menu
    Copy the full SHA
    ffd0ecd View commit details
    Browse the repository at this point in the history