Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce rally-tracks compatibility testing #1564

Merged
merged 12 commits into from
Aug 30, 2022

Commits on Aug 22, 2022

  1. Add ability to run track repo tests within the Rally repo.

    This commit modifies and extends `pytest-rally` so that it can be invoked from
    within the Rally repo. This enables Rally developers and CI jobs to test
    Rally changes against arbitrary revisions of local track repositories,
    using arbitrary versions of Elasticsearch.
    
    The actual contents of the tests live in the track repository. The
    plugin will run any tests found in the `it` subdirectory of the provided track
    repository by default, including those that are auto-generated by
    `pytest-rally`.
    
    By default, tests will be run against the `master` branch of the track
    repository checked out in `$RALLY_HOME/benchmarks/tracks/default` (typically
    `rally-tracks`), using a build of the `main` branch of Elasticsearch.
    
    To run tests with these defaults, run the following from the root of this
    repository:
    
    `pytest it/track_repo_compatibility`
    
    Here is an example invocation that overrides these defaults:
    
    ```
    pytest it/track_repo_compatibility \
         --track-repository=/path/to/repo \
         --track-revision=some-branch \
         --distribution-version=8.3.2
    ```
    Mike Baamonde committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    f5190a1 View commit details
    Browse the repository at this point in the history
  2. Add a rally-tracks-compat tox environment.

    Mike Baamonde committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    cdb27d7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7a07e32 View commit details
    Browse the repository at this point in the history
  4. Add rally-tracks-compat pull request Jenkins job.

    Mike Baamonde committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    ce9737f View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2022

  1. Make sure that rally-tracks is actually on disk.

    Mike Baamonde committed Aug 23, 2022
    Configuration menu
    Copy the full SHA
    fc8e4aa View commit details
    Browse the repository at this point in the history
  2. Improve error handling.

    Mike Baamonde committed Aug 23, 2022
    Configuration menu
    Copy the full SHA
    93ae346 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    fe81648 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2022

  1. local debugging

    Mike Baamonde committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    9258e28 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    257794b View commit details
    Browse the repository at this point in the history
  3. Revert "local debugging"

    This reverts commit 9258e28.
    Mike Baamonde committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    56ca636 View commit details
    Browse the repository at this point in the history
  4. Formatting.

    Mike Baamonde committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    f437d8a View commit details
    Browse the repository at this point in the history
  5. Remove hookimpl decorators.

    Mike Baamonde committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    6d084f5 View commit details
    Browse the repository at this point in the history