Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow user to specify branch of systemtests being triggered by trigger_systemtests.py #143

Draft
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

BenjaminRodenberg
Copy link
Member

closes #141

@BenjaminRodenberg
Copy link
Member Author

BenjaminRodenberg commented Dec 19, 2019

Exemplary usage of the feature implemented here is shown in precice/fenics-adapter#60:

Excerpt from .travis.yml

- curl -LO --retry 3 https://raw.githubusercontent.com/precice/systemtests/i_141/trigger_systemtests.py
- travis_wait 60 $PY trigger_systemtests.py --adapter fenics --wait --branch develop

repo = adapters_info[ args.adapter ].repo
trigger_travis_build( generate_failure_callback(), args.owner,repo)
repo = adapters_info[args.adapter].repo
trigger_travis_build(generate_failure_callback(branch=args.branch), args.owner, repo)
Copy link
Contributor

@Eder-K Eder-K Dec 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This conditional branch that contains this line is only ever reached in the following case:

  1. Some external repository, let's say fenics-adapter, commits something that triggers Travis to execute the .travis.yml file. This file contains the crucial line python trigger_systemtests.py --adapter fenics, which triggers a build on systemtests (and with this PR merged, on a specific branch of it). Note that the --wait argument is omitted, meaning that the fenics-adapter build on Travis will just send the request and instantly return as success.

  2. A systemtests build is launched to run the test that was requested. If this test now fails we reach the after_failure_action

    if trigger_failure:
    after_failure_action += " python trigger_systemtests.py --failure --owner {USER} --adapter {ADAPTER}"

    So, the systemtests build will send a request back to fenics-adapter which originally triggered the whole sequence.

At this point we have reached the highlighted line 282.

  1. On fenics-adapter, a mock build is launched. This build will just fail and do nothing else, so that the actual feedback for a failed test is negative on the original repository as well.

trigger_travis_build(generate_failure_callback(branch=args.branch), args.owner, repo)

Because of this, we have to be careful with which branch we choose here, as this must not be a branch of systemtests. Currently, this mock build would always run on the master branch of the fenics-adapter. It would be nice to have this actually run on the specific branch that was originally commited to in point 1. Note that this is again a different branch than the systemtests branch that will be adjustable with this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the mock request is generated on systemtests, we would need it to have received information in some way about which branch of the outside repository triggered it. One way might be to have the original request contain the following as after failure action

if trigger_failure:
    after_failure_action += " python trigger_systemtests.py --failure --owner {USER} --adapter {ADAPTER}" + "--branch {}".format(os.environ["TRAVIS_BRANCH"])

This way, when systemtests is executing trigger_systemtests.py as a result of a failed test, it will have received an args.branch value that correctly contains the branch of the outside repo that originally issued the test. Then we could actually call generate_failure_callback(branch=args.branch)

This way of formatting seems rather ugly, but it doesn't interfere with the formatting step done later on in

for key, systest_template in systest_templates.items():
job[key] = systest_template.format(TESTNAME=tests, USER=user, TEST=tests,
ADAPTER=adapter, BASE=base)

We also don't require any further variables this way.

Suggestions?

trigger_systemtests.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make sure that trigger_systemtest.py uses the correct branches
2 participants