Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial scripts and pipeline for stress test discovery/build/push #1851

Merged
8 commits merged into from
Jul 29, 2021

Conversation

benbp
Copy link
Member

@benbp benbp commented Jul 27, 2021

This adds some initial automation to:

  1. Log in to various resources (subscription, aks cluster, container registry)
  2. Automatically discover all stress tests in a directory
  3. Build and push docker images
  4. Update and build helm chart dependencies
  5. Deploy stress tests to a cluster

The scripts are designed to be run either locally or via CI.

I have some testing to do on the pipeline config, but everything else is fully ready for review.

Resolves Azure/azure-sdk#3168

@benbp benbp requested a review from a team as a code owner July 27, 2021 22:10
@check-enforcer-staging
Copy link

This pull request is protected by Check Enforcer.

What is Check Enforcer?

Check Enforcer helps ensure all pull requests are covered by at least one check-run (typically an Azure Pipeline). When all check-runs associated with this pull request pass then Check Enforcer itself will pass.

Why am I getting this message?

You are getting this message because Check Enforcer did not detect any check-runs being associated with this pull request within five minutes. This may indicate that your pull request is not covered by any pipelines and so Check Enforcer is correctly blocking the pull request being merged.

What should I do now?

If the check-enforcer check-run is not passing and all other check-runs associated with this PR are passing (excluding license-cla) then you could try telling Check Enforcer to evaluate your pull request again. You can do this by adding a comment to this pull request as follows:
/check-enforcer evaluate
Typically evaulation only takes a few seconds. If you know that your pull request is not covered by a pipeline and this is expected you can override Check Enforcer using the following command:
/check-enforcer override
Note that using the override command triggers alerts so that follow-up investigations can occur (PRs still need to be approved as normal).

- '!sdk/**/session-records'
- '!sdk/**/SessionRecords'

- task: AzureCLI@2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What value are we gaining by using the AzureCLI task here? Is the service connection information enough for us to do all the deployments or don't we need to stull pass in most of the secrets? Also what about other clouds do they need other configurations or is that not a scenario at this point.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't gotten to that point yet but my thinking was to add an access policy to the buildout bicep configs to allowlist one of the devops service connections (that way there are no manual steps on buildout). I may still create a new identity just for this.

I think it's unlikely that we'll target other clouds, as our aim here is to stress the client code, rather than to test its operability across clouds. Though if we do have a need for that, I don't think we'll ever need to host the stress cluster itself in another cloud, so any relevant credentials are not configured in this context (they are auto-synced into the cluster via keyvault).

@benbp
Copy link
Member Author

benbp commented Jul 28, 2021

@weshaggard The script pieces should be much more simplified now. I added a couple helpers for exit code + control flow handling, please let me know what you think.

Also, I'm considering moving these scripts into /eng/common, that way it will be a little easier for developers in any repo to use them for their dev loop.

}

Write-Output "Creating namespace $($pkg.Namespace) if it does not exist..."
kubectl create namespace $pkg.Namespace --dry-run=client -o yaml | kubectl apply -f -
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you use one of your run helpers here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was having trouble dealing with > and | operators when passing them through to Run(), so there are a few places in the script where I bypass it to keep Run() simple. In all places the subsequent command will fail if the previous command fails and is not caught, however.

Copy link
Member

@weshaggard weshaggard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couple minor comments but looks much better.

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@ghost
Copy link

ghost commented Jul 29, 2021

Hello @azure-sdk!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

@ghost ghost merged commit 0eb62b8 into Azure:main Jul 29, 2021
@benbp benbp deleted the benbp/stress-ci-scripts branch July 29, 2021 20:17
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[stress testing] Automate local and CI build+deploy for stress tests
3 participants