Fully automate stress cluster buildout and add support for azure file share mounting #2106

benbp · 2021-10-14T21:42:56Z

Fully automate cluster buildout. Add azure file share mount to stress tests.
- Moving the test/ad-hoc cluster back to the playground subscription
- Upgrading kubernetes cluster version to 1.21.x to pull in support for the azure csi file driver
- Adding high memory agent nodes to the base deployment
- Enabling node autoscaler in the base deployment
Publish stress watcher image in CI. Run docker build on PR
- Using common image location across stress clusters to simplify buildout+deployment
Add stress test debug file share usage example

Resolves #1903

azure-sdk · 2021-10-14T21:47:22Z

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

eng/containers/ci.yml

azure-sdk · 2021-10-14T22:01:14Z

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

… tests.

eng/containers/ci.yml

Co-authored-by: Wes Haggard <[email protected]>

benbp · 2021-10-14T22:46:21Z

@weshaggard I added some docs if you care to take a look: https://github.com/Azure/azure-sdk-tools/blob/27e997f44b251400927c1264e9a3839a6d7cabf3/tools/stress-cluster/cluster/README.md#development

azure-sdk · 2021-10-14T22:49:19Z

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

azure-sdk · 2021-10-14T23:01:44Z

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

azure-sdk · 2021-10-15T17:12:33Z

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

eng/containers/ci.yml

weshaggard · 2021-10-18T21:25:44Z

tools/stress-cluster/cluster/README.md

 # Dependencies

+- [Powershell Core](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-linux?view=powershell-7.1#ubuntu-2004) (if using Linux)


I'd recommend powershell core for all OS's, as there is a lot of improvements in networking over top of windows powershell as well.

weshaggard

Looks reasonable. The true test for the docs will be when someone actually uses them.

azure-sdk · 2021-10-18T21:53:08Z

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

…fra development/deployment

azure-sdk · 2021-10-22T20:48:31Z

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

ghost · 2021-10-22T22:07:35Z

Hello @azure-sdk!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.

benbp requested review from ckairen and a team as code owners October 14, 2021 21:42

benbp added the Central-EngSys This issue is owned by the Engineering System team. label Oct 14, 2021

benbp self-assigned this Oct 14, 2021

benbp changed the title ~~benbp/stress volume~~ Fully automate stress cluster buildout and add support for azure file share mounting Oct 14, 2021

benbp requested review from weshaggard and anuchandy October 14, 2021 21:53

benbp force-pushed the benbp/stress-volume branch from 0979698 to 3785762 Compare October 14, 2021 21:56

benbp commented Oct 14, 2021

View reviewed changes

eng/containers/ci.yml Outdated Show resolved Hide resolved

benbp added 3 commits October 14, 2021 18:01

Fully automate cluster buildout. Add azure file share mount to stress…

8ac50dc

… tests.

Publish stress watcher image in CI. Run docker build on PR

13e9a11

Add stress test debug file share usage example

4cc64b7

benbp force-pushed the benbp/stress-volume branch from 3785762 to 4cc64b7 Compare October 14, 2021 22:01

benbp added pillar-reliability The issue is related to reliability, one of our core engineering pillars. (includes stress testing) Stress This issue is related to stress testing, part of our reliability pillar. labels Oct 14, 2021

weshaggard reviewed Oct 14, 2021

View reviewed changes

eng/containers/ci.yml Outdated Show resolved Hide resolved

weshaggard reviewed Oct 14, 2021

View reviewed changes

eng/containers/ci.yml Outdated Show resolved Hide resolved

benbp and others added 2 commits October 14, 2021 18:43

Update stress test infrastructure docs

1e0b00a

Add succeeded() condition to docker push PullRequest condition

27e997f

Co-authored-by: Wes Haggard <[email protected]>

benbp force-pushed the benbp/stress-volume branch from 7bc1654 to 27e997f Compare October 14, 2021 22:43

Improve dev environment deploy instructions

6ae6dad

benbp force-pushed the benbp/stress-volume branch from bda2226 to 6ae6dad Compare October 14, 2021 22:56

Add stress test file volume helm template

322ab71

scbedd reviewed Oct 18, 2021

View reviewed changes

eng/containers/ci.yml Show resolved Hide resolved

weshaggard reviewed Oct 18, 2021

View reviewed changes

weshaggard approved these changes Oct 18, 2021

View reviewed changes

Fixes for stress file share. Dependency and doc updates

bf500f7

scbedd approved these changes Oct 18, 2021

View reviewed changes

benbp added 2 commits October 22, 2021 01:53

[stress testing] Require powershell core for all OSes doing stress in…

f2d773c

…fra development/deployment

Update stress test cluster access instructions

6647967

azure-sdk added the auto-merge label Oct 22, 2021

ghost merged commit dd1c8ca into Azure:main Oct 22, 2021

benbp deleted the benbp/stress-volume branch October 22, 2021 22:08

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fully automate stress cluster buildout and add support for azure file share mounting #2106

Fully automate stress cluster buildout and add support for azure file share mounting #2106

benbp commented Oct 14, 2021 •

edited

Loading

azure-sdk commented Oct 14, 2021

azure-sdk commented Oct 14, 2021

benbp commented Oct 14, 2021

azure-sdk commented Oct 14, 2021

azure-sdk commented Oct 14, 2021

azure-sdk commented Oct 15, 2021

weshaggard Oct 18, 2021

weshaggard left a comment

azure-sdk commented Oct 18, 2021

azure-sdk commented Oct 22, 2021

ghost commented Oct 22, 2021

		# Dependencies

		- [Powershell Core](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-linux?view=powershell-7.1#ubuntu-2004) (if using Linux)

Fully automate stress cluster buildout and add support for azure file share mounting #2106

Fully automate stress cluster buildout and add support for azure file share mounting #2106

Conversation

benbp commented Oct 14, 2021 • edited Loading

azure-sdk commented Oct 14, 2021

azure-sdk commented Oct 14, 2021

benbp commented Oct 14, 2021

azure-sdk commented Oct 14, 2021

azure-sdk commented Oct 14, 2021

azure-sdk commented Oct 15, 2021

weshaggard Oct 18, 2021

Choose a reason for hiding this comment

weshaggard left a comment

Choose a reason for hiding this comment

azure-sdk commented Oct 18, 2021

azure-sdk commented Oct 22, 2021

ghost commented Oct 22, 2021

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

benbp commented Oct 14, 2021 •

edited

Loading

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.