-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Flaky CI - Mimic #4454
Fix Flaky CI - Mimic #4454
Conversation
requirements.txt
Outdated
@@ -32,6 +32,7 @@ selenium==3.141.0 | |||
requests==2.25.0 | |||
azure-storage-blob==12.4.0 | |||
azure-core==1.8.0 | |||
msrest==0.6.21 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: this is a development issue I just discovered b/c I'm working on a new laptop. We need to pin msrest to 0.6.21 to get it to work with our version of azure-core. (see here). We might want to add this to a different PR, though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved to new PR.
I've rerun the test several times and it seems to pass every time, so I believe the issue is fixed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice fix!
Addresses one part of issue #4433 by addressing the flakiness of the Mimic test.
What happened was, after we removed the data hash, we had to write code in the test_cli.py file to produce something equivalent for tests that relied on having a check that the contents of two bundles were the same. To do this, we download the bundle and then hash the files in the directories in a function in the test_cli.py file.
However, that function did not
wait
on the uuid of the bundle being data hashed, and so sometimes the downloaded contents wouldn't match the contents of the bundle being mimicked, thereby causing occasional failures.(Note: This is still a hypothesis. I could not reproduce the issue locally but could on CI, but I have re-run the mimic CI test many times after this change and it has yet to fail.)