Flakey tests tracking

[isubasinghe]

A copy of #7133 to address more flakey tests

This issue is for tracking flakey tests, so we can see how flakey they are.

Flakey tests are often not actually flakey, they instead revealing and edge-case bug that is caused by rare events, such as API disruption.

If a test appears to be flakey, then we simply disable it and wait for the author of the test (and therefore the feature) to investigate it. Either way, it is ideally their responsibility.

Why do we need to aggressively skip flakey tests?

Consider a test suite with 5 tests that each fail on 5% of the time. How often will the whole suite fail? Hint: it is much greater that 5x5%!

https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html

• @alexec from Flakey tests #7133

Tests

• ArtifactsSuite/TestArtifactGC

please comment all flakey tests here and I will update this list.

[juliev0]

Thanks for tracking these. I actually have a similar issue here. I didn't realize Alex had created an issue before.

Hmm, I wonder if we should have a label for this?

[isubasinghe]

Ah thanks for letting me know about that @juliev0, I will close this issue in that case.

But yes a "misc"or dedicated label would be nice

[juliev0]

Ah thanks for letting me know about that @juliev0, I will close this issue in that case.

But yes a "misc"or dedicated label would be nice

I noticed an "area/build" label applied to the other issue - I guess that's supposed to be for this.