x/build/cmd/watchflakes: should report x/ repo breakage caused by Go repo change

[dmitshur]

A part of issue #58819 was was making watchflakes report, rather than skip, cases of consistent test failures.

golang.org/x/exp/typeparams.TestAPIConsistency is currently broken (e.g., https://ci.chromium.org/b/8740322578467634353) and has been consistently failing on all builders since CL 575455 was submitted about 6 days ago, but it doesn't seem to be reported by watchflakes.

I haven't looked very closely, but the problem might be that the breaking change was in the main Go repo rather than the x/exp repo itself, so looking at the latest x/exp commit in the https://ci.chromium.org/p/golang/g/x-exp-gotip/console view shows it as green, and there haven't been newer commits that landed in x/exp. Many failing commits are visible in the "by Go commit" view at https://ci.chromium.org/p/golang/g/x-exp-gotip-by-go/console, though.

Perhaps there's more to do for watchflakes to report consistent failures when they happen due to a change in the main Go repo. CC @cherrymui, @cagedmantis.

[gabyhelp]

Related Issues and Documentation

• build: golang.org/x/telemetry failures #67481 (closed)
• build: golang.org/x/tools/go/packages failures #67899 (closed)
• x/build/cmd/watchflakes: failures posted with misleading timestamps #62116
• build: golang.org/x/telemetry/internal/crashmonitor failures #68215 (closed)
• build: golang.org/x/telemetry/crashmonitor failures #67448 (closed)
• build: golang.org/x/telemetry/internal/crashmonitor failures #67449 (closed)
• build: golang.org/x/tools/cmd/godoc failures #68013 (closed)
• build: golang.org/x/tools/cmd/godoc failures #67489 (closed)
• build: golang.org/x/telemetry/godev/internal/content failures #67898 (closed)
• build: golang.org/x/tools/gopls/internal/cmd failures #68193 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[cherrymui]

Yeah, "subrepo by go commit" is one thing I want to get on but haven't. There is a TODO https://cs.opensource.google/go/x/build/+/master:cmd/watchflakes/main.go;l=29