tools: a number of targets are not cached and make bazel test //... slow

[stevekuznetsov]

The majority of targets are cached without issue, but a couple (notably pylint) make bazel test //... really slow and annoying. Here's a run with one BAZEL.build file changed between this run and the last one:

$ bazel test //...
INFO: Analysed 2110 targets (538 packages loaded).
INFO: Found 1987 targets and 123 test targets...
INFO: Elapsed time: 87.446s, Critical Path: 68.61s
INFO: Build completed successfully, 121 total actions
//boskos:go_default_test                                        (cached) PASSED in 1.4s

...

//velodrome/transform/plugins:go_default_test                   (cached) PASSED in 1.2s
//hack:verify-pylint                                                     PASSED in 64.3s
//hack:verify-spelling                                                   PASSED in 14.6s
//hack:verify_boilerplate                                                PASSED in 3.8s

Executed 3 out of 123 tests: 123 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

We should figure out how to make pylint and spelling and boilerplate incremental or faster.

/cc @ixdy @BenTheElder @rmmh

[BenTheElder]

they python targets are not really cacheable because they take in the whole repo and aren't properly hermetic, we've got lints in a separate job while we decide what to do with them.

not sure about the velodrome / boskos tests.

[stevekuznetsov]

Looks like pylint and spelling already parallelize, but boilerplate does not -- no super simple throughput wins that I can see. Maybe there is some bazel magic that can make pylint not run when no Python files have changed?

The velodrome and boskos got cached correctly, just didn't elide them from the output.

[ixdy]

The verify tests will cache, but only if nothing in your repo has changed.

They're pretty un-bazely in that they depend on //:all-srcs, which depends on the entire tree; this isn't really a use-case for which bazel is designed. As a result, they have to be re-run anytime any file in the repo changes.

There was a proposed PR (kubernetes/repo-infra#46) to try to improve things slightly so we can have //:all-go-srcs and //:all-py-srcs etc, though this starts to get unwieldy pretty quickly.

In the past, we've tagged these as manual, so bazel test ... doesn't run them automatically. This has caused some complaints, since then the checks might fail in PR tests, since you didn't remember to run them locally.

(I personally think we shouldn't be running them with bazel test ...; maybe we can tag them as verify and then by default skip them, but allow a bazel test --config=verify which includes them, perhaps.)

[BenTheElder]

#7041 took care of all of that short of making them manual. you can run bazel test ... with --config=unit or --config=lint

…

On Tue, Mar 27, 2018 at 11:23 AM Jeff Grafton ***@***.***> wrote: The verify tests will cache, but only if nothing in your repo has changed. They're pretty un-bazely in that they depend on //:all-srcs, which depends on the entire tree; this isn't really a use-case for which bazel is designed. As a result, they have to be re-run anytime any file in the repo changes. There was a proposed PR (kubernetes/repo-infra#46 <kubernetes/repo-infra#46>) to try to improve things slightly so we can have //:all-go-srcs and //:all-py-srcs etc, though this starts to get unwieldy pretty quickly. In the past, we've tagged these as manual, so bazel test ... doesn't run them automatically. This has caused some complaints, since then the checks might fail in PR tests, since you didn't remember to run them locally. (I personally think we shouldn't be running them with bazel test ...; maybe we can tag them as verify and then by default skip them, but allow a bazel test --config=verify which includes them, perhaps.) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7437 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA4Bq4Yl-vJQk2RNjONwyYPrmSUtSmYyks5tioO6gaJpZM4S9YJl> .

[stevekuznetsov]

Yeah I guess I was hoping for one button that would do Go unit + Go lint + boilerplate + bazel verify, basically everything that I would generally hit if I were iterating on Prow, but not other targets. Maybe I'm just imagining a world where prow is in it's own repo :)

[rmmh]

It's possible to do incremental/cached pylint results too, but probably not inside of bazel.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[stevekuznetsov]

No action items on this

/close