[Tool] New tool to download android dependencies

[reidbaker]

This pr is pushed for high level feedback/conversation. I will add tests before serious review.
should be read in conjuction with https://flutter-review.googlesource.com/c/recipes/+/46980

• Create new top level command to run flutter dependencies on changed packages
• when running android tests download dependencies before running tests

flutter/flutter#120119

Pre-launch Checklist

• I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
• I read the [Tree Hygiene] wiki page, which explains my responsibilities.
• I read and followed the [relevant style guides] and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
• I signed the [CLA].
• The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
• I listed at least one issue that this PR fixes in the description above.
• I updated pubspec.yaml with an appropriate new version according to the [pub versioning philosophy], or this PR is [exempt from version changes].
• I updated CHANGELOG.md to add a description of the change, [following repository CHANGELOG style].
• I updated/added relevant documentation (doc comments with ///).
• I added new tests to check the change I am making, or this PR is [test-exempt].
• All existing and new tests are passing.

[flutter-dashboard]

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat (don't just cc him here, he won't see it! He's on Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

[stuartmorgan]

This looks great! If there's a way to share the cache for the whole test run rather than per-plugin that would be even better (or is Gradle actually doing that under the hood?), but either way this is a clear improvement, and should really help us quickly diagnose network and server flake.

[reidbaker]

https://flutter-review.googlesource.com/c/recipes/+/46980

So our testing system is complex but gradle lays out the files it relies on here by default all tests on a machine will share the gradle cache unless we configure it differently. Searching I was not able to find a server lifecycle doc to see how often those machines are wiped [1]. Searching for where we might configure gradle home I found a bunch of places where we set GRADLE_USER_HOME and all the the values we set are the same [cache]/gradle value. So those places should share a gradle directory which holds both dependencies and gradle wrapper version and cleans itself of unused versions every 7-30 days depending on the type of thing in the cache.

[1] Moma search, exploring https://flutter.googlesource.com/infra/

[keyonghan]

Searching for where we might configure gradle home I found a bunch of places where we set GRADLE_USER_HOME and all the the values we set are the same [cache]/gradle value. So those places should share a gradle directory which holds both dependencies and gradle wrapper version and cleans itself of unused versions every 7-30 days depending on the type of thing in the cache.

Do you have a build example which downloads gradle dependencies that I can take a look? [cache]/gradle relies on swarming named cache, and we should have a corresponding named cache entry defined so that different tests can share the cache on the same testbed. Based on existing infra config, I do not see any named cache pointing to [cache]/gradle.

[johnmccutchan]

Update this comment now that this command is more generic.

[reidbaker]

I think I did. If you notice the line below you can see that I document the other things this command could do but this comment is correct with what it does now.

My expectation is that if/when we add dart support the tool comment will be updated to say

/// Run 'gradlew dependencies' on android.
/// Run 'pub get' on dart packages.
/// Run 'pod install' and 'swift package download --all'  depending on the configuration. 

For now I updated the top line to have a summary then the specifics. Hopefully that addresses your concern.

[reidbaker]

Searching for where we might configure gradle home I found a bunch of places where we set GRADLE_USER_HOME and all the the values we set are the same [cache]/gradle value. So those places should share a gradle directory which holds both dependencies and gradle wrapper version and cleans itself of unused versions every 7-30 days depending on the type of thing in the cache.

Do you have a build example which downloads gradle dependencies that I can take a look? [cache]/gradle relies on swarming named cache, and we should have a corresponding named cache entry defined so that different tests can share the cache on the same testbed. Based on existing infra config, I do not see any named cache pointing to [cache]/gradle.

This is the only pr that should have the new step to download dependencies. That said all existing android test would have gradle dependencies because I think we include kotlin by default.

Here is a recent build https://ci.chromium.org/ui/p/flutter/builders/try/Linux_android%20android_build_all_packages%20master/505/overview

I am not sure if I understand your comment about the existing infra config all I have found is in https://cs.opensource.google/search?q=GRADLE_USER_HOME%20-f:gradle-wrapper.properties&start=1.

[reidbaker]

Tests failing https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8775849073271137377/+/u/Run_package_tests/download_android_deps/stdout

Unhandled exception:
Could not find an option named "packages-for-branch".

[keyonghan]

android_platform_tests.yaml LGTM