ValidateJavaVersion is expensive

[jeromelaban]

Android application type

.NET Android (net7.0-android, etc.)

Affected platform version

net 7 and later

Description

The task ValidateJavaVersion takes about 23 seconds to run for every project of a solution, even when running on a beefy machine (Standard_L16s_v3 with premium SSDs).

https://github.com/xamarin/xamarin-android/blob/4fffbf686bad5479a8ef7954d7d9d0d68941f034/src/Xamarin.Android.Build.Tasks/Tasks/ValidateJavaVersion.cs#L14

Interestingly, the caching mechanism here does seem to work, but not in a meaningful way:

https://github.com/xamarin/xamarin-android/blob/4fffbf686bad5479a8ef7954d7d9d0d68941f034/src/Xamarin.Android.Build.Tasks/Tasks/ValidateJavaVersion.cs#L104

Steps to Reproduce

Build any android library and observe the task takes a long time to run. In a large solution, this accounts for about 4 minutes of build time.

Did you find any workaround?

None at this time.

Relevant log output

No response

[dellis1972]

out of curisity how long does java -version take to run on the command line?
also what output does it produce for your sdk? (I see you are using a custom java sdk).

[microsoft-github-policy-service]

Hi @jeromelaban. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

[jeromelaban]

So running the commands directly give a 7.4s for java.exe and 3.8s for javac.exe. Do you get similar values on your end?

About the Java SDK, we're using is the one by default on azure devops hosted agents, but maybe we've been using the wrong one for a while... which one's the most appropriate in that context?

[dellis1972]

so on my mac I get the following

 time java -version
openjdk version "17.0.8" 2023-07-18 LTS
OpenJDK Runtime Environment Microsoft-8035246 (build 17.0.8+7-LTS)
OpenJDK 64-Bit Server VM Microsoft-8035246 (build 17.0.8+7-LTS, mixed mode, sharing)
java -version  0.04s user 0.03s system 81% cpu 0.094 total

So that is really odd its taking so long. I'll have to try on a windows box somewhere and see what I see.

[jonathanpeppers]

One thing to note is that we cache the results of these calls with an AppDomain lifetime with RegisterTaskObject. So you'd see this on the first build per MSBuild node unless the Java SDK path changes.

We do call both javac -version and java -version, maybe we could drop one of those?

[jonathanpeppers]

@jeromelaban can you tell if the Java SDK path is changing between calls? Maybe this invalidates the cache for all of them?

[microsoft-github-policy-service]

Hi @jeromelaban. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

[jeromelaban]

@jonathanpeppers The path does not seem to change between invocations.

Here's a binlog, if there are specifics you're looking for:
build-5.0.0-dev.pool-update.2751-netcoremobile-net8-UWP.binlog.zip

[jonathanpeppers]

The caching is working:

ValidateJavaVersion
    Using cached value for `C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\11.0.20-101\x64\bin\java.exe -version`: 11.0.20.1
    Found Java SDK version 11.0.20.1.
    Using cached value for `C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\11.0.20-101\x64\bin\javac.exe -version`: 11.0.20.1
    Found Java SDK version 11.0.20.1.

But what is weird is how the non-cached calls like like 20s:


(It seems like I see one of these per MSBuild node)

@jeromelaban can you check what AzDO image you are using?

It does seem to be using a different Temurin JDK than what VS installs:

JavaSdkPath = C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\11.0.20-101\x64

We normally use Microsoft Open JDK in C:\Program Files\Microsoft\jdk-11.0.19.7-hotspot.

[jeromelaban]

The caching is working, but not always, that's the curious part. It could be because the build is using multiple msbuild nodes.

I'm currently using this particular image: https://github.com/actions/runner-images/tree/releases/win22/20230918. I've checked with more recent builds with the same image, and this particular task can vary between 8s and 20s.

Now with regards to selecting the Open JDK that comes with VS, do you select it with boots? or do you expect to have this specific path?

[jonathanpeppers]

I wonder if some version of C:\Program Files\Microsoft\jdk-11.0.19.7-hotspot is already preinstalled.

But the AzDO images have JAVA_HOME set, and it overrides it?

[dellis1972]

@jeromelaban out of curiosity can you add the msbuild flag -m:1 to force it to use only one node. See if that changes the cache usage?

[jonpryor]

I wonder if it would be "good enough" to instead parse $(_JavaSdkDirectory)\release, which would replace a java -version invocation with reading a file with contents:

IMPLEMENTOR="Microsoft"
IMPLEMENTOR_VERSION="Microsoft-40648"
JAVA_VERSION="11.0.16.1"
JAVA_VERSION_DATE="2022-08-12"
LIBC="default"
MODULES="…"
OS_ARCH="x86_64"
OS_NAME="Darwin"
SOURCE=""

…which we already have code to read: https://github.com/xamarin/xamarin-android-tools/blob/8a971d94a3fa2f0e8f69c5cf742c6836c14be1cd/src/Xamarin.Android.Tools.AndroidSdk/JdkInfo.cs#L208-L234

and we already create a JdkInfo instance in <ResolveSdks/>, so maybe that value should be cached instead?

https://github.com/xamarin/xamarin-android/blob/a1b1f7be388b69506d4c3521a4e357f88e43080b/src/Xamarin.Android.Build.Tasks/Tasks/ResolveSdksTask.cs#L97

Do we even need <ValidateJavaVersion/> anymore? Considering that the <ResolveSdks/> task runs first, and runs anyway, should we just "merge" these tasks?

[jeromelaban]

@jeromelaban out of curiosity can you add the msbuild flag -m:1 to force it to use only one node. See if that changes the cache usage?

It does though not everywhere (maybe some nodes are still created anyways), but it's not particularly applicable in larger projects :)