Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Xamarin.Android.Build.Tasks] remove Distinct() from <ResolveLibraryProjectImports/> #3296

Merged

Conversation

jonathanpeppers
Copy link
Member

The <ResolveLibraryProjectImports/> task has some logic that needs
to look at the item metadata of each @(Reference):

  • %(AndroidSkipResourceExtraction) to skip unzipping anything
  • %(AndroidSkipResourceProcessing) to skip <ConvertResourcesCases/>

To make matters weird, we had some LINQ:

foreach (var assemblyPath in Assemblies
        .Select (a => a.ItemSpec)
        .Distinct ()) {

And we would basically lose the metadata information...

To handle this problem, we had two HashSet's we used to lookup the
values:

assembliesToSkipCaseFixup = new HashSet<string> (StringComparer.OrdinalIgnoreCase);
assembliestoSkipExtraction = new HashSet<string> (StringComparer.OrdinalIgnoreCase);
bool metaDataValue;
foreach (var asm in Assemblies) {
    if (bool.TryParse (asm.GetMetadata (AndroidSkipResourceProcessing), out metaDataValue) && metaDataValue)
        assembliesToSkipCaseFixup.Add (asm.ItemSpec);
    if (bool.TryParse (asm.GetMetadata (GetAdditionalResourcesFromAssemblies.AndroidSkipResourceExtraction), out metaDataValue) && metaDataValue)
        assembliestoSkipExtraction.Add (asm.ItemSpec);
}

I added a test to see what happens when we use multiple @(Reference)
such as:

  • <PackageReference/>
  • packages.config and a regular <Reference/>
  • <Reference/> with a full path

It turns out there are MSBuild targets that disambiguate duplicate
assembly references:

https://github.com/NuGet/NuGet.BuildTasks/blob/640c8e13a9b7ab6e86264a296638fbf3cc016ad1/src/Microsoft.NuGet.Build.Tasks/Microsoft.NuGet.targets#L186-L207

The ResolveNuGetPackageAssets does a Remove against duplicate
@(Reference), even if there are no NuGet packages present in the
project.

It seems safe for us to just drop the Distinct() and the
HashSet's. The code is a bit simpler after doing this.

I'm also seeing a small ~25ms performance improvement with this change.

@dellis1972
Copy link
Contributor

@jonathanpeppers this needs rebasing I think. We are getting the packaging error because the CI system is not building the x86_64 native libraries.

@jonathanpeppers
Copy link
Member Author

This one failed in a weird way:

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Preparing the binary bundle
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Local bundle path: /Users/vsts/agent/2.153.2/work/1/s/bundle-v21-h86e7db4-Release-Darwin-libzip=b95cf3f,mono=52f6ea1.7z
Bundle directory from command line: /Users/vsts/agent/2.153.2/work/1/s

Step Xamarin.Android.Prepare.Step_PrepareBundle failed: Xamarin.Android bundle indicated on the command line does not exist (/Users/vsts/agent/2.153.2/work/1/s)
System.InvalidOperationException: Step Xamarin.Android.Prepare.Step_PrepareBundle failed: Xamarin.Android bundle indicated on the command line does not exist (/Users/vsts/agent/2.153.2/work/1/s) ---> System.InvalidOperationException: Xamarin.Android bundle indicated on the command line does not exist (/Users/vsts/agent/2.153.2/work/1/s)
  at Xamarin.Android.Prepare.Step_PrepareBundle.Execute (Xamarin.Android.Prepare.Context context) [0x00156] in /Users/vsts/agent/2.153.2/work/1/s/build-tools/xaprepare/xaprepare/Steps/Step_PrepareBundle.cs:42 
  at Xamarin.Android.Prepare.Step.Run (Xamarin.Android.Prepare.Context context) [0x00031] in /Users/vsts/agent/2.153.2/work/1/s/build-tools/xaprepare/xaprepare/Application/Step.cs:42 
  at Xamarin.Android.Prepare.Scenario.Run (Xamarin.Android.Prepare.Context context, Xamarin.Android.Prepare.Log log) [0x00095] in /Users/vsts/agent/2.153.2/work/1/s/build-tools/xaprepare/xaprepare/Application/Scenario.cs:33 
   --- End of inner exception stack trace ---
  at Xamarin.Android.Prepare.Scenario.Run (Xamarin.Android.Prepare.Context context, Xamarin.Android.Prepare.Log log) [0x00122] in /Users/vsts/agent/2.153.2/work/1/s/build-tools/xaprepare/xaprepare/Application/Scenario.cs:44 
  at Xamarin.Android.Prepare.Context.Execute () [0x000a7] in /Users/vsts/agent/2.153.2/work/1/s/build-tools/xaprepare/xaprepare/Application/Context.cs:777 
  at Xamarin.Android.Prepare.App.Run (System.String[] args) [0x0081d] in /Users/vsts/agent/2.153.2/work/1/s/build-tools/xaprepare/xaprepare/Main.cs:172 

Going to restart.

…rojectImports/>

The `<ResolveLibraryProjectImports/>` task has some logic that needs
to look at the item metadata of each `@(Reference)`:

* `%(AndroidSkipResourceExtraction)` to skip unzipping anything
* `%(AndroidSkipResourceProcessing)` to skip `<ConvertResourcesCases/>`

To make matters weird, we had some LINQ:

    foreach (var assemblyPath in Assemblies
            .Select (a => a.ItemSpec)
            .Distinct ()) {

And we would basically lose the metadata information...

To handle this problem, we had two `HashSet`'s we used to lookup the
values:

    assembliesToSkipCaseFixup = new HashSet<string> (StringComparer.OrdinalIgnoreCase);
    assembliestoSkipExtraction = new HashSet<string> (StringComparer.OrdinalIgnoreCase);
    bool metaDataValue;
    foreach (var asm in Assemblies) {
        if (bool.TryParse (asm.GetMetadata (AndroidSkipResourceProcessing), out metaDataValue) && metaDataValue)
            assembliesToSkipCaseFixup.Add (asm.ItemSpec);
        if (bool.TryParse (asm.GetMetadata (GetAdditionalResourcesFromAssemblies.AndroidSkipResourceExtraction), out metaDataValue) && metaDataValue)
            assembliestoSkipExtraction.Add (asm.ItemSpec);
    }

I added a test to see what happens when we use multiple `@(Reference)`
such as:

* `<PackageReference/>`
* `packages.config` and a regular `<Reference/>`
* `<Reference/>` with a full path

It turns out there are MSBuild targets that disambiguate duplicate
assembly references:

https://github.com/NuGet/NuGet.BuildTasks/blob/640c8e13a9b7ab6e86264a296638fbf3cc016ad1/src/Microsoft.NuGet.Build.Tasks/Microsoft.NuGet.targets#L186-L207

The `ResolveNuGetPackageAssets` does a `Remove` against duplicate
`@(Reference)`, even if there are no NuGet packages present in the
project.

It seems safe for us to just drop the `Distinct()` and the
`HashSet`'s. The code is a bit simpler after doing this.

I'm also seeing a small ~25ms performance improvement with this change.
@dellis1972
Copy link
Contributor

Failing Tests are

Xamarin.Android.JcwGen_Tests.logcat output
Xamarin.Android.JcwGen_Tests.Possible Crash / Release

They don't look related though :/

@jonpryor
Copy link
Member

They don't look related though :/

Indeed, they don't look related. They are disturbing; from msbuild-20190628T160328-Target-RunApkTests.binlog in xa-test-results-v9.4.99.150_Darwin-x86_64_pr_e23618b8-Release.zip:

  06-29 00:14:11.739  1880  2024 W ActivityManager: Unable to find instrumentation info for: ComponentInfo{Xamarin.Android.JcwGen_Tests/xamarin.android.jcwgentests.TestInstrumentation} (TaskId:207)

Unfortunately we don't normally package up the test .apk files, so there's no easy way to verify the sanity of the package.

The msbuild-*-Xamarin.Android.JcwGen-Tests.binlog files don't show any immediate errors, so it really doesn't make sense to me why this test would be failing at all.

Kicking off a rebuild: https://jenkins.mono-project.com/job/xamarin-android-pr-pipeline-release/1518/

@jonpryor jonpryor merged commit c2aadba into dotnet:master Jul 1, 2019
@github-actions github-actions bot locked and limited conversation to collaborators Jan 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants