Re-enable trimming of library tests on Apple mobile

[kotlarmilos]

Description

This PR re-enables the trimming of library tests on Apple mobile platforms. This change prevents the default settings of ILLink switches from targets that are imported. The regression was introduced in #103594.

Fixes #91923

[kotlarmilos]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[ivanpovazan]

Nice catch! These are difficult to detect when something goes wrong at runtime.

[akoeplinger]

I was under the impression we'd only do AOT on helix, not actual trimming. @steveisok how does this work?

[akoeplinger]

The new default for the property should affect many more cases than just AOT on helix so this doesn't sounds like the correct place for this fix.

[steveisok]

I was under the impression we'd only do AOT on helix, not actual trimming. @steveisok how does this work?

If you want trimming + AOT on helix, both have to occur in the same place. Otherwise, you have to send individual copies of the trimmed shared framework for each test from the build machine.

[steveisok]

That might not be as painful as it seemed initially.

[kotlarmilos]

@akoeplinger Since we perform trimming on Helix, we can move this to the Directory.Build.props file. However, it will not have effect on other components.

[kotlarmilos]

I've added a default switch to disable trimming on build machines as well. I haven't found a common location where these switches are set in Mono. Since we don't ship Apple mobile support from the runtime repository, the switch is set in the SDK/Xamarin.

A potential place to set this switch could be the AOT compiler props, but they are invoked after the trimming.

/cc: @sbomer

[sbomer]

The ideal solution would be for the AOT compiler props in this repo to be imported before the trimming props, which is how this works in the SDK. But barring that the Directory.Build.props seems like a good place.

[kotlarmilos]

The ILLink feature switches are moved to ConfigureTrimming target.

[kotlarmilos]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[kotlarmilos]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[kotlarmilos]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[kotlarmilos]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[kotlarmilos]

Let's revive this PR. Enabling trimming for Apple mobile consolidates it with the default shipping setup and reduces build time on CI.

[ivanpovazan]

Thanks for trying to revive this!

I agree this is beneficial for reducing our CI resource usage, but before going forward, we might want to also reprioritize and bring closer attention to having better crash/failure logs on iOS platforms testing dotnet/xharness#1188 so that potential new failures caused by trimming are not shadowed by various "Failed USB connection" or "TCP connection failures".

Another idea/option would be to run two jobs with: EnableAggressiveTrimming=true and EnableAggressiveTrimming=false on rolling builds, so that if there is a failure when trimming is enabled it can be compared against a non-trimmed variant to improve root cause detection of a failure.

Finally (and ideally) we would have a single source of truth for setting expected default feature switch values for iOS platforms when running tests, which we would keep in sync with: https://github.com/xamarin/xamarin-macios/blob/43768cfcb7e5ff9554a9bfd057f7b4d7a3c10cc7/dotnet/targets/Xamarin.Shared.Sdk.targets#L120

[ivanpovazan]

Couple of notes:

• If we want to actually trim codepaths guarded by these feature switches we would also need to set Trim attribute to true on these RuntimeHostConfigurationOption items, as ILLink picks up only those that have Trim=true
• Another, probably more robust, approach would be to set the actual feature switch properties instead of altering RuntimeHostConfigurationOption but that would need to happen really early in the build - at the time of property evaluation, like for example how we set DebuggerSupport on line 35 above. So we would have

  <PropertyGroup Condition="'$(EnableAggressiveTrimming)' == 'true' and '$(BuildTestsOnHelix)' != 'true'">
  ...
  </PropertyGroup>
  <PropertyGroup Condition="'$(EnableAggressiveTrimming)' == 'true' and '$(BuildTestsOnHelix)' == 'true'">
      <DynamicCodeSupport>false</DynamicCodeSupport>
      ... 
  </PropertyGroup>

instead of these RuntimeHostConfigurationOption overwrites

[ivanpovazan]

I think this is redundant. If the build on build machines was properly configured ie all feature switches have their values properly set for testing, we carry over RuntimeHostConfigurationOption to Helix through _AppleUsedRuntimeHostConfigurationOption item group and set the values in Target Name=_PrepareForAppleBuildAppOnHelix Project=ProxyProjectForAOTOnHelix.proj when we build on Helix.

[ivanpovazan]

FWIW
I think these need to be in a separate property group as I believe BuildTestsOnHelix=true in our setup.
So based on my previous comment, I think we will need a new property group conditioned with '$(EnableAggressiveTrimming)' == 'true' and '$(BuildTestsOnHelix)' == 'true' and then add these switches there.

[kotlarmilos]

Thanks, I missed the condition.

[kotlarmilos]

Another idea/option would be to run two jobs with: EnableAggressiveTrimming=true and EnableAggressiveTrimming=false on rolling builds, so that if there is a failure when trimming is enabled it can be compared against a non-trimmed variant to improve root cause detection of a failure.

Do we get the same ability to detect a failure by comparing a PR build (with trimming) with rolling build (without trimming)? I would like to avoid reducing cost by introducing an additional cost :)

Finally (and ideally) we would have a single source of truth for setting expected default feature switch values for iOS platforms when running tests, which we would keep in sync with: https://github.com/xamarin/xamarin-macios/blob/43768cfcb7e5ff9554a9bfd057f7b4d7a3c10cc7/dotnet/targets/Xamarin.Shared.Sdk.targets#L120

Yes, that would be ideal. The main question is how to achieve this without manual sync.

[ivanpovazan]

Do we get the same ability to detect a failure by comparing a PR build (with trimming) with rolling build (without trimming)? I would like to avoid reducing cost by introducing an additional cost :)

Hm yeah, I see your point.
The problem we are having now is:

• someone makes a change
• PR fails on iOS platforms due to a trimming issue
• the failure looks unrelated to the change, Known Build Error is opened, and the PR is merged (or worse PR is merged on red)
• the hard-to-detect failure stays, but is flagged by Build Analysis tool
• we have hard time tracking back what the problem was

To tackle this, it would be great if we would have an ability to rerun the same state with trimming disabled, to rule out that as an issue. With something like /azp rerun ios-platforms-no-trimming we would control the extra cost, and run it only if the failure occurs.

This is just an idea, probably out-of-scope, just thinking of ways of how to improve the triaging experience.

[kotlarmilos]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[kotlarmilos]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).