Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[net6] iOS release binaries are dynamically linking to .dylib #11145

Closed
spouliot opened this issue Apr 7, 2021 · 5 comments · Fixed by #12323
Closed

[net6] iOS release binaries are dynamically linking to .dylib #11145

spouliot opened this issue Apr 7, 2021 · 5 comments · Fixed by #12323
Assignees
Labels
dotnet-pri0 .NET 6: required for stable release iOS Issues affecting Xamarin.iOS
Milestone

Comments

@spouliot
Copy link
Contributor

spouliot commented Apr 7, 2021

see dotnet/targets/Xamarin.Shared.Sdk.targets

<ResolvedFileToPublish
   Update="@(ResolvedFileToPublish)"
  RelativePath="$([MSBuild]::MakeRelative($(MSBuildProjectDirectory)$(PublishDir),$(_DylibPublishDir)))%(Filename)%(Extension)"
  Condition="'%(Extension)' == '.dylib'" />

This copies all .dylib into the final app bundle (.app).

This greatly increase the size of the .app since there are duplicates and all (or at least part) of the code is also statically linked into the main executable binary. This is visible from the reports included in #10249

This is not allowed by App Store rules (dynamic linking is only possible to user frameworks, not dylib).

Note: Removal of the .dylib leads to crash at runtime so there is dynamic linking being done.

MySingleView -> /Users/poupou/git/xamarin/xamarin-macios/tests/dotnet/size-comparison/MySingleView/dotnet/bin/Debug/net6.0-ios/ios-arm64/MySingleView.dll
  Optimizing assemblies for size, which may change the behavior of the app. Be sure to test after publishing. See: https://aka.ms/dotnet-illink
  Installing application bundle 'com.xamarin.mycomparableapp' on 'Mercure'
  Application bundle 'com.xamarin.mycomparableapp' installed on 'Mercure'
  Launched application 'com.xamarin.mycomparableapp' on 'Mercure' with pid 3113
  xamarin_vm_initialize (2, 0x16d643130, 0x16d643120): rv: 0
  2021-04-06 20:37:50.830 MySingleView[3113:3659312] Xamarin.iOS: An exception occurred when calling Runtime.Initialize:
  libSystem.Native (System.DllNotFoundException)
     at System.Threading.LowLevelMonitor.Initialize()
     at System.Threading.WaitSubsystem.ThreadWaitInfo..ctor(Thread thread)
     at System.Threading.Thread.EnsureWaitInfo()
     at System.Threading.Thread.InitializeCurrentThread()
     at System.Threading.Thread.get_CurrentThread()
     at System.Threading.SynchronizationContext.SetSynchronizationContext(SynchronizationContext syncContext)
     at UIKit.UIApplication.Initialize()
     at ObjCRuntime.Runtime.InitializePlatform(InitializationOptions* options)
     at ObjCRuntime.Runtime.Initialize(InitializationOptions* options)
  2021-04-06 20:37:50.830 MySingleView[3113:3659312] Unhandled managed exception: libSystem.Native (System.DllNotFoundException)
  
  =================================================================
  	Native Crash Reporting
  =================================================================
  Got a SIGABRT while executing native code. This usually indicates
  a fatal error in the mono runtime or one of the native libraries 
  used by your application.
  =================================================================
  
  =================================================================
  	Native stacktrace:
  =================================================================
  	0x103009c4c - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : mono_dump_native_crash_info
  	0x102ff33d8 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : mono_handle_native_crash
  	0x103009020 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : sigabrt_signal_handler
  	0x1f54d7298 - /usr/lib/system/libsystem_platform.dylib : <redacted>
  	0x1f54dcb50 - /usr/lib/system/libsystem_pthread.dylib : pthread_kill
  	0x1b2e5bb74 - /usr/lib/system/libsystem_c.dylib : abort
  	0x102d41388 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : xamarin_find_protocol_wrapper_type
  	0x102eb31f8 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : mono_invoke_unhandled_exception_hook
  	0x102ff2de8 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : mono_handle_exception_internal
  	0x102ff1734 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : mono_handle_exception
  	0x103007454 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : mono_arm_throw_exception
  	0x102c5e0cc - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : throw_exception
  	0x102f06450 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : mono_raise_exception
  	0x102d410ac - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : xamarin_process_managed_exception
  	0x102d40f04 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : xamarin_process_managed_exception_gchandle
  	0x102d417ec - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : xamarin_initialize
  	0x102d52434 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : xamarin_main
  	0x103059644 - /private/var/containers/Bundle/Application/49878A4D-C476-4844-A1B4-5BE60D05E741/PublicStaging.app/MySingleView : main
  	0x1a96626b0 - /usr/lib/system/libdyld.dylib : <redacted>
  
  =================================================================
  	Basic Fault Address Reporting
  =================================================================
  Memory around native instruction pointer (0x1d7982414):0x1d7982404  ff 0f 5f d6 c0 03 5f d6 10 29 80 d2 01 10 00 d4  .._..._..)......
  0x1d7982414  03 01 00 54 7f 23 03 d5 fd 7b bf a9 fd 03 00 91  ...T.#...{......
  0x1d7982424  ff 71 ff 97 bf 03 00 91 fd 7b c1 a8 ff 0f 5f d6  .q.......{...._.
  0x1d7982434  c0 03 5f d6 90 29 80 d2 01 10 00 d4 03 01 00 54  .._..).........T
  
  =================================================================
  	Managed Stacktrace:
  =================================================================
  =================================================================
@spouliot spouliot added iOS Issues affecting Xamarin.iOS dotnet-pri0 .NET 6: required for stable release labels Apr 7, 2021
@spouliot spouliot added this to the .NET 6 milestone Apr 7, 2021
spouliot added a commit to spouliot/xamarin-macios that referenced this issue Apr 7, 2021
…builds

The original goal was to remove the `.dylib` from the app bundle.
However some are required, even in release, so that will need to be
fixed first [1]

[1] xamarin#11145
@filipnavara
Copy link
Contributor

filipnavara commented Apr 7, 2021

The underlying cause is tracked by #10950.

UPD: Hmm, maybe I spoke too fast. On iOS the dllmap API should likely still do the job.

@filipnavara
Copy link
Contributor

filipnavara commented Apr 7, 2021

Turns out I wasn't too far off. The calls to mono_dllmap_insert don't actually work because the name doesn't match the one used in the managed code. The managed code requires the lib prefix. If I update the calls it still fails but this time because it cannot find libcoreclr.so which is artifact of this code: https://github.com/dotnet/runtime/blob/bbb3dc4669b5c47ce7558026d68c201040191abd/src/mono/mono/metadata/native-library.c#L1070-L1074

The fallback path should not be hit at all though. It's the System.Native symbols not present in the main executable (verified with empty output of nm tests/dotnet/size-comparison/MySingleView/dotnet/bin/Debug//net6.0-ios/ios-arm64/MySingleView.app/MySingleView | grep SystemNative).

@filipnavara
Copy link
Contributor

filipnavara commented Apr 7, 2021

And the last missing piece is that this code doesn't account for the lib prefix in the managed code either.

So, in summary:

  • Calls to mono_dllmap_insert need to be adjusted to use the lib prefix to match the managed code
  • ListExportedSymbols linker step needs to be fixed to account for the lib prefix
  • Copying of .dylibs can be removed

@spouliot
Copy link
Contributor Author

spouliot commented Apr 8, 2021

@filipnavara interesting approach. You're using the override to get the dynamic linking resolved into the main executable. I'm not sure if that works with bitcode (for tvOS).

IIRC what we've been doing (until recently) is having the "__Internal" present before AOT'ing which allows the AOT compiler to do direct calls (and solve our bitcode problem for tvOS and watchOS).

It would be interesting to compare both (build perf, runtime perf and size) but I'm be happy to start your PR 😄

@filipnavara
Copy link
Contributor

Technically it would be possible to build the dotnet/runtime with __Internal names or do the substitution in the linker. I just started out with the code that was already there but didn't work because of a naming change that happened last year (dotnet/runtime#33236). tvOS is still blocked by dotnet/runtime#48508 so I cannot test it.

spouliot added a commit that referenced this issue Apr 8, 2021
…builds (#11153)

The original goal was to remove the `.dylib` from the app bundle.
However some are required, even in release, so that will need to be
fixed first [1]

[1] #11145
rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this issue Jul 26, 2021
…ng statically. Fixes xamarin#10950, xamarin#11145 and xamarin#12100.

* Add support for Mono Components.

* Modifies how we look up symbols from native libraries shipped with Mono:

This also meant propagating how libmono is linked from the MSBuild code to the Application
class so that our existing logic is able to correctly determine which native mono
lib to use.

Fixes xamarin#10950.
Fixes xamarin#11145.
Fixes xamarin#12100.
@rolfbjarne rolfbjarne self-assigned this Aug 2, 2021
rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this issue Aug 2, 2021
…ng statically. Fixes xamarin#10950, xamarin#11145 and xamarin#12100.

* Add support for Mono Components.

* Modify how we look up symbols from native libraries shipped with Mono: we keep
  track of which native libraries we linked with, and depending on how we linked
  to those assemblies, we look the symbols up at runtime in either the current executable
  (if linking statically), or the actual library (where the P/Invoke says they're
  supposed to be).

* This means that we have to propagate how libmono is linked from the MSBuild code
  to the Application class so that our existing logic is able to correctly determine
  which native mono lib to use.

* Modify how we list the P/Invokes we need to preserve by taking into account the
  list of native libraries from Mono we have to link with (for .NET). For legacy
  Xamarin, I've reverted the logic to how it was before we started adding .NET support.

Fixes xamarin#10950.
Fixes xamarin#11145.
Fixes xamarin#12100.
rolfbjarne added a commit that referenced this issue Aug 3, 2021
…ng statically. Fixes #10950, #11145 and #12100. (#12323)

* Add support for Mono Components.

* Modify how we look up symbols from native libraries shipped with Mono: we keep
  track of which native libraries we linked with, and depending on how we linked
  to those assemblies, we look the symbols up at runtime in either the current executable
  (if linking statically), or the actual library (where the P/Invoke says they're
  supposed to be).

* This means that we have to propagate how libmono is linked from the MSBuild code
  to the Application class so that our existing logic is able to correctly determine
  which native mono lib to use.

* Modify how we list the P/Invokes we need to preserve by taking into account the
  list of native libraries from Mono we have to link with (for .NET). For legacy
  Xamarin, I've reverted the logic to how it was before we started adding .NET support.

Fixes #10950.
Fixes #11145.
Fixes #12100.
@ghost ghost locked as resolved and limited conversation to collaborators Apr 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
dotnet-pri0 .NET 6: required for stable release iOS Issues affecting Xamarin.iOS
Projects
None yet
3 participants