ReadyToRun + WPF + PublishTrimmed=true does not generate valid executable

[tapika]

Description

git clone https://github.com/tapika/swupd.git

Edit:
build.ps1

Find line "$publishTrimmed = 'false'" - comment it out using '#' (Without trimming executable works correctly)

Run command:

build.bat buildexe_chocogui_win7

go into folder src\ChocolateyGui\bin\publish_win7-x64

Produced build weights around 64 Mb, which is maybe 25% of original executable (with PublishTrimmed=true)

When trying to launch in debugger - it will produce following error:

The target process exited without raising a CoreCLR started event. Ensure that the target process is configured to use .NET Core. This may be expected if the target process did not run on .NET Core.
The program '[13292] ChocolateyGui.exe' has exited with code -2147450726 (0x8000809a).
The program '[13292] ChocolateyGui.exe: Program Trace' has exited with code 0 (0x0).

src\ChocolateyGui\ChocolateyGui.csproj

contains a lot of configurations of different trimming options - you can either comment
them out or fix them as they should work.

What I by myself find to be rather difficult - is configuring trimming with right options
for right libraries.

Based on debugging experience it's rather difficult or even impossible to say
which error results because of which missing dll / class / etc...

If you reconfigure using different trimming options - you will get other errors after that.

Another example - if you modify csproj, and remove all other ManagedAssemblyToLink xml tags and leave only lines:

      <ManagedAssemblyToLink>
        <TrimMode>link</TrimMode>
      </ManagedAssemblyToLink>

after that application fails to start with following exception:

This exception was originally thrown at this call stack:
    System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(int, System.IntPtr)
    MS.Internal.Text.TextInterface.Native.Util.ConvertHresultToException(int)
    MS.Internal.Text.TextInterface.Factory.GetFontCollection(System.Uri)
    MS.Internal.FontCache.DWriteFactory.GetFontCollectionFromFileOrFolder(System.Uri, bool)
    MS.Internal.FontCache.DWriteFactory.GetFontCollectionFromFolder(System.Uri)
    MS.Internal.FontCache.FamilyCollection.FromUri(System.Uri)
    System.Windows.Media.FontFamily.LookupFontFamilyAndFace(MS.Internal.FontCache.CanonicalFontFamilyReference, ref System.Windows.FontStyle, ref System.Windows.FontWeight, ref System.Windows.FontStretch)
    System.Windows.Media.FontFamily.FindFirstFontFamilyAndFace(ref System.Windows.FontStyle, ref System.Windows.FontWeight, ref System.Windows.FontStretch)
    System.Windows.Media.Typeface.ConstructCachedTypeface()
    System.Windows.Media.Typeface.CachedTypeface.get()
    ...
    [Call Stack Truncated]

This exception I was not able to solve no matter what. Suspect need to use <TrimMode>copyused</TrimMode> for some of dlls, but which ?!

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

git clone https://github.com/tapika/swupd.git

Edit:
build.ps1

Find line "$publishTrimmed = 'false'" - comment it out using '#' (Without trimming executable works correctly)

Run command:

build.bat buildexe_chocogui_win7

go into folder src\ChocolateyGui\bin\publish_win7-x64

Produced build weights around 64 Mb, which is maybe 25% of original executable (with PublishTrimmed=true)

When trying to launch in debugger - it will produce following error:

The target process exited without raising a CoreCLR started event. Ensure that the target process is configured to use .NET Core. This may be expected if the target process did not run on .NET Core.
The program '[13292] ChocolateyGui.exe' has exited with code -2147450726 (0x8000809a).
The program '[13292] ChocolateyGui.exe: Program Trace' has exited with code 0 (0x0).

src\ChocolateyGui\ChocolateyGui.csproj

contains a lot of configurations of different trimming options - you can either comment
them out or fix them as they should work.

What I by myself find to be rather difficult - is configuring trimming with right options
for right libraries.

Based on debugging experience it's rather difficult or even impossible to say
which error results because of which missing dll / class / etc...

If you reconfigure using different trimming options - you will get other errors after that.

Another example - if you modify csproj, and remove all other ManagedAssemblyToLink xml tags and leave only lines:

      <ManagedAssemblyToLink>
        <TrimMode>link</TrimMode>
      </ManagedAssemblyToLink>

after that application fails to start with following exception:

This exception was originally thrown at this call stack:
    System.Runtime.InteropServices.Marshal.ThrowExceptionForHR(int, System.IntPtr)
    MS.Internal.Text.TextInterface.Native.Util.ConvertHresultToException(int)
    MS.Internal.Text.TextInterface.Factory.GetFontCollection(System.Uri)
    MS.Internal.FontCache.DWriteFactory.GetFontCollectionFromFileOrFolder(System.Uri, bool)
    MS.Internal.FontCache.DWriteFactory.GetFontCollectionFromFolder(System.Uri)
    MS.Internal.FontCache.FamilyCollection.FromUri(System.Uri)
    System.Windows.Media.FontFamily.LookupFontFamilyAndFace(MS.Internal.FontCache.CanonicalFontFamilyReference, ref System.Windows.FontStyle, ref System.Windows.FontWeight, ref System.Windows.FontStretch)
    System.Windows.Media.FontFamily.FindFirstFontFamilyAndFace(ref System.Windows.FontStyle, ref System.Windows.FontWeight, ref System.Windows.FontStretch)
    System.Windows.Media.Typeface.ConstructCachedTypeface()
    System.Windows.Media.Typeface.CachedTypeface.get()
    ...
    [Call Stack Truncated]

This exception I was not able to solve no matter what. Suspect need to use <TrimMode>copyused</TrimMode> for some of dlls, but which ?!

Author:	tapika
Assignees:	-
Labels:	area-Host, untriaged
Milestone:	-

[ThomasGoulet73]

Hey @tapika,

The problem is that your EnsureAllAssembliesAreLinked target does not do anything at the moment. The multiple ManagedAssemblyToLink sets TrimMode to copyused, which is already the default. The multiple TrimmerRootAssembly must be outside of the target to be used.

You can either replace copyused for false or move your TrimmerRootAssembly outside of your target.

I was able to remove the target EnsureAllAssembliesAreLinked completely in favor of this code:

<ItemGroup>
  <TrimmerRootAssembly Include="mscorlib" />
  <TrimmerRootAssembly Include="System.Collections" />
  <TrimmerRootAssembly Include="System.Diagnostics.Debug" />
  <TrimmerRootAssembly Include="System.Runtime" />
  <TrimmerRootAssembly Include="System.Runtime.CompilerServices.VisualC" />
  <TrimmerRootAssembly Include="System.Runtime.Extensions" />
  <TrimmerRootAssembly Include="System.Runtime.InteropServices" />
</ItemGroup>

Hope this helps!

[tapika]

Ok, that was impressive - 160 Mb is already something. App even starts and seems to work more or less right. On exit however there is an exception:

---------------------------
Unhandled Exception
---------------------------
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.

 ---> System.TypeInitializationException: The type initializer for 'MS.Internal.SystemXmlLinqExtension' threw an exception.

 ---> System.NullReferenceException: Object reference not set to an instance of an object.

   at MS.Internal.SystemXmlLinqExtension..cctor()

   --- End of inner exception stack trace ---

   at MS.Internal.SystemXmlLinqExtension..ctor()

But I can try out later on to add more TrimmerRootAssemblies.

But I want to squeeze that 160 Mb further. For choco chocolatey.console\chocolatey.console.csproj I have used link quite heavily. Can you propose configuration with more aggressive trimming options, so far we managed to chew 30% away.

[agocke]

Hey @tapika, I'm glad you've had luck with .NET 5, but unfortunately WPF is not trim-compatible for .NET 6. As you've seen, as the trimmer gets more intelligent, it starts to trim away things that WPF needs to keep running. WPF hasn't had any modifications to deal with trimming, so it can't communicate what's necessary to the trimmer.

More info on incompatibilities at https://docs.microsoft.com/en-us/dotnet/core/deploying/trimming/incompatibilities. For a tracking issue on the trimming problems in WPF, see dotnet/wpf#3811

[tapika]

Ok, now managed to study ReadyToRun bit deeper.

Unlike you suggested - Chocolatey GUI is based on netcoreapp3.1 platform, not net5.0 / net6.0.

And adding couple of assemblies to trim - also exit exception went away.

    <TrimmerRootAssembly Include="System.Threading.ThreadPool" />
    <TrimmerRootAssembly Include="System.Xml.Linq" />

For some reason I did not need to add other assemblies - suspect because
chocolatey was optimized using TrimMode=link, and Chocolatey GUI using TrimMode=copyused.

I've fixed couple app internal bugs (which came because of .net core update) and will continue
to fix them later on. (issues with running UI code on non-UI thread).

Now install / uninstall seems to be working more or less. (At least tested on putty package)

Resulting file size 168 Mb (/p:PublishTrimmed=true), which is slightly better that 230 Mb (/p:PublishTrimmed=false).

I think 168 Mb is still bit too big package size, and I started to wonder whether I
can compress that one. Tried one utility mentioned in

https://www.hanselman.com/blog/brainstorming-creating-a-small-single-selfcontained-executable-out-of-a-net-core-application

I've managed to compress executable into 58 Mb, and app was still working.

One disadvantage I saw in whole story is that directory is hardcoded
and application ends up being extracted in %appdata%..\Local\warp\packages\myProgramName.exe.
(See also https://github.com/dgiagio/warp/issues/55 )

But if you cross compare with ReadyToRun - with ReadyToRun folder is also hardcoded
and it's something like %TEMP%\.net\<appname>\bxro3vee.pjx | dgiey5el.yii | or another cryptic hash \

And that directory is also not cleaned (like in wrap case https://github.com/dgiagio/warp/issues/30).

Wondering whether it's possible to provide extraction path folder to ReadyToRun and also control cleanup logic.

(I can raise new issues as necessary)

Then finally I've find out that .net 6.0 should support compression out of box - it's mentioned in here:
https://docs.microsoft.com/en-us/dotnet/core/deploying/single-file

See EnableCompressionInSingleFile=true.

I've got interested and attempted to upgrade Chocolatey GUI to .net 5 & 6 - I've partially managed to do that one
and committed my changes in side branch - if you want - you can try them out:

git clone -b develop https://github.com/tapika/swupd

To build on .net 5.0 platform:

build buildexe_chocogui_win7 -net net5.0

It does build .exe (which is 157Mb) -
but additionally to that it also produces a lot of native .dll's next
to executable like: mi.dll, miutils.dll, mscordaccore.dll and so on.

Same problem persists also in .net 6.0 platform - so maybe I need to create new bug-issue for this one.

Interesting thing that build works - as I have configured UI to use ProjectsCommon.props / net5.0-windows10.0.19041
target framework, which should be working only in windows 10, not in windows 7. (maybe another bug)

build buildexe_chocogui_win7 -net net6.0

gives error:

C:\Program Files\dotnet\sdk\6.0.100-rc.2.21505.57\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(162,5): error NET
SDK1180: Specified runtime identifier 'win7-x64'' implies Windows 7 compatibility. Single File publishing is not compatible with Windows 7.

Why ? On netcore3.1 & net5.0 this seems to be working ?

build buildexe_chocogui_win81 -net net6.0

Seems to go further, but also fails, I guess this was the problem you've mentioned about disabling WPF trimming.

C:\Program Files\dotnet\sdk\6.0.100-rc.2.21505.57\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(178,5): error NET
SDK1168: WPF is not supported or recommended with trimming enabled. Please go to https://aka.ms/dotnet-illink/wpf for more details.

By uncommenting lines:

build.ps1

        #if($buildTarget -eq 'chocogui')
        #{
        #    $publishTrimmed = 'false'
        #}

build seems to go through (with native dll present problem mentioned above)

With .exe produced size is 94 Mb.

It's compressed , but is it possible to adjust compression level or using different compression algorithm ?

Using 7-zip from here: https://github.com/mcmilk/7-Zip-zstd/releases + brotli compression for example it's possible to squeeze .exe to 36 - 41 Mb, which gives better compression ratio than .net 6.0.

Is self extracting executable of ReadyToRun a public source code ?

[vitek-karas]

Wondering whether it's possible to provide extraction path folder to ReadyToRun and also control cleanup logic.

DOTNET_BUNDLE_EXTRACT_BASE_DIR environment variable overrides the default extraction location (details). There's currently no cleanup logic or hook. Apps which extract lot of files are pretty slow on the first start, so to make them run reasonably fast on the second start, we rely on reusing the already extracted files. Deleting them every time would make the app start slowly every time.

Note that on .NET 6 publishing as single-file will not extract any managed assemblies. Only native libraries which are explicitly added to the bundle (IncludeNativeLibrariesForSelfExtract) would be extracted.

SDK1180: Specified runtime identifier 'win7-x64'' implies Windows 7 compatibility. Single File publishing is not compatible with Windows 7.
Why ? On netcore3.1 & net5.0 this seems to be working ?

Because on .NET 6 the runtime is part of the executable (statically linked) and not extracted. On Win7 the runtime needs the api-ms-..dll shims, and making them part of the executable is really tricky. On .NET 5 and below these files are extracted to disk. We really wanted to have single-file not extract anything to disk by default in .NET 6, so we prioritized that over Win7 support.

It's compressed , but is it possible to adjust compression level or using different compression algorithm ?

Currently no. For simplicity we used the deflate algorithm available in CoreCLR runtime already. We also prioritized decompression speed over everything else for now.

Is self extracting executable of ReadyToRun a public source code ?

Yes - the design docs for the feature are here.

[tapika]

Wondering whether it's possible to provide extraction path folder to ReadyToRun and also control cleanup logic.

DOTNET_BUNDLE_EXTRACT_BASE_DIR environment variable overrides the default extraction location

What if I would like that path to be configurable by the one who creates that bundle ?
At the moment default path include some sort of checksum directory, which does changes from one build to another.
I would prefer files to be extracted in c:\ProgramData\ChocolateyGUI and keep them in there.

Only thing which worries me - is that amount of files can change from one build to another, and it's not safe to keep previous versions of older bundle. I would prefer that all directories would remain in control of application, but all .dll files which exists on target path would be preserved only if they are part of bundle.

Not clearing is ok from my perspective.

Note that on .NET 6 publishing as single-file will not extract any managed assemblies. Only native libraries which are explicitly added to the bundle (IncludeNativeLibrariesForSelfExtract) would be extracted.

SDK1180: Specified runtime identifier 'win7-x64'' implies Windows 7 compatibility. Single File publishing is not compatible with Windows 7.
Why ? On netcore3.1 & net5.0 this seems to be working ?

Because on .NET 6 the runtime is part of the executable (statically linked) and not extracted. On Win7 the runtime needs the api-ms-..dll shims, and making them part of the executable is really tricky. On .NET 5 and below these files are extracted to disk. We really wanted to have single-file not extract anything to disk by default in .NET 6, so we prioritized that over Win7 support.

I would prefer to have compression present, even configured to maximum + decompression handled to file system still.
If load to ram - then decompression would need to be performed every time, if save to disk - then only first time.
If working with hard disk - then also no need to break windows 7 compatibility.

If I create any new issue - to which .net versions it can be implemented - only to .net 6 or also older (.net 5 / .net 3.1 ?)

It's compressed , but is it possible to adjust compression level or using different compression algorithm ?

Currently no. For simplicity we used the deflate algorithm available in CoreCLR runtime already. We also prioritized decompression speed over everything else for now.

I could recommend to check sheet from here:

https://quixdb.github.io/squash-benchmark/#results

And github itself:

https://github.com/quixdb/squash

Using API similar to or even exactly squash plugins can hide you compression behind one clean cut api.
Only change which can be done - is maybe to change from dynamic linking to static linking, so can be integrated into
host application.

If I'll raise issue to support other compression codecs (I would propose to start from brotli) - would that be accepted for implementation ?

[vitek-karas]

What if I would like that path to be configurable by the one who creates that bundle ?

#35249 - this is basically the same discussion.

Overall I think this is getting into the gray area of:

• Provide a way to build a single executable with reasonable performance characteristics as a way to simplify distribution of tools
• Provide a deployment mechanism which is customizable enough for various needs

Currently our goals are in the former bucket. The single-file feature is not meant to replace installers and similar technologies.

If I create any new issue - to which .net versions it can be implemented - only to .net 6 or also older (.net 5 / .net 3.1 ?)

Most likely answer is .NET 7+. We typically don't ship new functionality in servicing releases (too much risk associated), unless there's a really good reason to do so.

If I'll raise issue to support other compression codecs (I would propose to start from brotli) - would that be accepted for implementation ?

As with any other feature this depends on many trade offs:

• Performance
• Size impact
• Complexity of the change (for example, currently it's really hard to deliver optional hosting features as we only build one single file executable, so for example adding multiple compression algorithms would mean carrying the decompressors for all of them - or tackle the much harder problem of modular singlefile executable)
• Prioritization against all the other feature requests

[tapika]

• Provide a way to build a single executable with reasonable performance characteristics as a way to simplify distribution of tools
• Provide a deployment mechanism which is customizable enough for various needs

Currently our goals are in the former bucket. The single-file feature is not meant to replace installers and similar technologies.

As installer you probably mean packaging technology, which purpose is to bring application for first time on computer ?

First time is good, but if you think about it - application may need also second, third, and so on times to bring itself to computer - like software update. If installer technology does not satisfy developer needs, then custom install+software update technologies will be born to replace installer technologies.

:-)

Anyway, I think I've managed to have now readytorun with size 60 Mb (choco) - that is cli only, and 168Mb with UI (choco gui). if 168Mb weights too much, I can use 60Mb to bring .net core or .net framework install and continue from there. I think end-user will survive that for first installation he will have no UI, but after that UI will be present. And even that 168 Mb is not that bad if whole application installation is measured in Gb:s.

[tapika]

I think this ticket can be closed. Parameter handling is not perfect, but at least it works.