Add feature switch for disabling startup hooks

[mateoatr]

Fix #36526. This introduces a new feature switch which allows the ILLinker to trim the startup hooks logic.

[ghost]

Tagging subscribers to this area: @safern, @ViktorHofer
See info in area-owners.md if you want to be subscribed.

[jkotas]

Startup hook is one of many places that call AssemblyLoadContext.LoadFromPath that is the root cause of this problem. Do we expect to have a feature switch for each of these places? Should we also (or instead) have a switch that turns AssemblyLoadContext.Default.LoadFromAssemblyPath into NotSupportedException?

[MichalStrehovsky]

I would structure this so that if someone specifies a startup hook (there's something actionable in STARTUP_HOOKS) and startup hook support was linked out, they get some sort of feedback about it (exception?).

[jkotas]

I think it is better to ignore it silently (no exception).

Having ability to disable startup hooks per app is useful on its own, independent of linking. For example, STARTUP_HOOKS can be set globally for APM, but there is one utility where you do not want it to kick in.

[eerhardt]

Any chance we can add a test that has both a startup hook defined, and sets the app context switch to false, and verifies that the startup hook is not executed?

[eerhardt]

Startup hook is one of many places that call AssemblyLoadContext.LoadFromPath that is the root cause of this problem.

@jkotas - Isn't it any dynamic load of an assembly? The linker needs to see every assembly that will be loaded in the app, or else it may trim code that the dynamically loaded assembly needed.

[jkotas]

Yes, it is any dynamic load. All dynamic assembly loads from disk file go through AssemblyLoadContext.LoadFromPath (conceptually at least). I would be ok with treating in-memory loads same way too. The in-memory loads are slightly different case, but the distinction is very subtle to explain. Disabling all dynamic loads is very simple to explain.

[mateoatr]

Should we also (or instead) have a switch that turns AssemblyLoadContext.Default.LoadFromAssemblyPath into NotSupportedException?

I think this is a good option, and I'd rather have one feature switch to disable all the places where a dynamic load can happen than one for each. Unless there's a scenario in which we'd like to use StartupHookSupport but still be able to load random assemblies somewhere else, I think I should close this PR to work on this "bigger" feature switch.

[MichalStrehovsky]

The Assembly.LoadFrom APIs and friends are already annotatated as linker unfriendly. Linker will warn whenever the codepath that calls into them is reachable. The objective of this pull request is to get rid of that warning - startup hook is compiled into an empty console app, so everyone who does trimming would see the warning.

I'm not sure I like the idea of having a "feature switch" that makes LoadFrom and friends throw, with what I assume is no linker warnings - it's a pretty big hammer. Everyone who does any sort of linking is going to have to disable this (because they get warnings from the StartupHook use from the get-go).

Once LoadFrom is disabled globally, people won't get a warning when they e.g. add a NuGet package that calls into LoadFrom in some obscure scenario. The end result is going to be a broken app.

It feels like too big of a hammer.

[jkotas]

Ah, I have not realized that this change is required to fix IL warning that you get for LoadFromPath use inside startup hook today. It makes sense now.

To answer my own questions:

Do we expect to have a feature switch for each of these places?

Yes, we expect to have a feature switch for each use of LoadFromPath that is optional feature like startup hook. Each of these optional features will be disabled for trimmed apps by default.

switch that turns AssemblyLoadContext.Default.LoadFromAssemblyPath into NotSupportedException?

Warnings that the linker issues for LoadFromAssemblyPath today cover this case already. Having a switch like this may be sometimes useful in addition to the warnings we get today, but it is not helping with what we are focused on now (drive IL linker warnings to zero).

[jkotas]

this change is required to fix IL warning that you get for LoadFromPath use inside startup hook today

How do we make startup hook disabled by default for trimmed apps so that you get no warnings out of the box?

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke
See info in area-owners.md if you want to be subscribed.

[mateoatr]

How do we make startup hook disabled by default for trimmed apps so that you get no warnings out of the box?

We can use this feature switch in the SDK everytime PublishTrimmed=true.

[MichalStrehovsky]

We can use this feature switch in the SDK everytime PublishTrimmed=true.

Yeah, but that doesn't pass the feature switch that disables startup hooks. Startup hooks are enabled by default on .NET Core, unfortunately.

So we need to make a call on whether we disable them by default when PublishTrimmed is specified, disable them by default in general (so that there's no secretly disabled functionality that one needs to dig out of docs when PublishTrimmed is specified), or whether people need to do two things to actually get usable trimming - specify PublishTrimmed and specify the feature switch.

I wish when startup hooks got added, they were added as opt-in. :(

[jkotas]

I wish when startup hooks got added, they were added as opt-in. :(

Making startup hooks opt-in would not full-fill the requirements that they were added for originally.

[jkotas]

disable them by default when PublishTrimmed is specified

I think it is the right option.

[MichalStrehovsky]

disable them by default when PublishTrimmed is specified

I think it is the right option.

We've been taking a position that PublishTrimmed should never lead to observable changes in behavior. If a change in behavior is possible, there needs to be a warning. Having a warning for all apps would be annoying so maybe we don't have any other choice than to break that promise here, but I don't want to take it lightly.

Making startup hooks opt-in would not full-fill the requirements that they were added for originally.

Would that be a possible problem for trimmed apps?

[marek-safar]

We've been taking a position that PublishTrimmed should never lead to observable changes in behaviour.

I don't think that's entirely true. We publish in 5.0 with observable changes in behaviour, one example is UseSystemResourceKeys but there are many more which by default change app behaviour.

disable them by default when PublishTrimmed is specified

If I understood this feature it requires also runtime.json settings. Could we add a warning when there are such settings and PublishTrimmed is used?

[vitek-karas]

Just to clear things a little:

• UseSystemResourceKeys is not on by default, so just adding PublishTrimmed=true will NOT enable it. So by default there's no observable difference (at least from this feature)
• Startup hooks don't require anything anywhere to work, they're on by default... and can't be turned off. That's why it's a problem, linker will generate warnings with no way for the developer to "correct" them.

[marek-safar]

UseSystemResourceKeys is not on by default

As far as users are concerns this is on by default for example for Blazor workload when published with trimming. https://github.com/dotnet/aspnetcore/blob/219ecd6880120215137f4d6ef0dd2aa62c780af2/src/Components/WebAssembly/Sdk/src/targets/Microsoft.NET.Sdk.BlazorWebAssembly.Current.targets#L58

Startup hooks don't require anything anywhere to work

The doc talks about env variable to be set to make it work https://github.com/dotnet/runtime/blob/master/docs/design/features/host-startup-hook.md#proposed-behavior. Though not sure how accurate that doc is

[vitek-karas]

I consider Blazor "special" in this regard. For one thing, publishing Blazor also turns trimming on by default - another difference from non-Blazor apps (at least for now).

The env. variable is the mechanism to actually use the startup hook feature, but the code is active all the time. It's similar to event pipe - the feature is there all the time, but does basically nothing, unless something external connects to it. Similarly startup hooks are there all the time, and are activated via env. variable.

[MichalStrehovsky]

As far as users are concerns this is on by default for example for Blazor workload

Right, but Blazor is a niche scenario where we established users can get by without the message. Blazor is incompatible with the regular .NET in many other ways. I would be more careful about making it the default elsewhere.

The doc talks about env variable to be set to make it work

The env variable is specified at runtime, not at compile time. We don't have a good spot to emit a warning - that's why I was suggesting throwing an exception - it feels better to indicate that we're not going to do something the user wants us to do than to silently not do it.

[jkotas]

One of the deployment patterns for startup hooks is to have them set in the global environment. The first thing that startup hook does is a check whether it is meant to be running for the app it got loaded for. If not, it exits and does nothing. Throwing an exception would break this deployment pattern.

Startup hooks are a niche scenario, even more niche than Blazor. It would be nice to avoid teaching everybody about their interactions with trimming by having good defaults.

[vitek-karas]

Regarding the default for trimmed apps - I would very much like to have this off by default for trimmed apps (along with couple other feature switches like COM for example). The problem is the mechanics. I strongly believe that feature switches should behave consistently regardless of trimming or even publishing. Meaning my app should run the same way when I do dotnet run as well as when I do dotnet publish /p:PublishTrimmed=true and then run the executable. Anything else and we're basically back to where we are with .NET Native experience which is undesirable.

In order to achieve that we need to have some way for the developer to specify "I want this app to support trimming" (or something like that), which is not the same as "Trim this app now". Currently the PublishTrimmed=true is the "action" - Trim this app now, since it only affects dotnet publish. What we need is the "intent" - I want this app to support trimming. We could use the PublishTrimmed property for this as well (the name would not be ideal, but we can live with that), we would just need to explain that the meaning is somewhat different - it's the "intent", which also happens to turn on trimming during publish. Or we could introduce a new property for this purpose... or something else.

I think we will need similar mechanism for single-file, AOT and so on. I view feature switches as a tool we can use for other scenarios, not just trimming. In fact the BinaryFormatter support is also a feature switch which is used in lot of contexts without trimming in the picture (it does also have trimming part to it, but that's just a nice side effect). The thing which scares me is overusing feature switches - they're pretty easy to add, but if we have too many it's going to be a mess.

I'm trying to write all of this down in some kind of a design doc which will tackle the end-to-end experience, unfortunately I don't have it in any shape which would make sense to share right now.

[jkotas]

Meaning my app should run the same way when I do dotnet run as well as when I do dotnet publish /p:PublishTrimmed=true and then run the executable

I agree with this when it comes to the inner working of app. I am less concerned when it comes to how the app interacts with the environment.

For example, take self-contained publish vs. framework dependent publish. These two options are meant to be fully transparent. It holds for the most part, but breaks when it comes to interaction with the environment. For example, the app may check for specific files in the AppDomain.BaseDirectory and crash if it does not find them.

I see the startup hooks in this environmental category. Other features that fall into this category include managed profilers or "watch window" debugging experience.

[marek-safar]

Meaning my app should run the same way when I do dotnet run as well as when I do dotnet publish /p:PublishTrimmed=true and then run the executable.

I'm not so sure about that. I think there are legit scenarios which will never care about trimming, mostly complex apps with a shared framework like VS. Why should we limit everyone to a common ground if the needs are different for large and small apps?

For example, take self-contained publish vs. framework dependent publish. These two options are meant to be fully transparent.

I don't think that hods anymore. It might be possible for some scenarios but as a general rule, I don't think we can say that. I think self-contained became a synonym for workload-specific output and not something that is a choice for any configuration.

In order to achieve that we need to have some way for the developer to specify "I want this app to support trimming" (or something like that), which is not the same as "Trim this app now".

We have that already but we don't use it that way but we should. I agree we should leverage PublishTrimmed everywhere in the build process and ensure that every workload which will use trimming has it set in project settings.

The thing which scares me is overusing feature switches - they're pretty easy to add, but if we have too many it's going to be a mess.

Agree but it's called feature-switch and it's about features, some are small whereas others are large. In this case, I think it's fine to have it (perhaps rename it to host startup hook) as it's a feature even though is not well documented and supported by CoreCLR only.

[vitek-karas]

I'm not so sure about that. I think there are legit scenarios which will never care about trimming, mostly complex apps with a shared framework like VS. Why should we limit everyone to a common ground if the needs are different for large and small apps?

I don't see the problem here to be honest. If a given app doesn't care about trimming that's fine. Some features will be "broken" when trimming (startup hooks as an example), those features will produce warnings. But if I can build my app without warnings its behavior should be consistent across all of the builds where I don't get warnings. I think @jkotas take on this is the right one - the behavior will not be identical, different build configurations are intentionally modifying some behavior of the app (debug enables better debugging experience, self-contained changes deployment requirements, ...). But features which are not intentionally affected by the change of configuration should NOT be affected. Crazy example: building the app as Debug should not run it with a different default culture when compared to Release.

It's kind of hard to make a broad statement, since there will always be exceptions and gotchas, but I would much like to make a relatively simple statement about the consistency of behavior across different build/publish configurations. To me it's about expectations - clicking a single check-box in VS to enable single-file doesn't "feel" dangerous, so it should not break my app (or if it will, it should tell me -> warnings).

Self-contained versus framework-dependent

I think that some verticals will not support framework-dependent and that's OK, but those which will should behave consistently across the two configurations. To that end the verticals which don't support framework-dependent should warn if I try to force it - building Blazor client WASM app should fail/warn if I explicitly state --self-contained false. (It actually does fail today, but for different reasons, which is not ideal).

[vitek-karas]

@samsp-msft can you help us here with the message wording?

This is the message for a warning which will show up if an app allows startup hooks but asks for trimming. Since startup hooks by design load "random" code into the process, that code may have dependencies on things which were trimmed and thus it won't work.

The goal of the message should be:

• Describe what the problem is - short version "startup hooks are not compatible with trimming"
• At least hint at possible solution - in this case use the StartupHookSupport feature switch to disable startup hooks.

My attempt:
"This application enables startup hooks which might require assemblies, types or members removed during trimming. Startup hooks can be disabled with the StartupHookSupport feature switch."

[samsp-msft]

Potential Incompatibility: The StartupHookSupport feature switch has been enabled for this app which is being trimmed. Startup hook code is not observable by the trimmer and so required assemblies, types and members may be removed.

[vitek-karas]

LGTM pending the warning message change

[vitek-karas]

Looks good.

Just so that you're aware. Once this is merged and the new runtime makes it to the sdk repo, the CI will fail on it in sdk, because of this test: https://github.com/dotnet/sdk/blob/0ae5136153da2d364288be1d9cdd4804d8279adf/src/Tests/Microsoft.NET.Publish.Tests/GivenThatWeWantToRunILLink.cs#L288

You will need to update the test in the 'update dependencies' PR to push it through.