.NET 8: No longer build Windows-specific assets (except on Windows)

[richlander]

.NET has various construction challenges. One of them is that various product assets are multi-targeted for net4* targets, which we build, taking both time and space. The product would be faster to build and smaller if we did not build those targets. We should find a way to achieve that.

These assets are mostly (or entirely) unvaluable and unused. Also, today source-build targets Linux, where net4* is the least likely to be valuable.

Related: dotnet/sdk#16895

The .NET SDK enables building net4* assets. It adds a PackageReference to Microsoft.NETFramework.ReferenceAssemblies. We can continue to offer that. It is largely unrelated to this topic.

It would be useful to do three things:

• Determine the cost of these assets, in built time and/or output size.
• Validate the premise that these assets are unused/unvaluable.
• Can we just target net8 and netstandard*. What scenarios require targeting other TFMs?

The initial findings is that removing .NET Framework targeting would:

• Reduce source size by 3GB.
• Reduce build time by TBD.
• Reduce build output size by TBD.

@MichaelSimons @mmitche @jaredpar @omajid @tmds

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[mmitche]

We'll need to figure out where the net4* usage is coming from. I took a quick look yesterday at runtime and aspnetcore, and here is what I saw offhand:

• Infrastructure - Some tools and tasks that use the desktop runtime. Some (most) of these are probably movable to core. Obviously any of these tools wouldn't work on Linux or Mac anyway. My recommendation would be to triage them and try to move all infra/tooling to core.
• Tests - There were a bunch of test projects that multi-targeted. Those could go away I think
• Product - I found at least one place in aspnetcore (src/SiteExtensions/Runtime/Microsoft.AspNetCore.Runtime.SiteExtension.pkgproj) that targets 462.

I am all for a full move to core.

[richlander]

We either need to "full move to core" or have a new feature that ignores certain families of targets. Possibly we'll need a bit of both.

[mmitche]

We can also make rules per platform. Basically say: Windows can use desktop targeting packs as a pre-built input, but it's not allowed on Linux. So when building for Linux, we don't build those RIDs.

It's likely we will find cases where we need to build something for desktop (e..g tooling that can't be ported). We shouldn't penalize all platforms though.

[jkotas]

There are many packages in dotnet/runtime that target multitarget .NET Framework. Look for $(NetFrameworkMinimum) to find the full set.

Windows can use desktop targeting packs as a pre-built input, but it's not allowed on Linux. So when building for Linux, we don't build those RIDs.

Yes, that's definitely an option. In other words, we would say that the Microsoft-built nuget packages cannot be built via offline source build.

[mmitche]

@jkotas Is there a reason to keep this targeting around long term?

[jkotas]

For as long as we want to keep supporting .NET Framework with the OOB dotnet/runtime packages. It is a complex topic. For example, we have some evidence that .NET Framework support allows people to modernize faster since they do not have to move from .NET Framework to .NET Core at once.

[richlander]

For as long as we want to keep supporting .NET Framework with the OOB dotnet/runtime packages.

That's goodness.

Can those packages be built from dotnet/runtime and not souce-build, at all?

[mmitche]

For as long as we want to keep supporting .NET Framework with the OOB dotnet/runtime packages.

That's goodness.

Can those packages be built from dotnet/runtime and not souce-build, at all?

We shouldn't do that. That keeps the runtime build around, as well as any dependency flow and build that consumes these packages. In other words, you're basically in the same situation as today, perhaps with more complexity.

I think the right call here is to remove the framework packages from the source build reference packages, and conditionalize the build so that those TFMs are not built when targeting platforms that do not allow an online restore (or need them). Basically anything but Windows.

There really cannot be independent repo official builds for anything but some isolated leaf components like the tooling.

[richlander]

OK. Makes sense.

Does generating the net[old] targeting packs need to be done from this repo? That can surely be a foreign pre-built. Right?

[mmitche]

They're already on NuGet. Just need restore them (the SDK does this automatically I think).

[richlander]

Right.

[terrajobst]

Generally speaking, we should tie multi-targeting for net4x to whether or not we offer netstandard2.0. That's because .NET Framework 4.6.1 is considered implementing .NET Standard 2.0 but it only rally works starting with .NET Framework 4.7.2.

Said differently: If you target netstandard2.0, you should also target net461.

Edit: Looks like we're talking about different things here. This is about building the .NET platform from source, which today only happens on Linux, for Linux, specifically Red Hat. I'd agree that in that context building any Windows assets is a waste of resources, which includes net4x.

[ericstj]

Cc @ViktorHofer
We can build dotnet/runtime without those packages. That’s how the vertical builds work today. I suspect there are plenty of components up stack that depend on them though. We could build partial versions of the packages just for upstack build. What’s the thinking around these packages in general? I don’t expect source build really needs them. It doesn’t produce any NuGet packages to publish on NuGet.org so it shouldn’t need any of these packages with cross targeting builds. An alternative would be to make sure the upstack build consume transport packages wherever it was consuming a cross-targeting “official” package.

[richlander]

Switch your mindset to source-build being the only build. That's the underlying (and unstated) assumption of this issue.

[ericstj]

That’s fine. It works fine with that. We have a build leg of the product that does produce NuGet packages (some of which have cross platform joins, in addition to cross framework joins). I’m suggesting that Source build for a single distro shouldn’t need to care about those. So it’s probably better for us to think about them as different outputs that we try to keep out of the vertical builds.

[mmitche]

@ericstj The intention is to eliminate the cross-platform joins (specifically multi-machine joins) and make each vertical independently buildable on a single machine, and that source build is the only build producing artifacts under the SDK umbrella. The current Microsoft official build goes away entirely. The intention is to get out of the situation where we have two different methods of building altogether. What that means is that the verticals need to be able to produce the artifacts for their respective platforms. Some platforms could also produce platform-agnostic artifacts and package them up, though of course we would only ship those artifacts to NuGet.org from one of the verticals.

The goal is a single-machine vertical build for each platform that doesn't require complex orchestration that typically ends up tied to our infrastructure implementation, is robust long-term, can be built locally by developers (or other organizations, etc.). This may require some new features in NuGet (e.g. the ability to create global tool packages in a method similar to metapackages?).

This will require some rework in some areas. For instance, today workload manifests are produced in the runtime build, requiring a join among all platforms (to generate the VS insertion manifest). This is only required for VS authoring though. @joeloff's proposed solution in this instance is to note that VS insertion artifacts across all repos are simply derived and repackaged from the primary .NET artifacts (MSIs, zips, etc.). We would move generation of VS-specific artifacts into another build, which would just consume and package up the .NET artifacts with the right metadata. This is consistent with how we work with other consumers (e.g. we don't produce VSCode artifacts, docker images, corext packages, etc. directly in the build).

[omajid]

Hey, @terrajobst , we have some good news about:

This is about building the .NET platform from source, which today only happens on Linux, for Linux, specifically Red Hat.

There's actually many other platforms that have adapted source-build or are in the middle of doing so. For example:

• Alpine Linux (testing): https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/29253
• Arch Linux: https://archlinux.org/packages/community/x86_64/dotnet-sdk/
• Fedora: https://src.fedoraproject.org/rpms/dotnet6.0
• Homebrew (macOS): https://formulae.brew.sh/formula/dotnet

We are working with some folks to try and get this into additional distributions, including Debian/Ubuntu, as well.

[richlander]

FYI: This query demonstrates targeting in dotnet/runtime. I found a naked use of net48 as well, but the following pattern seems to be the primary one.

<TargetFrameworks>$(NetCoreAppCurrent);$(NetFrameworkMinimum)</TargetFrameworks>

Macros would be very easy to collapse in terms of erasing the NETFX targets.

https://github.com/dotnet/runtime/search?q=NetFrameworkMinimum

[ericstj]

@mmitche - what I'm saying is not disagreeing with you at all. I'm saying that instead of removing .NETFramework from the packages, you just need to think of those packages as a different vertical. Packages are immutable and making them behave differently for source build just to facilitate codeflow and never ship them is a pretty big hack. Instead we should be thinking about those packages as produced by a vertical that's capable of producing their content. For source build we avoid depending on packages that need to have more content than is necessary for a vertical product build. In this way we'll actually naturally land in a place where we clean up the vertical source builds to produce just what we need.

[mmitche]

@ericstj Alright, sounds good. Thanks for the clarification.

[richlander]

I changed the issue title to make it more general and implementation-agnostic.

[tmds]

+1000 for eliminating the net4* targets and getting rid of 3GB of IL sources.

Note that previously we needed these targets to build assemblies used by mono-based OmniSharp.
OmniSharp has switched to .NET (Core), so we don't need them anymore.

Ideally, a source-build of .NET X.0 should require netX.0 as a target framework only.

And, the ref packages (for .NET Core, and ASP.NET Core) for that target framework should be buildable from the runtime/aspnetcore repo sources and not require separate IL dumps.

[MichaelSimons]

And, the ref packages (for .NET Core, and ASP.NET Core) for that target framework should be buildable from the runtime/aspnetcore repo sources and not require separate IL dumps.

@tmds - FYI, This improvement was already made in .NET 6.0. The ref packages are getting built as part of the product.

[MichaelSimons]

This work was completed in #3014.