Add reference assembly for System.Private.CoreLib.dll

[akoeplinger]

We now compile against the reference assembly in all places where we were compiling against the mono/coreclr System.Private.CoreLib.dll implementation assembly before.

The new reference assembly consumes sources from the existing contracts to avoid checking in a generated version of SPC.dll (this would add ~20k lines of .cs which is mostly duplicated with System.Runtime.cs)

The reference .cs file of contracts where all types are actually living in SPC like System.Runtime.cs or System.Numerics.Vectors.cs were moved into src\libraries\System.Private.CoreLib\ref\contracts\ to keep them co-located (@ericstj suggested this in a discussion we had, but happy to change it back if most people don't like it).

Since a few contracts have only partially moved types to SPC we wrap contract types with #if !BUILDING_CORELIB_REFERENCE so we can hide them when compiling the SPC reference assembly. This needed a few GenAPI changes which are implemented here: dotnet/arcade#10003.

Note that this means that the types which live in CoreLib are moved to the end of the ref .cs file which causes a GitHub diff to show up, but it is actually a no-op.

Regenerating the ref .cs files works the same way as before, by running the GenerateReferenceAssemblySource target in the contract's src\ folder.

Fixes #67660

/cc @stephentoub @tannergooding @jkotas @vitek-karas @ericstj @ViktorHofer

[dotnet-issue-labeler]

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

[ghost]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

---

We now compile against the reference assembly in all places where we were compiling against the mono/coreclr System.Private.CoreLib.dll implementation assembly before.

The new reference assembly consumes sources from the existing contracts to avoid checking in a generated version of SPC.dll (this would add ~20k lines of .cs which is mostly duplicated with System.Runtime.cs)

The reference .cs file of contracts where all types are actually living in SPC like System.Runtime.cs or System.Numerics.Vectors.cs were moved into src\libraries\System.Private.CoreLib\ref\contracts\ to keep them co-located (@ericstj suggested this in a discussion we had, but happy to change it back if most people don't like it).

Since a few contracts have only partially moved types to SPC we wrap contract types with #if !HIDE_CONTRACT_TYPES so we can hide them when compiling the SPC reference assembly. This needed a few GenAPI changes which are implemented here: dotnet/arcade#10003.

Note that this means that the types which live in CoreLib are moved to the end of the ref .cs file which causes a GitHub diff to show up, but it is actually a no-op.

Fixes #67660

/cc @stephentoub @tannergooding @jkotas @vitek-karas @ericstj @ViktorHofer

Author:	akoeplinger
Assignees:	-
Labels:	area-Infrastructure-libraries, new-api-needs-documentation
Milestone:	-

[jkotas]

moved into src\libraries\System.Private.CoreLib\ref\contracts\ to keep them co-located (@ericstj suggested this in a discussion we had, but happy to change it back if most people don't like it).

Thoughts:

• It makes the contract and the tests non-collocated. I think I would have a mild preference for keeping the contracts in the same location as they are today.
• If we were to move them, do we need the contracts subdirectory - why not just src\libraries\System.Private.CoreLib\ref?

[stephentoub]

I think I would have a mild preference for keeping the contracts in the same location as they are today.

+1

[ViktorHofer]

As an alternative to manually creating the reference assembly and maintaining it, could we add a switch to Roslyn to not emit private fields into the reference assembly produced via RefOut, in order to get a stable MVID? We would only want to use such a switch for unshipping "private" reference assemblies like System.Private.CoreLib anyway.

Asking as I could imagine that would need to do this for other private assemblies like System.Private.Uri or System.Private.Xml as well in case they would become arch/platform specific in the future.

cc @jaredpar

[jkotas]

Disable compiling against SPC ref assembly for NativeAOT

Are we still going to get a validation that the public surface matches with this?

The extra methods in NativeAOT CoreLib should be in several types. It may be better to add MatchingRefApiCompatBaseline.txt-like suppression for these types instead of maintaining two parallel systems.

[akoeplinger]

@jkotas I'm looking at it, this was mostly to get something building so I could get test coverage, I'll continue looking at how best to handle NativeAOT

[ericstj]

GenAPI changes and usage look ok to me. I’d like folks working in this codebase to provide their sign off.

[tannergooding]

I didn't review in depth, but I don't see any changes that impact anything. It's just reordering where various entries are, so provided GenAPI changes are checked in it won't impact anyone needing to make subsequent changes to the ref assemblies.

[jonathanpeppers]

@akoeplinger is there any plan to backport this to .NET 6?

We're hitting this issue in MAUI + .NET 7 when you change the the TargetFramework to net6.0-*. So you end up with a .NET 7 linker and .NET 6 runtime packs in that case.

[vitek-karas]

@sbomer - another instance of using 7.0 linker for 6.0 apps - completely different problem though.
I would argue this is even more of a case for using 6.0 linker for 6.0 apps regardless of the SDK... anyway 😉

[steveisok]

@sbomer - another instance of using 7.0 linker for 6.0 apps - completely different problem though. I would argue this is even more of a case for using 6.0 linker for 6.0 apps regardless of the SDK... anyway 😉

@vitek-karas How realistic is making a change to use the 6.0 linker for 6.0 apps? We're either going to have to do something like that or backport this change. Using the 6.0 linker seems like the least impactful from my perspective. Thoughts?

[vitek-karas]

@sbomer has been looking into that, but it's relatively complicated and thus risky.

[sbomer]

As @vitek-karas said - it will be difficult and risky to make that change in time for the release. We would need to start publishing a package for the linker and add SDK logic to download the right version, probably using the runtime pack mechanism that we use for ILCompiler. There are a lot of details to get right, with changes across multiple repos. Often for this kind of change we only catch errors when dependencies flow into upstream repos.

I'll work on this in case we decide to go that route, but we should also consider:

• Do we have an idea for the risk and feasibility of backporting this change instead?
• What would be the impact of shipping in the current state (requiring a 6.0 SDK to work around the issue)?

[jonathanpeppers]

@sbomer so the result when building a Release-mode Android app, we end up with an assembly per architecture:

Where if you use a global.json and pin to the 6.0.400 SDK, you get 1 of each of these. We "dedup" them based on the MVID during the build.

I will get an app size comparison, but the duplicates cause another issue we need to solve first: dotnet/android#7302

I suspect the app might be near 4x larger?

[steveisok]

• Do we have an idea for the risk and feasibility of backporting this change instead?

I don't believe it's super risky backporting this change. @akoeplinger are there any concerns that you know of / have?

[marek-safar]

I think we ran out of time for any solution beyond the backport. Can we get that rolling ASAP?