[master] Update dependencies from dotnet/runtime #14574

dotnet-maestro · 2020-11-17T13:13:48Z

This pull request updates the following dependencies

From https://github.com/dotnet/runtime

Subscription: ea219f43-0754-4d2f-576e-08d76e1d56cb
Build: 20201216.14
Date Produced: 12/17/2020 8:00 AM
Commit: be27e294a12514638bb5337c7fe5cc72d93a443a
Branch: refs/heads/master

Updates:
- Microsoft.NETCore.Platforms: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- Microsoft.NETCore.DotNetHostResolver: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- Microsoft.NET.HostModel: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- Microsoft.Extensions.DependencyModel: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- Microsoft.NETCore.App.Runtime.win-x64: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- Microsoft.NETCore.App.Host.win-x64: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- Microsoft.NETCore.App.Ref: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- System.CodeDom: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- System.Reflection.MetadataLoadContext: from 5.0.0 to 6.0.0-alpha.1.20616.14
- System.Resources.Extensions: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- System.Security.Cryptography.ProtectedData: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- System.Text.Encoding.CodePages: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14
- VS.Redist.Common.NetCore.SharedFramework.x64.6.0: from 6.0.0-alpha.1.20560.10 to 6.0.0-alpha.1.20616.14

…1117.3 Microsoft.NETCore.Platforms , System.CodeDom , Microsoft.NET.HostModel , Microsoft.Extensions.DependencyModel , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.App.Host.win-x64 , Microsoft.NETCore.DotNetHostResolver , Microsoft.NETCore.App.Ref , System.Resources.Extensions , System.Security.Cryptography.ProtectedData , System.Text.Encoding.CodePages , VS.Redist.Common.NetCore.SharedFramework.x64.6.0 From Version 6.0.0-alpha.1.20560.10 -> To Version 6.0.0-alpha.1.20567.3

…1117.4 Microsoft.NETCore.Platforms , System.CodeDom , Microsoft.NET.HostModel , Microsoft.Extensions.DependencyModel , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.App.Host.win-x64 , Microsoft.NETCore.DotNetHostResolver , Microsoft.NETCore.App.Ref , System.Resources.Extensions , System.Security.Cryptography.ProtectedData , System.Text.Encoding.CodePages , VS.Redist.Common.NetCore.SharedFramework.x64.6.0 From Version 6.0.0-alpha.1.20560.10 -> To Version 6.0.0-alpha.1.20567.4

ViktorHofer · 2020-11-17T16:17:55Z

cc @jkoritzinsky

…1117.16 Microsoft.NETCore.Platforms , System.CodeDom , Microsoft.NET.HostModel , Microsoft.Extensions.DependencyModel , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.App.Host.win-x64 , Microsoft.NETCore.DotNetHostResolver , Microsoft.NETCore.App.Ref , System.Resources.Extensions , System.Security.Cryptography.ProtectedData , System.Text.Encoding.CodePages , VS.Redist.Common.NetCore.SharedFramework.x64.6.0 From Version 6.0.0-alpha.1.20560.10 -> To Version 6.0.0-alpha.1.20567.16

ViktorHofer · 2020-11-18T10:50:52Z

error NU1102: (NETCORE_ENGINEERING_TELEMETRY=Restore) Unable to find package Microsoft.NETCore.DotNetAppHost with version (>= 6.0.0-alpha.1.20567.16)

…1118.2 Microsoft.NETCore.Platforms , System.CodeDom , Microsoft.NET.HostModel , Microsoft.Extensions.DependencyModel , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.App.Host.win-x64 , Microsoft.NETCore.DotNetHostResolver , Microsoft.NETCore.App.Ref , System.Resources.Extensions , System.Security.Cryptography.ProtectedData , System.Text.Encoding.CodePages , VS.Redist.Common.NetCore.SharedFramework.x64.6.0 From Version 6.0.0-alpha.1.20560.10 -> To Version 6.0.0-alpha.1.20568.2

ViktorHofer · 2020-11-18T13:14:11Z

@jkoritzinsky any idea why the DotNetAppHost package is missing?

mmitche · 2020-11-18T15:14:02Z

@ViktorHofer @jkoritzinsky It was removed here dotnet/runtime@a505c46

/cc @Anipik

mmitche · 2020-11-18T15:15:37Z

At least, I'm pretty sure. It's in this build: https://dev.azure.com/dnceng/internal/_build/results?buildId=884890&view=artifacts&type=publishedArtifacts but not this one https://dev.azure.com/dnceng/internal/_build/results?buildId=885334&view=artifacts&type=publishedArtifacts

ViktorHofer · 2020-11-18T15:19:00Z

I'm relatively certain this is because of Jeremy's shared framework sdk change:

mmitche · 2020-11-18T15:22:10Z

I'm relatively certain this is because of Jeremy's shared framework sdk change:

Ahh yeah that commit is in between but wasn't built.

jkoritzinsky · 2020-11-18T16:34:50Z

That was unintentional. I thought I kept all of the legacy host packages. Apparently accidentally dropped one. I’ll add it back.

ViktorHofer · 2020-11-18T17:43:19Z

When you say legacy package, do you mean that dotnet/sdk should use another one ideally?

jkoritzinsky · 2020-11-18T17:49:16Z

By legacy I mean a package that used to be used in product construction before .NET Core 3.0. If the SDK still has a usage for the packages, then they’re the right ones to use.

…1118.7 Microsoft.NETCore.Platforms , Microsoft.NET.HostModel , Microsoft.Extensions.DependencyModel , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.DotNetHostResolver , Microsoft.NETCore.App.Host.win-x64 , Microsoft.NETCore.App.Ref , System.CodeDom , System.Resources.Extensions , System.Security.Cryptography.ProtectedData , System.Text.Encoding.CodePages , VS.Redist.Common.NetCore.SharedFramework.x64.6.0 From Version 6.0.0-alpha.1.20560.10 -> To Version 6.0.0-alpha.1.20568.7

ViktorHofer · 2020-11-19T00:19:49Z

Do we need to update the System.Text.Json version being used in the sdk?

src\Resolvers\Microsoft.DotNet.MSBuildSdkResolver\Microsoft.DotNet.MSBuildSdkResolver.csproj(0,0): error NU1605: (NETCORE_ENGINEERING_TELEMETRY=Restore) Detected package downgrade: System.Text.Json from 6.0.0-alpha.1.20568.7 to 4.7.2. Reference the package directly from the project to select a different version. 
 Microsoft.DotNet.MSBuildSdkResolver -> Microsoft.DotNet.Cli.Utils -> Microsoft.Extensions.DependencyModel 6.0.0-alpha.1.20568.7 -> System.Text.Json (>= 6.0.0-alpha.1.20568.7) 
 Microsoft.DotNet.MSBuildSdkResolver -> System.Text.Json (>= 4.7.2)

jkoritzinsky · 2020-11-19T01:04:17Z

cc: @dsplaisted

…1119.2 Microsoft.NETCore.Platforms , Microsoft.NET.HostModel , Microsoft.Extensions.DependencyModel , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.DotNetHostResolver , Microsoft.NETCore.App.Host.win-x64 , Microsoft.NETCore.App.Ref , System.CodeDom , System.Resources.Extensions , System.Security.Cryptography.ProtectedData , System.Text.Encoding.CodePages , VS.Redist.Common.NetCore.SharedFramework.x64.6.0 From Version 6.0.0-alpha.1.20560.10 -> To Version 6.0.0-alpha.1.20569.2

dsplaisted · 2020-11-19T20:05:42Z

@ericstj @joperezr Is Microsoft.Extensions.DependencyModel supposed to be depending on the latest version of System.Text.Json?

We've had to be careful about how we handle these dependencies to ensure that they can load correctly for MSBuild tasks running on Full Framework.

ericstj · 2020-11-19T20:27:46Z

In 5.0.0 Microsoft.Extensions.DependencyModel depended on System.Text.Json 5.0.0:
https://www.nuget.org/packages/Microsoft.Extensions.DependencyModel/

In 6.0 it depends on the 6.0.0 version.

joperezr · 2020-11-20T00:01:08Z

Yeah, as @ericstj explained by default in dotnet/runtime repo all dependencies between live-built packages are to the live-built version (latest version) after a re-baseline of the repo, which we do after every big release. That means that now that we have switched dotnet/runtime repo to target 6.0, we have re-baselined all live-built packages to use 6.0.0 as well.

dsplaisted · 2020-11-20T01:57:04Z

The restore failure is happening when the TargetFramework for the project is net6.0. This insertion is bringing in a version of the Microsoft.Extensions.DependencyModel which has dependencies on the System.Text.Encodings.Web and System.Text.Json packages for net6.0, where the previous version of the package had no package dependencies in for net6.0. This is because the old version of the package had an empty framework dependency group for net5.0:

<group targetFramework="net5.0" />

The new package doesn't have this group, so NuGet ends up selecting the .NETCoreApp2.1 package dependencies.

This seems like an unintentional change and it is breaking us. @ericstj / @joperezr Can you investigate this change or find the right person to do so? Thanks!

ViktorHofer · 2020-11-24T15:20:08Z

@ericstj @joperezr can you please take a look and check if the added dependencies are right?

ericstj · 2020-11-24T18:16:43Z

Microsoft.Extensions.DependencyModel which has dependencies on the System.Text.Encodings.Web and System.Text.Json packages for net6.0

Indeed I can see this here: https://www.nuget.org/packages/Microsoft.Extensions.DependencyModel/

This is controlled by the inboxOn calculations by the packaging system to drop the package references where these are inbox. @Anipik we need to add inbox here (and likely others) for 6.0.0: https://github.com/dotnet/runtime/blob/2e8c4c78d5dc628afd94053a008ab203e648a2b9/src/libraries/pkg/baseline/packageIndex.json#L6629
Opened dotnet/runtime#45163 to track this.

@dsplaisted why would you be referencing System.Text.Json package at all on net5.0 or net6.0. It's inbox, just use the one in the framework, it's what's going to win from conflict resolution anyway.

sdk/src/Resolvers/Microsoft.DotNet.MSBuildSdkResolver/Microsoft.DotNet.MSBuildSdkResolver.csproj

Line 84 in d257737

    
           <PackageReference Include="System.Text.Json" Version="4.7.2" Condition="'$(UseSystemTextJson)'=='True'"/>

You could do that to unblock yourself here.

dsplaisted · 2020-11-25T00:30:49Z

@ericstj Good point, I think b3c9c3f should fix this

mmitche · 2020-12-16T16:43:49Z

Aside - is there a way to stop the bot picking up new runtime versions?

In the original Maestro, we had a label that could be applied to the PR - Maestro-Stop-Updating. I'm not sure if the current version has the ability or not.

No, but the subscription can be disabled in the interim.

danmoseley · 2020-12-16T17:29:56Z

Can someone who deals with File IO take a look at this?

I will look as soon as I'm out of meetings today, if nobody does ahead of time. It would be surprising if something so basic broke; I don't see any relevant recent changes in a quick look.

dsplaisted · 2020-12-16T17:48:59Z

Can someone who deals with File IO take a look at this?

I will look as soon as I'm out of meetings today, if nobody does ahead of time. It would be surprising if something so basic broke; I don't see any relevant recent changes in a quick look.

@danmosemsft I created a simple repro:

static void Main(string[] args)
{
    foreach (var subdir in new DirectoryInfo(args[0]).GetDirectories())
    {
        Console.WriteLine(subdir.FullName);
    }
}

I get different output between .NET 5 and .NET 6 (Shared Framework 6.0.0-alpha.1.20614.10). .NET 5:

/mnt/c/git/cli
/mnt/c/git/CsWinRT
/mnt/c/git/demo
/mnt/c/git/docs
...

.NET 6

cli
CsWinRT
demo
docs
...

danmoseley · 2020-12-16T19:24:21Z

Looking at that now.

eerhardt · 2020-12-16T19:43:08Z

I have a suspicion that the subdir.FullName issue is a crossgen problem. I rebuilt System.IO.FileSystem locally using the same SHA as that build, and I use my local .dll (which isn't crossgen'd) and the problem doesn't repro.

jeffschwMSFT · 2020-12-16T20:04:27Z

@mangod9

danmoseley · 2020-12-16T20:07:38Z

As @eerhardt pointed out - here's the repro:

dan@LAPTOP-P6UJDVTA:~/1$ cat nuget.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <!--To inherit the global NuGet package sources remove the <clear/> line below -->
    <clear />
    <add key="nuget" value="https://api.nuget.org/v3/index.json" />
<add key="dotnet6" value="https://dnceng.pkgs.visualstudio.com/public/_packaging/dotnet6/nuget/v3/index.json" />
  </packageSources>
</configuration>
dan@LAPTOP-P6UJDVTA:~/1$ cat 1.csproj
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
   <RuntimeIdentifier>linux-x64</RuntimeIdentifier>
    <RuntimeFrameworkVersion>6.0.0-alpha.1.20614.10</RuntimeFrameworkVersion>
  </PropertyGroup>

  <ItemGroup>
    <KnownFrameworkReference Remove="Microsoft.AspNetCore.App" />
  </ItemGroup>

</Project>
dan@LAPTOP-P6UJDVTA:~/1$ cat Program.cs
using System;
using System.IO;

class Program
{
    static void Main(string[] args)
    {
        foreach (var subdir in new DirectoryInfo("/home/dan/1").GetDirectories())
        {
            Console.WriteLine(subdir.FullName);
        }
    }
}
dan@LAPTOP-P6UJDVTA:~/1$ export COMPlus_ReadyToRun=1
dan@LAPTOP-P6UJDVTA:~/1$ dotnet run
obj
bin
dan@LAPTOP-P6UJDVTA:~/1$ export COMPlus_ReadyToRun=0
dan@LAPTOP-P6UJDVTA:~/1$ dotnet run
/home/dan/1/obj
/home/dan/1/bin
dan@LAPTOP-P6UJDVTA:~/1$

danmoseley · 2020-12-16T20:09:20Z

Should we be running unit tests against crossgenned bits? As far as I can see, this would break the unit tests.

ViktorHofer · 2020-12-16T20:10:21Z

Should we be running unit tests against crossgenned bits? As far as I can see, this would break the unit tests.

For libraries we don't today as we don't crossgen in the libraries build but in the installer. Kind of tracked via dotnet/runtime#487

danmoseley · 2020-12-16T20:17:40Z

For libraries we don't today as we don't crossgen in the libraries build but in the installer. Kind of tracked via dotnet/runtime#487

Right, it's come up several times. This would essentially be a move to catch crossgen, not libraries bugs - so I think @mangod9 would be the best person to decide what the priority should be. In this case, we were fortunate that SDK tests found it -- as far as I know the bits we ship to customers never have the libraries teams' tests run on them at all.

ViktorHofer · 2020-12-16T20:21:47Z

the bits we ship to customers never have the libraries teams' tests run on them at all.

Correct, we don't run automated tests for official builds.

This would essentially be a move to catch crossgen, not libraries bugs - so I think @mangod9 would be the best person to decide what the priority should be.

We might want to do it sooner or later anyway as it reduces complexity and duplication of our test bootstrapping logic. @jkoritzinsky and I were looking at using the bundle that currently lives in the installer subset of dotnet/runtime for that.

danmoseley · 2020-12-16T20:30:00Z

My vote would be sooner rather than later as upstack tests probably only verify 10% of what our unit tests do (since it's not their job) - it feels like we are relying on previews to discover such bugs, and we want previews to be more predictable.

ViktorHofer · 2020-12-16T20:31:16Z

Absolutely agree. Will make sure that this item is prioritized accordingly.

mangod9 · 2020-12-16T20:34:11Z

+@trylek, who will soon be working on compiling libraries using cg2. Would be good to ensure that we can run library tests on compiled bits. Thx.

mangod9 · 2020-12-16T20:39:21Z

As for the subdir.FullName issue, does it repro on windows too?

dsplaisted · 2020-12-16T20:43:31Z

As for the subdir.FullName issue, does it repro on windows too?

No, it appears only to repro on Mac and Linux

mangod9 · 2020-12-17T07:31:01Z

I looked at this to narrow down when things broke, and it appears possibly related to this change: dotnet/runtime#45625. Looks to have been rolled back recently since it was causing other failures as part of this PR: dotnet/runtime#46172

this is possibly the cause:

This PR blocks crossgenning UnmangedCallersOnly methods in crossgen1 because the custom attribute parsing code is not currently included in crossgen1 and I didn't want to fight to get that hooked up since we're looking at deprecating/removing crossgen1 in the .NET 6 time frame anyway.

@jkoritzinsky possibly this should wait till we get cg2 rolled out broadly ?

mangod9 · 2020-12-17T07:32:53Z

@jkotas fyi

…1216.14 Microsoft.NETCore.Platforms , Microsoft.NETCore.DotNetHostResolver , Microsoft.NET.HostModel , Microsoft.Extensions.DependencyModel , Microsoft.NETCore.App.Runtime.win-x64 , Microsoft.NETCore.App.Host.win-x64 , Microsoft.NETCore.App.Ref , System.CodeDom , System.Reflection.MetadataLoadContext , System.Resources.Extensions , System.Security.Cryptography.ProtectedData , System.Text.Encoding.CodePages , VS.Redist.Common.NetCore.SharedFramework.x64.6.0 From Version 6.0.0-alpha.1.20560.10 -> To Version 6.0.0-alpha.1.20616.14

jkotas · 2020-12-17T14:09:24Z

@jkoritzinsky possibly this should wait till we get cg2 rolled out broadly ?

@jkotas fyi

This is not common. There are number of other less common cases that cg2 handles better than cg1. One more or less won't make a huge difference. I think this is fine.

jeffschwMSFT · 2020-12-17T15:38:15Z

@mangod9 @jkoritzinsky what is our plan to get this addressed? Are we going to take the latest runtime which contains the revert?

akoeplinger · 2020-12-17T15:40:30Z

Are we going to take the latest runtime which contains the revert?

The latest runtime bump from about three hours ago already contains the fix: dotnet/runtime#46176

mangod9 · 2020-12-17T15:56:24Z

Yeah build 6.0.0-alpha.1.20616.14 doesnt repro the issue anymore.

dsplaisted · 2020-12-17T17:16:33Z

It passed CI and merged! 🎉🎉🎉

Thanks folks.

danmoseley · 2020-12-17T17:26:56Z

@marek-safar @jeffschwMSFT...tag! Which of you takes the next one.

jeffschwMSFT · 2020-12-17T17:28:23Z

I can be it. Given the holidays, it will likely be early next year. :)

dotnet-maestro bot added 2 commits November 17, 2020 13:13

jkoritzinsky mentioned this pull request Nov 17, 2020

Update archives package to fix zip/tar naming. dotnet/runtime#44827

Merged

ViktorHofer approved these changes Nov 18, 2020

View reviewed changes

captainsafia mentioned this pull request Nov 19, 2020

[master] Update dependencies from dotnet/efcore dotnet/runtime dotnet/aspnetcore#27845

Merged

ericstj mentioned this pull request Nov 24, 2020

Inbox package trimming not happening on net6.0 dotnet/runtime#45163

Closed

Remove unnecessary reference to System.Text.Json

b3c9c3f

dsplaisted mentioned this pull request Dec 17, 2020

[master] Update dependencies from dotnet/aspnetcore #14909

Merged

dotnet-maestro bot and others added 2 commits December 17, 2020 12:39

Remove ILLink warnings that were fixed by dotnet/runtime#45984

3d51a6a

dotnet-maestro bot merged commit 21a2f4e into master Dec 17, 2020

dotnet-maestro bot deleted the darc-master-4a612234-0bdd-45db-8d7f-6c5bed075213 branch December 17, 2020 17:01

[master] Update dependencies from dotnet/runtime #14574

[master] Update dependencies from dotnet/runtime #14574

Conversation

dotnet-maestro bot commented Nov 17, 2020 • edited Loading

From https://github.com/dotnet/runtime

ViktorHofer commented Nov 17, 2020

ViktorHofer commented Nov 18, 2020 • edited Loading

ViktorHofer commented Nov 18, 2020

mmitche commented Nov 18, 2020

mmitche commented Nov 18, 2020

ViktorHofer commented Nov 18, 2020 • edited Loading

mmitche commented Nov 18, 2020

jkoritzinsky commented Nov 18, 2020

ViktorHofer commented Nov 18, 2020 • edited Loading

jkoritzinsky commented Nov 18, 2020

ViktorHofer commented Nov 19, 2020

jkoritzinsky commented Nov 19, 2020

dsplaisted commented Nov 19, 2020

ericstj commented Nov 19, 2020

joperezr commented Nov 20, 2020

dsplaisted commented Nov 20, 2020

ViktorHofer commented Nov 24, 2020

ericstj commented Nov 24, 2020 • edited Loading

dsplaisted commented Nov 25, 2020

mmitche commented Dec 16, 2020

danmoseley commented Dec 16, 2020

dsplaisted commented Dec 16, 2020

danmoseley commented Dec 16, 2020

eerhardt commented Dec 16, 2020

jeffschwMSFT commented Dec 16, 2020

danmoseley commented Dec 16, 2020

danmoseley commented Dec 16, 2020

ViktorHofer commented Dec 16, 2020 • edited Loading

danmoseley commented Dec 16, 2020

ViktorHofer commented Dec 16, 2020 • edited Loading

danmoseley commented Dec 16, 2020

ViktorHofer commented Dec 16, 2020

mangod9 commented Dec 16, 2020

mangod9 commented Dec 16, 2020 • edited Loading

dsplaisted commented Dec 16, 2020

mangod9 commented Dec 17, 2020

mangod9 commented Dec 17, 2020

jkotas commented Dec 17, 2020

jeffschwMSFT commented Dec 17, 2020

akoeplinger commented Dec 17, 2020 • edited Loading

mangod9 commented Dec 17, 2020

dsplaisted commented Dec 17, 2020

danmoseley commented Dec 17, 2020

jeffschwMSFT commented Dec 17, 2020

dotnet-maestro bot commented Nov 17, 2020 •

edited

Loading

ViktorHofer commented Nov 18, 2020 •

edited

Loading

ViktorHofer commented Nov 18, 2020 •

edited

Loading

ViktorHofer commented Nov 18, 2020 •

edited

Loading

ericstj commented Nov 24, 2020 •

edited

Loading

ViktorHofer commented Dec 16, 2020 •

edited

Loading

ViktorHofer commented Dec 16, 2020 •

edited

Loading

mangod9 commented Dec 16, 2020 •

edited

Loading

akoeplinger commented Dec 17, 2020 •

edited

Loading