Restore fails if netstandard1.x package reference is present #3044
Comments
|
Referencing current HEAD in case it moves between now and repro investigation: RssBandit/RssBandit@628bd5c |
|
I was confused on the email thread, this isn't conflict resolution as it's a downgrade during restore. |
|
This should be fixed when we do https://github.com/dotnet/cli/issues/10105. Previously I thought that trimming previous NuGet packages would merely be a perf optimization and conflict resolution would handle selecting the right assets. However this demonstrates a case where there is a functional failure if we don't trim previous packages an old bug can resurface with .NET Core 1.x package references when a RID is specified. @onovotny I believe adding the following PackageReference to RssBandit.csproj should work around the issue: <PackageReference Include="Microsoft.NETCore.App" Version="2.2.0" ExcludeAssets="all" PrivateAssets="all" /> |
|
Do I want version 2.2.0 or 3.0.0? |
|
2.2.0. We will likely not have this package on NuGet.org for 3.0.0. |
|
@dsplaisted The obvious, simpler, solution to this is to implement CleanFull, which we discussed in a separate topic last week as a meta-step I used when upgrading from .NET Core 2.1 to 2.2: #3046 (comment)
Trimming the NuGet object graph is the "clever" solution, but probably also too clever by half and adds complexity to an already complex sub-system. I've created a ticket for this suggestion: #3067 |
|
Nevermind. I see that the issue here is that the packagereference is in the csproj. I misunderstood @nguerrera link to the git commit hash - it's the commit that fixes the issue as well as what caused the issue in the first place. I thought @onovotny was saying that he upgraded to 2.0.4 and still got errors due to cached packages still existing in the packages directory. |
|
A simple repro for this is simply a .NET Core console app with a RuntimeIdentifier and a reference to the log4net package: <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RuntimeIdentifier>win-x86</RuntimeIdentifier>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="log4net" Version="2.0.8" />
</ItemGroup>
</Project>The problem is that the log4net package depends on System.Runtime.Serialization.Formatters version 4.3.0, but on the 4.0.x or 4.1.x versions of a lot of other contracts. With a RID specified, RID-specific dependencies get injected which cause package downgrades such as the following:
To fix this, we may need something like NuGet/Home#7344. We thought an "empty" Microsoft.NETCore.App package could help with this, but because it doesn't have a runtime.json with RID-specific references to the 2.x equivalents of "runtime packs", it didn't fix the issue in the same way that the explicit reference to the 2.2.0 package suggested as a workaround here did. |
|
Warning: The value of this comment might be very low. I'd be careful about over-engineering this. Looking at .NET Core 3.0, I'm already starting to feel like library maintainers will be screwed once more. A few weeks ago I saw Brice Lambson tweeting about how awesome .netstandard2.1 was now that he could write once target everywhere. But since the direction is for .netstandard to be about "API surface area", many of us are upset at the complexity this will introduce in our projects: We see it as going back to the dark ages once more. For example, I can't "just" take the To be honest, I don't know what the right solution is and I don't envy the intrepid soul who is tasked with solving the problem, since some of it is idiosyncratic (how binding redirection only works on strong named assemblies and how strong name assemblies came about prior to the advent of the web semver2.0.0 standard and even Java's OSGI standards). For me, I am filing this case away as interesting and something to noodle on. |
|
@jzabroski The complexity in this issue is coming from trying to make things simpler and better ("less over-engineered") in .NET Core 3.0, while still supporting packages that were created back in .NET Core 1.x when .NET Core was represented by a complex graph of NuGet packages. |
|
I may be wrong, but I think the "trivially hard" case is the one I run into regularly:
In other words, while you may solve it for the framework (which is clever), how will you solve it for the more common scenarios? If you think that's not a big problem, consider the most common TaskRunner is Json.Net's If my comments are wrong here, I apologize - but I have spent A LOT of time thinking about this lately. |
|
@jzabroski Exact version matching (and binding redirects to get around it) are required on .NET Framework, but not on .NET Core. So the issues you refer to (which were indeed painful) are mostly solved for .NET Core. |
|
It's also possible to hit this if you end up with dependencies on contracts from different "bands" of .NET Core 1.x. log4net happens to have these dependencies directly, but it could happen with other combinations of packages. These references also cause the issue: <PackageReference Include="System.IO.FileSystem" Version="4.0.1" />
<PackageReference Include="System.Reflection" Version="4.3.0" />
|
|
A better workaround than adding a reference to Microsoft.NETCore.App is probably to reference Microsoft.NETCore.Targets: <PackageReference Include="Microsoft.NETCore.Targets" Version="2.1.0" ExcludeAssets="all" PrivateAssets="all" /> |
|
Is there an ETA on a "transparent" / silent / built-in fix for this? Still happens with preview 6. |
|
+1 on this, still getting customer reports of this issue. I thought we agreed to fix this by bringing in the empty packages? Now I see you've linked this to a NuGet feature to dummy out packages. Would love that feature, don't get me wrong, but we asked for the same thing in 2.0 and didn't get it (and thus we have conflict resolution). What's your plan to solve this problem in 3.0. This is a regression from 2.1. |
It is worth noting that the experience regresses when retargeting to 3.0, but does not break without project changes. We hold this distinction in mind strongly. We do reserve the right to require additional steps upon retargeting. Just like the runtime does not guarantee that apps will work 100% of the time on roll forward or retargeting.
That doesn't turn out to work. We have to know when to pull in the empty package, which we tried to do based on whether there are any package references, but it turns out that we would have to wait for restore to know if there are any because the only package references could be transitive through a project reference. We went through about 3 different iterations of workarounds and none of them worked. At some point, we have to cut our losses. The workaround can be applied manually. If we pulled in the empty package all the time, we'd break offline build of FDD with only framework references, which is a key value prop. Or we'd have to go back to having offline nuget sources/fallback folders as part of the SDK, which was a huge burden and also something we're very intent on keeping out from 3.0 onwards.
We are actively avoiding this strategy. We deliberately worked closely with NuGet for FrameworkReference instead of working around limitations. Conflict resolution added quite a maintenance burden. The NuGet fix for this was on the list of requests from the beginning of the FrameworkReference discussions with NuGet going back about a year. NuGet has avoided doing this particular work item in 3.0 on the assumption that this does not impact mainline scenarios. Prior to that, there was a mistaken assumption that the NuGet fix would only cause perf penalty of leaving more work to conflict resolution. There was some data gathering about which packages in all of nuget.org would cause a conflict like this due to mismatched 1.x packages in the graph, and I believe it was deemed tolerable for 3.0 given that data and given that a workaround exists. If there are data points that contradict this, then we need the details on that. And we should share that data with the NuGet team. For the most part, it is believed that there is a better workaround of upgrading components to get 1.x era packages out of your graph. In the specific case of System.Device.Gpio, it is netstandard2.0, but requires transitive 1.x dependencies. Can we fix that? I would also push on log4net to have a clean netstandard2.0 TFM without dependencies on 1.x packages. This adds value irrespective of the fix here. The performance of the build will be better to name one. |
|
I hit this today upgrading ML.NET to 3.0. I was able to distill it down to this simple repro. Restoring this project: <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.11.3" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.8.0" />
</ItemGroup>
</Project>with the 3.0.100-preview8-013437 SDK results in: If I literally change any line in the .csproj, it works:
I think many people are going to run into this issue and we should strongly consider getting some mitigation in place. |
|
We need NuGet/Home#7344 to fix this. As I said, there has been some data gathering suggesting that could be deferred based on this being not common. We included that feature request from the beginning of the FrameworkReference work. We tried some workarounds in the SDK and none of them work. cc @rrelyea The recent uptick in reports on this seems to go against the data gathering that suggested this wouldn't be common. |
|
Hit this today on 3.0-rc1 |
|
@nguerrera It's hard as an outsider to help fix the problems without knowledge of the user stories and test cases. Do you have failing test cases and a general suite of tests with an Xunit tag around this feature that need to pass to consider this "feature complete"? I think, given the number of false-starts here and misjudge of complexity, test-driven development and getting the tests to fail first and then write the code might work better. |
|
@jzabroski This issue is hard as an insider too. It's simply hard. If this issue is important to you, the best way to help get it fully resolved is to engage on NuGet/Home#7344. That is the correct fix. |
|
Respectfully, that seems down in the weeds. To read NuGet/Home#7344 I need to then also read about https://github.com/dotnet/cli/issues/10006, which proposes breaking up .NET meta-packages into "targeting packs" and "run-time packs". But that issue doesn't clearly define what these two words mean. I had to jump from https://github.com/dotnet/cli/issues/10006 to https://github.com/dotnet/cli/issues/10086 to https://github.com/dotnet/cli/issues/10007 to figure out what a "runtime pack" was, and I'm left understanding:
User stories would be: In this sense, a "targeting pack" is basically a .NET Core 3 plug-in which takes a nuget package reference and installs an object graph (.deps.json) and its concrete assets into the lib directory. That to me seems like the easiest fix here... |
|
tl;dr: I think there is duplicate/cross-effort here between what you are doing and the CLR team designing roll forward functionality. Simplify things by inverting your dependencies and treating targeting packs as plug-ins. I hate to over-simplify and think I solved all your problems in 30 minutes*, but this is my gut reaction to all this design effort. * 30 minutes disclaimerI've actually spent more hours than I can count thinking about how to manage dependencies, going back 15 years ago and OSGi early discussions, talking to the designer of cfEngine about dependency management, the architect of NetBeans IDE about dependency management, reading through 6LowPan meeting notes on self-describing formats for configuring optimal, low-energy consumption services, etc. |
|
The design for runtime packs and targeting packs is covered by dotnet/designs#50. It is already implemented. NuGet/Home#7344 is about a very specific piece allowing a way to ask nuget to exclude packages that are overridden by platform. It is not as simple as you suggest because there is a deliberate requirement for "targeting packs" to not be so coupled to nuget. |
|
@nguerrera Thanks, I read through your dotnet Design Doc, but toward the bottom of the pull request review, I see your caveat lector that says:
Given .NET Core 3.0 is live, can you please update this PR and fix the "out-of-date" issues? IDK as a .NET developer how I am expected to understand how the .NET world works when there is no explanation of it in a single place. |
….11 (#3044) - Microsoft.DotNet.Cli.Runtime - 5.0.100-alpha1.19502.11
|
dotnet/extensions just hit a similar problem today. NETStandard.Library 1.6.0 is brought in transitively which brings in 1.x era packages, which bring in the 1.x era private native shims. Due to name changes we renamed these, but still understand the old names (without lib prefix). It'd be nice to have an implicit NETStandard.Library 2.0.3 reference, similar to the Microsoft.NETCore.Platforms reference. |
|
Im having a similar issue today. A simple repro is to create a .NET 6.0 project and add a reference to latest Debug and publishing goes as expected for me (on win 11 with VS2022), but as soon as I try to build the auto-scaffolded
VS
Please help! |
|
@cdavernas Have you tried applying the workaround: <PackageReference Include="Microsoft.NETCore.Targets" Version="2.1.0" ExcludeAssets="all" PrivateAssets="all" /> |
|
Having the same issue with .NET 7.0 and Serilog: https://stackoverflow.com/q/74453395/1768303 This solves the problem. Is there a better solution? Should <PackageReference Include="Microsoft.NETCore.Platforms" Version="7.0.0" />
<PackageReference Include="Microsoft.NETCore.Targets" Version="5.0.0" PrivateAssets="All" />I am not even sure how exactly this fix works. Can some please shed some light on it? |
Well, Microsoft.NETCore.Targets has been almost empty since 5.0.0, hence why it was removed later. You can use Nuget.org Package Explorer to see its contents here: https://nuget.info/packages/Microsoft.NETCore.Targets/5.0.0 It has a lib folder, which contains a netstandard1.0 folder. It used to then contain a runtimes.json file in that folder that stated the runtimes the library targeting netstandard1.0 could run against.
I can't speak to why Microsoft stopped publishing Microsoft.NETCore.Targets, or why your particular solution works here. What Else Can I Say?There is no one, good explanation of how to manage dependencies in .NET. The whole system is just at least as bad as any other major ecosystem out there.
|
|
Thank you @jzabroski, very helpful. On a side note, microsoft.netcore.targets.3.1.0.nupkg is pretty much the same as v5.0.0. |
SDK version: 3.0.100-preview4-010963
repro:
EDIT by @dsplaisted: This is the workaround for this bug:
The text was updated successfully, but these errors were encountered: