BUG: SDK-3.1.300 dotnet publish .sln now chooses the older DLL in version conflict

[vidarw]

SDK 3.1.300 shows a different behavior when running dotnet publish on the solution file.

3.1.300 now selects the lowest versioned DLL when there is a version conflict between projects. This causes the projects depending on the newer DLLs to break, where the old bevhavior usually is expected to work fine due to a >= version dependency. We have seen this behavior both on direct package dependencies, but also on implicit dependencies from installed packages.

SDK 3.1.202 and earlier work as expected by selecting the most recent version of the two conflicting DLL versions to the publishing directory.

I have uploaded an example repository to demonstrate the behavior with a Modern App with a dependency on Newtonsoft.Json 12.0.x living in the same solution as a Legacy App with the same Newtonsoft.Json dependency targeting the older 10.0.x version.
Project can be found here: https://github.com/vidarw/dotnet-sdk-3.1.300-nuget-issue

docker run --rm -it -e /bin/sh -v C:\source\dotnet-sdk-3.1.300-nuget-issue:/app mcr.microsoft.com/dotnet/core/sdk:3.1.300
cd /app
dotnet publish -o ./3.1.300-docker

vs.

docker run --rm -it -e /bin/sh -v C:\source\dotnet-sdk-3.1.300-nuget-issue:/app mcr.microsoft.com/dotnet/core/sdk:3.1.202
cd /app
dotnet publish -o ./3.1.202-docker

and compare the version of the dlls. In my example the Newtonsoft.Json.dll will return 12.0.3 in 3.1.202 and 10.0.3 in 3.1.300 respectively.

The expected behavior is to get the most recent version in the published directory. If the change in behavior for some reason is intentional - a warning should be added to the console output.

[sunghwan2789]

I am having this issue, too.

[Erythnul]

@dsplaisted Is any progress being made on this issue? We've run into the same problem on our Azure Pipelines builds using dotnet publish.
When we publish a solution a wrong version of Newtonsoft is added to the zip, when we publish one of the csproj files directly it works as intended.

[billybraga]

@dsplaisted Is any progress being made on this issue? We've run into the same problem on our Azure Pipelines builds using dotnet publish.
When we publish a solution a wrong version of Newtonsoft is added to the zip, when we publish one of the csproj files directly it works as intended.

Try running the publish command with "-v d" (verbose detailed) and look for the line that says "copy xx/Newtonsoft.Json.dll to xx/publish..."

I experienced this problem with a nuget package that package the dll without defining it as a nuget dependency.

[blacksnake-rus]

@dsplaisted Is any progress being made on this issue? We've run into the same problem on our Azure Pipelines builds using dotnet publish.
When we publish a solution a wrong version of Newtonsoft is added to the zip, when we publish one of the csproj files directly it works as intended.

Try publishing without overriding the output folder. It works for me

[dsplaisted]

The problem is likely that you have multiple projects publishing to the same output folder, and its a matter of timing which one wins in copying its version of Newtonsoft.Json.dll. As @blacksnake-rus mentions, you can probably fix this by avoiding using the -o command line option.

If you can share a binlog of the publish operation where it's copying the wrong thing, it will be possible to verify whether this is the case.

[vidarw]

The problem is likely that you have multiple projects publishing to the same output folder, and its a matter of timing which one wins in copying its version of Newtonsoft.Json.dll. As @blacksnake-rus mentions, you can probably fix this by avoiding using the -o command line option.

If you can [https://aka.ms/binlog](share a binlog) of the publish operation where it's copying the wrong thing, it will be possible to verify whether this is the case.

The obvious workaround would of course be to just publish the single project and not a solution. Quitting the -o option is imo. way more destructive for existing build pipelines. I find this problem way too consistent over projects and computers to simply be a timing issue.

Here are binlogs, where 300 has the wrong older DLL in the output:

https://github.com/vidarw/dotnet-sdk-3.1.300-nuget-issue/blob/master/202-msbuild.binlog
https://github.com/vidarw/dotnet-sdk-3.1.300-nuget-issue/blob/master/300-msbuild.binlog

[dsplaisted]

I looked at the binlogs and they confirm that the issue is both projects are trying to write to the same folder. If you open up the 300 log in MSBuild Structured Log Viewer, you will see that it reports double writes for the different versions of the DLL to the same path:

I wasn't able to determine what is different about the 202 build, but I think it is probably just a matter of timing - ie the ModernApp project finishes publishing more quickly so that by the time the LegacyApp publishes, it already sees the newer version of the DLL and thus doesn't copy it. Or the LegacyApp finishes first and then the ModernApp overwrites the DLL.

[vidarw]

Come on... This is clearly a issue that should be handled by the SDK/compiler in some way or form?

I certainly understand that the problem is caused by stupid build practices - but it should at least produce a warning when you dotnet publish a solution file (which is after all a valid input) using the -o parameter or produce visible warnings to the user when DoubleWrites are detected?

[Erythnul]

@dsplaisted I think closing this is premature. Publishing a solution to one folder may not be the perfect way to work with your files, but if you won't support it it should be removed from the sdk.
My scenario involves transitive dependencies, not projects that directly reference different nuget packages versions, perhaps that will change things? Our solution doesn't even directly reference older nuget packages, it references nuget packages that reference nuget packages etc. Visual studio properly detects the right version with the transitive dependencies, I'm not even sure how an older version of the files is even an option considering higher versions should be resolved for literally every project and are resolved correctly when published individually. I'll get a binlog if you want.

[dsplaisted]

I agree we should fix something with the -o option and solution files. I thought we had other issues tracking that, but when I looked I found various related issues but not one specifically for -o and solution files, so I filed #15607.

@Erythnul If you can share more details along with a binlog or a repro of your issue I can take a look. Are you specifying an output path as a command-line parameter when building the solution? What's the incorrect behavior you're seeing?