Acquisition story for Sdks

[rainersigwald]

As of MSBuild 15.1 RC2, to be usable in a project file, an Sdk must be "installed" into a specific directory for MSBuild to find. This means that the possible Sdks are limited to what ships with MSBuild's install mechanism--today, generally Visual Studio or dotnet CLI.

A third party should be able to build an Sdk, and "non-core" Sdks should be acquirable via some mechanism. Ideally, the only difference between the "core" Sdks and an arbitrary Sdk is time-of-download.

This presents a number of challenges. See most of the discussion on #1392 and #1423.

[gulshan]

IMHO, the acquisition story should be-

• Nuget should be used, avoiding reinventing the wheel, but completely separated from dev nuget packages.
• SDKs can live in a separate specialized Nuget repository like Myget.
• SDKs should be cached and then always reused in subsequent uses- should be downloaded only once.

[natemcmaster]

I'm not saying NuGet should be the distribution mechanism...

But if that's what you go with, here's a feature that could be useful:
NuGet 3.5 created the concept of "package types" cref http://docs.nuget.org/ndocs/create-packages/creating-a-package#setting-a-package-type. This package type is an arbitrary identifier and can include a name and version. SDK authors could add this to nuspec

<metadata>
   <packageTypes>
      <packageType name="MsBuildSdk" version="15.1.0" />
   </packageTypes>
</metadata>

[wjk]

For what it's worth, I don't think SDKs should be distributed as NuGet packages; this just seems like it is bending the NuGet technology just a little too far from its standard uses for my taste. In addition, if a project's SDK is missing, and therefore the project cannot be properly evaluated, it creates a chicken-and-egg issue if the project itself (directly or indirectly) contains logic to locate and download the SDK (such as a PackageReference item).

On the other hand, I completely agree that we need a way to install a third-party SDK. In my opinion, it should be as simple as a folder in some central location where SDKs can be installed. As to how the folder containing the SDKs should be located, I have two ideas:

• If the SDK must be located before the project itself is evaluated, I would look first in $(MSBuildProjectDirectory), and then in each parent directory going up the tree, until a file (I'll call it .msbuild.sdks.json for purposes of this discussion) is found. This file is a simple JSON dictionary (or other simple, hand-editable format) that maps the names of SDKs to the folder where they are located, relative to the .msbuild.sdks.json file.
• If, on the other hand, the project can be at least partially evaluated before the SDK is loaded (enough to recursively parse explicitly included <Import /> tags in the project), I would simply use an item type called MSBuildSdkPath that lists every directory where SDKs can be located. MSBuild would then look in every directory so specified for a subdirectory with the name of the SDK. This algorithm allows SDKs to be specified in a central *.props file, and therefore be shared between all the project files in a repository.

I was thinking on how to use custom SDKs for my closed-source .NET Core development, and currently the only way I can integrate a custom SDK is to build all of the .NET Core SDK from source, and therefore creating a Sdks directory I can write files into. Although I am planning on using a custom-built .NET Core SDK and Runtime going forward for other reasons, the features being discussed in this thread would nonetheless make my life much easier. Thanks!

[jnm2]

Please do not require SDKs to be centrally installed as the only mechanism for locating them.

NuGet brings two huge benefits:

• Cake is designed to acquire NuGet tooling before MSBuild is even invoked, so it's the most natural way to obtain it. I'd have Cake pull https://www.nuget.org/packages/Microsoft.Build.Runtime and I'd also like to be able to pull SDKs the same way.
• You'd most importantly have the exact SDK version be source-controlled in the build script. When you queue a build on a build server, you should not have to log in and make sure files are copied to a central SDK installation folder for your particular SDK versions. The acquisition process should be completely automated by changing the version in the source-controlled build script. NuGet is a natural fit for this workflow.

[wjk]

@jnm2 I'm afraid I wasn't quite clear above. I am not advocating for the SDKs to be installed in a central location for the computer (or the build server). What I was asking for was that I could list the SDKs and their versions in a packages.config or similar, download them in a pre-build script (either using Cake or some other program) exactly as you are suggesting, and then for MSBuild to locate and use the downloaded assets. You and I are really discussing the same idea. All I was asking for above was a way for MSBuild to locate the downloaded SDKs in a way other than having to install them in a central location as I have to do today.

[wjk]

Also, my comment about NuGet being a bad idea for acquiring SDKs above is, after giving it more thought, incorrect. NuGet can and should be used to download files for an SDK. What I was contemplating when I wrote that was @natemcmaster's idea that we should use custom NuGet infrastructure to indicate that a package is an SDK. What I would do is keep the SDK files in a subdirectory of the package (similarly to what we do today with build directories to wire MSBuild targets into full-framework projects) and have that be the method of recognizing and extracting an SDK from a NuGet package. Convention over configuration and all that.

[rainersigwald]

I just pointed folks to this issue yesterday (sorry!), but I'm going to close it in favor of #1493, which has the current proposed design and answers some of the questions posed here. Specifically, what @wjk mentioned is covered by the (incomplete) idea of "lineups". Answering "what differentiates an Sdk from a normal NuGet package?" is covered by #1439.

[bbowman]

Mentioning here too, the WinObjC project https://github.com/Microsoft/WinObjC just switched to use NuGet as our SDK distribution mechanism so I'm in favor of something along those lines for a formal MSBuild third party SDK mechanism. @wjk @natemcmaster