Producing an MSBuild task package

[livarcocc]

From @natemcmaster on April 14, 2017 18:41

Can we get better support for projects designed to produce MSBuild tasks?

Pain points

• packing. You can get a task assembly and files into the package, but requires deep understanding of how to alter the default layout of the package in csproj.
  • TargetFrameworks. A package that only has task assemblies/targets is compiled for net46 and netcoreapp1.0, but the package itself can be compatible with any project, regardless of its TFM.
• task references. When loading a task assembly, MSBuild does not use the NuGet cache to find dependencies. This means we have to package assemblies to sit side-by-side on disk so task loading works.
• tasks with native dependencies. Never been able to make this work.

Workarounds
Task assembly projects end up looking like this

  <PropertyGroup>
    <!--
      The netstandard1.0 and net45 TFMs don't actually compile tasks.
      Only here so the generated nuspec includes the TFM so it appears as "compatible" on NuGet.org
      and in VS NuGet GUI.

      netstandard1.6 => loaded on dotnet.exe
      net46 => loaded on MSBuild.exe
    -->
    <TargetFrameworks>netstandard1.6;net46;netstandard1.0;net45</TargetFrameworks>
    <!-- Be quiet NuGet. I don't want assemblies in lib/ and yes I'm sure this is right. -->
    <NoPackageAnalysis>true</NoPackageAnalysis>
    <!-- forces SDK to copy dependencies into build output to make packing easier -->
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
    <BuildOutputTargetFolder>tools</BuildOutputTargetFolder>
  </PropertyGroup>

  <ItemGroup>
    <!-- ensure nuspec doesn't contain any package references as we bundle their assemblies in our package -->
    <PackageReference Update="@(PackageReference)" PrivateAssets="All" />
  </ItemGroup>

  <!-- 
    Copy dependencies into the tools/$(TargetFramework)/ package folders.
    MSBuild does not resolve task runtime dependencies from PackageReference
  -->
  <Target Name="PackTaskDependencies" BeforeTargets="GenerateNuspec">
    <!--
    The include needs to happen after output has been copied to build output folder
    but before NuGet generates a nuspec.
    -->
    <ItemGroup>
      <_PackageFiles Include="bin\$(Configuration)\*\Newtonsoft.Json.dll;bin\$(Configuration)\*\NUglify.dll">
        <PackagePath>tools\%(RecursiveDir)</PackagePath>
        <Visible>false</Visible>
        <BuildAction>Content</BuildAction>
      </_PackageFiles>
    </ItemGroup>
  </Target>

Some MSBuild task projects in the wild:

madskristensen/BundlerMinifier
aspnet/BuildTools
natemcmaster/Yarn.MSBuild

Copied from original issue: dotnet/sdk#1125

[livarcocc]

cc @emgarten @rohit21agrawal

[livarcocc]

From @rohit21agrawal on April 14, 2017 21:27

we already added some extension points that should make this easier : NuGet/NuGet.Client#1255

We can't make a generic Task package type since all packages would differ widely. Do you have any concrete suggestions on what you would like to see that would make it easier for you to create task packages?

[livarcocc]

From @natemcmaster on April 14, 2017 21:59

In a world of unlimited resources, I'd love to see this:

<Project Sdk="Microsoft.Build.Task">
</Project>

Microsoft.NET.Sdk is geared towards runtime packages, not MSBuild task packages. Packages designed to carry MSBuild tasks are fundamentally different from packages that only carry runtime bits.

[livarcocc]

From @natemcmaster on April 14, 2017 22:18

@rohit21agrawal if you're looking for short-term fixes to the pack task...

One of the problems today is that package authors have to choose between nuspec and csproj=>nuspec generation. Keeping csproj and nuspec aligned is difficult, esp. for users new to MSbuild. Wrangling MSBuild to make csproj=>nuspec generation work right is difficult, even for advanced MSBuild users. I had to read NuGet's source code to figure some of this out.

Anything you can do to make nuspec + csproj easier would help with this ask.

One idea:
Make all msbuild properties available in nuspec. Currently you have to manually marshal these via <NuspecProperties>

<package>
  <metadata>
    <id>${MSBuild.PackageId}</id>
    <version>${MSBuild.PackageVersion}</version>
  </metadata>
</package>

[livarcocc]

From @rohit21agrawal on April 16, 2017 19:19

@natemcmaster instead of making csproj + nuspec easier, i'd like to understand more about what we can do to make the need for nuspec go away.
what are some of the things that you can't achieve via csproj currently and for which you absolutely do need the nuspec?

[livarcocc]

From @natemcmaster on April 17, 2017 18:14

@rohit21agrawal

• Packing the output of multiple projects. Example: Microsoft.EntityFrameworkCore.Tools.nuspec
• "Fat packages". i.e. I want a nupkg with the output of "dotnet publish". Example: NuGetPackageVerifier.nuspec
• <summary>. cref Packing csproj with long description issues unresolvable warning about missing summary text #4587.
• Explicitly defining the <dependencies> section.
  • Example: I want my MSBuild task package to say <dependencies><group targetFramework="Any"></group></dependencies>, but nuspec generation always uses <TargetFrameworks> which is set to the target frameworks for which the task assembly compiles. In this case, <TargetFrameworks> does not accurately represent which frameworks the package is compatible with.

[livarcocc]

From @conniey on April 18, 2017 20:17

I think this issue overlaps with: dotnet/msbuild#1756

[livarcocc]

Given that this conversation is happening around pack, I am moving this issue over to NuGet. Please, re-activate or file new issues for specific asks for the SDK to support this.

[clairernovotny]

Here's another example of this with nasty workarounds:
https://github.com/paulcbetts/refit/blob/04db09ffb67ba500ee2bc3370bc68b81437b46a0/Refit/Refit.csproj#L28-L46

We need to call dotnet publish on the task package we want to include in our library package (we do build-time code generation). It leads to build-time weirdness and is far too hard to figure out. One of the issues is ensuring that the dependent publish is only called once, so that's why it's in a tfm condition. Was all trial-and-error to figure out.

[matkoch]

@onovotny I'm trying to do something similar. Could you please explain how you ended up with build\MSBuild<TargetFramework> being the destination folder? Is this documented somewhere? Unfortunately, the way you've created the package didn't work for me at first try. Have to check later :/

[clairernovotny]

There is nothing special about the directory name or location; it comes from detecting core msbuild vs full here:

https://github.com/paulcbetts/refit/blob/master/Refit/targets/refit.targets#L12-L13

We also needed a custom msbuild task base type (ContextAwareTask) to handle assembly loading in the correct directory:
https://github.com/paulcbetts/refit/tree/master/InterfaceStubGenerator.BuildTasks

Then in our main library we call publish on each TFM to get all the files we need in the publish output directory:
https://github.com/paulcbetts/refit/blob/master/Refit/Refit.csproj#L39-L50

And finally, include them in the package in the right location.

Hope this helps!

[clairernovotny]

The point of all of this though is that's far too complicated and painful to create MSBuild tasks that work on both Full and Core MSBuild.

[matkoch]

@onovotny Thanks for the explanation. It seems that my target file is not getting included. Are there any obstacles you know about that? First tried with build\net45 and build\netstandard1.4 (as in refit), then with just build\ ... but still nothing. For now I'm not using any task assembly; just a <Error Text="..." />.

[clairernovotny]

Your props/targets file has to match the package id for it to be automatically included. For Refit, that'd be Refit.targets. Do yours match?

[jnm2]

Docs: https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package#including-msbuild-props-and-targets-in-a-package

[matkoch]

🤦‍♂️

Thanks to both of you, @onovotny and @jnm2 .

[jnm2]

I remember tripping over the same magic early this year when I wrote my first MSBuild target and tried to package it.

[matkoch]

This should be part of NuGet package analysis actually. /cc @emgarten