Stop removing items that were included in the project file

[dsplaisted]

Instead, only apply the patterns that used to be called DefaultRemoves to the items that are implicitly included. If someone adds an include for a file to their project file, we should respect that.

Fixes #623
Fixes #588

The DefaultRemoves property was originally used so that when you had wildcard includes in your project file, you wouldn't have to specifically exclude the bin, obj, etc. folders. When we switched to implicit includes, I left it in with the idea that it would help you have a cleaner project file if you did want to explicitly see a wildcard include in your project, and it would help protect you from accidentally including items that you "shouldn't" from the output folders.

However, it's become apparent that it comes with two huge downsides:

• If you actually want items that would match the patterns in DefaultRemoves, you will include them in the project file, and that won't work, and you will have absolutely no idea why and there's no good way to debug / figure it out
• If you do manage to figure out why the items aren't being included, there's no good way to override the behavior to let you include a file. You have to turn off the DefaultRemoves entirely, which then means your implicit globs will suddenly pick up a bunch of files that they shouldn't, unless you jump through even more hoops in your project file.

This PR makes it so that if you put an item in your project file, it will be respected. The items that will be included implicitly remain the same as before.

This PR also takes the opportunity to remove the packages folder from the list of folders that the default includes don't apply to. It's unclear why this was originally added in the CLI. @davidfowl had this to say about it:

It was likely because of a mix of older packages folder (nuget v2) and xproj. Though that should only have been at the solution level...

Removing this exclusion will make default items in projects with code or other assets in a "packages" folder work as people will expect.

This change will need to be made together with a change to the Web SDK, as now it will be its responsibility to remove these items from its default includes. I will also submit a PR for that.

[dsplaisted]

I've filed the Web SDK PR: dotnet/websdk#94

[rainersigwald]

This seems fine but I'm a tiny bit worried about doing it in combination with the higher-pri change.

[dsplaisted]

Point taken, I'll back that part out and send it as a separate PR.

[dsplaisted]

@MattGertz for approval

Scenario

Include items in the project file that are in the output paths, or have an extension that we exclude from the implicit item includes. For example, a developer wants to include an item from the output folder in the NuGet package:

  <ItemGroup>
    <Content Include="$(OutputPath)\$(AssemblyName).runtimeconfig.json">
      <Pack>true</Pack>
      <PackagePath>lib\$(TargetFramework)</PackagePath>
    </Content>
  </ItemGroup>

Currently, this will be ignored entirely, with no good way for the developer to understand why.

Or, they want to copy a folder to the output directory:

<Content Include="TestProjects\**\*" CopyToOutputDirectory="PreserveNewest" />

Currently, any files in the folder with an extension ending in "proj" (ie .csproj, .proj, etc) would not be copied, though the rest of the files in the folder would be.

Bug

#623 and #588

Workarounds

See #623. This is ugly and most people won't even get as far as figuring out what's wrong

Risk

Low - Tests should cover any impacted scenarios

Performance Impact

Low - the change moves a file filter from happening after the main contents of the project file is evaluated to before then

Regression Analysis

n/a

[MattGertz]

@dsplaisted @srivatsn I don't understand the full customer scenario here -- can you please add clarifications on the customer impact to the template?

[dsplaisted]

@MattGertz I've updated the scenario as requested

[natidea]

Shiproom approved to merge