Build managed projects and tests with csproj's #2259

mellinoe · 2017-05-02T22:23:14Z

Replace all test projects and test assets with csproj files.
- Update some test code to use only properties that the 2.0 CLI understands.
- Pass through additional properties in the test code in order to restore and build projects for the correct RID and version of Microsoft.NETCore.App (Fixes https://github.com/dotnet/core-setup/issues/2127)
Replace src/managed/*project.json with csproj files.
- Update GetBuildArgsByFrameworks so it understands how to read and parse csproj files instead of project.json files.
Update dir.proj's to correctly restore and build the csproj files (using DotnetSdkToolCommand instead of DotnetToolCommand).

mellinoe · 2017-05-03T00:32:08Z

I pushed a couple of commits which should fix some Linux problems. I'm going to need some help understanding the remaining test issues -- they don't make a lot of sense to me.

src/managed/Microsoft.Extensions.DependencyModel/Microsoft.Extensions.DependencyModel.csproj

+    <Reference Include="Microsoft.CSharp" />
+  </ItemGroup>
+
+  <ItemGroup Condition=" '$(TargetFramework)' == 'netstandard1.3' ">


src/managed/dir.proj

@@ -24,7 +24,7 @@
      <Output ItemName="buildCmdArgs" TaskParameter="BuildArgs" />
    </GetBuildArgsByFrameworks>

-    <Exec Command="$(DotnetToolCommand) build $(Args) %(buildCmdArgs.Identity)" 
+    <Exec Command="$(DotnetSdkToolCommand) build $(Args) %(buildCmdArgs.Identity)" 


src/pkg/packaging/dir.proj

@@ -160,18 +160,19 @@

  <Target Name="GenerateNugetPackages" DependsOnTargets="InitPackage">
    <ItemGroup>
-      <PackageProjects Include="$(ProjectDir)src\managed\Microsoft.DotNet.PlatformAbstractions\project.json" />
-      <PackageProjects Include="$(ProjectDir)src\managed\Microsoft.Extensions.DependencyModel\project.json" />
+      <PackageProjects Condition="'$(OS)' == 'Windows'" Include="$(ProjectDir)src\managed\Microsoft.DotNet.PlatformAbstractions\Microsoft.DotNet.PlatformAbstractions.csproj" />


src/test/Assets/TestProjects/LightupLib/LightupLib.csproj

+    <RuntimeFrameworkVersion>$(MNAVersion)</RuntimeFrameworkVersion>
+  </PropertyGroup>
+
+  <ItemGroup Condition=" '$(TargetFramework)' == 'netcoreapp2.0' ">


src/test/TestUtils/RepoDirectoriesProvider.cs


            string baseArtifactsFolder = artifacts ?? Path.Combine(_repoRoot, "Bin");

            _targetRID = Environment.GetEnvironmentVariable("TEST_TARGETRID");
            _buildRID = Environment.GetEnvironmentVariable("BUILDRID");
+            _mnaVersion = Environment.GetEnvironmentVariable("MNA_VERSION");


src/test/TestUtils/TestProjectFixture.cs

            if ( ! _testProject.IsRestored())
            {
-                RestoreProject(fallbackSources);
+                string extraMSBuildProperties = $"/p:RuntimeIdentifiers={rid} /p:RuntimeIdentifier={rid}";


src/test/dir.proj

-          EnvironmentVariables="NUGET_PACKAGES=$(PackagesDir);TEST_ARTIFACTS=$(SystemPathTestsOutputDir);TEST_TARGETRID=$(TestTargetRid);BUILDRID=$(TargetRid)"
+    <Exec Command="$(DotnetTestToolCommand) test $(TestArgs) --logger &quot;trx;LogFileName=$(TestResultsXml)&quot;"
+          WorkingDirectory="$(TestWorkingDirectory)"
+          EnvironmentVariables="NUGET_PACKAGES=$(PackagesDir);TEST_ARTIFACTS=$(SystemPathTestsOutputDir);TEST_TARGETRID=$(TestTargetRid);BUILDRID=$(TargetRid);MNA_VERSION=$(ProductVersion)"


mellinoe · 2017-05-03T19:45:58Z

With the two latest commits, I have a clean build on my Ubuntu machine. Hopefully we will see some green in CI now.

src/test/HostActivationTests/GivenThatICareAboutStandaloneAppActivation.cs

@@ -50,6 +50,12 @@ public void Running_Build_Output_Standalone_EXE_with_DepsJson_and_RuntimeConfig_

            var appExe = fixture.TestProject.AppExe;

+            // TODO: Use FS.Chmod when build utility project is converted to csproj.


src/test/Microsoft.Extensions.DependencyModel.Tests/PackageResolverTest.cs

@@ -17,7 +17,8 @@ public class PackageResolverTest
    {
        private static string PackagesPath = Path.Combine("package", "directory", "location");

-        [Fact]
+        // This test does not work on .NET Core 2.0.


tools-local/tasks/GetBuildableFrameworks.cs

@@ -31,21 +32,20 @@ public override bool Execute()
            List<string> args = new List<string>();
            foreach (var projectJsonPath in ProjectJsonPaths)
            {
-                using (TextReader projectFileReader = File.OpenText(projectJsonPath.ItemSpec))
+                string dir = Path.GetDirectoryName(projectJsonPath.ItemSpec);


mellinoe · 2017-05-03T20:05:21Z

I think we will need to modify the netci.groovy script, because we are not producing xunit test results anymore. Rather, we are producing "TRX" logs.

EDIT: I've made that change, but the CI jobs will still all show failures because of this.

weshaggard · 2017-05-03T21:00:01Z

I"m good with merging this if you have confirmed it works and follow-up after your netci changes are in to ensure everything is green.

mellinoe · 2017-05-03T21:18:20Z

The only thing I need to fix is the x86 run. I think I understand why it failed, though.

… code.

src/managed/Microsoft.Extensions.DependencyModel/Microsoft.Extensions.DependencyModel.csproj

+    <VersionPrefix>2.0.0</VersionPrefix>
+    <TargetFrameworks>net451;netstandard1.3;netstandard1.6</TargetFrameworks>
+    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
+    <AssemblyName>Microsoft.Extensions.DependencyModel</AssemblyName>


src/managed/Microsoft.Extensions.DependencyModel/Microsoft.Extensions.DependencyModel.csproj

+    <PackageId>Microsoft.Extensions.DependencyModel</PackageId>
+    <RepositoryType>git</RepositoryType>
+    <RepositoryUrl>git://github.com/dotnet/core-setup</RepositoryUrl>
+    <PackageTargetFallback Condition=" '$(TargetFramework)' == 'netstandard1.3' Or '$(TargetFramework)' == 'netstandard1.6' ">$(PackageTargetFallback);portable-net45+wp80+win8+wpa81+dnxcore50</PackageTargetFallback>


src/managed/Microsoft.Extensions.DependencyModel/Microsoft.Extensions.DependencyModel.csproj

+  </ItemGroup>
+
+  <ItemGroup Condition=" '$(TargetFramework)' == 'net451' ">
+    <Reference Include="System" />


eerhardt · 2017-05-03T22:17:59Z

  ProjectJsonPaths="@(PackageProjects)">

Should rename ProjectJsonPaths

Refers to: src/managed/dir.proj:23 in bd5f1ce. [](commit_id = bd5f1ce, deletion_comment = False)

src/pkg/packaging/dir.proj

    </ItemGroup>

    <PropertyGroup>
-      <BuildBasePathArg>--build-base-path $(IntermediateOutputForPackaging)</BuildBasePathArg>
+      <BuildBasePathArg></BuildBasePathArg>


src/test/Assets/TestProjects/LightupClient/LightupClient.csproj

+
+  <PropertyGroup>
+    <TargetFramework>netcoreapp2.0</TargetFramework>
+    <AssemblyName>LightupClient</AssemblyName>


src/test/TestUtils/TestProjectFixture.cs

                restoreArgs.Add(fallbackSource);
            }
            restoreArgs.Add("--disable-parallel");

+            restoreArgs.Add($"/p:MNAVersion={_repoDirectoriesProvider.MicrosoftNETCoreAppVersion}");


src/test/TestUtils/TestProjectFixture.cs

            if ( ! _testProject.IsRestored())
            {
-                RestoreProject(fallbackSources);
+                string extraMSBuildProperties = $"/p:TestTargetRid={rid}";


src/test/Assets/TestProjects/PortableTestApp/PortableTestApp.csproj

+    <AssemblyName>PortableTestApp</AssemblyName>
+    <PackageId>PortableTestApp</PackageId>
+    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
+    <PackageTargetFallback>$(PackageTargetFallback);dotnet5.4;portable-net451+win8</PackageTargetFallback>


src/test/Assets/TestProjects/StandaloneApp/StandaloneApp.csproj

+    <AssemblyName>StandaloneApp</AssemblyName>
+    <OutputType>Exe</OutputType>
+    <PackageId>StandaloneApp</PackageId>
+    <RuntimeIdentifiers>$(TestTargetRid)</RuntimeIdentifiers>


src/test/Assets/TestProjects/StandaloneTestApp/StandaloneTestApp.csproj

+    <PackageId>StandaloneTestApp</PackageId>
+    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
+    <PackageTargetFallback>$(PackageTargetFallback);dotnet5.4;portable-net451+win8</PackageTargetFallback>
+    <RuntimeIdentifiers>$(TestTargetRid)</RuntimeIdentifiers>


src/test/bootstrap/.cliversion

@@ -1 +1 @@
-1.0.0-preview3-003886
+2.0.0-preview1-005899


src/test/build/shared-build-targets-utils/Utils/DotNetCli.cs

@@ -96,7 +96,7 @@ private void ComputeSharedFxPaths()
            GreatestVersionSharedFxPath = sharedFxVersionDirectories
                .OrderByDescending(p => p.ToLower())
                .First();
-


src/test/HostActivationTests/GivenThatICareAboutStandaloneAppActivation.cs

+            // TODO: Use FS.Chmod when build utility project is converted to csproj.
+            if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
+            {
+                Command.Create("chmod", "+x", appExe).Execute().EnsureSuccessful();


src/test/HostActivationTests/HostActivationTests.csproj

+    <TargetFramework>netcoreapp2.0</TargetFramework>
+    <AssemblyName>HostActivationTests</AssemblyName>
+    <PackageId>HostActivationTests</PackageId>
+    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>


...Microsoft.Extensions.DependencyModel.Tests/Microsoft.Extensions.DependencyModel.Tests.csproj

+    <ProjectReference Include="..\..\managed\Microsoft.DotNet.PlatformAbstractions\Microsoft.DotNet.PlatformAbstractions.csproj" />
+    <ProjectReference Include="..\..\managed\Microsoft.Extensions.DependencyModel\Microsoft.Extensions.DependencyModel.csproj" />
+
+    <!-- TODO: Should this dependency be removed? -->


src/test/TestUtils/TestProject.cs

@@ -50,8 +50,8 @@ public class TestProject
            _sharedLibraryExtension = sharedLibraryExtension;
            _sharedLibraryPrefix = sharedLibraryPrefix;
            _projectName = Path.GetFileName(_projectDirectory);
-            _projectJson = Path.Combine(_projectDirectory, "project.json");
-            _projectLockJson = Path.Combine(_projectDirectory, "project.lock.json");
+            _projectJson = Path.Combine(_projectDirectory, $"{_projectName}.csproj");


src/test/TestUtils/TestUtils.csproj

+
+  <PropertyGroup>
+    <TargetFramework>netcoreapp1.0</TargetFramework>
+    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>


netci.groovy

@@ -84,7 +84,7 @@ platformList.each { platform ->
    Utilities.standardJobSetup(newJob, project, isPR, "*/${branch}")

    if (!(os == 'Windows_NT' && architecture == 'arm')) {
-        Utilities.addXUnitDotNETResults(newJob, '**/*-testResults.xml')
+        Utilities.addMSTestResults(newJob, '**/*-testResults.xml')


tools-local/tasks/GetBuildableFrameworks.cs

                    {
+                        Console.WriteLine("OSGroup: " + OSGroup);


tools-local/tasks/GetBuildableFrameworks.cs

@@ -31,21 +32,20 @@ public override bool Execute()
            List<string> args = new List<string>();
            foreach (var projectJsonPath in ProjectJsonPaths)
            {
-                using (TextReader projectFileReader = File.OpenText(projectJsonPath.ItemSpec))
+                string dir = Path.GetDirectoryName(projectJsonPath.ItemSpec);


tools-local/tasks/GetBuildableFrameworks.cs

                        {
-                            args.Add($"--framework {framework.Name} {dir}");
+                            args.Add($"--framework {framework} {projectJsonPath}");


mellinoe · 2017-05-04T00:42:39Z

@dotnet-bot Test Windows_NT x64 Release Build

mellinoe · 2017-05-04T01:03:39Z

I think this is at an okay state to go in now. All of the CI "failures" are because of the mismatched test results files, and shouldn't show up after the update to netci.groovy gets consumed.

dnfclas added the cla-already-signed label May 2, 2017

weshaggard reviewed May 3, 2017

View reviewed changes

mellinoe commented May 3, 2017

View reviewed changes

weshaggard reviewed May 3, 2017

View reviewed changes

mellinoe added 14 commits May 3, 2017 14:49

Build/publish "managed" projects with 2.0 SDK

c2ed89b

Fix packaging of "managed" projects.

badc12e

Start porting tests to 2.0 CLI

e2d9d8d

Continue working on tests and add some logging

1849321

Continue porting tests

ddeeda4

Make all test projects build and run with the MSBuild-based toolset.

798ddda

Pass the version for Microsoft.NETCore.App through test execution

4c087f0

Remove all extra PackageReferences from the migrated test assets

189bf25

Fix build problems outside of Windows.

d097f88

Remove explicit "RuntimeFrameworkVersion" property in test projects.

b1331b8

Make sure tests target netcoreapp2.0

a0a0283

Fix remaining test issues.

082b977

Don't set DOTNET_SKIP_FIRST_TIME_EXPERIENCE on Windows.

02e2907

Pass TestTargetRid instead of RuntimeIdentifier(s) in restore in test…

30a8dbc

… code.