Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Two flaky tests that need investigated #38268

Open
MiYanni opened this issue Jan 25, 2024 · 1 comment
Open

Two flaky tests that need investigated #38268

MiYanni opened this issue Jan 25, 2024 · 1 comment
Assignees
Labels
Test Debt untriaged Request triage from a team member

Comments

@MiYanni
Copy link
Member

MiYanni commented Jan 25, 2024

Summary

While being domestic cat, I noticed a couple of tests that are inconsistent/flaky. These tests will cause build failures, but a rerun of the PR CI pipeline will often result in success. These test failures can be seen when searching the Helix test log for [FAIL]. The Helix test log link can be accessed when checking the build failure by following the Failure log: link:

image

Tests

  • In the Windows_NT Build_Release job, the test GivenDotnetRunIsInterrupted.ItTerminatesTheChildWhenKilled shows this failure:
    Microsoft.DotNet.Cli.Run.Tests.GivenDotnetRunIsInterrupted.ItTerminatesTheChildWhenKilled [FAIL]
      Expected boolean to be true, but found False.
      Stack Trace:
           at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
           at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
           at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
           at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
           at FluentAssertions.Primitives.BooleanAssertions`1.BeTrue(String because, Object[] becauseArgs)
        /_/test/dotnet-run.Tests/GivenDotnetRunIsInterrupted.cs(152,0): at Microsoft.DotNet.Cli.Run.Tests.GivenDotnetRunIsInterrupted.ItTerminatesTheChildWhenKilled()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
      Output:
        > C:\h\w\AD9409B3\p\d\dotnet.exe run
        
        Exit Code: -1
  • In the Ubuntu_22_04 Build_Release job, the test GivenThatWeHaveAPackageReferenceWithAliases.CanBuildProjectWithMultiplePackageReferencesWithAliases shows this failure:
�[31;1m    Microsoft.NET.Build.Tests.GivenThatWeHaveAPackageReferenceWithAliases.CanBuildProjectWithMultiplePackageReferencesWithAliases [FAIL]
�[m�[37m      Expected command to pass but it did not.
�[m�[37m      File Name: /datadisks/disk1/work/A9B509B9/p/d/dotnet
�[m�[37m      Arguments: msbuild /t:Build /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj /restore
�[m�[37m      Exit Code: 1
�[m�[37m      StdOut:
�[m�[37m      MSBuild version 17.10.0-preview-24073-02+d51ae5297 for .NET
�[m�[37m        Determining projects to restore...
�[m�[37m        Restored /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj (in 1.83 sec).
�[m�[37m      /datadisks/disk1/work/A9B509B9/p/d/sdk/9.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(313,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj]
�[m�[37m      /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.cs(9,13): error CS0118: 'First.A' is a namespace but is used like a type [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj]
�[m�[37m      StdErr:
�[m�[37m      
�[m�[37m      
�[m�[30;1m      Stack Trace:
�[m�[37m           at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
�[m�[37m           at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
�[m�[37m           at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
�[m�[37m           at FluentAssertions.Execution.AssertionScope.FailWith(String message)
�[m�[37m        /_/test/Microsoft.NET.TestFramework/Assertions/CommandResultAssertions.cs(29,0): at Microsoft.NET.TestFramework.Assertions.CommandResultAssertions.Pass()
�[m�[37m        /_/test/Microsoft.NET.Build.Tests/GivenThatWeHaveAPackageReferenceWithAliases.cs(91,0): at Microsoft.NET.Build.Tests.GivenThatWeHaveAPackageReferenceWithAliases.CanBuildProjectWithMultiplePackageReferencesWithAliases()
�[m�[37m           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
�[m�[37m           at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
�[m�[30;1m      Output:
�[m�[37m        > /datadisks/disk1/work/A9B509B9/p/d/dotnet msbuild /t:Pack /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/A.csproj /restore
�[m�[37m        MSBuild version 17.10.0-preview-24073-02+d51ae5297 for .NET
�[m�[37m          Determining projects to restore...
�[m�[37m          Restored /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/A.csproj (in 246 ms).
�[m�[37m        /datadisks/disk1/work/A9B509B9/p/d/sdk/9.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(313,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/A.csproj]
�[m�[37m          A -> /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/bin/Debug/net9.0/A.dll
�[m�[37m          The package A.1.0.0 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important.
�[m�[37m          Successfully created package '/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---E8E10EA8_2/A/bin/Debug/A.1.0.0.nupkg'.
�[m�[37m        > /datadisks/disk1/work/A9B509B9/p/d/dotnet msbuild /t:Pack /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/B.csproj /restore
�[m�[37m        MSBuild version 17.10.0-preview-24073-02+d51ae5297 for .NET
�[m�[37m          Determining projects to restore...
�[m�[37m          Restored /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/B.csproj (in 232 ms).
�[m�[37m        /datadisks/disk1/work/A9B509B9/p/d/sdk/9.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(313,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/B.csproj]
�[m�[37m          B -> /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/bin/Debug/net9.0/B.dll
�[m�[37m          The package B.1.0.0 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important.
�[m�[37m          Successfully created package '/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---BAEEF93F_2/B/bin/Debug/B.1.0.0.nupkg'.
�[m�[37m        > /datadisks/disk1/work/A9B509B9/p/d/dotnet msbuild /t:Build /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj /restore
�[m�[37m        MSBuild version 17.10.0-preview-24073-02+d51ae5297 for .NET
�[m�[37m          Determining projects to restore...
�[m�[37m          Restored /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj (in 1.83 sec).
�[m�[37m        /datadisks/disk1/work/A9B509B9/p/d/sdk/9.0.100-ci/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(313,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj]
�[m�[37m        /datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.cs(9,13): error CS0118: 'First.A' is a namespace but is used like a type [/datadisks/disk1/work/A9B509B9/w/BA120A0E/e/testExecutionDirectory/CanBuildProje---32D5EEC0_2/Project/Project.csproj]
�[m�[37m        Exit Code: 1
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged Request triage from a team member label Jan 25, 2024
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Test Debt untriaged Request triage from a team member
Projects
None yet
Development

No branches or pull requests

2 participants