Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong test run data is passed to the Livingdoc report. #2746

Open
smmaya opened this issue Apr 25, 2024 · 1 comment
Open

Wrong test run data is passed to the Livingdoc report. #2746

smmaya opened this issue Apr 25, 2024 · 1 comment
Labels

Comments

@smmaya
Copy link

smmaya commented Apr 25, 2024

SpecFlow Version

3.9.57

Which test runner are you using?

xUnit

Test Runner Version Number

2.5.8

.NET Implementation

.NET 6.0

Project Format of the SpecFlow project

Sdk-style project format

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

Visual Studio Test Explorer

SpecFlow Section in app.config or content of specflow.json

No response

Issue Description

There seems to be a issue with what is being passed to the html report.
Each time I run all tests (or a test, depending on hooks), a new TestExecution.html is created based on the TestExecution.json, nothing new here.
BUT, what I see in the actual report is not the last run, but the one before the last one.

I have no idea if this is a date/time issue as the json "GenerationTime": "0001-01-01T00:00:00" has this value???
The time on the report is 2 minutes late as the files were created at 15h14
2024-04-25 15_15_08-net8 0

2024-04-25 15_27_33-TestExecutionReport html and 6 more pages - Work - Microsoft​ Edge

This is what I call it with:
2024-04-25 15_34_52-USC Schedule Integration Staging SpecFlowTest csproj - Microsoft Visual Studio

Any help is much appreciated.

Steps to Reproduce

Say you set the livingdoc hooks to [AfterFeature] to be faster so you don't wait for all tests to run
And that we have those two tests:

Test A

  • Step 1
  • Step 2

Test B

  • Step 1
  • Step 2

Run Test A => if this is the first time you run it, no html report created yet, you will get an empty report
Run Test B => now in the report you will see the test run results for the Test A
2024-04-25 15_16_32-_C__git_SpiralSuite_ScheduleStaging_Automation_USC Schedule Integration Staging

Link to Repro Project

No response

@smmaya smmaya added the Bug label Apr 25, 2024
@smmaya
Copy link
Author

smmaya commented May 7, 2024

OK, 2 weeks passed with no response, but I managed to find the cause for my livingdoc html report being created BEFORE the TestExecution.json being updated.
In my LivingDocHooks class, I simply changed from process.WaitForExit(); to process.WaitForExitAsync(); and now I am getting the right data in the report.
You can close this bug, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant