Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The generated annotations have wrong line, file and test name properties #11

Open
Vampire opened this issue Jul 13, 2020 · 3 comments
Open

Comments

@Vampire
Copy link

Vampire commented Jul 13, 2020

If you might have a look at https://github.com/Vampire/svn2git/actions/runs/167642031
There are three tests failing
Your action generates three annotations for that
But the annotations are somewhat wrong
Line and File are wrong
test name is wrong
As the report generated by the AutoModality/action-xunit-viewer action seems to be fine, I guess it is not a problem of the XML files being wrong.
The XML reports are uploaded as artifacts, so you can have a look at them.

@jmisur
Copy link
Contributor

jmisur commented Jul 27, 2020

Hi @Vampire,
This action does not support specs (yet). As a lot of reporters are misusing that XML report structure, we always need to adapt for new usages. For example classname in this report is actually a file name, but in Java report it's a real class name.
Where do you see correct lines for failed test in the HTML report? All I see is raw logs.

@Vampire
Copy link
Author

Vampire commented Jul 27, 2020

This action does not support specs (yet).

The point is, I don't care about "specs".
Your action says the error was in a spec but this is not nearly correct.
The errors were in test/command-line.bats line 30,
test/command-line.bats line 40 and
test/copy-directories.bats line 27.

I have no idea why your action says the error is on line 1 of file test/libs/bats-core/contrib/rpm/bats.spec which is not even one of my files, just a file of the test harness I'm using.
Now that I think about it you maybe take the last dot-separated part which is bats and search for a file with any extension that has name bats which in this case is wrong of course. No idea how the 1 makes it there.

And you also prefix that bats to the test name which of course is wrong in that case.

Where do you see correct lines for failed test in the HTML report? All I see is raw logs.

Yeah, it's probably that raw logs that I have seen.
Actually in the raw logs shown by your action the line is contained too of course.

So summarized, actually this issue asks about supporint BATS generated JUnit XML reports. :-)

@jmisur
Copy link
Contributor

jmisur commented Aug 20, 2020

As this is very foreign to me, a PR would be appreciated, or at least some form of description about how bats works, some examples of test reports and from where can we extract specific pieces for annotations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants