Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report portal is not handling TestNG test method invocationCount correct. #109

Open
the-octopus opened this issue May 11, 2020 · 2 comments
Assignees
Milestone

Comments

@the-octopus
Copy link

Scenario: When run a test method more than one iteration by configuring the invocationCount property of the test method.
Correct behavior: the test method should appear in report portal as one level that has nested iterations.

Current behavior: each iteration from the same method appears as a standalone test method level.

for more details please check this [URL comment](#96 (comment))

@HardNorth HardNorth added this to the 5.1 milestone May 18, 2020
@HardNorth HardNorth modified the milestones: 5.1, 6.0 May 20, 2020
@HardNorth
Copy link
Member

HardNorth commented May 21, 2020

That change requires server updates.

Classic retry looks like this: skipped->skipped->passed/failed.
It's counted as a single test in our statistics. It is marked as failed only if the last one in the retry chain is failed, when there is no attempts left.

InvocationCount looks like this: passed/failed->passed/failed->passed/failed
It should be counted as 3 tests. And if one of them failed the whole batch should be marked as failed.

Currently RP server supports only retries, not invocations. So invocations will stay as separate items until we implement such feature on the server side.

@HardNorth
Copy link
Member

@evjlobanova Please include such feature in the future scope as we agreed.

@HardNorth HardNorth assigned evjlobanova and unassigned HardNorth May 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants