Ignore test_failures if the overall status is 'succeeded' #77
Conversation
| ].flatten.compact.map do |summary| | ||
| result = Result.new(summary.message, parse_location(summary.document_location_in_creating_workspace)) | ||
| Result.new(format_test_failure(result, summary.producing_target, summary.test_case_name), | ||
| if action.action_result.status != 'succeeded' then |
There was a problem hiding this comment.
I'm not sure if this fully fixes the issue though. For example, If I have 2 tests, one which passed on 2nd iteration and another which failed all iterations, then this would post 2 test failures to the PR instead of just 1. Could lead to confusing results?
There was a problem hiding this comment.
@jwelton you are 100% right, this really only fixes the case where the overall action succeeds with after a retry. Looking further into this, I'm wondering if this needs to be handled in the xcresult gem itself...
There was a problem hiding this comment.
This looks to be the area where trainer handles it.
https://github.com/fastlane/fastlane/blob/3d2d208c7cbec41b03e2e6ceed93590dbf90629f/trainer/lib/trainer/test_parser.rb#L295
|
@diogot, any chances of merging this? |
|
Hi, if anyone add tests for this this feature I can merge this PR. |
Addresses #73
Only parse test_failures if the overall status is not 'succeeded'. This ensures that tests that pass via test repetition do not show up as a false negative.