Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support displaying test results in markdown using GitHub Actions Job Summaries #463

Merged
merged 14 commits into from
Jun 25, 2024
Merged

Support displaying test results in markdown using GitHub Actions Job Summaries #463

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
83b7f42
Persist test summary on disk
ritchxu Apr 23, 2022
49667db
Add badge title customization
ritchxu Apr 23, 2022
bd77050
Support none for list-suites
ritchxu Apr 24, 2022
3608ee0
Add option of use-actions-summary
ritchxu Apr 24, 2022
724497a
Fix edge case of one test assembly
ritchxu May 6, 2022
d545618
Update action.yml to use node20
ritchxu Feb 20, 2024
0b7d35f
Merge branch 'dorny:main'
ritchxu Mar 3, 2024
c1768c8
Skip markdown file
ritchxu Apr 6, 2024
3816496
Merge pull request #2 from ritchxu/ritchxu/pr-feedbacks
ritchxu Apr 6, 2024
775c900
Update README.md and Actions file
ritchxu Apr 6, 2024
574868a
Undo trx parsing changes temporarily
ritchxu May 29, 2024
1db4305
Reaction to PR feedback
ritchxu Jun 25, 2024
84e60ba
Merge branch 'upstream-main' into ritchxu/support-actions-summary
ritchxu Jun 25, 2024
9d0f09a
Fix code formatting issues
jozefizso Jun 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This [Github Action](https://github.com/features/actions) displays test results from popular testing frameworks directly in GitHub.

✔️ Parses test results in XML or JSON format and creates nice report as Github Check Run
✔️ Parses test results in XML or JSON format and creates nice report as GitHub Check Run or GitHub Actions job summaries

✔️ Annotates code where it failed based on message and stack trace captured during test execution

Expand Down Expand Up @@ -151,9 +151,18 @@ jobs:
# Detailed listing of test suites and test cases will be skipped.
only-summary: 'false'

# Allows you to generate reports for Actions Summary
# https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/
use-actions-summary: 'true'

# Customize the title of badges shown for each Actions Summary.
# Useful when distinguish summaries for tests ran in multiple Actions steps.
badge-title: 'tests'

# Limits which test suites are listed:
# all
# failed
# none
list-suites: 'all'

# Limits which test cases are listed:
Expand Down
7 changes: 5 additions & 2 deletions __tests__/__outputs__/dart-json.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
![Tests failed](https://img.shields.io/badge/tests-1%20passed%2C%204%20failed%2C%201%20skipped-critical)
|Report|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|fixtures/dart-json.json|1 ✅|4 ❌|1 ⚪|4s|
## ❌ <a id="user-content-r0" href="#r0">fixtures/dart-json.json</a>
**6** tests were completed in **4s** with **1** passed, **4** failed and **1** skipped.
|Test suite|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|[test/main_test.dart](#r0s0)|1✅|3❌||74ms|
|[test/second_test.dart](#r0s1)||1❌|1⚪|51ms|
|[test/main_test.dart](#r0s0)|1 ✅|3 ❌||74ms|
|[test/second_test.dart](#r0s1)||1 ❌|1 ⚪|51ms|
### ❌ <a id="user-content-r0s0" href="#r0s0">test/main_test.dart</a>
```
Test 1
Expand Down
5 changes: 4 additions & 1 deletion __tests__/__outputs__/dotnet-nunit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
![Tests failed](https://img.shields.io/badge/tests-3%20passed%2C%205%20failed%2C%201%20skipped-critical)
|Report|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|fixtures/dotnet-nunit.xml|3 ✅|5 ❌|1 ⚪|230ms|
## ❌ <a id="user-content-r0" href="#r0">fixtures/dotnet-nunit.xml</a>
**9** tests were completed in **230ms** with **3** passed, **5** failed and **1** skipped.
|Test suite|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|[DotnetTests.NUnitV3Tests.dll.DotnetTests.XUnitTests](#r0s0)|3✅|5❌|1⚪|69ms|
|[DotnetTests.NUnitV3Tests.dll.DotnetTests.XUnitTests](#r0s0)|3 ✅|5 ❌|1 ⚪|69ms|
### ❌ <a id="user-content-r0s0" href="#r0s0">DotnetTests.NUnitV3Tests.dll.DotnetTests.XUnitTests</a>
```
CalculatorTests
Expand Down
5 changes: 4 additions & 1 deletion __tests__/__outputs__/dotnet-trx.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
![Tests failed](https://img.shields.io/badge/tests-5%20passed%2C%205%20failed%2C%201%20skipped-critical)
|Report|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|fixtures/dotnet-trx.trx|5 ✅|5 ❌|1 ⚪|1s|
## ❌ <a id="user-content-r0" href="#r0">fixtures/dotnet-trx.trx</a>
**11** tests were completed in **1s** with **5** passed, **5** failed and **1** skipped.
|Test suite|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|[DotnetTests.XUnitTests.CalculatorTests](#r0s0)|5✅|5❌|1⚪|118ms|
|[DotnetTests.XUnitTests.CalculatorTests](#r0s0)|5 ✅|5 ❌|1 ⚪|118ms|
### ❌ <a id="user-content-r0s0" href="#r0s0">DotnetTests.XUnitTests.CalculatorTests</a>
```
✅ Custom Name
Expand Down
132 changes: 69 additions & 63 deletions __tests__/__outputs__/fluent-validation-test-results.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,70 +1,75 @@
![Tests passed successfully](https://img.shields.io/badge/tests-803%20passed%2C%201%20skipped-success)
<details><summary>Expand for details</summary>

|Report|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|fixtures/external/FluentValidation.Tests.trx|803 ✅||1 ⚪|4s|
## ✅ <a id="user-content-r0" href="#r0">fixtures/external/FluentValidation.Tests.trx</a>
**804** tests were completed in **4s** with **803** passed, **0** failed and **1** skipped.
|Test suite|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|[FluentValidation.Tests.AbstractValidatorTester](#r0s0)|35✅|||12ms|
|[FluentValidation.Tests.AccessorCacheTests](#r0s1)|4✅||1⚪|4ms|
|[FluentValidation.Tests.AssemblyScannerTester](#r0s2)|2✅|||2ms|
|[FluentValidation.Tests.CascadingFailuresTester](#r0s3)|38✅|||23ms|
|[FluentValidation.Tests.ChainedValidationTester](#r0s4)|13✅|||6ms|
|[FluentValidation.Tests.ChainingValidatorsTester](#r0s5)|3✅|||1ms|
|[FluentValidation.Tests.ChildRulesTests](#r0s6)|2✅|||7ms|
|[FluentValidation.Tests.CollectionValidatorWithParentTests](#r0s7)|16✅|||13ms|
|[FluentValidation.Tests.ComplexValidationTester](#r0s8)|17✅|||26ms|
|[FluentValidation.Tests.ConditionTests](#r0s9)|18✅|||9ms|
|[FluentValidation.Tests.CreditCardValidatorTests](#r0s10)|2✅|||2ms|
|[FluentValidation.Tests.CustomFailureActionTester](#r0s11)|3✅|||1ms|
|[FluentValidation.Tests.CustomMessageFormatTester](#r0s12)|6✅|||3ms|
|[FluentValidation.Tests.CustomValidatorTester](#r0s13)|10✅|||6ms|
|[FluentValidation.Tests.DefaultValidatorExtensionTester](#r0s14)|30✅|||38ms|
|[FluentValidation.Tests.EmailValidatorTests](#r0s15)|36✅|||18ms|
|[FluentValidation.Tests.EmptyTester](#r0s16)|9✅|||5ms|
|[FluentValidation.Tests.EnumValidatorTests](#r0s17)|12✅|||24ms|
|[FluentValidation.Tests.EqualValidatorTests](#r0s18)|10✅|||3ms|
|[FluentValidation.Tests.ExactLengthValidatorTester](#r0s19)|6✅|||2ms|
|[FluentValidation.Tests.ExclusiveBetweenValidatorTests](#r0s20)|19✅|||6ms|
|[FluentValidation.Tests.ExtensionTester](#r0s21)|4✅|||1ms|
|[FluentValidation.Tests.ForEachRuleTests](#r0s22)|34✅|||47ms|
|[FluentValidation.Tests.GreaterThanOrEqualToValidatorTester](#r0s23)|14✅|||5ms|
|[FluentValidation.Tests.GreaterThanValidatorTester](#r0s24)|13✅|||4ms|
|[FluentValidation.Tests.InclusiveBetweenValidatorTests](#r0s25)|18✅|||4ms|
|[FluentValidation.Tests.InheritanceValidatorTest](#r0s26)|11✅|||18ms|
|[FluentValidation.Tests.InlineValidatorTester](#r0s27)|1✅|||2ms|
|[FluentValidation.Tests.LanguageManagerTests](#r0s28)|21✅|||28ms|
|[FluentValidation.Tests.LengthValidatorTests](#r0s29)|16✅|||17ms|
|[FluentValidation.Tests.LessThanOrEqualToValidatorTester](#r0s30)|13✅|||4ms|
|[FluentValidation.Tests.LessThanValidatorTester](#r0s31)|16✅|||6ms|
|[FluentValidation.Tests.LocalisedMessagesTester](#r0s32)|6✅|||3ms|
|[FluentValidation.Tests.LocalisedNameTester](#r0s33)|2✅|||1ms|
|[FluentValidation.Tests.MemberAccessorTests](#r0s34)|9✅|||5ms|
|[FluentValidation.Tests.MessageFormatterTests](#r0s35)|10✅|||2ms|
|[FluentValidation.Tests.ModelLevelValidatorTests](#r0s36)|2✅|||1ms|
|[FluentValidation.Tests.NameResolutionPluggabilityTester](#r0s37)|3✅|||2ms|
|[FluentValidation.Tests.NotEmptyTester](#r0s38)|10✅|||7ms|
|[FluentValidation.Tests.NotEqualValidatorTests](#r0s39)|11✅|||7ms|
|[FluentValidation.Tests.NotNullTester](#r0s40)|5✅|||1ms|
|[FluentValidation.Tests.NullTester](#r0s41)|5✅|||2ms|
|[FluentValidation.Tests.OnFailureTests](#r0s42)|10✅|||8ms|
|[FluentValidation.Tests.PredicateValidatorTester](#r0s43)|5✅|||2ms|
|[FluentValidation.Tests.PropertyChainTests](#r0s44)|7✅|||1ms|
|[FluentValidation.Tests.RegularExpressionValidatorTests](#r0s45)|15✅|||6ms|
|[FluentValidation.Tests.RuleBuilderTests](#r0s46)|29✅|||96ms|
|[FluentValidation.Tests.RuleDependencyTests](#r0s47)|14✅|||3s|
|[FluentValidation.Tests.RulesetTests](#r0s48)|21✅|||14ms|
|[FluentValidation.Tests.ScalePrecisionValidatorTests](#r0s49)|6✅|||4ms|
|[FluentValidation.Tests.SharedConditionTests](#r0s50)|42✅|||42ms|
|[FluentValidation.Tests.StandalonePropertyValidationTester](#r0s51)|1✅|||0ms|
|[FluentValidation.Tests.StringEnumValidatorTests](#r0s52)|10✅|||5ms|
|[FluentValidation.Tests.TrackingCollectionTests](#r0s53)|3✅|||2ms|
|[FluentValidation.Tests.TransformTests](#r0s54)|4✅|||3ms|
|[FluentValidation.Tests.UserSeverityTester](#r0s55)|7✅|||3ms|
|[FluentValidation.Tests.UserStateTester](#r0s56)|4✅|||3ms|
|[FluentValidation.Tests.ValidateAndThrowTester](#r0s57)|14✅|||25ms|
|[FluentValidation.Tests.ValidationResultTests](#r0s58)|8✅|||8ms|
|[FluentValidation.Tests.ValidatorDescriptorTester](#r0s59)|5✅|||1ms|
|[FluentValidation.Tests.ValidatorSelectorTests](#r0s60)|10✅|||9ms|
|[FluentValidation.Tests.ValidatorTesterTester](#r0s61)|73✅|||74ms|
|[FluentValidation.Tests.AbstractValidatorTester](#r0s0)|35 ✅|||12ms|
|[FluentValidation.Tests.AccessorCacheTests](#r0s1)|4 ✅||1 ⚪|4ms|
|[FluentValidation.Tests.AssemblyScannerTester](#r0s2)|2 ✅|||2ms|
|[FluentValidation.Tests.CascadingFailuresTester](#r0s3)|38 ✅|||23ms|
|[FluentValidation.Tests.ChainedValidationTester](#r0s4)|13 ✅|||6ms|
|[FluentValidation.Tests.ChainingValidatorsTester](#r0s5)|3 ✅|||1ms|
|[FluentValidation.Tests.ChildRulesTests](#r0s6)|2 ✅|||7ms|
|[FluentValidation.Tests.CollectionValidatorWithParentTests](#r0s7)|16 ✅|||13ms|
|[FluentValidation.Tests.ComplexValidationTester](#r0s8)|17 ✅|||26ms|
|[FluentValidation.Tests.ConditionTests](#r0s9)|18 ✅|||9ms|
|[FluentValidation.Tests.CreditCardValidatorTests](#r0s10)|2 ✅|||2ms|
|[FluentValidation.Tests.CustomFailureActionTester](#r0s11)|3 ✅|||1ms|
|[FluentValidation.Tests.CustomMessageFormatTester](#r0s12)|6 ✅|||3ms|
|[FluentValidation.Tests.CustomValidatorTester](#r0s13)|10 ✅|||6ms|
|[FluentValidation.Tests.DefaultValidatorExtensionTester](#r0s14)|30 ✅|||38ms|
|[FluentValidation.Tests.EmailValidatorTests](#r0s15)|36 ✅|||18ms|
|[FluentValidation.Tests.EmptyTester](#r0s16)|9 ✅|||5ms|
|[FluentValidation.Tests.EnumValidatorTests](#r0s17)|12 ✅|||24ms|
|[FluentValidation.Tests.EqualValidatorTests](#r0s18)|10 ✅|||3ms|
|[FluentValidation.Tests.ExactLengthValidatorTester](#r0s19)|6 ✅|||2ms|
|[FluentValidation.Tests.ExclusiveBetweenValidatorTests](#r0s20)|19 ✅|||6ms|
|[FluentValidation.Tests.ExtensionTester](#r0s21)|4 ✅|||1ms|
|[FluentValidation.Tests.ForEachRuleTests](#r0s22)|34 ✅|||47ms|
|[FluentValidation.Tests.GreaterThanOrEqualToValidatorTester](#r0s23)|14 ✅|||5ms|
|[FluentValidation.Tests.GreaterThanValidatorTester](#r0s24)|13 ✅|||4ms|
|[FluentValidation.Tests.InclusiveBetweenValidatorTests](#r0s25)|18 ✅|||4ms|
|[FluentValidation.Tests.InheritanceValidatorTest](#r0s26)|11 ✅|||18ms|
|[FluentValidation.Tests.InlineValidatorTester](#r0s27)|1 ✅|||2ms|
|[FluentValidation.Tests.LanguageManagerTests](#r0s28)|21 ✅|||28ms|
|[FluentValidation.Tests.LengthValidatorTests](#r0s29)|16 ✅|||17ms|
|[FluentValidation.Tests.LessThanOrEqualToValidatorTester](#r0s30)|13 ✅|||4ms|
|[FluentValidation.Tests.LessThanValidatorTester](#r0s31)|16 ✅|||6ms|
|[FluentValidation.Tests.LocalisedMessagesTester](#r0s32)|6 ✅|||3ms|
|[FluentValidation.Tests.LocalisedNameTester](#r0s33)|2 ✅|||1ms|
|[FluentValidation.Tests.MemberAccessorTests](#r0s34)|9 ✅|||5ms|
|[FluentValidation.Tests.MessageFormatterTests](#r0s35)|10 ✅|||2ms|
|[FluentValidation.Tests.ModelLevelValidatorTests](#r0s36)|2 ✅|||1ms|
|[FluentValidation.Tests.NameResolutionPluggabilityTester](#r0s37)|3 ✅|||2ms|
|[FluentValidation.Tests.NotEmptyTester](#r0s38)|10 ✅|||7ms|
|[FluentValidation.Tests.NotEqualValidatorTests](#r0s39)|11 ✅|||7ms|
|[FluentValidation.Tests.NotNullTester](#r0s40)|5 ✅|||1ms|
|[FluentValidation.Tests.NullTester](#r0s41)|5 ✅|||2ms|
|[FluentValidation.Tests.OnFailureTests](#r0s42)|10 ✅|||8ms|
|[FluentValidation.Tests.PredicateValidatorTester](#r0s43)|5 ✅|||2ms|
|[FluentValidation.Tests.PropertyChainTests](#r0s44)|7 ✅|||1ms|
|[FluentValidation.Tests.RegularExpressionValidatorTests](#r0s45)|15 ✅|||6ms|
|[FluentValidation.Tests.RuleBuilderTests](#r0s46)|29 ✅|||96ms|
|[FluentValidation.Tests.RuleDependencyTests](#r0s47)|14 ✅|||3s|
|[FluentValidation.Tests.RulesetTests](#r0s48)|21 ✅|||14ms|
|[FluentValidation.Tests.ScalePrecisionValidatorTests](#r0s49)|6 ✅|||4ms|
|[FluentValidation.Tests.SharedConditionTests](#r0s50)|42 ✅|||42ms|
|[FluentValidation.Tests.StandalonePropertyValidationTester](#r0s51)|1 ✅|||0ms|
|[FluentValidation.Tests.StringEnumValidatorTests](#r0s52)|10 ✅|||5ms|
|[FluentValidation.Tests.TrackingCollectionTests](#r0s53)|3 ✅|||2ms|
|[FluentValidation.Tests.TransformTests](#r0s54)|4 ✅|||3ms|
|[FluentValidation.Tests.UserSeverityTester](#r0s55)|7 ✅|||3ms|
|[FluentValidation.Tests.UserStateTester](#r0s56)|4 ✅|||3ms|
|[FluentValidation.Tests.ValidateAndThrowTester](#r0s57)|14 ✅|||25ms|
|[FluentValidation.Tests.ValidationResultTests](#r0s58)|8 ✅|||8ms|
|[FluentValidation.Tests.ValidatorDescriptorTester](#r0s59)|5 ✅|||1ms|
|[FluentValidation.Tests.ValidatorSelectorTests](#r0s60)|10 ✅|||9ms|
|[FluentValidation.Tests.ValidatorTesterTester](#r0s61)|73 ✅|||74ms|
### ✅ <a id="user-content-r0s0" href="#r0s0">FluentValidation.Tests.AbstractValidatorTester</a>
```
✅ Can_replace_default_errorcode_resolver
Expand Down Expand Up @@ -1054,4 +1059,5 @@
✅ Unexpected_message_check(withoutErrMsg: "bar", errMessages: [])
✅ Unexpected_severity_check
✅ Unexpected_state_check
```
```
</details>
7 changes: 5 additions & 2 deletions __tests__/__outputs__/jest-junit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
![Tests failed](https://img.shields.io/badge/tests-1%20passed%2C%204%20failed%2C%201%20skipped-critical)
|Report|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|fixtures/jest-junit.xml|1 ✅|4 ❌|1 ⚪|1s|
## ❌ <a id="user-content-r0" href="#r0">fixtures/jest-junit.xml</a>
**6** tests were completed in **1s** with **1** passed, **4** failed and **1** skipped.
|Test suite|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|[__tests__\main.test.js](#r0s0)|1✅|3❌||486ms|
|[__tests__\second.test.js](#r0s1)||1❌|1⚪|82ms|
|[__tests__\main.test.js](#r0s0)|1 ✅|3 ❌||486ms|
|[__tests__\second.test.js](#r0s1)||1 ❌|1 ⚪|82ms|
### ❌ <a id="user-content-r0s0" href="#r0s0">__tests__\main.test.js</a>
```
Test 1
Expand Down
10 changes: 8 additions & 2 deletions __tests__/__outputs__/jest-react-component-test-results.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
![Tests passed successfully](https://img.shields.io/badge/tests-1%20passed-success)
<details><summary>Expand for details</summary>

|Report|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|fixtures/external/jest/jest-react-component-test-results.xml|1 ✅|||1000ms|
## ✅ <a id="user-content-r0" href="#r0">fixtures/external/jest/jest-react-component-test-results.xml</a>
**1** tests were completed in **1000ms** with **1** passed, **0** failed and **0** skipped.
|Test suite|Passed|Failed|Skipped|Time|
|:---|---:|---:|---:|---:|
|[\<Component /\>](#r0s0)|1✅|||798ms|
|[\<Component /\>](#r0s0)|1 ✅|||798ms|
### ✅ <a id="user-content-r0s0" href="#r0s0">\<Component /\></a>
```
✅ <Component /> should render properly
```
```
</details>
Loading