Format junit-xml need to have passed test results, not only failed test

[alex-3sr]

• terrascan version: 1.3.3
• Operating System: Linux Docker

Description

Hi all,
I'm actually experimenting Terrascan as docker Mode on Azure DevOps, for use the recent Junit XML fomat.
The Junit XML is good, and I'm able to push test result in Azure DevOps sucessfully.
But I notice that I've only failed results, not the one who passed successfully.

So it result that If I've only 3 failed, I got a pass result of 0% in anycases. We need have success and failed, for get a % note of global results.

What I Did

I use a Docker run task in DevOps for run this command :

terrascan scan --iac-dir /tf --policy-type azure --iac-type terraform --iac-version v14 --output junit-xml

And here the XML Junit file output ->

<testsuites tests="260" name="TERRASCAN_POLICY_SUITES" failures="3" time="0">
  <testsuite tests="260" failures="3" time="0" name="TERRASCAN_POLICY_SUITE" package="/tf">
    <properties>
      <property name="Terrascan Version" value="v1.3.3"></property>
    </properties>
    <testcase classname="az-keyvault.tf" name="[ERROR] resource: &#34;keyvault&#34; at line: 1, violates: RULE - accurics.azure.EKM.20" severity="HIGH" category="Encryption and Key Management">
      <failure message="Description: Ensure that logging for Azure KeyVault is &#39;Enabled&#39;, File: az-keyvault.tf, Line: 1, Severity: HIGH, Rule Name: reme_keyVaultAuditLoggingEnabled, Rule ID: accurics.azure.EKM.20, Resource Name: keyvault, Resource Type: azurerm_key_vault, Category: Encryption and Key Management" type=""></failure>
    </testcase>
    <testcase classname="az-sql-server-fw.tf" name="[ERROR] resource: &#34;sqlfwruletempazure&#34; at line: 2, violates: RULE - accurics.azure.NS.5" severity="HIGH" category="Network Security">
      <failure message="Description: Ensure entire Azure infrastructure doesn&#39;t have access to Azure SQL ServerEnsure entire Azure infrastructure doesn&#39;t have access to Azure SQL Server, File: az-sql-server-fw.tf, Line: 2, Severity: HIGH, Rule Name: reme_sqlPublicAccess, Rule ID: accurics.azure.NS.5, Resource Name: sqlfwruletempazure, Resource Type: azurerm_sql_firewall_rule, Category: Network Security" type=""></failure>
    </testcase>
    <testcase classname="az-rg.tf" name="[ERROR] resource: &#34;rgproject&#34; at line: 1, violates: RULE - accurics.azure.NS.272" severity="LOW" category="Network Security">
      <failure message="Description: Ensure that Azure Resource Group has resource lock enabled, File: az-rg.tf, Line: 1, Severity: LOW, Rule Name: reme_resourceGroupLock, Rule ID: accurics.azure.NS.272, Resource Name: rgproject, Resource Type: azurerm_resource_group, Category: Network Security" type=""></failure>
    </testcase>
  </testsuite>
</testsuites>

This Junit output file, for my point of view, need to have all other testcases who passed successfully.
Or Maybe, there are an argument that I don't know/found for get it.

Thanks for your help, and remain at disposal
Regards
Alexandre

[patilpankaj212]

Hello @alex-3sr,
Thanks for the report.
junit-xml output doesn't not support showing passed tests. But we do plan to add it.

[devang-gaur]

So it result that If I've only 3 failed, I got a pass result of 0% in anycases. We need have success and failed, for get a % note of global results.

((Total_tests - failures) % Total_tests) * 100 ?

In this case, ((260 - 3) / 260) * 100 = 98.8461538462 % success.

[alex-3sr]

So it result that If I've only 3 failed, I got a pass result of 0% in anycases. We need have success and failed, for get a % note of global results.

((Total_tests - failures) % Total_tests) * 100 ?

In this case, ((260 - 3) / 260) * 100 = 98.8461538462 % success.

Yes, we can calculate it like this, sure, but It's not me who calc it, it's integrated on Azure DevOps test runs feature.
For example, on this test results, I've 8 failed and 220 passed, but got this ->

If I use an another test run, like Checkov for example, where I've both passed and failed, I got this ->

For my point of view, we need to have passed tests in Junit XML output file

Regards
Alexandre

[alex-3sr]

Hi,

With last 1.4.0 we have now --show-passed arguments, and works very well ;) Thanks, this is perfect !
I close this one so, no more action is required.

Regards and have a nice week-end
Alexandre