sarif output appears to be duplicated and invalid

[flcdrg]

Describe the bug

Output from jf build number --vuln=true --fail=true --server-id "server" --format sarif does not validate when uploaded to https://sarifweb.azurewebsites.net/Validation

The actual output is duplicated and contains invalid elements.

Current behavior

This is the output from the above command.

{
  "version": "2.1.0",
  "$schema": "https://json.schemastore.org/sarif-2.1.0-rtm.5.json",
  "runs": [
    {
      "tool": {
        "driver": {
          "informationUri": "https://jfrog.com/xray/",
          "name": "JFrog Xray",
          "rules": [
            {
              "id": "XRAY-260082",
              "shortDescription": null,
              "help": {
                "markdown": ".NET and Visual Studio Denial of Service Vulnerability. This CVE ID is unique from CVE-2022-23267, CVE-2022-29145."
              },
              "properties": {
                "security-severity": "7.5"
              }
            }
          ]
        }
      },
      "results": [
        {
          "ruleId": "XRAY-260082",
          "ruleIndex": 0,
          "message": {
            "text": "[CVE-2022-29117] Upgrade microsoft.owin:4.2.0 to [4.2.2]"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": " Package Descriptor"
                }
              }
            }
          ]
        },
        {
          "ruleId": "XRAY-260082",
          "ruleIndex": 0,
          "message": {
            "text": "[CVE-2022-29117] Upgrade microsoft.owin:4.2.0 to [4.2.2]"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": " Package Descriptor"
                }
              }
            }
          ]
        }
      ]
    }
  ]
}
{
  "version": "2.1.0",
  "$schema": "https://json.schemastore.org/sarif-2.1.0-rtm.5.json",
  "runs": [
    {
      "tool": {
        "driver": {
          "informationUri": "https://jfrog.com/xray/",
          "name": "JFrog Xray",
          "rules": [
            {
              "id": "XRAY-260082",
              "shortDescription": null,
              "help": {
                "markdown": ".NET and Visual Studio Denial of Service Vulnerability. This CVE ID is unique from CVE-2022-23267, CVE-2022-29145."
              },
              "properties": {
                "security-severity": "7.5"
              }
            },
            {
              "id": "XRAY-138885",
              "shortDescription": null,
              "help": {
                "markdown": "Newtonsoft Json.NET (Newtonsoft.Json) JSON Deserialization Nested Object Recursion Handling Stack Exhaustion DoS Weakness"
              },
              "properties": {
                "security-severity": "0.0"
              }
            }
          ]
        }
      },
      "results": [
        {
          "ruleId": "XRAY-260082",
          "ruleIndex": 0,
          "message": {
            "text": "[CVE-2022-29117] Upgrade microsoft.owin:4.2.0 to [4.2.2]"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": " Package Descriptor"
                }
              }
            }
          ]
        },
        {
          "ruleId": "XRAY-138885",
          "ruleIndex": 1,
          "message": {
            "text": "[XRAY-138885] Upgrade newtonsoft.json:12.0.1 to [13.0.1]"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": " Package Descriptor"
                }
              }
            }
          ]
        }
      ]
    }
  ]
}

No, I didn't paste twice - the output seems to be duplicated (such that it is not even valid JSON!

Removing the duplication, it also contains errors according to the online validator.

• runs[0].tool.driver.rules[0].shortDescription: The schema requires one of the types [Object], but a token of type 'String' was found
• runs[0].tool.driver.rules[0].help: The required property 'text' is missing.
• runs[0].tool.driver: The tool 'JFrog Xray' does not provide any of the version-related properties 'version', 'semanticVersion', 'dottedQuadFileVersion'. Providing version information enables the log file consumer to determine whether the file was produced by an up to date version, and to avoid accidentally comparing log files produced by different tool versions
• runs[0].results[0].locations[0].physicalLocation.artifactLocation.uri: The string ' Package Descriptor' is not a valid URI reference. URIs must conform to RFC 3986.
• runs[0].results[1].locations[0].physicalLocation.artifactLocation.uri: The string ' Package Descriptor' is not a valid URI reference. URIs must conform to RFC 3986.

Reproduction steps

Ran command in description against a .NET project with a vulnerable NuGet package.

Expected behavior

Valid sarif outputted

JFrog CLI-Core version

2.34.6

JFrog CLI version (if applicable)

2.34.6

Operating system type and version

Windows 2019

JFrog Artifactory version

Current hosted version

JFrog Xray version

Current hosted version

[flcdrg]

I suspect the duplication might be due to using --vuln=true

That's not particularly useful if it is just concatenating two JSON outputs together if that is the case

[sverdlov93]

Hi @flcdrg ,
Thanks for reporting this issue.
We're sorry for the inconvenience caused by this.
First, it's indeed an issue with our sarif format creation and should be fixed.
Regardless of that issue, can you please share the reason behind using --vuln flag here?

[flcdrg]

I was under the impression that would result in addition information being included the the report, which sounded useful?

[sverdlov93]

Running jf bs with --vuln flag is actually running 2 scans and prints 2 different tables.
The first table is the regular build scan security violations according to Xray watches and policies assigned to that build.
The second table is an additional table of all vulnerabilities regardless of any Xray watch and policy.
The all-vulns table should not appear on the sarif data.

[sverdlov93]

Hi @flcdrg
JFrog CLI 2.35.0 is released and should fix the above issue.
Looking forward to your feedback on it.

[Lykathia]

Still an issue, at least w.r.t. invalid output. This is using CLI version 2.40.0

Error details: instance.runs[0].tool.driver.rules[1].help requires property "text"

  {
    "path": [
      "runs",
      0,
      "tool",
      "driver",
      "rules",
      1,
      "help"
    ],
    "property": "instance.runs[0].tool.driver.rules[1].help",
    "message": "requires property \"text\"",
    "schema": {
      "description": "A message string or message format string rendered in multiple formats.",
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "text": {
          "description": "A plain text message string or format string.",
          "type": "string"
        },
        "markdown": {
          "description": "A Markdown message string or format string.",
          "type": "string"
        },
        "properties": {
          "description": "Key/value pairs that provide additional information about the message.",
          "$ref": "#/definitions/propertyBag"
        }
      },
      "required": [
        "text"
      ]
    },
    "instance": {
      "markdown": "A denial of service vulnerability exists when .NET Framework or .NET Core improperly handle web requests, aka '.Net Framework and .Net Core Denial of Service Vulnerability'. This CVE ID is unique from CVE-2019-0820, CVE-2019-0980."
    },
    "name": "required",
    "argument": "text",
    "stack": "instance.runs[0].tool.driver.rules[1].help requires property \"text\""
  }

Error details: instance.runs[0].tool.driver.rules[1].shortDescription is not of a type(s) object
  {
    "path": [
      "runs",
      0,
      "tool",
      "driver",
      "rules",
      1,
      "shortDescription"
    ],
    "property": "instance.runs[0].tool.driver.rules[1].shortDescription",
    "message": "is not of a type(s) object",
    "schema": {
      "description": "A message string or message format string rendered in multiple formats.",
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "text": {
          "description": "A plain text message string or format string.",
          "type": "string"
        },
        "markdown": {
          "description": "A Markdown message string or format string.",
          "type": "string"
        },
        "properties": {
          "description": "Key/value pairs that provide additional information about the message.",
          "$ref": "#/definitions/propertyBag"
        }
      },
      "required": [
        "text"
      ]
    },
    "instance": null,
    "name": "type",
    "argument": [
      "object"
    ],
    "stack": "instance.runs[0].tool.driver.rules[1].shortDescription is not of a type(s) object"
  }

text is missing and shortDescription is null in the original output.

[ipowellBT]

While not completely valid SARIF, as of version 2.52.1 of the jfrog cli, the SARIF produced by jf audit and jf docker scan is accepted by GitHub Security Centre.