Skip to content

Commit

Permalink
SyftCLIScanner: support SBOM generation with syft CLI
Browse files Browse the repository at this point in the history
Packit currently supports SBOM generation with syft tooling by utilizing
syft's go library. This has caused packit maintainers significant
maintainence burden. This commit adds a mechanism for buildpack authors
to utlize the syft CLI instead to generate SBOM. The intention here is
that with widespread adoption of this, we can phase out the codebase
that uses the syft go libary and thereby relieve the maintainers of this
pain.

Until recently, syft did not allow consumers to specify the exact schema
version of an SBOM mediatype they want generated (the tooling currently
supports passing a version for CycloneDX and SPDX -
github.com/anchore/syft/issues/846#issuecomment-1908676454). So packit
was forced to vendor-in (copy) large chunks of upstream syft go code
into packit in order to pin SBOM mediatype versions to versions that
most consumers wanted to use. Everytime a new version of Syft comes out,
maintainers had to painfully update the vendored-in code to work with
upstream syft components (e.g.
github.com//pull/491).

Furthermore, it is advantageous to use the syft CLI instead of syft go
library for multiple reasons. With CLI, we can delegate the entire SBOM
generation mechanism easily to syft. It should help buildpacks avoid any
CVEs that are exposed to it via syft go libaries. The CLI tool is well
documented and widely used in the community, and it seems like the syft
project is developed with with a CLI-first approach. The caveat here is
that buildpack authors who use this method should include the Paketo
Syft buildpack in their buildplan to have access to the CLI during the
build phase.

Example usage:

\# detect
\# unless BP_DISABLE_BOM is true
requirements = append(requirements, packit.BuildPlanRequirement{
                Name: "syft",
                Metadata: map[string]interface{}{
                        "build": true,
                },
})

\# build
syftCLIScanner := sbomgen.NewSyftCLIScanner(
		pexec.NewExecutable("syft"),
		scribe.NewEmitter(os.Stdout),
)

\# To scan a layer after installing a dependency
_ = syftCLIScanner.GenerateSBOM(myLayer.Path,
	context.Layers.Path,
	myLayer.Name,
	context.BuildpackInfo.SBOMFormats...,
)

\# OR to scan the workspace dir after running a process
_ = syftCLIScanner.GenerateSBOM(context.WorkingDir,
	context.Layers.Path,
	myLayer.Name,
	context.BuildpackInfo.SBOMFormats...,
)

- A new package sbomgen is created instead of adding the functionality
  to the existing sbom package because it helps buildpacks remove pinned
  "anchore/syft" lib from their go.mod which were flagged down by CVE
  scanners.
- I have not implemented pretty-fication of SBOM that the codepath that
  use syft go lib implements. This seems to be adding bloat to the app
  image and not supported via CLI. Consumers of SBOM can easily prettify
  the SBOM JSONs.
- In the codepath that use the syft go lib, license information is
  manually injected from buildpack.toml data into the SBOM. This is not
  available with the SyftCLIScanner. I couldn't find any reasoning for
  why this was done in the first place.
- I have intentionally not reused some code in methods that's mixed up
  with the syft go library with an intention to easily phase out that
  codebase in the near future.
  • Loading branch information
arjun024 committed Sep 17, 2024
1 parent 884a7b7 commit 1ab5b00
Show file tree
Hide file tree
Showing 6 changed files with 753 additions and 0 deletions.
32 changes: 32 additions & 0 deletions sbomgen/fakes/executable.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
package fakes

import (
"sync"

"github.com/paketo-buildpacks/packit/v2/pexec"
)

type Executable struct {
ExecuteCall struct {
mutex sync.Mutex
CallCount int
Receives struct {
Execution pexec.Execution
}
Returns struct {
Err error
}
Stub func(pexec.Execution) error
}
}

func (f *Executable) Execute(param1 pexec.Execution) error {
f.ExecuteCall.mutex.Lock()
defer f.ExecuteCall.mutex.Unlock()
f.ExecuteCall.CallCount++
f.ExecuteCall.Receives.Execution = param1
if f.ExecuteCall.Stub != nil {
return f.ExecuteCall.Stub(param1)
}
return f.ExecuteCall.Returns.Err
}
51 changes: 51 additions & 0 deletions sbomgen/formats.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package sbomgen

import (
"fmt"
"mime"
"strings"
)

const (
CycloneDXFormat = "application/vnd.cyclonedx+json"
SPDXFormat = "application/spdx+json"
SyftFormat = "application/vnd.syft+json"
)

// Format is the type declaration for the supported SBoM output formats.
type Format string

// Extension outputs the expected file extension for a given Format.
// packit allows CycloneDX and SPDX mediatypes to have an optional
// version suffix. e.g. "application/vnd.cyclonedx+json;version=1.4"
// The version suffix is not allowed for the syft mediatype as the
// syft tooling does not support providing a version for this mediatype.
func (f Format) Extension() (string, error) {
switch {
case strings.HasPrefix(string(f), CycloneDXFormat):
return "cdx.json", nil
case strings.HasPrefix(string(f), SPDXFormat):
return "spdx.json", nil
case f == SyftFormat:
return "syft.json", nil
default:
return "", fmt.Errorf("Unknown mediatype %s", f)
}
}

// Extracts optional version. This usually derives from the "sbom-formats"
// field used by packit-based buildpacks (@packit.SBOMFormats). e.g.
// "application/vnd.cyclonedx+json;version=1.4" -> "1.4" See
// github.com/paketo-buildpacks/packit/issues/302
func (f Format) VersionParam() (string, error) {
_, params, err := mime.ParseMediaType(string(f))
if err != nil {
return "", fmt.Errorf("failed to parse SBOM mediatype. Expected <mediatype>[;version=<ver>], Got %s: %w", f, err)
}

version, ok := params["version"]
if !ok {
return "", nil
}
return version, nil
}
45 changes: 45 additions & 0 deletions sbomgen/formats_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
package sbomgen_test

import (
"testing"

"github.com/paketo-buildpacks/packit/v2/sbomgen"
"github.com/sclevine/spec"

. "github.com/onsi/gomega"
)

func testFormats(t *testing.T, context spec.G, it spec.S) {
var Expect = NewWithT(t).Expect
var f sbomgen.Format

context("Formats", func() {
context("no version param", func() {
it("gets the right mediatype extension and version", func() {
f = sbomgen.CycloneDXFormat
ext, err := f.Extension()
Expect(err).NotTo(HaveOccurred())
Expect(ext).To(Equal("cdx.json"))
Expect(f.VersionParam()).To(Equal(""))
})
})

context("with version param", func() {
it("gets the right mediatype extension and version", func() {
f = sbomgen.SPDXFormat + ";version=9.8.7"
ext, err := f.Extension()
Expect(err).NotTo(HaveOccurred())
Expect(ext).To(Equal("spdx.json"))
Expect(f.VersionParam()).To(Equal("9.8.7"))
})
context("Syft mediatype with version returns empty", func() {
it("returns error", func() {
f = sbomgen.SyftFormat + ";version=9.8.7"
ext, err := f.Extension()
Expect(err).To(MatchError(ContainSubstring("Unknown mediatype application/vnd.syft+json;version=9.8.7")))
Expect(ext).To(Equal(""))
})
})
})
})
}
42 changes: 42 additions & 0 deletions sbomgen/init_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package sbomgen_test

import (
"testing"
"time"

"github.com/onsi/gomega/format"
"github.com/sclevine/spec"
"github.com/sclevine/spec/report"
)

func TestUnitSBOM(t *testing.T) {
format.MaxLength = 0

suite := spec.New("sbomgen", spec.Report(report.Terminal{}))
suite("Formats", testFormats)
suite("SyftCLIScanner", testSyftCLIScanner)
suite.Run(t)
}

type externalRef struct {
Category string `json:"referenceCategory"`
Locator string `json:"referenceLocator"`
Type string `json:"referenceType"`
}

type pkg struct {
ExternalRefs []externalRef `json:"externalRefs"`
LicenseConcluded string `json:"licenseConcluded"`
LicenseDeclared string `json:"licenseDeclared"`
Name string `json:"name"`
Version string `json:"versionInfo"`
}

type spdxOutput struct {
Packages []pkg `json:"packages"`
SPDXVersion string `json:"spdxVersion"`
DocumentNamespace string `json:"documentNamespace"`
CreationInfo struct {
Created time.Time `json:"created"`
} `json:"creationInfo"`
}
Loading

0 comments on commit 1ab5b00

Please sign in to comment.