Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SIP for signing Spin releases #1217

Merged
merged 1 commit into from
Mar 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
210 changes: 210 additions & 0 deletions docs/content/sips/012-signing-spin-releases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
title = "SIP 012 - "Signing the Spin releases"
template = "main"
date = "2023-03-01T01:01:01Z"

---

Summary: This improvement proposal describes the reasoning and implementation
for signing Spin releases.

Owners: [email protected]

Created: March 2, 2023

## Background

Signing release artifacts for software offers a way for consumers of the software
to verify the integrity of the package they downloaded. The signature should
offer two guarantees: the author of the artifact is indeed the one expected by
the consumer, and the content of the artifact has not been tampered with since
its creation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have any comment on the technology, which I realise is the focus of the SIP, but I'd find it really useful to understand what guarantees it provides, what attacks it mitigates, what malicious actions it allows a consumer to detect. The proposal mentions "validate the integrity" but clearly means something more than file integrity (which is presumably guaranteed with the digest). Is it about the attack of someone posting a false file somewhere and claiming it to be Spin?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the file integrity (which is now verifiable using a transparency log, rather than just using a file), a user can validate that the build was actually generated by the fermyon/spin repo, through a GitHub action workflow they can inspect at a given SHA or tag, and that us or someone else has not replaced the binary since the GitHub infrastructure built it at the time of the SHA or tag.

Besides this guarantee, this can be later expanded to include even more attestations (such as SLSA, which could include even more provenance data).


The goal of this SIP is to define the tools used for generating signatures for
the Spin project, and to provide a standard process for signing release artifacts
for Spin and related projects.
Copy link
Contributor

@kate-goldenring kate-goldenring Mar 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consider changing the way we expect plugins to be signed as well? SHA checksums seem sufficient for plugins because they are verifying that the remote executable/source code is what is defined in the manifest (matches the SHA). The difference here is that the manifest SHA is used to verify the plugin rather than the user. This seems worth evaluating with regards to the note on "related projects"


## Proposal

This SIP proposes that for every [Spin release](https://github.com/fermyon/spin/releases),
we add metadata that helps a user validate the integrity of the package they
download.
Specifically, this proposal suggests the use of [Sigstore](https://www.sigstore.dev/),
_a new standard for signing, verifying, and protecting software_, and in particular,
the use of [the new release of Cosign v2.0](https://blog.sigstore.dev/cosign-2-0-released),
which stabilizes [keyless signatures using an OIDC provider](https://docs.sigstore.dev/cosign/keyless/).

In keyless mode, Sigstore makes an OIDC provider the root of trust for the
signature by creating short-lived x509 certificates bound to an OIDC identity.
The certificates are authenticated and auditable, which makes the resulting
signatures auditable as well.

This process uses a few central pieces that are assumed trusted:

- an OIDC provider (such as GitHub)
- [Fulcio](https://docs.sigstore.dev/fulcio/overview) — free root certificate
authority that issues temporary certificates bound to an OIDC identity,
published to Rekor
- [Rekor](https://docs.sigstore.dev/rekor/overview) — transparency and timestamp
service, provides a ledger that can be audited
- `cosign sign` — the CLI that connects an OIDC identity to Fulcio to generate the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to get an idea of the external services involved when invoking this command... Fulcio? Others?

Perhaps a rare event but what is our approach when we're unable to sign due to external service issues?

Copy link
Member Author

@radu-matei radu-matei Mar 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fulcio and Rekor are both public good services that this mechanism uses while performing a signature, and that users would invoke while checking signatures.

If either is down, our signature process will fail.
Given that fairly large projects such as Kubernetes have already adopted this process, and that this has been marked as stable, my assumption is that the services should be relatively stable.

While we should sign canary builds, it's only stable releases that we are asking users to trust. Given that, rare events when we would be unable to sign stable releases should be indeed rare.
(if our experience is different as we test this, happy to re-evaluate this)

certificate, then uses the certificate to sign the artifact, publishing the
signing certificate to Rekor
- `cosign verify` — the CLI that verifies the signature of a given artifact by
auditing the transparency log

![Sigstore workflow, as described by sigstore.dev](../../static/image/sigstore.png)

This document proposes that the build and release infrastructure for the Spin project be the entity
that signs the release artifacts before finalizing a new release using
[GitHub's OIDC workflow](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect).
This would tell a user that the artifact they are downloading was built by the
Spin project GitHub infrastructure, and that it has not been tampered with since
its creation.

The following workflow describes the process:

- `permission: id-token: write` needs to be added to the GitHub action creating
the release (see [permissions for `GITHUB_TOKEN`](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token))
- a new step is added in the Spin release process that signs every artifact and
captures the certificate and signature:

```bash
$ cosign sign-blob \
# write the output certificate to a file that will be later added to the release
--output-certificate out/spin-v1.0.0-linux-amd64-keyless.pem \
# write the signature to a file that will be later added to the release
--output-signature out/spin-v1.0.0-linux-amd64-keyless.sig \
# skip interactive confirmation
--yes \
spin
```

- when uploading the assets to the new release, the certificate and signatures
need to be added side-by-side with the actual artifact
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we clarify the expected interpretation of 'side-by-side'? I'm assuming this should be interpreted as: include the .sig and .pem files in the same archive containing the artifact and related files (eg LICENSE, README, etc.)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on what we want to sign — the archive containing everything, or just the binary.

Since we already compute the content digest of the archive, it might make sense to sign that. If we sign the archive, then the signature and certificate would be distributed as separate release artifacts.

If we only sign the binary, then we could distribute the signature and certificate inside the archive.


- instructions are added for users on how to use `cosign` to verify the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tangential idea: I wonder if a badge service might be neat here. A GET req to the badge service would trigger an agent to download assets, perform the verification and return status. Badge could go on the Spin README, etc.

authenticity of the new Spin release.

### Alternative implementations

- the obvious alternative for signing releases would be to use [OpenPGP signatures](https://infra.apache.org/release-signing.html).
However, this has a few disadvantages, in particular related to key management
(both from the perspective of the Spin project, which would need to carefully
manage a private key, and for users who need to get the public key). In general,
using this mechanism to sign software has proven to be difficult to convince users
to adopt because of the complexity around GPG.
- using Cosign with keys — this would still require the Spin project to maintain
a private key

The keyless mode of Cosign can eliminate the key management issue through
ephemeral keys, which is why this method is preferred to alternatives.
vdice marked this conversation as resolved.
Show resolved Hide resolved

### Security considerations

[Sigstore's trust model](https://docs.sigstore.dev/security/) relies on a few
key trusted components: the OIDC provider to prove identity, the trust root
based on [TUF](https://theupdateframework.io/), [Rekor](https://docs.sigstore.dev/rekor/overview/),
and [Fulcio](https://docs.sigstore.dev/fulcio/overview/).

If any of the components above are compromised, the security guarantees of _any_
signature is no longer valid. However, because of the transparency logs,
such compromises can be detected.

Unauthorized access to the GitHub actions used to generate the releases could
allow an attacker to tamper with the release — however, that attack surface is
present regardless of what signature mechanism we choose — which suggests increased
attention to the workflows used to build and generate release artifacts.

### Appendix: minimal example in GitHub Actions

[The following repository](https://github.com/radu-matei/keyless-cosign-demo) can
be used as a minimal example for this workflow:

```yml
jobs:
sign:
runs-on: ubuntu-latest

permissions:
id-token: write

name: Sign artifact and publish signature and certificate
steps:
- uses: actions/checkout@master
with:
fetch-depth: 1

- name: Install Cosign
uses: sigstore/cosign-installer@main
with:
cosign-release: v2.0.0

- name: Create an artifact
run: |
mkdir out
echo 'hello world' > out/artifact

- name: Sign the artifact with GitHub OIDC token
run: cosign sign-blob --output-certificate out/crt.pem --output-signature out/artifact.sig out/artifact --yes
Copy link
Member

@vdice vdice Mar 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking ahead to our use in Spin and how we produce a number of <os>-<arch> builds. Is the crt.pem expected to be different for each artifact? I take it both artifact.sig and artifact itself, naturally, would be.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, each <os>-<arch> build (whether that is directly the Spin binary or the archive containing the readme and license) would have a signature and certificate associated.


- name: Upload assets as GitHub artifact
uses: actions/upload-artifact@v3
with:
name: artifact
path: out/*
...(continue with adding artifacts to the release)
```

### Verifying signatures

After the action is executed, the new artifact is released and its signature
published using Sigstore. To validate the signature, a user will then need to
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation detail I suppose, but it seems like here down to the rest of the doc should be available to users via documentation as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, a lot of this requires instructions for how users would interact and perform checks on the signatures.

download the release package (which now contains the artifact itself, the
certificate used to sign, and the signature itself), then validate the signature
using `cosign`:

```bash
$ tree .
├── artifact
├── artifact.sig
└── crt.pem

$ cosign verify-blob \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cosign appears to have built-in support for github oidc now via --certificate-github-workflow-* flags. (which I haven't actually tested...)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--certificate-github-workflow-sha appears to be part of the signature, not sure about others, haven't been able to validate them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me, it would be super useful to have the user experience of this clearly called out - the "GitHub actions example" is not a natural place to look for it grin

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users should have this in a documentation section on the developer website.
For the SIP, I renamed the section.

--signature artifact.sig --certificate crt.pem \
# the identity for the certificate is tied to the official repository of the Spin project
--certificate-identity https://github.com/fermyon/spin/.github/workflows/release-sign.yml@refs/heads/main \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
artifact

Verified OK
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to have a spin self update command or plugin that handles this verification part for the user if possible in the future.

```

Because of the `--certificate-identity` and `--certificate-oidc-issuer` flags,
the verification step also validates that not only the artifact has not been
tampered with, but the identity used when performing the signature is associated
with the GitHub repository and project.

Attempting to verify the signature of a different artifact will result in an error:

```bash
$ cosign verify-blob \
--signature artifact.sig --certificate crt.pem \
--certificate-identity https://github.com/fermyon/spin/.github/workflows/release-sign.yml@refs/heads/main \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Brainstorming how to present the correct value here for users, eg refs/heads/main for main (or refs/tags/canary for the corresponding canary release) and refs/tags/<tag> for other vX.X.X tagged releases.

Maybe the automation agent (GH action) attaches a verification-notes.txt asset or some such to a given release with command samples and notes, etc, filled-in with the proper strings. Or perhaps a default section added to the release notes by the author...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation and release notes would contain optional instructions on getting the certificate and signature, and the command required to perform the signature validation.

--certificate-oidc-issuer https://token.actions.githubusercontent.com \
fake-artifact

Error: verifying blob [fake-artifact]: searching log query: [POST /api/v1/log/entries/retrieve][400] searchLogQueryBadRequest &{Code:400 Message:unmarshalling entry: verifying signature: invalid signature when validating ASN.1 encoded signature}
```

Attempting to verify a signature using a different identity also results in an error:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For completeness, do we want to illustrate the scenario when one or both of the .sig and .pem files are fake/incorrect (but artifact, cert-identifier and cert-oidc-issuer correct)?


```bash
$ cosign verify-blob \
--signature artifact.sig --certificate crt.pem \
# the identity for the certificate is tied to the official repository of the Spin project
--certificate-identity <another-identity> \
--certificate-oidc-issuer <another-oidc-issuer> \
artifact

Error: verifying blob [artifact]: none of the expected identities matched what was in the certificate, got subjects [https://github.com/fermyon/spin/.github/workflows/release-sign.yml@refs/heads/main] with issuer https://token.actions.githubusercontent.com
```
Binary file added docs/static/image/sigstore.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.