Workload Attestation with JWTs

[APTy]

Following up on some discussion to iron out whether JWTs are feasible for workload attestation.

Motivation
To allow for generic workload attestation in cases where the orchestration platform doesn't provide an appropriate API for looking up workload attributes, but there is a way to generate a JWT and provide it to the workload somehow.

API
Since the Workload API doesn't allow attestation data to be passed by the workload, the JWT would need to live in a well-known location, likely co-located with the application binary. e.g.

/home/foo/foo
/home/foo/foo-attestation.jwt

Selectors
The selectors would define the policy by which a workload could request its identity, based on the issuer and the subject fields of the JWT.

selector	value
jwt:iss	The name of the launcher of the workload
jwt:sub	The workload's name

This would allow for workload registration as follows:

$ spire-server entry create \
    -parentID spiffe://example.org/host \
    -spiffeID spiffe://example.org/foo \
    -selector jwt:iss:spiffe://example.org/job-spawner \
    -selector jwt:sub:spiffe://example.org/foo

Open questions

1. JWTs are short-lived. This could be seen as a pro or a con. Since JWTs are short-lived, it would be difficult for a long-lived workload to use a JWT for the entirety of the workload's lifetime.

2. If we can extend the Workload API to allow for workloads to provide attestation data such as signed documents (akin to node attestation), we can provide the JWT directly to the SPIRE agent, as opposed to requiring the SPIRE agent to find the JWT on the filesystem.

Thoughts?

EDIT (2019-01-14): Actually I wonder if there is a way to use part of the jwt as the parent id? e.g. jwt:parent:${parentID}, in which case we could have 1 registration per subject, and have the parent policy be part of the JWT. If that makes sense

[kevinswiber]

Just a quick comment.

1. JWTs are short-lived. This could be seen as a pro or a con. Since JWTs are short-lived, it would be difficult for a long-lived workload to use a JWT for the entirety of the workload's lifetime.

Best practices for JWTs are that they should be "short-lived," depending heavily on the use case. One reason is because they are self-validating tokens, and having a shorter lifetime means determining the acceptable delta between an assigned revocation of credentials and the actual enforcement of that revocation.

However, if there's a separate revocation mechanism for JWTs, the constraint of short expiration times can be reduced dramatically. I'm still fairly new to JWTs with SPIFFE, and perhaps it's too big of a change, but streaming back responses with some kind of JWT Revocation List might be an idea worth considering.

EDIT:

I forgot to mention that the way apps cope with this today is by using a token refresh mechanism. So the JWT is short-lived, but the refresh token can be long-lived. This is possible, too, but would require a runtime update.

[APTy]

the way apps cope with this today is by using a token refresh mechanism.

Right, so in this case, we could extend the life of workloads by having the orchestration platform mount a directory and place the JWT there, such that it can update the JWT periodically.

[evan2645]

I'm curious to know a little more about your use case. Are you targeting a specific orchestrator? A generic solution that would work for others in the community would be great (e.g. a mesos integration story), if possible.

Another possibility is to pair registration api calls with more static selectors... For instance, scheduler integration registers workloads with tightly-scoped parent IDs, using selectors like docker image ID, binary name/path/sha, etc. Selectors like those could negate the need for a locally-available orchestrator API.

More thoughts below

JWTs are short-lived. This could be seen as a pro or a con. Since JWTs are short-lived, it would be difficult for a long-lived workload to use a JWT for the entirety of the workload's lifetime.

I think replay is a concern here too? If I can get a copy of the JWT, I can masquerade as that workload from other places in the infrastructure. The JWT would need to be bound to the workload (e.g. by embedding the workload SHA in the document, which must be validated).

If we can extend the Workload API to allow for workloads to provide attestation data such as signed documents (akin to node attestation), we can provide the JWT directly to the SPIRE agent, as opposed to requiring the SPIRE agent to find the JWT on the filesystem.

The SPIFFE Workload API is designed to be fairly opinionated. For instance, certificates returned through the X509-related service(s) MUST be an SVID... the reasoning here is that the semantics of an SVID are well defined (how to parse it, what parts are important, how to use it, validate it etc). In order to provide portability and interop guarantees, we need strict definition around things that go in and out of the workload api. As such, if we poke a hole for something like this, I think we'd need to define exactly the behavior of this document (e.g. is it a JWT-SVID? How to validate it? What parameters are required?). Otherwise, the functionality is tied to the SPIRE implementation (or worse, a plugin implementation).

[APTy]

I actually like the idea of using things like docker image ID, as well as the binary path/sha attestations that already exist, as a solution for some of the cases I was considering.

Still, JWTs could possibly function as a scaling lever - think about using a "parent" claim on a JWT signed by your "admin" workload - which would allow you to issue JWTs totally horizontally without ever needing to touch the registration API. For short-lived jobs (e.g. yarn + spark) where you may have millions of jobs per day, it may even be necessary, though I haven't tested whether the SPIRE server can/can't handle that sort of load with the attestation options that exist out of the box.. I'm just brainstorming a bit.

Regarding Mesos attestation, it's a bit tricky but I think the APIs exist to build out a plugin for it.. I'm looking into that but happy to contribute it back to the project!

[dkiser]

Interesting idea.

TL;DR; Of the opinion model could work, but shouldn't be a generic workload attestation plugin, but may be used within the implementation of a workload attestation plugin.

I'd argue that introducing a mechanism like this (globally) goes against the design intent of the workload attesters plugins being self-contained and specific.

Here are a few comments:

• If an orchestration platform has the ability to provide a JWT to a workload (wether centrally or locally somehow), then where, how, and what contents of this JWT are within the responsibility of the orchestration platform, meaning forcing one standard here may not satisfy current or future orchestration platforms. The key/value pairs of a selector would be different between orchestration platform A and B.

• Introducing a global option as proposed complicates the core security model. Promoting the use of the workload attestation plugins allow each specific integration to have its own contained security model with tradeoffs and compensating controls distinct from the core. For example, lets say that orchestration platform A, has an agent B that lives on every node. B can provide custom attributes to workloads it launches (possibly by a JWT in a specific location for every workload). Workload attester B would have to have some compensating control to trust attributes provided by B, and the larger threat model for the system would have to provide compensating controls to trust that B is only orchestrated by A. How these controls are layered should be opinionated by the creators of A, B, and the workload attestation plugin for B, lending itself to the plugin model that already exists.

[evan2645]

Regarding Mesos attestation, it's a bit tricky but I think the APIs exist to build out a plugin for it.. I'm looking into that but happy to contribute it back to the project!

That would be awesome ❤️ 🙌

[APTy]

Thanks for discussion all. I'm going to close this for now until there's a more defined use case