Support secret validation

[killuazhu]

Right now, all the tokens / secrets found are potential secret which require human to audit each of them carefully.

Would this project be interested in supporting or accepting PR w.r.t token validation?
For example, for a potential token we found, let's say a slack token, the tool can provide an option to actually extract the token and hit the service endpoint to validate if the token is working or not.

The validation and working status can be reflected in the final report. During audit, these information can be used to prioritize the review sequence.

[KevinHock]

That's a great idea @killuazhu, I would definitely support and accept that. It'd probably have to be under --some-flag aside from that I'd be excited to have that.

[domanchi]

I'm curious -- how would you define which endpoints to query, to validate a secret?

[killuazhu]

I'm curious -- how would you define which endpoints to query, to validate a secret?

Very good questions. The endpoint can be considered as one factor for a multi-factor token. To properly validate a token, you need to extract all the factors from the source code.

If a token has a very specific format, like AWS access key id, you might be able to infer the endpoint from that token type. Otherwise, you would need to look at the context (for example, 5 lines next to the token line) to extract all the factors including endpoint. For some other cases like basic auth, you should be able to extract the endpoint from the URL.

[domanchi]

Putting here for posterity:

It's going to be hard to do this for AWS credentials, because we look for the ACCESS_KEY_ID as a canary for the actual secret. But with both, you can do:

import boto3
import botocore

def is_valid(key, secret):
    client = boto3.client(
        'sts',
        aws_access_key_id=key,
        aws_secret_access_key=secret,
    )
    try:
        client.get_caller_identity()
        return True
    except botocore.exceptions.ClientError:
        return False

[killuazhu]

That would be a good example of multi factor token. Once your extract both of them, you will be able to validate.

[domanchi]

#194 addresses the scaffolding for this.

[domanchi]

I'm leaving this ticket open, because there are two more things that IMO needs to be done before secret validation is truly an option:

• : logic to merge baselines for compatibility (since Verifiable secrets scaffolding #194 introduces a is_verified key in the baseline). This should just go through the secrets listed in the baseline, and add a key is_verified: False to each of them.
• : verify argument in CLI usage to verify secrets in the baseline post-hoc. This would allow for complete upgrades of old baselines, since you would be able to do:

$ detect-secrets verify .secrets.baseline

[dgzlopes]

Found this repo today. Contains ways to check if API keys are valid. Leaving it here in case it's useful!

[KevinHock]

That looks super useful @dgzlopes, thanks!

[domanchi]

With the v1 migration, when the baseline is upgraded, the verification status of a secret is automatically set to false. I have added @dgzlopes' link to the documentation, so it is more discoverable than buried in an issue's comment.

The verify CLI option is still a possibility, but I will open a new issue to track that. Otherwise, closing this issue as completed.