Exit codes

[di]

Right now, pip-audit exits with an exit code of 0 regardless of whether the audit has passed or failed:

$ pip-audit
- Auditing zipp (3.4.1)
Package  Version ID             Fix Versions
-------- ------- -------------- ------------
py       1.9.0   PYSEC-2020-92  1.10.0
Pygments 2.6.1   PYSEC-2021-140 2.7.4
Pygments 2.6.1   PYSEC-2021-141 2.7.4
PyYAML   5.3.1   PYSEC-2021-142 5.4
sqlparse 0.3.1   PYSEC-2021-333 0.4.2
urllib3  1.25.9  PYSEC-2021-108 1.26.5

$ echo $?
0

Ideally, pip-audit would exit with a nonzero exit code if vulnerabilities were found.

[woodruffw]

Sounds good. We can also make our exit codes slightly semantic (1 for exactly one vuln found, 2 for more than one?) if that's desirable.

[di]

We can also make our exit codes slightly semantic (1 for exactly one vuln found, 2 for more than one?) if that's desirable.

I think it might eventually make sense to have different nonzero exit codes, although I'm not sure making a distinction between the quantity of vulnerabilities would be useful. For now, let's just support 0 for no vulnerabilities, 1 for one or more vulnerabilities.

We should also think about this in the context of #82 as well.

[irishismyname]

Can you recommend a way to bypass this for pipeline jobs? Exit code 1 is preventing us from uploading the CycloneDX report as a job artifact in GitLab.

[woodruffw]

The easiest thing to do is probably to swallow the exit code entirely:

pip-audit ... || exit 0

That should keep your CI from exiting prematurely. Longer term, we could think about making the current exit code behavior optional if it's something multiple users would like 🙂