Temporary ignoring of vulnerabilities

[KeithWM]

First of all, thanks for the great package. It is a very useful tool in keeping our code safe, much appreciated.

My feature request is inspired by the fact that we are using pip-audit in our CI/CD pipeline and this means that if a vulnerability is detected, this has to be dealt with immediately. Sometimes there is no "proper" fix available yet and we resort to simply ignoring the vulnerability with --ignore-vuln. This doesn't bother me much, but I would prefer it if this ignore could be made temporary, so that we will be forced to revisit the issue at a later date.

Keen to hear your thoughts on this idea. One of our team might even be able to contribute to the implementation, if desired. I could also just be not seeting some existing feature that would solve our problem.

[woodruffw]

Hi @KeithWM, thanks for the feature request!

Out of curiosity: is your CI/CD pipeline built on GitHub Actions and, if so, are you using pypa/gh-action-pip-audit?

The reason why I ask is because this kind of feature might be a better fit for the action, rather than pip-audit itself -- pip-audit itself currently has no notion of its own time, and encoding "fix by" metadata into its CLI arguments might be a little bit clunky 🙂

[KeithWM]

Hi Woodruff,

No, we are using Bitbucket pipelines.

How would you build the feature into the github action? Maybe a similar idea would be possible for Bitbucket pipelines.

[woodruffw]

How would you build the feature into the github action? Maybe a similar idea would be possible for Bitbucket pipelines.

For GitHub Actions, we could probably do something like this:

- uses: pypa/gh-action-pip-audit@v1.0.8
  with:
    ignore-vulns: |
      PYSEC-AAAA-BBBB:YYYY-MM-DD
      CVE-AAAA-BBBB:YYYY-MM-DD

i.e. each ID listed under ignore-vulns could be listed with an optional YYYY-MM-DD date, after which the ignore rule is no longer respected.

That being said, I find that pretty clunky, and I'm not a huge fan of adding syntax to the action that isn't also present in the CLI 🙂

Ultimately, this kind of statefulness might be outside of pip-audit (and the action's) scope: I think the "right" way to do this might be to have some additional periodic workflow in your CI/CD that checks a custom JSON/YAML/whatever file you define, and invokes pip-audit with the appropriate --ignore-vuln flags depending on the date. Do you think that would work for your case?

[KeithWM]

I think you're right that it is better for pip-audit to be agnostic of time and date. In Bitbucket pipelines, I think we could do some shell scripting to first populate a list of --ignore-vuln options based on the datetime and then add these (if any) to the pip-audit call.

Maybe a feature that could lie within pip-audit is that you can specify to ignore a vulnerability until a fix is available?

[woodruffw]

Maybe a feature that could lie within pip-audit is that you can specify to ignore a vulnerability until a fix is available?

Yeah, this is possible (although we've had some design concerns in the past about allowing users to ignore vulnerabilities by default). #243 has some more context 🙂

TL;DR there is that users might pass --ignore-unfixed or similar just to get things working, and then subsequently ignore serious vulnerabilities that are "unfixed" but can be remediated in other ways (like outright removal, or downgrading, etc.).

Edit: and #209 is likely still the route we'll want to take there, although I haven't re-evaluated the TOML landscape recently to see if pip has vendored a parser yet 🙂

[KeithWM]

Good points. There is a big difference though between a blanket --ignore-unfixed and a --ignore-vuln-until-fixed ABC-123. Even then, that could allow packages that never fix their vulnerabilities to remain ignored forever. That's why I'd like the user to be able to set a date, thereby actively acknowledging how long they are willing to run the risk posed by the vulnerability. I did come up with some scripting that would (probably) work in our Bitbucket CI/CD, but it's really clunky.

[woodruffw]

Yeah, none of this is ideal 😞

Thinking about it some more, --ignore-vuln-until-fixed <ID> probably isn't the worst we could do here -- there's a bunch of user demand for their feature, and having it be per-ID rather than a blanked flag (like --ignore-unfixed would be) covers most of our concerns.

CC @di for thoughts -- I'm pretty ambivalent 🙂

[KeithWM]

I don't like the "until fixed" option at all, it puts you at the mercy of the maintainers of the dependency package. I've made a quick mockup of what I think might be nice in a fork. I will created PR #679 as a suggestion.

EDIT: Also created PR #680, which is probably the tidier implementation.