Is this responsible disclosure?

[aytey]

As a maintainer of one repo that has a PR request automatically created by you, I am wondering if the process that was followed was to announce these issues was "responsible"?

The text on the PR was quite direct:

First question, do we need to perform vulnerability disclosure? It depends!

Is the vulnerable code only in tests or example code? No disclosure required!
Is the vulnerable code in code shipped to your end users? Vulnerability disclosure is probably required!

For the people that are affected by the final line, should there not have been some element of "responsible disclosure" of these issues, before the PRs were created for anyone to see?

To be clear: notifying people about these kind of issues is great (so I have no issue with you finding the problem in our code base); the concern I have here is that the metaphorical cat is immediately "out of the bag" after you raised the PR, giving us no chance to deal with the issue in a timely and "secure" manner.

See, e.g., https://en.wikipedia.org/wiki/Coordinated_vulnerability_disclosure

[JLLeitschuh]

Hi,

Thank you for reaching out, I really do appreciate it.

No, you are correct, this isn't "responsible" or "coordinated" disclosure. It's what's called "full disclosure". I agree that it's less than ideal, but I'm working on identifying and fixing widespread common security vulnerabilities in Open Source Software.

The problem is that as an open source software security researcher, I (exactly like Open Source maintainers), I only have so much time. I'm able to find vulnerabilities impacting hundreds, or sometimes thousands of open source projects with tools like GitHub code search and CodeQL. The problem that I have is that my knowledge of vulnerabilities doesn't scale very well.

Individualized vulnerability disclosure takes time and care. It's a long and tedious process, and I have a significant amount of experience with it (I have over 50 CVEs to my name). Even tracking down the reporting channel (email, Jira, ect..) can take time and isn't automatable. Unfortunately, when facing prblems of this scale, individual reporting doesn't work well either.

Additionally, if I just spam out emails or issues, I'll just overwhelm already over taxed maintainers, I don't want to do this either.

By creating a pull request, you are proving a maintainer something highly actionable to actually fix the identified vulnerability. The pull request.

If github gave me a way to create a private pull request or issue with this vulnerability information, I'd jump at it, but unfortunately, that's not currently possible.

I've thought a lot about the best way to scale my knowledge of security vulnerabilities to have the highest net-positive on the open source community and arguably more importantly, down stream consumers of that software. I believe that the benefits of fully disclosing a vulnerability in this way, specifically due to the scale I'm operating at, when attempting to mediate widespread vulnerabilities like this, outweigh the potential risks, especially when the alternative is, in most cases, no disclosure at all.

[aytey]

Do you think maybe it would have been better to create an issue saying "your repo has a potential security vulnerability, I don't want to announce this publicly, can you reach-out to me? If I don't hear from you in XX days, I'll create a PR for that fixes this issue".

Surely this gives people the time and opportunity to reach-out to you, and if they don't, you can then automatically create the PRs?

I'm not against what you're doing at all, I just think this is a very brutal way of doing it and is potentially unfair to the open-source maintainers that have now had their laundry "aired in public", before they have time to arrange fixes/releases resolving the issues you found.

[JLLeitschuh]

Do you think maybe it would have been better to create an issue saying "your repo has a potential security vulnerability, I don't want to announce this publicly, can you reach-out to me? If I don't hear from you in XX days, I'll create a PR for that fixes this issue".

When dealing with hundreds or potentially even thousands of these projects, I'd have to build infrastructure to handle all of this coordination. This is a good end-goal to attempt to achieve, but the infrastructure isn't there.

Also, I updated my message above with some more context. Might be worth a re-read.

I really appreciate the dialogue here.

[aytey]

When dealing with hundreds or potentially even thousands of these projects, I'd have to build infrastructure to handle all of this coordination. This is a good end-goal to attempt to achieve, but the infrastructure isn't there.

So, basically, you're saying "I'd rather make pain for you (the maintainer) than pain for me (in building the infrastructure)"?

Again, nothing wrong with that, I just want to make sure we all understood that you've taken this approach to make things easier for you rather than easier for the maintainers (but, with the hope that, overall, you make the world a better place).

[JLLeitschuh]

Again, nothing wrong with that, I just want to make sure we all understood that you've taken this approach to make things easier for you rather than easier for the maintainers (but, with the hope that, overall, you make the world a better place).

To reiterate, there's only one of me, and there are hundreds, or possibly thousands of maintainers sending these PRs to. Making it easier for all maintainers would make it impossible for me.

Another problem is, if I generate hundreds of issues for a single issue all at the same time, as soon as one project discloses the presence of that vulnerability in their software, any attacker could deduce that that that same vulnerability probably also existed in all the other projects I reported to.

Further, almost all of the information about the list of projects vulnerable to this is already public information on LGTM.com:
https://lgtm.com/rules/1515014784717/alerts/

In 80% of the case, I write a CodeQL query for my research, and get it merged, before doing bulk pull request generation. I suggest enabling the CodeQL GitHub action on your repositories you'd like to get alerted about my vulnerability research before I start attempting bulk PR generation.

[aytey]

I suggest enabling the CodeQL GitHub action on your repositories you'd like to get alerted about my vulnerability research before I start attempting bulk PR generation.

Would it have made sense then for you to look at adding a PR to all those repos that enabled CodeQL (I'm guessing this is "as simple" as adding a GitHub action as part of a PR)? That way, people will find and learn about not only the issue you found, but all the other issues that CodeQL can check for?

I agree that this then leaves the fixing up to the maintainers (or you could also open-up your "fix" PR), but at least it then gives maintainers the opportunity to learn about and diagnose issues following their own "process", rather than having to deal with it in the very "urgent" way that your PRs have led to.

[pjfanning]

Many Github projects provide email addresses in the SECURITY.md, that could be scraped and an autogenerated git patch files with an autogenerated issue description could be emailed instead. You might need to follow links in the SECURITY.md to get to the page with the email address. For example, https://github.com/apache/xmlbeans/blob/trunk/SECURITY.md has a link to https://www.apache.org/security/ - this latter page has the [email protected] email address. For ASF projects, you could just hardcode that email address.

Github also has its security advisory feature. You could create draft advisories in your forks and notify the team by emailing them using the approach described above. This is an alternative to emailing the git patch.

These approaches avoid disclosing the issue publicly.

[JLLeitschuh]

Github also has its security advisory feature. You could create draft advisories in your forks and notify the team by emailing them using the approach described above. This is an alternative to emailing the git patch.

This is not a great long-term solution as I hope that GitHub will soon be allowing private disclosure. The biggest problem here is that GitHub security advisories don't have an API to create them. As such, it's impossible to automate the creation of GitHub Security Advisories without using something like Selenium. However, if you use selenium, I presume that, at-scale, GitHub would detect any bot operating on their UI in this way and ban it.