GSoC 2020 Project Idea: Add new checkers to the CVE Binary Tool

[terriko]

The CVE Binary tool team is hoping to participate in Google Summer of Code (GSoC) under the Python Software Foundation umbrella. You can read all about what this means at http://python-gsoc.org/. This issue, and any others tagged 'gsoc' are not generally available bugs, but related to project ideas for GSoC.

Project Idea : Add new checkers to the CVE Binary Tool

Project description: The CVE Binary Tool, at launch, only had checkers for 10 different open source libraries. But if you look through our issues (search for new checkers) you can see that there are lots of others we'd like to have. For this project, you'll implement as many of those as you can, including producing tests. (This was also a project idea last year but there are always new checkers!)

(Don't worry that we're going to run out of checkers -- if we exhaust the current list, we can definitely find more to add.)

Skills: Python, git, experience reading source code for other languages a bonus

Difficulty level: Easy

Related Readings/Links: How to add new checkers

Potential mentors: @terriko @pdxjohnny

Getting Started: Python requires that all students submit a code sample as part of your application. For your first pull request, we recommend you write a new test case for an existing checker. The easiest one to do as a first pull request is a scanner test for a new version of a library we can already detect, but there are there are two types of easy tests you might want to try: CVE mapping test and CVE file test. Note: the way we add tests has changed recently, so please make sure to read the instructions!

Here's the instructions for the easier mapping test:

• Existing tests are in test/
• You can see the scanner tests in 'tests/test_scanner.py'
• To add a new one, make a new test case that detects a few CVEs known to occur in this version, and a few that are known not to occur in that version. For example, this is how the current tests look like. You should add the details of the new test case in the @pytest.mark.parametrize decorator of test_binaries test::

    @pytest.mark.parametrize(
        "binary, package, version, are_in, not_in",
        [
            (
                "test-bluetoothctl-5.42libbluetooth.so.out",
                "bluetoothctl",
                "5.42",
                [
                    # for known CVE
                    "CVE-2016-9797",
                    "CVE-2016-9798",
                    "CVE-2016-9799",
                    "CVE-2016-9800",
                    "CVE-2016-9801",
                    "CVE-2016-9802",
                    "CVE-2016-9803",
                    "CVE-2016-9804",
                    "CVE-2016-9917",
                    # "CVE-2016-9918",
                ],
                [
                    # for older version
                    "CVE-2016-7837"
                ],
            ),
            (
                "test-icu-3.8.1.out",
                "international_components_for_unicode",
                "3.8.1",
                ["CVE-2007-4770", "CVE-2007-4771"],
                ["CVE-2019-3823"],
            ),
            (
                "test-icu-dos.out",
                "international_components_for_unicode",
                "3.8.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1.1",
                ["CVE-2014-9911"],
                ["CVE-2019-3823"],
            ),
            (
                "test-curl-7.34.0.out",
                "curl",
                "7.34.0",
                [
                    "CVE-2019-3823",
                    "CVE-2018-14618",
                    "CVE-2017-1000101",
                ],  # CVE-2017-1000101 needs supplemental data
                [],
            ),  
            .....
            .....
            .....
    def test_binaries(self, binary, package, version, are_in, not_in):
        self._binary_test(binary, package, version, are_in, not_in)

• Not sure what CVEs apply to a version? The checkers themselves often have links, and have the vendor/product pair so you can find them in the national vulnerability database as well. Here's a link to the openssl checker for you to look at and the rest are in cve-bin-tool/checkers/
• You'll also need to make a fake file designed to trick the checker into thinking it has found that version of the library. You can see these files in test/binaries/
• Please note that sometimes the database we're using doesn't have perfect mapping between CVEs and product versions -- if you try to write a test that doesn't work because of that mapping but the description in the CVE says that version should be vulnerable, don't discard it! Instead, please make a note of it in a github issue can investigate and maybe report it upstream.

Extra credit: Got your test working and want to try something more? You can also try adding a checker before the project starts. See the related readings above for instructions.

[terriko]

Some places where we are missing tests that you might want to try first: #274 #273 #271 #270 #237.

[SinghHrmn]

Hi, I am interested to work on this @terriko

Can I take this as my GSoC 2020 project?

[terriko]

@SinghHrmn The way gsoc works is that we give you some ideas, like this one, then you submit a project proposal through Google's system. We choose students once we've seen all the proposals submitted through Google. So yes, you're most defeinitely welcome and encouraged to use this as the basis for your proposal, but so is everyone else up until the point where the submission system closes.

[k-udupa2000]

@terriko Is there a possibility that this project is not selected for gsoc this year in a situation where proposals with other ideas are submitted?

[terriko]

@k-udupa2000 All of GSoC decision making is based on the distribution of our best applicants. If one of our best applicants wants to do this project, it'll get selected.

[terriko]

In more detail:

What happens behind the scenes is that after applications close, we read all of the submissions and decide how many slots to request based on which proposals and students we want to work with. Since we're a sub-org of Python, our numbers get compiled together with numbers from other sub-orgs, and then Google gives Python a number to distrubute as the Python org admins see fit.

Basically what I'm trying to say is that slots aren't really tied to projects, they're tied to students.

If we get an amazing student who wants to do this project, we'll almost certainly get a slot and it'll happen. If we get a good but not outstanding student, we might get a slot (or it might go to an amazing student in another sub-org). If we get no viable students, then the project doesn't happen. This is true of basically any project in GSoC -- the projects only happen if there's a student people want to work with. If we get two amazing students for a single project, sometimes we can split it up into multiple projects and sometimes we can only take one.

Incidentally: "amazing" doesn't mean technical ability alone. There have definitely been cases across GSoC where a project wasn't filled because the top student was technically adept enough but so unpleasant to work with that none of the mentors was willing to commit time to work with him. And vice versa, there have been cases where the "amazing" student still had plenty to learn but were such a good community member that the mentors argued that they'd be happy to spend extra time with them.

[twentyse7en]

@terriko That was a excellent explanation. It really helped me to get a overall picture about the selection process. I was really struggling to get started.

[SaurabhK122]

@terriko, could you like give an idea as to how many checkers have to be added as part of the project?

[terriko]

We're done with gsoc 2020 and we have a lot of new checkers thanks to @SaurabhK122 now. There's always room for more, but I'm going to close this project idea as complete for 2020.