A Scanning Solution is Only as Good as the Vulnerability Data That Drives it
June 26, 2019 • RBS
We had some great conversations at JFrog’s user conference, SwampUp 2019. Brian Martin, our Vice President of Vulnerability Intelligence, took part in an all-star keynote of experts where he discussed how our VulnDB® service helps secure JFrog Xray user pipelines.
The integration of VulnDB allows DevOps teams to discover, receive notifications on, and help remediate vulnerabilities in third-party libraries and dependencies early in the development cycle. As JFrog puts it, “a security scanning solution is only as good as the database of vulnerabilities that drives it.” Driven by Risk Based Security’s comprehensive data, Xray with VulnDB is the best security intelligence solution on the market for developers.
Why does data from VulnDB give you an edge?
VulnDB is the most comprehensive source of vulnerability data available, with almost 69,000 vulnerabilities that are not found in CVE or the National Vulnerability Database (NVD). As Brian shared in his SwampUP keynote presentation, an average of about 70 new vulnerabilities are disclosed every day. This is an alarming volume, especially if your organization isn’t seeing the complete picture. That’s why our rallying cry is #BetterDataMatters. VulnDB is so much more advanced than any other database because we are looking for vulnerabilities and we speak with the DevOps community to ensure we are monitoring the libraries they are using. VulnDB includes more vulnerabilities, and carries more metadata and research on entries. This allows you to arm your organization with the most complete and up-to-date information available so you can make data-driven decisions to effectively manage and prioritize risk mitigation.
Taking this to mind, let’s look at some real-world applications. Recently, Sophos put out a very thought provoking article. The article made some very interesting points:
- Most vulnerabilities aren’t exploited, and if they are, they tend to have a high CVSS score.
- There is apparently no relationship between the proof-of-concept (PoC) exploit code being published online and the start of real-world attacks.
- In order to patch vulnerabilities, a “reference tagging” machine learning model is the most efficient method.
Sophos based their conclusions on data provided in a whitepaper that researchers from Cyntia, Virginia Tech, and the RAND Corporation published. The findings were extremely engaging, however, the data used to support these claims is lacking…comprehensiveness.
Looking further into the data provided, it’s apparent that the researchers relied very heavily on security sensors based on CVE IDs, meaning that only vulnerabilities within CVE were being considered. This means that there are almost 69,000 vulns being missed in this study. To make matters worse, security scanning devices tend to cover half of the vulnerabilities in CVE, which makes the subset of data even smaller.
In addition, Risk Based Security believes the machine learning models used in the study may have mis-categorized focused attacks. In situations where someone determines that a remote target is running specific software, then tries a comprehensive list of attacks against it, a detected attack would likely be labeled incorrectly. Since the researchers were basing their findings off of CVE data, it is highly likely that their sensors were not aware of specific vulnerabilities, resulting in a label of “Generic XSS” for example. This could have skewed results.
Last, these type of reports typically don’t share their full methodology, let alone what they are capable of matching against. This means that there is no way to reproduce or validate their findings. Unfortunately, if the research is solely based on CVE data it means that several vendors performing a similar study will also provide the same rough figures. CVE is the industry “standard,” yet it is missing a huge amount of vulnerabilities, with many of them possessing high CVSS scores and affecting major vendors. As previously stated, “a security scanning solution is only as good as the database of vulnerabilities that drives it.”
#BetterDataMatters. We would be very interested if the findings presented would be the same if more up-to-date data was used.
Interested in learning more about Risk Based Security’s suite of products? Click here to schedule a demo and see how much more comprehensive VulnDB is compared to other security solutions on the market.