May 2, 2017 • RBS

Categories: Security News

The first thing we did when starting this blog series was to reach out to the CVSS SIG mailing list to find out if there had been any detailed analysis of CVSSv2 vs CVSSv3 base scoring. We were pleased to get a response from Cisco, explaining that in April 2016, Omar Santos had written a  blog post called “The Evolution of Scoring Security Vulnerabilities“.

Here are some of the key points from the post:

  • The study analyzed the difference between CVSSv2 and CVSSv3 scores using the scores provided by the National Vulnerability Database (NVD). A total of 745 vulnerabilities identified by CVEs and disclosed in 2016 were analyzed.
  • The goal was to identify the percentage of vulnerabilities that had a score increase or decrease, based on the two versions of the protocol (CVSSv2 vs. CVSSv3).
  • Score Increase from Medium to High or Critical
    • 144 vulnerabilities increased from Medium to High or Critical. That represents 19.33% of all studied vulnerabilities and 38% of the 380 Medium-scaled vulnerabilities (under CVSSv2 scores). The average base score of these vulnerabilities was 6.1 with CVSSv2 with an increase to an average base score of 8.2 when scored with CVSSv3.
  • Score Increase from Low to Medium
    • 35 vulnerabilities increased from Low to Medium. That represents only 4.7% of all studied vulnerabilities, but 88% of the 40 Low-scaled vulnerabilities (under CVSSv2 scores). The average base score of these vulnerabilities was 3.0 with CVSSv2 with an increase to an average base score of 5.5 when scored with CVSSv3,

In the conclusion of the post, Omar Santos states: ”The CVSS enhancements mean that we will see more vulnerabilities being rated as high or critical throughout the security industry.”

At the end of October 2016, Omar Santos published a follow-up post called “The Evolution of Scoring Security Vulnerabilities: The Sequel

Here are some of the key points from the post:

  • The total number of vulnerabilities studied was 3862. These were vulnerabilities disclosed from January 1, 2016 thru October 6, 2016 and the source of the data is NVD.
  • The average base score increased from 6.5 (CVSSv2) to 7.4 (CVSSv3).
  • 44% of the vulnerabilities that scored Medium in CVSSv2 increased to High when scored with CVSSv3.
  • 28% of the vulnerabilities that scored High in CVSSv2 increased to Critical when scored with CVSSv3.
  • 1077 vulnerabilities moved from Low or Medium to High or Critical. That is a 52% increase in High or Critical vulnerabilities.

We were quite pleased to see the work that Omar Santos published and the amount of included details. We were initially concerned that doing any comparison of CVSSv2 vs CVSSv3 by relying on NVD scores would futile. The reason is that we have scored over 15,000 vulnerabilities in each of the past two years in our VulnDB product and have seen that NVD has scored a lot of issues incorrectly or inconsistently over those years with CVSSv2. We have noticed that they continue to score some vulnerabilities incorrectly using CVSSv3 as well.

Prior to reading Omar Santos’ articles, we were brainstorming our own ideas and trying to determine what we’d look at in the analysis when comparing CVSSv2 to CVSSv3 scoring provided by NVD.

  • How many are the exact same score?
  • What percentage of the vulnerabilities have the same Impact score?
  • What percentage of the vulnerabilities have the same Exploitability score?
  • What percentage of vulnerabilities stay in the same range (Low, Medium, High, Critical)?
  • On average how far different are the scores?
  • How many of the ratings (Low, Medium, High, Critical) are the exact same?

We wanted to know if we could add any value to the scoring conversation and decided to take a look at scoring for all of 2016. In December 2015, NVD announced that they started scoring with CVSSv3. Prior to any conducting any analysis, it was important to understand how NVD does their scoring.

NVD Vulnerability Severity Ratings

NVD provides severity rankings of “Low,” “Medium,” and “High” in addition to the numeric CVSS scores but these qualitative rankings are simply mapped from the numeric CVSS scores:

CVSS V3 Ratings

  1. Vulnerabilities are labeled “Low” severity if they have a CVSS base score of 0.0-3.9.
  2. Vulnerabilities will be labeled “Medium” severity if they have a base CVSS score of 4.0-6.9.
  3. Vulnerabilities will be labeled “High” severity if they have a CVSS base score of 7.0-8.9.
  4. Vulnerabilities will be labeled “Critical” severity if they have a CVSS base score of 9.0-10.0.

CVSS V2 Ratings

  1. Vulnerabilities are labeled “Low” severity if they have a CVSS base score of 0.0-3.9.
  2. Vulnerabilities will be labeled “Medium” severity if they have a base CVSS score of 4.0-6.9.
  3. Vulnerabilities will be labeled “High” severity if they have a CVSS base score of 7.0-10.0.

Incomplete Data

With some vulnerabilities, all of the information needed to create CVSS scores may not be available. This typically happens when a vendor announces a vulnerability but declines to provide certain details. In such situations, NVD analysts assign CVSS scores using a worst case approach. Thus, if a vendor provides no details about a vulnerability, NVD will score that vulnerability as a 10.0 (the highest rating).

When looking closer at the data, we can see that NVD started scoring CVSSv3 with CVE-2015-6934, which was published on December 20, 2015 at 10:59:00 PM. That initially had us believe that we would have a full year of dual scoring for a 2016 analysis.

Once we started analysis, we saw something that caused us some immediate concern with the plan: For some unexplained reason, NVD did not fully score all vulnerabilities using both CVSSv2 and CVSSv3 from that point onward. 2016 started out with dual scoring and then suddenly there were gaps starting at CVE-2016-0401, which only provided CVSSv2 scoring for that particular vulnerability.

The good news was that it seemed to quickly get back under control with almost all CVEs having both CVSSv2 and CVSSv3 scores, so we continued on with our plan. In doing a full analysis for all of 2016 we saw that NVD scored the following:

  • CVSSv2 – 5,135 vulnerabilities
  • CVSSv3 – 4,929 vulnerabilities

While not every vulnerability published by NVD in 2016 had both scores, it was determined that just 209 vulnerabilities were missing CVSSv3 scoring. We were pleased to discover this, as we felt it gave us a decent sampling of both scores for further analysis of a few points.

The following is a distribution of all CVSSv2 and CVSSv3 scores from 2016:

The chart immediately aligned with what we expected to see based on our analysis of the new standard. It confirmed that with a larger sample size Omar Santos’ findings are still true. The changes to CVSSv3 has increased the overall base scoring of vulnerabilities based on the numbers from 2016.

Low 447 142
Medium 2,622 1,705
High 2,066 2,188
Critical 894

To look at the data in another view, here is a quick bar chart of the same data:

Here we can see the percentage breakout:

Low 8.70% 2.88%
Medium 51.06% 34.59%
High 40.23% 44.39%
Critical 18.14%

What did we see from the 2016 analysis?

  • Low severity vulnerabilities decreased by 5.82% (only 142 vulnerabilities!) when scoring CVSSv3
  • Medium severity vulnerabilities decreased by 16.47% when scoring CVSSv3
  • High severity vulnerabilities increased by 4.16% when scoring CVSSv3
  • Critical severity vulnerabilities increased by 18.14% when scoring CVSSv3
    • Since Critical didn’t exist in CVSSv2, it had to increase! =)

The initial reactions that some may have are:

  • So what… what is the big deal that scores have increased!
  • Isn’t it a great thing that the base scores have increased?
  • Doesn’t this make sure that vulnerabilities are fixed quickly?

Kymberlee Price discussed prioritization and how it matters in a 2015 Black Hat talked titled “Stranger Danger! What Is The Risk From 3rd Party Libraries?” One of the main points she discussed applies very much so to the increased base score ratings.

In the 2016 analysis, using CVSSv3 we see that High and Critical severity vulnerabilities account for 3,082 vulnerabilities (62.53%). We also note that almost no vulnerabilities are scored as Low severity (only 2.88%).

CVSSv3 Scoring Impacts

In August 2007, the Payment Card Industry Data Security Standard required the use of “the NVD Common Vulnerability Scoring System impact scores for use within approved scanning vendor tools.” In the document, it states the following about CVSS.

Generally, to be considered compliant, a component must not contain any vulnerability that has been assigned a CVSS base score equal to or higher than 4.0.

So we have to consider that PCI compliance generally dictates a failure if any vulnerabilities with a CVSS score of 4.0 or above are found. Based on that requirement and using CVSSv3, in order to be PCI compliant, an organization would have to address more than 97% of the vulnerabilities reported in 2016!

While security is important, and most organizations appear to be focused on fixing issues more than ever before, the reality is that there is only so much time to invest into security patching.  System administrators are asking security teams to help prove that the issues they are raising are really required to be dealt with so quickly rather than waiting for routine maintenance windows to be addressed.

The analysis also underlines a problem discussed in previous blog post: There will rarely be a vulnerability with a remote attack vector that in the real-world is considered Low severity, but  actually is also rated Low in CVSSv3.

In fact, this was already a problem with CVSSv2. While a minor local vulnerability may fall into the Low severity range, almost no minor vulnerabilities with a remote vector do. CVSSv3 has now made this serious failing of the CVSS scoring system even worse.

Consider a basic vulnerability that allows disclosing the version of an installed product. These are borderline vulnerabilities; many security practitioners do not consider them vulnerabilities, but the industry has generally decided that disclosing such information is bad security practice and should be considered a minor security weakness. Most would agree that such an issue should score as Low severity, but this is the CVSSv2 and CVSSv3 scores for such weaknesses:

CVSSv2: AV:N/Au:N/AC:L/C:P/I:N/A:N = 5.0 (Medium)

CVSSv3: AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N = 5.3 (Medium)

There are many similar weaknesses with remote attack vectors that also end up in the Medium severity range instead of Low. But as mentioned, very few weaknesses with remote attack vectors are able to score as Low severity.

It can been argued and seen as a huge problem that a standard, which is entirely focused to help prioritize vulnerabilities with levels ranging from Low, Medium, High, and Critical in real-world cases rarely will score a vulnerability as Low! It could even be further argued that the standard is broken as it does not truly help organizations understand and prioritize the most critical vulnerabilities that are disclosed.

Scoring concerns aside, next up we will discuss some of the things that we really like about CVSSv3.

CVSS – Is Version 3 All Bad?

About Risk Based Security

Risk Based Security (RBS) provides detailed information and analysis on Vulnerability Intelligence, Vendor Risk Ratings, and data breaches. Our products, Cyber Risk Analytics (CRA), VulnDB and YourCISO, provide organizations access to the most comprehensive threat intelligence knowledge bases available, including advanced search capabilities, access to raw data via API, and email alerting to assist organizations in taking the right actions in a timely manner.

Our products
The Platform
Risk Based Intelligence
Learn more
Vulnerability Intelligence
Learn more
Cyber Risk Analytics
Threat Intelligence
Learn more
Risk Management
Learn more