The Slow Burn of Meltdown and Spectre: Exploits, Lawsuits, and Perspective
January 9, 2018 • RBS
Last week, we published our initial blog about the recently disclosed Meltdown and Spectre vulnerabilities. In that post we focused on the perils of rushing patches, attribution and the disclosure process, research collisions, and the impact on cloud offerings. In this update, we continue to examine higher-level aspects of the disclosures a week after the initial fallout.
Patch and Performance
Perhaps a bigger news aspect than the vulnerabilities themselves, were reports saying that patches to mitigate these flaws may slow down processors by up to 30%. At the same time, Intel claimed that “performance impact of these updates is highly workload-dependent and, for the average computer user, should not be significant”. Now that administrators have had time to monitor systems after various vendor patches, we’re seeing a more accurate report of the impacts.
With many organizations using cloud services for a significant part of their public facing offerings, many have been especially concerned about the patches’ potential hit to performance, which could increase monthly costs dramatically. The graph above comes from Ian Chan who noticed the spike after the patches were presumably deployed to their AWS EC2 instances, and noticing that the spikes range from 5 to 20%. Amanpreet Singh shared a graph showing his Redis / Elasticache install on AWS Managed Services, with a similar look. Similarly, Ruben Berenguel noticed spikes but points out that in addition to the slow down, the costs for the services will go up approximately 10% for the month. This is one aspect of patching in the cloud that many organizations have overlooked. In his case, they are noticing new instances spawning because the current ones can’t handle the workload. Companies that use the cloud for public facing services, such as the situation at Epic Games, may face login issues and service instability after the patches.
For traditional on premise servers and home machines, the initial benchmarks and testing are also showing significant spikes as well. Peter Czanik mentioned increased compile times on his Fedora system, where syslog_ng went from a 4 minute compile time to a 21 minute compile time. He also mentions that he believes compiling Java is the most affected. The Guru of 3D blog did extensive testing and benchmarks on a Windows 10 machine. Their results show that while there are noticeable changes, some aspects of the system rumored to be impacted (e.g. memory performance) are not showing any real difference, while other aspects like File I/O and disk performance are taking a hit. They go on to test browser performance and the results indicate minimal changes even under heavy usage, while gaming / GPU performance shows negligible change before and after the patch.
There are mixed results for one-off users doing CryptoCurrency mining. One particular Reddit thread has users suggesting there could be significant performance hits while others do not see the same impact. Regardless, even a small performance hit on a mining system could mean serious financial implications and in some cases make some systems not profitable (e.g. electricity consumption costs exceed currency output). The thread also reminds users to think about threat modeling and consider if they even need the patches, while others say that configuration options could help offset the post-patch performance hits. Kirk Pepperdine has a great summary about PCID (Process-Context Identifiers) functionality and how that may help minimize patch performance issues.
Finally, as noted in our first blog, the Meltdown patch is still causing serious compatibility issues with some software. The latest is Microsoft’s Meltdown patch, which appears to break the PulseSecure VPN client. Microsoft’s patches seemingly play poorly with AMD-based PCs and has been confirmed as causing performance dips on older versions of Windows.
Exploitation, Detection, and Mitigation
Coming to grips with Meltdown and Spectre reminds us of a fundamental truth – focusing on how to respond to the vulnerabilities is critical. Understanding the risk from an exploitation angle is frequently first on the mind. The time between a vulnerability disclosure and the time for someone to publish a functional exploit, often referred to as Total Time to Exploit, is one metric we track in VulnDB. Unsurprisingly, the time for working proof-of-concepts / exploits to be released was fairly short, with exploits already being released for both Meltdown and Spectre. As always, when looking for public exploits, it is important that organizations evaluate the code before running it. For example, Jerry Gamblin tweeted about a Meltdown PoC on January 5th, receiving 349 Retweets and 626 likes. We have a feeling very few people actually read the code:
The detection front has been more interesting, given initial speculation and statements that Meltdown could not be detected and Spectre would depend on how the attack was carried out. Since the disclosure, we’ve seen several tools that claim they can assist in detecting exploitation such as a set of Snort rules developed by the Talos team, a blog about detecting Meltdown using Capsule8, and an online Spectre detection utility from Tencent. In addition to detecting an attack, utilities are being released to verify you have the mitigations in place such as the SpecuCheck tool for Windowsand exploit sample tracking via Hybrid-Analysis.
Finally, Intel has released a comprehensive analysis of the vulnerabilities in a paper titled “Intel Analysis of Speculative Execution Side Channels” (PDF). Perhaps more interesting is a detailed Twitter thread by Joe Fitz, a former Intel engineer, on why fixing these vulnerabilities at the chip level is so difficult. This may give perspective on why, despite knowing about the issues for six or more months, an Intel patch wasn’t immediately ready prior to the disclosure.
One aspect of the Meltdown and Spectre disclosures that has been fairly unique is the lawsuits. It’s typical to see a multitude of consumer lawsuits filed in the days immediately following a data breach announcement, but it’s rare to see legal activity arising out of vulnerability disclosures. Within days of disclosure, Intel found themselves facing the beginning of at least three separate class-action lawsuits. Based on discussions with more legally-knowledgeable associates, RBS offers some general commentary for consideration about the merits of the three lawsuits in California, Oregon, and Indiana. We are not lawyers and this is not legal advice, just speculative commentary.
The cases out of Indiana and Oregon are primarily based on claims of state level unfair or deceptive Practices. These state level unfair trade practices acts are often called the “Little FTC Acts.” In the Indiana case, the plaintiffs are also arguing a breach of implied warranty claim, which may offer an opportunity for the court to expand contract law to recognize that every contract includes an implicit promise of security. While we are not familiar with cases that include implied warranty of the security of a product, legal scholars have advocated this approach, as security becomes more of a concern to consumers. However, if Intel made no explicit promises of security to computer manufacturers and their forthcoming solution does not significantly impact the speed of processors on the market, Intel may find themselves victorious. However, if they are found to have made explicit promises of security to manufacturers and especially consumers, or their ultimate solution significantly degrades the performance of processors already deployed, a case against Intel may have merit.
Legally, the case out of California is perhaps the most interesting, the most aggressive, and on the surface, the most likely to win of the three cases. This case brings up unfair competition as well as Little FTC Acts claims with the most robust pleadings of the three. Further, the case points out a refusal to recall vulnerable processors and dug into the contracts behind it. If this case proceeds based on the available information, we may see a settlement with Intel sooner than later. Ultimately, Intel may opt to settle all of these cases and more depending on the number of lawsuits ultimately brought against them. Settling and offering coupons for upgraded processors may be more cost effective than court costs including the potential of losing cases and having to pay on the class-action suits, or if they also face any fines from regulators (e.g. the Federal Trade Commission (FTC)).
Over the next several years, these three cases and any subsequent cases will be of interest to hardware and software vendors as security becomes more of an expectation and as vendor’s claims of security are examined more thoroughly in the courts. As Michael Scott argues in his paper titled “Tort Liability for Vendors of Insecure Software: Has the Time Finally Come?” from August 2007, “tort law can provide an ideal mechanism for enforcing the reasonable expectations of software licensees and users, particularly in the area of software intended to secure computer systems and networks.”
Disclosure History Addendum
In our first blog, we gave a very brief synopsis around the disclosure of Meltdown and Spectre, pointing out some prior works that were warning signs of what was to come. Building on that thread, three more bits have come to light that make the history of these issues more interesting. In no particular order:
- Simha Sethumadhavan Tweeted that he gave a presentation to Intel around six years ago on the “time tools and techniques to detect and mitigate microarchitectural side channels (Side Channel Vulnerability Factor measurement and method, and the TimeWarp mitigation from ISCA12).”
- Joanna Rutkowska and Rafał Wojtczuk did research in 2010 along these lines, but didn’t publish since it was under NDA and they did not have a working attack.
- In 2014, Immunity, Inc. wrote a paper with a synopsis of “Intel CPU information leak” discusses similar issues.
- In August, 2017, Intel SGX for Linux was found to be vulnerable to cache-timing attacks that disclosed privileged information to a local attacker, just like the latest. That issue didn’t even receive a CVE assignment.
- Similar research was done years back on an Xbox 360, but not published until now.
Anders Fogh published a blog titled “Behind the scenes of a bug collision” that gives some of his history and speculation on how so many parties found the same issue. He also cites prior work going back to 2005 that likely helped set the stage for this. Andy Greenberg of Wired wrote an article covering the triple-discovery as well, with some additional details about how it unfolded. These examples show that the idea and legwork was there many years ago, yet the full vulnerabilities weren’t discovered and made public until a week ago. This is a good reminder of the state of vulnerability research and that there are hundreds of highly-skilled researchers out there, all capable of finding such issues.
Since the disclosure of Meltdown and Spectre, Intel has faced some serious criticism and even lawsuits as mentioned previously. On one hand, as noted above, fixing this vulnerability is not simple by any means. On the other hand, the prior research leading up to this disclosure can be argued as a huge warning sign to Intel and that perhaps they should have been more cognisant of the threat and impact. While many are focusing on this one disclosure and using it to make more sweeping statements about vulnerability collisions, others are having to point out why such statements are incorrect. With that, here are some thoughts and questions that are central to this disclosure.
Could Intel have predicted or prevented these vulnerabilities? Predict, mostly likely yes, but prevent is still to be determined. Side-channel attacks against cryptography are fairly common, and cryptography algorithms are supposed to be resilient against such attacks. Yet those don’t seem to make the news with the same furor as the recent issues, even when that algorithm may be used in widely deployed or sensitive systems. Is Intel diligent in their handling of security vulnerabilities? Yes. They maintain an extensive set of pages on their website with security advisories, work with researchers, and frequently answer third-party questions asking for clarity on their disclosures. With AMD and Arm finding themselves vulnerable to a degree, it begs the questions if they offer the same security resources and diligence. But so far, there seems to be very little finger-pointing in their direction. Granted, sometimes Intel could benefit by looking to improve their public response to such issues (note the ITWire article calls F00F a remote issue, but public information around time of its disclosure suggests otherwise).
Vulnerabilities in processors and related functionality happen more than you think. There were 33 vulnerabilities in Intel processors/software in 2017 alone. In fact, it could be easily argued that at least one of them was far more significantthan Meltdown or Spectre, especially given all of the vendors impacted. In the middle of Meltdown and Spectre making the news, two days before technical details were published, Cfir Cohen of the Google Cloud Security Team disclosed a vulnerability in AMD’s PSP. AMD describes it as “a dedicated processor that features ARM TrustZone® technology, along with a software-based Trusted Execution Environment (TEE) designed to enable third-party trusted applications.” Cohen’s vulnerability allows for unauthenticated remote code execution due to a buffer overflow. Yet somehow, this was missed by most and didn’t enjoy the same mainstream press while likely being a bigger risk to AMD users.
Vulnerabilities are disclosed every day, to the tune of over 20,000 new disclosures in 2017 alone. Just because a vulnerability receives a name, a website, and/or a marketing campaign does not necessarily mean it is high risk or that it will impact your organization. As always, we strongly encourage organizations to cut through the noise and focus on the details relevant to them, and make a decision based on that alone.
In addition to the usual news articles covering these disclosures, there are many others that are pointing out interesting aspects:
- Alasdair Allan brings up a good point about who may be impacted by this vulnerability more than others. He notes that CryptoCurrency exchanges may be particularly targeted as the private keys to wallets would be of high value.
- Daniel Gruss et al, researchers behind the “KASLR is Dead: Long Live KASLR” paper, submitted their research to BlackHat Briefings, but were rejected. Sometimes research that may be fairly critical doesn’t seem appealing to conferences.
- Richard Grisenthwaite from Arm Limited published a whitepaper titled “Cache Speculation Side-channels” covering the susceptibility of Arm implementations.
- Robert O’Callahan points out the continuing problems with the disclosure of Meltdown and Spectre, some of which we have noted in vendor advisories and commentary about the vulnerabilities (primarily around Spectre since it is multiple issues). Alex Ionescu humorously summarizes this confusion.
- Artturi Lehtiö Tweeted that IBM mainframes are vulnerable to these issues. IBM is still investigating per their blog.
- Shawn Webb gives his perspective on these vulnerabilities, with a focus from a FreeBSD and Hardened BSD slant.
- Hector Martin has created a GitHub repo to collect documentation and resources about the vulnerabilities and invites users to share by contributing pull requests.