Although we tend to think of the CVD process as ending with the disclosure of a vulnerability, if the fix is not deployed the rest of the exercise is futile. A patch that is quietly posted to a website and not well advertised is almost useless in protecting users from vulnerabilities.
Let's say that again, but clearer: Vendors make patches available. But systems are not secure until those patches are deployed.
Deploying patches typically implies provoking users, customers, and deployers to take positive action. Many software products are used by non-technical users. These users are often unaware of how to take remediative action for a vulnerability. A vendor's disclosure plan should consider how to reach the widest audience with actionable advice.
Products with secure automatic updates provide a good way to get a patch deployed quickly to a wide audience. However, not all users are able or willing to use automatic updates, so it is still important for vendors to draw attention to their fixes. Vendors should strive to implement easy and secure update methods in their products. In situations where this is not possible, the vendor's disclosure plan should be specific about how to spread the word of a new patch as quickly as possible.
Give Critical Infrastructure a Head Start When Possible
Some vulnerabilities are pervasive in the very infrastructure required for the patches or information about the vulnerability to be distributed. Vulnerabilities in foundational network protocols1, or problems such as denial of service against backbone routers2, remote code execution on Domain Name System (DNS) servers3, or virtualization escapes4 in cloud services serve as examples. Other vulnerabilities may disproportionately affect critical infrastructure services that directly impact public safety – for example the water system, power grid, or hospital medical gear. All these types of systems often require their operators to perform extra testing and impact analysis prior to deploying patches. It's not always practical to do so, but when possible, providing these kinds of deployers with advance notification of either the existence of the vulnerability or access to the fix can reduce the risk faced by the public and improve outcomes.
Amplify the Message
Sometimes it is necessary to draw more attention to a problem or fix. Critical vulnerabilities, including those that are already being exploited or are highly likely to be exploited, may warrant attracting attention beyond merely publishing a document on the vendor's support site. In such cases, additional measures should be taken to draw attention to the existence of the vulnerability or the availability of its fix. (See also 4.5 Gaining Public Awareness)
Vendors should consider using:
- Announcements via social media. Many defenders use services like Twitter or Reddit as part of their daily situation awareness process, routinely sharing useful links and references with each other.
- Mass media such as press releases, press conferences, and media interviews
- Working with a coordinator or government agency to draw attention to a vulnerability or its fix. In particular, National CSIRTs can often provide advice or assistance with publicity on important issues.
Once a vulnerability and/or its fix has been disclosed, both vendors and reporters should look for feedback concerning any problems with either the documentation or the fix. In some cases, this can take the form of technical monitoring (e.g., monitoring download logs from the vendor's update service, checking inventories of deployed system versions, or even scanning) to ascertain the rate of defender deployments. Even if such technical monitoring is not possible, not permitted, risky, costly, or otherwise impractical, it is usually possible to monitor for user feedback via support requests, online discussions, and so forth.
In the event of slow uptake of the fix, additional effort might be warranted to call attention the vulnerability (for example, using social media).
It is also possible that the remediation advice is incorrect, or may not apply to all scenarios. Therefore the vendor and reporter should monitor for public discussion or reports of problems, so that the disclosure advisory and remediation information can be updated as necessary. Remember, the goal for remediation is to fix vulnerable product instances or at least reduce the impact of the vulnerability. Consequently, if a significant portion of the vulnerable product instances have not been remediated, that goal has not been achieved.
- Havrilla, Jeffrey. "Multiple TCP/IP implementations may use statistically predictable initial sequence numbers
Vulnerability Note VU#498440." 13 March 2001. https://www.kb.cert.org/vuls/id/498440/
Juniper. "2018-10 Security Bulletin: Junos OS: Receipt of a specifically crafted malicious MPLS packet leads to a Junos kernel crash (CVE-2018-0049)." 10 October 2018. https://kb.juniper.net/InfoCenter/index?page=content&id=JSA10883&cat=SIRT_1&actp=LIST
Cohen, Cory. "ISC BIND 8 contains buffer overflow in transaction signature (TSIG) handling code Vulnerability Note VU#196945." 29 January 2001. https://www.kb.cert.org/vuls/id/196945/
XEN. "Xen Security Advisory CVE-2017-8903 / XSA-213; version 3; x86: 64bit PV guest breakout via pagetable use-after-mode-change." 2 May 2017. https://xenbits.xen.org/xsa/advisory-213.html
In an unexpected turn of events following the publication of this Guide, we were called on by the US House Committee on Energy and Commerce and the Senate Committee on Commerce, Science, and Transportation to address concerns regarding the coordinated disclosure of the Meltdown and Spectre vulnerabilities in early 2018. In particular, the committees were concerned about the timing of patch availability and deployment relative to the public disclosure of these vulnerabilities.
The committees went on to note:
In our response, we agreed. The relevant section of our reply is reproduced below.
Patch Availability is not Patch Deployment
The committees’ letter asks “...whether companies used precise terminology in describing the availability, not application, of patches” and points out “...the misapplication of such terms as ‘in place’ and ‘available’ when used to describe the status of vulnerability patches.”
CVD guidance can be more clear in both terminology and the boundary between the phases of patch availability and patch deployment. And while we agree that vendors should take care not to overstate the status of patch deployment, the best many vendors can do today is to make patches available, along with sufficient vulnerability information for users to make informed patching and other risk decisions.1 Ultimate responsibility for installing patches often falls to deployers,2 including end users.
While we appreciate the committees’ desire that “sound CVD strategies would seek to limit disclosure of vulnerability information before stakeholders are able to apply patches,” our experience indicates that it is impractical to privately notify all affected stakeholders without public disclosure. Thus, public disclosure is usually the best practice to inform affected parties—including end users—who may need to take action in order to apply patches to their software and devices.
The committees’ letter correctly points out that the deployers’ need to test patches “can lead to a lag time of weeks or months before a patch is applied.” We note that in the extreme, this lag time can become indefinite for reasons including:
For especially pervasive vulnerabilities such as Meltdown and Spectre, there is no clear optimal solution to balancing the diverse operational cadence across such a wide range of industries (including critical infrastructure sectors) with the need for timely public disclosure. It may be that the best we can expect is for consistent, accurate, thorough, and timely information to be provided in support of defender decisions.