...
- software vulnerability = a CVE ID in the NVD with a CVSSv3 vector string
- exploited = an IDS signature triggered for an attempt to exploit the CVE-ID over the network
- in the wild = a customer of Kenna Security AlienVault or Fortinet whose network is instrumented with their IDS systems and their data is shared with Kenna (I think it’s specifically and only Kenna, but as they’re a subsidiary of Cisco, it is a little bit hard to say if other Cisco brands contribute data. But that is really the point: I know it is IDS alert data, and only network IDS alert data, that provides the key signal for EPSS, but we cannot examine the details of how it is sourced, because it is not open.)
Also, EPSS is clear, later in its specification, that the time frame for the prediction is “in the next 30 days.” What is not clear from the documentation is that only about 10 percent of the vulnerabilities with CVE IDs even have IDS signatures. So 90 percent of CVE IDs could never be detected to be actively exploited this way. The way IDS signatures are created is complex. Moreover, the signature curators have their own priorities and own performance aspects to optimize, which means the coverage for the signatures is probably much better than random as long as your environment is similar to the environment the IDS vendor is managing. The flip side is that your coverage is plausibly worse than random if your environment is a mismatch.
In some important way, EPSS is doing something smart. It’s saying, Hey, we saw IDS alerts for attempts to exploit these CVE IDs, and here are a handful of things we didn’t see alerts for but that seem similar. That’s great if you have an environment similar to the environments of Kenna’s AlienVault's or Fortinet’s main and biggest customers. I don’t know what that is, but my guess is offices and other classic IT shops. They probably run mail and AD servers, databases, and Microsoft endpoints; are midsize; have employees who are English-speaking; are located primarily in North America; and are regular commercial-ish businesses.
...
For example, there are several CVE IDs in CISA’s known exploited vulnerabilities list with low EPSS scores, and there are plenty of CVE IDs with high EPSS scores not in that list. People seem to think that this discrepancy means one or the other is wrong. Actually, it probably does not inform rightness or wrongness about either. The discrepancy might be telling us that attackers use different methods to attack the organizations in CISA’s constituency than they use to attack Kenna’s AlienVault's or Fortinet’s constituency. This interpretation would be consistent with the fact that we know attackers target victims using specific infrastructure. Perhaps, however, it is just the result of the expected error rate reported about the EPSS model.
...
EPSS is great in that it is bringing attention to threat data. I agree 100 percent that paying attention to what attackers are exploiting is important in prioritizing vulnerabilities. The scores are probably not attuned to your environment or threat landscape, however, unless you are one of Kenna’s AlienVault's or Fortinet’s customers who is donating their IDS data to the project or know your environment is similar to those who are.
...