Measuring the Attack/Defense Balance
“Who’s winning on the internet, the attackers or the defenders?”
I’m asked this all the time, and I can only ever give a qualitative hand-wavy answer. But Jason Healey and Tarang Jain’s latest Lawfare piece has amassed data.
The essay provides the first framework for metrics about how we are all doing collectively—and not just how an individual network is doing. Healey wrote to me in email:
The work rests on three key insights: (1) defenders need a framework (based in threat, vulnerability, and consequence) to categorize the flood of potentially relevant security metrics; (2) trends are what matter, not specifics; and (3) to start, we should avoid getting bogged down in collecting data and just use what’s already being reported by amazing teams at Verizon, Cyentia, Mandiant, IBM, FBI, and so many others.
The surprising conclusion: there’s a long way to go, but we’re doing better than we think. There are substantial improvements across threat operations, threat ecosystem and organizations, and software vulnerabilities. Unfortunately, we’re still not seeing increases in consequence. And since cost imposition is leading to a survival-of-the-fittest contest, we’re stuck with perhaps fewer but fiercer predators.
And this is just the start. From the report:
Our project is proceeding in three phases—the initial framework presented here is only phase one. In phase two, the goal is to create a more complete catalog of indicators across threat, vulnerability, and consequence; encourage cybersecurity companies (and others with data) to report defensibility-relevant statistics in time-series, mapped to the catalog; and drive improved analysis and reporting.
This is really good, and important, work.
Subscribe to comments on this entry
Clive Robinson • July 30, 2025 8:33 AM
@ Bruce,
With regards,
Is a question with actually a simple answer,
“Neither and both, like a pendulum and flywheel that both balance, and provide momentum toward advancement.”
I’m known for saying,
“Technology is agnostic to use a directing mind puts it to, whether that is seen as good or bad by an observer depends on their point of view at any given moment in time”
Thus the three important elements of the advancement are,
1, The using / directing entity.
2, The judging / observing entity.
3, The morals and ethics underlying the judgment.
All of which change with time.
People also need to be mindful that neither good nor bad can exist in isolation one requires the other for judgment. Thus they form a line or spectrum by which all technology can be measured at that point in time.
Which is what,
Is in effect trying to quantify, but is in danger of missing the change of time and environment.
Also there is a reason we are “drowning in metrics” but we are not achieving anything with them.
There a “value” is not meaningful untill it can be not just accurately and independently quantitized but meaningfully compared.
That is they have to allow for meaningful ratios to be given so that progress or advancement in any given direction can be independently checked and used for test and analysis.
Without which the metrics are not even worth the paper they are noted on.