March 27th, 2016
How do you know whether or not the defenses you're deploying are actually working? All too often, we rely on the absence of evidence as evidence of absence. But sometimes the absence of evidence can be a helpful, so long as you are confident that you've been measuring the right thing.
Effective defenses in information security focus on reducing your attack surface, but it's critical to differentiate between attack surface and attack vectors. The former can be reduced, the latter need to be eliminated.
Reducing your attack surface by e.g. minimizing the number of systems exposed to the internet and ensuring a solid, uniform stack is meaningful; reducing the number of systems that offer a given SQL injection vulnerability but not eliminating it completely from all exposed systems is not.
"If you go from 36% on fire to 27% on fire, you're still on fire."
Attackers will always go for the low hanging fruit. And they will continue to pick this fruit until it's gone. The cheapest, successful attack will continue to be used until it stops being either. If you reduced the number of systems with an SQL injection vulnerability from 36% down to 27%, attackers will still use it, and you have not improved your security posture. You're still on fire.
Eliminating attack vectors is hard. It requires you to be thorough and follow up on all instances for a given vulnerability. Doing a half-assed job here does not increase your security. Don't be lazy.
How do you know whether or not you've made meaningful progress? Observe your attackers. If they start to change attacks, then that's a good sign that you've made their job more difficult. As long as you're not observing a reduction in attempts to exploit a particular weakness, you can be assured that this attack vector is still working for your adversaries. Once these particular attempts disappear, then you know you made their life's more difficult. Absence of evidence (of a specific attack) can become a useful signal.
In order to accurately measure this sort of thing, you need to be attack-driven, though. You need to understand how your adversaries view, probe, and penetrate your systems. Differentiate between attack surface and attack vectors. When your attackers shift tactics, then you know you've moved the needle.
March 27th, 2016