Experimental security

 

Which keeps you drier – walking or running in the rain? It turns out that doing a careful analysis of this problem isn't that hard. There's a paper by mathematician David Bell that walks through a complete solution. Like most things, if you think carefully about the problem, it turns out to be more complicated than you first think. In the case of keeping dry in the rain, it turns out that the optimal solution depends on the direction that the wind is blowing. If the wind is coming from in front of you, you keep driest by running. If it's coming from behind you, you keep driest by keeping pace with the wind. With most problems, however, a definitive solution isn't as easy to find. Information security is particularly tricky in this respect.

When you take a careful look at the risks that come from using computer systems, it's very difficult to find all of the risks. Even if you find them, understanding how serious they are can be hard. Understanding the best way to address them is even harder.

Because most people probably aren't aware of Bell's solution to the walk-or-run-in-the-rain problem and don't seem to be inclined to derive the optimal solution themselves, they often try other approaches. If what you see on the Internet is true, many people have resorted to comparing how wet they get when they walk in the rain to how wet they get when they run in the rain to estimate which approach is best. Most of these seem to arrive at the right answer – that it's better to run.

In information security, we have a similar problem. Even if we want to do a careful model to help find the optimal way to get the security that we want, we can't do it because we don't have enough accurate data about security risks. In the absence of reliable risk information, a similar approach to information security may be the best that we can do – just try different things and see which works the best. You might call this approach "experimental security." There may be no better approach.