Getting reliable data

Reliable data is hard to come by for almost anything connected with information security. There's little data that accurately quantifies information risks and how effectively information security technologies mitigate these risks. This makes it very difficult to make informed decisions about what security products to use.

One of the few sources of such data is the CSI Computer Crime & Security Survey, a survey that's done each year by the Computer Security Institute. This survey can give you an idea of how many security incidents the typical company experiences in a year and the average losses experienced due to security incidents. It also tells how widespread certain security technologies are. Unfortunately, the CSI reports don't talk about the correlation between losses and the use of security technologies. That's what I'd really like to see.

For example, 94 percent of businesses run a firewall and 13 percent reported having their systems penetrated. What would be interesting is the correlation between these two numbers. Do the businesses that don't run a firewall account for more than their share of the total number of penetrations or not?

Similarly, 53 percent of business report encrypting data at rest, and 17 percent reported losing sensitive customer data. Of those who encrypted data at rest, how many lost sensitive data? This could happen, for example, if data is encrypted in a database, but it's not encrypted outside the database. If that's the case, then the data is still vulnerable to being lost when it's pulled to a laptop or burned to a CD, because database encryption stops protecting data once it leaves the database.

So the CSI seems to be a good position to offer provide information on the effectiveness of security technologies. They have the raw data that they can estimate the necessary correlations and conditional probabilities from their data. Maybe they'll add this information to future reports.