Superconductor

Why we have buggy software

"You will find that a good group of adults has a lot more kindness, understanding, tolerance and forgiveness than a similar group of teenagers. I think the reason for this is because adults understand that they are fallible. All adults have made mistakes, they have all tried to do things and had them not work out, and they all know how that feels. Therefore, they cut each other a lot more slack. It is a much happier environment."

Marshall Brain, The Teenager’s Guide to the Real World

Lots of classic papers are full of grammar and spelling errors. Diffie and Hellman's classic paper "New Directions in Cryptography" has its share of errors. So does Dijkstra's "Go To Statement Considered Harmful." Some of these errors are so obvious that you sometimes end up wondering how they could have been overlooked. That's until you actually try to write something yourself.

That's when you learn that what you write tends to reproduce a sampling of your thought process. When you try to edit what you've written, what you see then seems to reconstruct this thought process. This means that you see what you were thinking instead of what's actually in front of you, which makes it very difficult to edit what you write. It often takes quite a while away from something that you've written to get your mind into a state that's different enough from its original state so that you avoid this problem.

Writing anything involving a significant level of mathematics is even worse. And it's even more difficult to edit. It seems that your brain does lots of pattern matching on mathematics. "Ah," it thinks, "that's just calculating an autocorrelation. I know what part's doing and I can move on to the next section." And when it does this, it's even easier for mistakes to get incorrectly overlooked. Spellcheckers have made it easier to find some spelling errors, but they've also made us lazy, and we’re now probably much less likely to see an error unless it's underlined in red.

Writing software seems to be plagued by a similar problem. If you think through a feature before writing software that implements it, you often find that debugging the software is often unnecessarily difficult because your brain assumes that what you wrote does things correctly, even though it doesn't. Some of these bugs turn into security flaws that appear in the National Vulnerability Database. And just like you might first be astounded that so many errors creep into writing, you might be tempted to be surprised that so much buggy software can be released in shipping products. Lots of these bugs are probably overlooked because our brains make too many assumptions or the wrong assumptions when looking at the source code. If you’ve written software, you probably understand this.

So we may be stuck with bugs in software and the security vulnerabilities that they cause. The only way to get rid of them may be to have software written and debugged by something other than a person, and I don't think that that will be happening any time soon.

Happy Turkey Day

Today is Thanksgiving, the American holiday that's loosely based on the feast that the settlers of Plymouth, Massachusetts had to celebrate surviving their first New England winter. It's traditional to serve turkey on this holiday, so it's sometimes referred to as "Turkey Day." There may be another day, however, that's just as deserving of that name. This is a day in August instead of November.

August is the month that ITU-T Recommendation X.509 (1997): Information Technology - Open Systems Interconnection - The Directory: Authentication Framework was approved. This is the standard that defined the format for digital certificates as well as the framework for using them. It was probably the first step in the wrong direction for public-key technology, and a step that has made public-key technology more difficult than it needs to be.

It's also probably the reason that the adoption of public-key technology was negligible for many years. During this time, PKI vendors insisted that products that supported the X.509 standard were the only ones worthy of adopting, despite the many serious problems that PKI technology has. Customers seemed to believe this. Consultants made lots of money writing documents with important-sounding titles like "PKI Strategic Plan," that described how the use of X.509 certificates could solve lots of pressing security problems. Many customers tried the technology, but not too many deployed it on a wide scale, despite the pressing need for encryption of lots of sensitive data.

In retrospect, it seems odd that customers didn't question vendors' claims. It was really little more than a handful of vendors claiming that only technology that they happened to sell was the only solution worth considering and customers accepting this without asking too many questions.

Public-key technology was a significant breakthrough when it was invented in the 1970s. It made some things practical for the first time that were extremely cumbersome and expensive to do with just symmetric cryptography. But as the technology evolved, the idea of digital certificates that the X.509 standard defines proved to too difficult and expensive to implement and support. Aside from the single use in SSL for authenticating web servers, the use of X.509 certificates has found little use outside government projects, where its high costs don’t seem to matter as much. It's a technology that has definitely proven to be a turkey. If we could only find the exact day in August on which the first version of the X.509 standard was approved, we could add "X.509 Day" to the list of days that observe significant historical events. "Turkey Day" might also be a good name for it.

Lessons from LDAP

One of the problems that any key management solution needs to solve is how to manage a distributed, hierarchical database of keys. LDAP, the Internet protocol used to communicate between directory servers and clients, is an example of another protocol for the management of distributed, hierarchical databases. The state of key management market today is very similar to the state of the LDAP market roughly 10 years ago, and looking at the capabilities of LDAP products may give some insights into how future key management systems may work.

Suppose that an LDAP client needs information that is stored in an LDAP server. To get this information, the client passes some sort of unique identifier to the server and requests the information associated with that unique identifier. It might pass a user's name, for example, and ask for their e-mail address. If the client is authorized access to this information, then the server returns it to the client.

If the server doesn't have the information that the client needs, it can redirect the client to a place where this information can be found. A metadirectory can even be used to integrate different directories so that several different directories can be easily managed by a single administrator. The dozens of application-specific directories that a typical enterprise needs to manage made metadirectories attractive.

Unfortunately, existing products don't support this ideal model very well. Many LDAP clients can't handle redirections at all. Metadirectories never became very successful due to the problems with integrating different sources of data. If the name "John Smith" is associated with the e-mail address "jsmith@example.com" in the directory used by an e-mail system and with the social security number "812-34-5678" in the directory used by the finance system, it may not always be the case the two John Smiths are the same. This made life very difficult for metadirectory vendors, and the technology never really became very successful. Some industry observers have even declared that metadirectory technology is now dead.

Just like the proliferation of application-specific directories around the time of the dot-com boom led to the introduction of metadirectories, the proliferation of application-specific key management systems has led to the development of standards for key management that should let key management servers from different vendors work together. With any luck, the evolving key management standards will eventually define a technology that ends up being as useful as LDAP and also avoids the problems that LDAP implementations still face today.

Suppose that a key management client requests a key from a key management server. This key needs to be uniquely identified in some way, and the client needs to pass some sort of unique identifier to the key server when it make a key request. It will probably also have to authenticate to the key server before it gives the client the key that it asked for. If the key server that the client requests a key from doesn't have that particular key, the server can redirect the client to the key server that does. And if you have a standards-based approach to key management, it's even possible for different applications to get keys from a single key server and for several different key servers to be managed by a single administrator.

The IEEE P1619.3 Standard for Key Management Infrastructure for Cryptographic Protection of Stored Data should be able to provide all of these features. Any compliant client will definitely be able to request a key from any compliant server. More advanced capabilities, like the ability to redirect clients to another key server and the ability may have to wait for future revision of the standard, but they're certainly on the standard's roadmap. So with any luck, P1619.3 can avoid the problems that LDAP experienced and become a useful standard for interoperable key management products.

Do we have enough data breaches?

Do we have enough data breaches?

That may be a question that you've never heard before. Instead, attention usually focuses on the massive amount of sensitive personal information that's lost through data breaches and the ways to address the problem. It's certainly possible to reduce the amount of sensitive data that's lost. You can encrypt storage devices like laptop hard drives and backup tapes, for example, so that if the storage is lost then the sensitive data that it stores isn't available to whoever ends up with the device.

The question that's rarely considered is whether or not this is actually worth doing. After all, many forms of encryption are expensive and hard to use, so it might be the case that the cost of encrypting your storage is greater than the damage that losing the stored data will cause. There's also the question of availability to address. If you can't decrypt data that you've encrypted, your encryption hasn't just protected the sensitive data from hackers – it's also cryptographically shredded it and made it unavailable to you also.

This is much like the situation that auditors face when trying to eliminate fraud. With no controls in place, you'll probably have lots of losses due to fraud. At the other extreme, you can have extremely strict controls in place, but you'll find that you’re spending more on the controls than the fraud that you’re eliminating. So there's an optimal amount of fraud, and auditors don’t expect you to have controls that reduce fraud past this optimal level.

In the case of protecting sensitive data, we have a very similar situation. With no controls at all in place, it's likely that all of your sensitive data will find its way into the hands of hackers. At the other extreme, you can have extremely strict information security measures in place. But in this situation you'll find that the costs imposed by the higher level of security is extremely high, and you're better off without such draconian measures. So you also need to find the point where the cost of the security measures isn't too high, but the amount of sensitive data lost also isn't too high. And just like auditors don't try to reduce fraud past the optimal level, you shouldn't try to reduce data breaches past the optimal level either.

This means that it's certainly possible that you're not having enough data breaches, and that it would make sense to reduce the level of security in your organization until you find the right balance between data loss and the cost of your security measures. This is almost certainly not the case. Most organizations still don't encrypt much information, and this is probably because some forms of encryption are indeed hard and expensive to use. If you've only looked at those technologies, then you might have come away with the impression that it was better to not use the technology and to take the risk of losing data.

Fortunately, encryption technology has gotten much better in the past few years. It's now simple enough to use that the costs of supporting it make it reasonable to use in more cases than before. Key management technology has also gotten better, so you can be sure that you'll be protecting your data with encryption instead of shredding it. So if you once looked at encryption as a way of protecting sensitive data and decided not to use it, it might be worth looking at the newer technologies. They're much better than they once were, which means that it’s now cost-effective to use them in ways that it wasn't in the past.

Work for the ABA

The American Bar Association has an interesting point of view about encrypting e-mail. In their Formal Opinion 99-413, they say that lawyers can send unencrypted e-mail over the Internet without violating the ABA Rules of Professional Conduct. This means that lawyers are allowed to send confidential client information by unencrypted e-mail. Here’s how they describe the basis for this decision:

The Committee believes that e-mail communications, including those sent unencrypted over the Internet, pose no greater risk of interception or disclosure than other modes of communication commonly relied upon as having a reasonable expectation of privacy. The level of legal protection accorded e-mail transmissions, like that accorded other modes of electronic communication, also supports the reasonableness of an expectation of privacy for unencrypted e-mail transmissions. The risk of unauthorized interception and disclosure exists in every medium of communication, including e-mail. It is not, however, reasonable to require that a mode of communicating information must be avoided simply because interception is technologically possible, especially when unauthorized interception or dissemination of the information is a violation of law.

So the ABA is willing to rely on the fact that people should expect their e-mail not to be read by anyone else that the intended recipient and the fact that intercepting and reading e-mail is illegal to protect confidential client information. This seems to be a fairly odd position. If you're in the health care industry, I don't think that the ABA's reasons would be enough to satisfy the requirements of HIPAA. And I'm fairly sure that an auditor checking to see if you're compliant with the PCI DSS wouldn't buy those arguments either. The ABA's guideline is the weakest that I've seen. It's not that difficult or expensive to encrypt e-mail these days. Perhaps the ABA should revisit this issue with this in mind.

Now THAT'S security

Several years ago, I was an engineering manager in a place that handled classified information. Being very security-conscious, we had security guards that would walk through our offices to look for any security violations. This could be classified information left out, or it could be other controlled items left out. Floppy disks and other magnetic media were controlled items because they might contain classified information, so leaving a floppy disk out would get you a security violation just like leaving out classified information would.

As the engineering manager, every time the guards found a security violation, I had to do annoying paperwork that explained how the violation happened and what steps we were going to take to make sure that it didn't happen again.

One day I came in to work and this very type found paperwork waiting for me because someone had left "unattended magnetic media" on his desk. When I went to the nearby security office to retrieve the offending media, I was surprised to find that it was just an empty jewel case for a CD. This reminded me of the following exchange from the TV show Red Dwarf:

Cat: "Hey, I got it! We laser our way through!"

Kryten: "An excellent suggestion, sir, with just two minor drawbacks. One, we don't have a power source for the lasers, and two, we don't have any lasers."

I was tempted to try a Kryten-inspired comment like this:

Me: "There are only two problems with this violation. One, a jewel case isn't really magnetic, and two, a jewel case isn't quite what most people think of as being 'media.' Sure, you can write on one with a Sharpie, but that as long as we don't write anything classified that should be OK."

Instead, I just filled out the paperwork and got back the empty jewel case. It probably wasted less time than trying to convince the security people that an empty jewel case really didn't need to considered a controlled item.

VHS, QWERTY and IBE?

It's commonly believed that having the right marketing is often more important than having a good product. The examples that are often used to support this position include the triumph of VHS videocassettes over Betamax and the triumph of the QWERTY keyboard over the Dvorak keyboard. The success of many Microsoft products is sometimes added to this list.

In their book Winners, Losers and Microsoft: Competition and Antitrust in High Technology, Stanley Liebowitz and Stephen Margolis argue that none of these examples will stand up to careful scrutiny, and that superior products almost always win in the marketplace. They give fairly solid arguments that support the position that VHS and QWERRY won on their merits, and claim that even Microsoft can’t use its marketing power to win when they have products that are inferior to the competition.

Now let's go back in time to five years ago. At that time, Voltage had no customers but had a good idea that related to how to make a better e-mail encryption product. Back then, people weren't sure what to make of the new "identity-based encryption" technology. It was based on difficult mathematics, so the average person in a corporate security department didn't really have much of a chance to understand how it worked. Even encryption specialists had a hard time understanding it at first. I know that I did.

But people didn't buy IBE because of the elegant mathematics that it uses. Instead, they found that it actually had some significant benefits compared to other encryption technologies. Not needing to look up keys is a big advantage. Not having to manage certificates is another. Maybe the most important of all is the fact that by calculating keys on the fly, the back end of an IBE system is extremely simple. This means that it's much simpler to support and operate, which also means that it costs less.

Enough people found these advantages compelling enough to convince them to buy a product that uses IBE, and there are now over 10 million users of the technology worldwide. If Liebowitz and Margolis are right, that’s probably proof that it's a useful technology. Maybe it will even up as a case study in a book that they write one day.

Enforcers and Easy Targets

Have you heard this joke? Two men are out backpacking. In the middle of the night, they hear a bear rummaging around their campground. They decide to make a run for saftey. One man immediately starts running, barefoot. The other puts on his boots. The barefoot man gets a head start, but the man with boots quickly catches up. The barefoot man asks, "Why did you waste valuable running time putting on your boots? Even with boots on you'll never outrun the bear." To which the boot man replies, "I don't need to outrun the bear, I only need to outrun you."

We all know that security is often simply making yourself a more difficult target than someone else. Even though you might not be 100% protected, the criminals won't attack you because they're busy with easier targets.

I think enforcers are similar. Whether it is the police or the SEC, the people in charge of reining in or catching the rule violators go after easier targets. If the target is too difficult, they don't even bother trying. It may seem that big crimes are the exception. That is, when someone is murdered or raped or beaten, or a bank is robbed, the police will make an attempt, even if it is a difficult case. Part of the reason is probably that if the enforcers did not attempt to catch them, the political fallout would be too great and they would be out of jobs. However, I also believe those are the types of crimes for which there is often plenty of evidence available. Those violators are easier targets.

Here are some stories.

On Oct. 9, 2002, the San Francisco Chronicle (among others) reported on the capture of a thief. The man (Julian Torres) was part of a ring that would "steal mail left in boxes or public bins, searching the envelopes for credit card and bank records..." What makes the story interesting, is that the police did not find him, a private citizen did. Despite complaints, police did almost nothing. In fact, the article reports that, "Torres himself was arrested in San Francisco [in 2001] with a fraudulant check and a stack of stolen mail, but prosecutors declined to file charges..." It was Jill Maggio, a small business owner who suffered losses at the hands of Torres, who collected enough evidence that would finally convince San Jose, CA, Police to move ahead. The enforcers did nothing until someone else did their work for them.

Possibly the police forces and prosecutors that could have gone after the thieves were just so overwhelmed by crime that they had to make difficult choices on which crimes to investigate. But in this case, the thieves were not particularly talented nor careful. It was not difficult to find them nor collect the evidence. It would seem this is an example of an easy target. But it wasn't. It looks like an easy target, but it was actually difficult. Think of it, if this was a difficult target, imagine if the thieves were experienced, professional, and kept a low profile.

There's the SEC. I worked for a company that was investigated by the SEC. The investigators spent lots of money but found nothing (they found nothing because the company did nothing wrong). Despite finding nothing, they kept at it, planning to keep at it until finding something, anything. Finally, their bosses told them to quit, so to save face, they got the company to promise never to do what it did again, even though it was shown to be in accordance with SEC rules.

It was during this episode that our chief lawyer pointed out that the SEC does not go after big companies. They don't have the budget. Most big companies can out-lawyer them. So the SEC goes after smaller, easier targets, hoping they can make examples of the smaller fish in order to keep others in line. So Enron, Worldcom, Tyco, and so on, are the ones doing the really bad things, but the SEC won't try to prevent those types of messes.

So let's say the point is you often have to be responsible for your own security, the authorities are not going to do it.

The dangers of a risk assessment

Performing a risk assessment is often listed as one of the first steps in information security life-cycle methodologies. Performing such a risk assessment is actually hard to do. There's little valid data that tells how often security vulnerabilities are exploited, and it's very hard to quantify the damage that’s done if a hacker actually exploits a vulnerability.

This means that estimating risks, which are defined to be the probability of an event multiplied by the loss associated with the event, often isn't practical in information security. It turns out that there may actually be another reason to do such a risk assessment, even if was feasible, and this reason relates to the potential legal complications that may arise if you do a careful risk assessment. This was first noted by W. Kip Viscusi in an internal discussion paper that he wrote while at Harvard Law School that was subsequently published in the Journal of Legal Studies as "Jurors, Judges, and the Mistreatment of Risk by the Courts."

As we previously mentioned, the Hand Rule tells us that you’'e not required to spend more than the value of a risk to mitigate it. So if it will cost you $2 million to mitigate a $1 million risk and you decide not to spend the $2 million, the Hand Rule tells that you can't be found negligent.

Viscusi’s research showed that jurors don't properly apply negligence rules like the Hand Rule, particularly in cases where the probabilities of events are small and losses are large. Jurors seem to be offended by trade-offs between costs and risks. In Viscusi's research, the only factor that showed a meaningful correlation with the size of damages awarded by synthetic juries (those composed of test subjects that were asked to decide damages under a number of different scenarios) was with whether or not a risk assessment was performed.

The personal characteristics of jurors didn't matter. The cost per life saved didn't matter. Even a high absolute level of risk didn't matter. The only factor that was significant was whether or not a risk assessment was performed.

Here has been no research similar to Viscusi's that asks about damages from data breaches or other security incidents, but the fact that jurors might be offended by a careful risk assessment should be chilling to people in information security organizations. Without a risk assessment, you may not spend your budget in a reasonable way, but with one, you may be leaving yourself open to other complications.

What's wrong with the world

“What is wrong is that we do not ask what is right.”

G. K. Chesterton, What's Wrong with the World

Discussions of information security almost always focus on what’s wrong instead of what's right, and this is perfectly understandable. Security vendors have an incentive to make things sound worse than they are, and tend to focus on the ways in which their products can change this situation for the better. Stories about gloom and despair seem to attract more readers than stories about things going well, so journalists also have an incentive to report on what's wrong instead of what's right.

Information security has actually greatly improved in some ways over the past few years. Many things much better than they once were, but this seems to be frequently overlooked. Usability of security products is much better that it was in the past. Costs of supporting some security technologies have dropped dramatically. Many security technologies are much more widely deployed now, making many networks far more secure that they once were.

The widespread use of SSL has enabled billions of dollars of on-line transactions that probably would not have happened in its absence. This has probably been a major factor in the success of Amazon.com and eBay, neither of which would probably been successful without the confidence that SSL gives customers.

Innovation continues. Only five years ago, Silicon Valley seemed to be full of start-ups developing new intrusion detection technologies. Most of these companies are gone now, but the best of their technologies lives on in shipping IDS products, and it's now harder for hackers to penetrate and exploit networks without getting caught.

Identity-based encryption has gone from being a clever idea in an academic paper to having over 10 million users worldwide. The ease of use and lower cost of ownership that it provides has made encryption feasible where it was once too expensive to seriously consider.

Things are still far from perfect, but it's probably a good idea to look at the things that are right every now and then.

The vendors' dilemma

Vendors of security products do not always provide accurate descriptions of the strengths and weaknesses of their offerings, although such behavior would benefit the industry as a whole. Mathematical game theory provides a framework for understanding why this happens, but it doesn't tell us how to avoid the problems that this can cause.

The prisoners' dilemma is a classic problem in game theory, the branch of mathematics that models the interactions of competitors and predicts their actions. In the prisoners' dilemma, two prisoners who collaborated in a crime are interrogated separately. The police do not have enough information to convict either of the prisoners, but offer each of them a light penalty in return for informing on the other, who will then receive a harsher penalty. So if both prisoners remain silent then both are released and suffer no penalty; if only one informs on the other then one suffers a harsh penalty while the informant gets off with a light penalty; but if both inform on the other, then they both receive a light penalty. The best case for both prisoners is for them to both to refuse to inform, but we can expect this not to happen.

John Nash, the mathematician whose life was depicted in the movie A Beautiful Mind, was awarded the Nobel Prize in Economics in 1994 for his contributions to game theory.  Nash showed that when the prisoners' dilemma is analyzed by rational participants we can expect to end up with both prisoners informing on their companion, so that both end up in a position that is not as good as they could achieved through cooperation. The uncertainty in their decision-making leads them to a decision that they would have avoided if they had better information.

The prisoners' dilemma can give us some insight into the way in which technology vendors compete for customers. Vendors typically know more about their technology than their potential customers do, and vendors are tempted use their superior knowledge and experience to gain an advantage over customers during the sales cycle.

If all vendors fully explained the weaknesses as well as the strengths of their technology, then customers could make informed choices. But if one vendor decides to give customers misleading or incomplete information in order to gain sales at the expense of their competition, then they alone gain while their competitors all lose. Much like we can expect the prisoners in the prisoners' dilemma decide that informing on the other, we can expect rational vendors to fully exploit the information advantage that they enjoy over their potential customers. This might be called "the vendors' dilemma." Game theory tells us that the result that we can expect is that all vendors take advantage of their position relative to their customers in an effort to minimize the impact of similar tactics that they expect their competition to be using.

So game theory tells us to expect vendors to present inaccurate and incomplete views of their technology to customers and that this can result in a market failure when customer demand drops due to their inability to find high-quality products that are worth their price. It is likely that some security products have experienced market failures attributable to these mechanisms.

It has been estimated that over 50 percent of Public-Key Infrastructure (PKI) products sold ends up as "shelfware," software that is purchased yet never deployed. PKI software is fairly expensive, and it is reasonable to assume that corporate IT organizations did not intend to make a significant purchase they would not deploy. So why did people buy PKI software?

PKI vendors (which included the author of this post at one time) told their customers that PKI technology could solve many of their security problems by providing strong authentication, unbreakable encryption and legally-enforceable digital signatures. What the PKI vendors did not tell their customers was that virtually no existing applications used the digital certificates that their PKI software created and managed, so that it was very difficult to actually create a sound business case for purchasing PKI software. And while the PKI vendors boasted about the capabilities of their PKI toolkits for PKI-enabling applications, they didn't mention the fact that the toolkits were just too complex for the average programmer to use.

The results were purchases of technology that could not live up to their expectations and whose limited benefits could not justify the cost of their deployment. Eventually the PKI market crashed. Both vendors and their customers felt the pain of this crash, all of which could have been avoided if vendors had been a bit more honest about the strengths and weaknesses of their technology.

The vendors' dilemma tells us that we cannot expect vendors to give us an accurate picture of the strengths and weaknesses of their products, but you should try to get the best estimate of these before buying anything.

Attacking AES

Encryption is probably one of the most misunderstood security technologies. Some of this is due to the technology itself; another part is due to other reasons. The security of encryption relies on mathematics that’s far too complicated for most people to understand. That’s the part that’s due to the technology itself. This property is unfortunate, but there seems to be no way around it. On the other hand, the fact that the details of the technology aren’t well understood by most people makes it hard to make sense of some of the arcane arguments about encryption that we see, and this can be the source of more misunderstandings.

If an encryption algorithm is "broken," for example, it just means that someone found an attack against it that can be done significantly faster that a brute-force attack. This doesn't mean that the new attack is practical for a hacker to actually do. Reducing the time for an attack from 10 billion years on the world's fastest supercomputer to only 10,000 years on it is a very significant result, but it's not one that a hacker can actually take advantage of.

Whether it takes a hacker 10,000 years or 10 billion years to beat a cryptographic algorithm, the result is essentially the same from the point of view of users – such attacks are impractical to carry out and hackers won't do them. This means that any attacks that they will do will almost certainly take advantage of a weakness in the way in which the encryption is used instead of the encryption itself. It's almost certainly easier to find a weakness in key management processes than in the encryption, so hackers are much more likely to focus their efforts there. Almost any alternative is better that attacking the cryptography directly.

Back in the dot-com boom, I had a conversation with a potential client that went roughly like this:

Client: "We want you to try to beat the DES encryption algorithm. We believe that it’s too weak for us to use."

Me: "I can do that for you, but it will take lots of expensive computing time. On the other hand, I can probably get any DES key that you use by a less computationally-intensive and cheaper way. Give me $2 million. I’ll keep $1 million of it and bribe one of your employees to give me the key that I want with what’s left."

It's actually fairly easy to build a special-purpose computer that can actually recover a DES key in no more than a few days, and you can do it for less than $2 million. But if you had to recover a 128-bit AES key, that approach won't work. In that case, the bribery is probably a better approach. It's cheaper than trying to build a special-purpose AES cracker, and it's also much more likely to succeed.

What type of spam works?

There's a lot of spam these days. A recent report from Symantec shows the relative frequency of the the types of spam are the most prevalent today. This is shown in Figure 1. It certainly looks like we have a wide variety of annoying spam to choose from, doesn't it?

Figure 1. Breakdown of types of spam messages sent by type.

According to widely-quoted data Ciphertrust, however, all types of spam aren't equally effective. If we weight the data from Symantec by the clickthrough rates estimated by Ciphertrust, we get an entirely different picture. This is shown in Figure 2. From this, it's fairly clear which types of spam are effective and which ones aren't. You have to wonder why some spammers even bother. Why waste your time on an ineffective spam campaign if you can change your product and get a much better response? Spammers are fairly clever at finding ways to get their messages past spam filters. How could they have missed this obvious optimization?

Figure 2. Breakdown of total clickthroughs on different types of spam messages.

Counterfactual what?

If quantum cryptography and quantum computing seem to defy any reasonable intuition, counterfactual computing pushes this to an extreme limit. In counterfactual computing, we use the fact that a quantum system is in multiple states at once, but use this in an unusual way.

Instead of having a quantum system that is in multiple state that correspond to different calculations, like we do with quantum computing, in counterfactual computing, we think of a computer as being in two states at once: on and off. So if we interact with a computer, we will be interacting with it as it were both on and off at the same time.

Carrying this to the next logical step, we can then interact with a computer that is turned off, but get information out of it as if it were actually on. This may seem to make absolutely no sense, but it has actually been demonstrated in a laboratory by researchers.

Fortunately, there seem to be limits to what this technology is capable of, and it may turn out to be impossible to take advantage of its bizarre properties. Otherwise we could imagine assuming that a computer is in states that correspond to either having finished cracking a 256-bit cryptographic key or not having finished this calculation, and be able to break military-strength cryptography just by interacting with a computer that is in these two states. This would allow such attacks without even having to spend any computing resources on the attack.

With any luck, researchers will find that counterfactual computing is little more than a laboratory curiosity that only works in small, well-defined experiments, and is not suitable for use in anything more than trivial demonstrations.

Quantum computing

Quantum computing takes advantage of the ability of quantum information to be in more than one state at a time, and allows the construction of computers with very different properties than the ones that exist today. They also have some fairly unusual theoretical properties. Computations on existing computers are not reversible. In the simplest case, two bits go into an AND gate, but only one bit comes out, and there is no way to reverse the computation and determine the inputs of an AND gate from the outputs.

On quantum computers, however, every calculation is inherently reversible. This means that it's theoretically possible to use a quantum computer to perform a calculation, print out the result, and then step through the steps of the calculation in reverse, ending up back where it started.

The security of modern cryptography depends on certain mathematical problems being easy to compute one way and hard to compute the other way. Multiplying two numbers together is easy, but factoring a large number is hard, which is the basis for the security of the RSA scheme. Quantum computers make it possible to turn the hard mathematical problems that provide the basis for public-key cryptography, like integer factoring and calculation of discrete logarithms, into easy problems. This would make decrypting as easy as encrypting, which eliminates the security provided by the algorithms.

Because this would make encrypting and decrypting equally easy, there would be no way to overcome this problem just by making keys longer. All of the commonly-used public-key algorithms would be affected by the existence of quantum computers.

Quantum computers can also make it easier to defeat symmetric cryptography, like DES or AES, but only by a relatively small amount, and it is easy to increase the sizes of keys to keep them secure, even if quantum computers are available to an adversary. In particular, algorithms that can run on a quantum computer can reduce the effective key size of a symmetric algorithm by half, reducing the strength of a 256-bit key down to only 128 bits, for example.

Unlike public-key algorithms, where quantum computers totally eliminate the security provided by the algorithms, it is easy to increase the strength of symmetric algorithms to compensate for the existence of quantum computers: just double the key size. While speaking at the 30^th Anniversary of Public Key Cryptography event recently, Brian Snow, the former Technical Director of the NSA’s Information Assurance Directorate, said that this was the reason that the standard for AES defines keys up to 256 bits in length, keys that provide more security that will ever be needed. If quantum computing ever becomes practical, 256-bit AES keys will still provide the equivalent of 128 bits of strength, which is still more than adequate for almost all purposes.

Quantum computers have been built that use a small number of quantum bits, but they are not yet capable of performing useful calculations. A quantum computer with seven quantum bits has managed to factor the number 15 using an algorithm designed to run on a quantum computer, a result that is interesting because it shows that quantum computers can actually be constructed, but building one with enough quantum bits to threaten even 1,024-bit RSA keys is still a daunting engineering challenge.

Some experts believe that it will prove impossible to build such computers with enough quantum bits to do such calculations. Only time will tell if useful quantum computers can be built that can threaten the security of existing public-key cryptography, but even if this turns out to be possible, it is probably still several decades in the future.

Going, going, gone

The huge amount of transactions on eBay shows that auctions have become a popular way of selling goods. This is realy nothing new, because auctions have been around for thousands of years. One of the most notable auctions of all time took place in AD 193 when the Praetorian Guard auctioned the entire Roman Empire to the highest bidder.

The winner of this auction, Didius Julianus, offered each soldier 25,000 sesterces, or 10 times their annual salary, and became the next emperor only to be overthrown and slain by Septimus Severus 66 days later. It seems likely that toward the end of his life that Didius Julianus came to regret his purchase, and felt what is commonly known as "buyer's remorse."

The consequences of winning most auctions are not usually as severe as those suffered by Didius Julianus, but economists tell us that we should expect buyer's remorse to be fairly common because we can expect auction winners to pay too much. Their reason for believing this assumes that bidders in an auction will not know the exact value of what they are bidding on, so that the bidder who overestimates this value the most will end up winning the auction and suffering the winner's curse of having paid too much for their purchase. The lower-than-expected returns earned by winners of auctions for oil drilling rights or wireless spectrum licenses seem to provide evidence that this does indeed happen.

Economists call the familiar "going, going, gone" auction that is used by Christie's and Sotheby's an English auction. In this type of auction, bids increase until only one bidder remains. Another common type of auction is the Dutch auction, which is named after the way in which flowers are sold in The Netherlands. In this type of auction, the price starts high and is progressively lowered until a buyer is found. Dutch auctions are also used by the Federal Reserve Bank of New York to sell options on overnight repurchase agreements, and were used to sell Google shares in Google's initial public offering.

Although the analogy is not perfect, there is a parallel between the market for information technology (IT) and a Dutch auction: new technology is usually introduced at a high price, but drops over time, just like the price of an item sold in a Dutch auction. At some point the price of the technology may become low enough so that some firms can justify its purchase. To consumers of IT, this looks much like a Dutch auction, except for the fact that prices may continue to drop after a purchase is made.

So if purchasing information technology is like a Dutch auction we can expect the winner's curse to affect IT purchases and expect that many firms will pay more than they should for IT because they overestimate the return on investment that they use to justify its purchase. Early adopters of technology seem particularly prone to this problem because they tend to pay higher prices for technology than others who wait until the technology drops in price.

The overestimation of the value of IT purchases may also be caused by problems in the deployment of the technology. The Standish Group, a consulting firm that specializes in tracking the rates of failure in IT projects, estimates that the chance of a trouble-free completion of any IT project is small.

Their annual CHAOS Report tracks the state of IT implementations, and recently estimated that the chances for a trouble-free deployment ranged from roughly 2 per cent for larger projects to roughly 46 per cent for smaller ones.

This report also found that while only 15 per cent of IT projects resulted in total failure, higher-than-expected costs were common, deployments often took longer than expected and often resulted in fewer capabilities than first planned. Our understanding of auctions may provide an explanation of why so many projects end up troubled.

Firms tend to allocate funding to the projects that have the highest return on investment. So we can think of different projects as bidders in an auction, with funds going to the projects that estimate the highest rate of return on the investment that they require. This means that firms will tend to fund projects for which they have overestimate their value the most.

Similarly, if a firm uses a system integrator for an IT implementation, the contract for the project is usually awarded to the lowest bidder. The lowest bidder tends to be the one who underestimated the true costs of a project the most, which then tends to result in projects with difficulties with costs and schedules once the inaccuracy of the estimates is discovered.

Understanding why some IT projects may result in difficulties can provide insight into ways to address the problem. If you are planning to deploy a relatively new technology, you should carefully consider your business case for adopting the technology. In some cases the benefits will be clear and you should proceed with the deployment of the new technology.

In other cases you may find that unrealistic expectations of the benefits of the new technology have led you to a bad decision and that you are on your way to experiencing the winner's curse as you pay too much for it. The safe strategy is to assume that you have overestimated the value of the new technology and to revise your projections downward to compensate for this to avoid a possible case of the winner's curse.

Similarly, you should ensure that all of your IT projects will have an appropriate return on investment in the event that its deployment encounters difficulties. Costs will often be higher than first anticipated, schedules will often slip and deployed technologies will often not offer all of the features that you had anticipated.

Most projects will encounter some of these difficulties, and being prepared for this will let you increase your chances of success and avoid feeling buyer's remorse for your IT investments. Again, the safe strategy is to assume that costs and schedules have been underestimated and to plan accordingly.

The time for action is past

Developing standards is supposed to be a process that moves towards reflecting a consensus of subject matter experts. It doesn't always work this way. The IEEE Security in Storage Working Group, for example, is often paralyzed by an inability of the group to make the simplest of decisions.

On SISWG calls I'm frequently reminded of a t-shirt that’s available from despair.com. Despair.com started making parodies of the motivational posters that you often see in corporate offices, and have more recently branched into their fine "DespairWear" line of clothing. You can find the DespairWear design that I'm frequently reminded of in SISWG meetings here.

Keeping data valuable

According to the Association for Information and Image Management, only 20 percent of corporate data is structured data, but structured data consumes 75 percent of corporate IT resources. Examples of structured data are the information in ERP systems, CRM systems, and finance systems. In each of these cases, the data is well understood. It's fairly easy to know both where it is and its exact format. This makes protecting it easy, whether you're using encryption or some alternative. Maybe saying that it's easy is an oversimplification.

It can still be tricky to integrate encryption with legacy systems that handle structured data because the size and format of data typically changes when you encrypt it. The new technology of Format-preserving Encryption goes a long way towards making legacy computing environments simpler to deal with, but even that will only protect 20 percent of your data. The remaining 80 percent is much harder to protect.

The remaining 80 percent of corporate data is unstructured. Examples of unstructured data are the information in e-mail, documents, spreadsheets. Even voicemails count as unstructured data. With unstructured data, you often don't know exactly where it is or what it's exact format it.

Suppose that you encrypt all of your unstructured data. Maybe you can do this with the DLP technology offered by vendors like EMC. Once you've encrypted your data, however, it may become much less valuable than it once was because you're probably unable to search it.

A significant part of the value of Google, after all, is due to the fact that they let you search lots of the world's data. If you couldn't do this, the world's data would be much less useful and much less valuable, and Google is valuable because they make the world’s data more valuable. Similarly, if you can't search your corporate data then it's less valuable than it could otherwise be. If you believe that most of the value of modern businesses is determined by the value of their information, this might make you think twice about trying to encrypt unstructured data. On the other hand, using identity-based encryption may provide a good way to encrypt data, yet still keep it searchable.

One feature of IBE is that all keys are calculated as needed. This means that you don't need to keep a database of private keys to do key recovery. This is because you can recalculate any private keys when they're needed. This means there's no need to store private in a secure key archive to do useful things like key recovery. This also lets you do clever things like doing content filtering of encrypted e-mail by delegating the permission to get IBE private keys to a filtering appliance.

It also can let you easily search encrypted data in much the same way. Delegate the permission to get IBE private keys to a search appliance and it can decrypt encrypted unstructured data, search it, and return the results of the search. You can even restrict the result of such a search to users that are authorized to decrypt the encrypted data that the search finds. By doing this, you can protect your unstructured data without greatly reducing its value, staying compliant the data security and privacy laws that complicate business these days, but keeping the value of one of your most valuable assets.

Unstructured data isn’t commonly encrypted today, but when it is, I wouldn't be surprised to find that encrypting it is a good application for IBE.

How much does US-VISIT really cost?

There's a recent article that describes how wasteful the US government's US-VISIT (U.S. Visitor and Immigration Status Indicator Technology) program has been. This Department of Homeland Security program uses new technologies, most notably biometrics, to collect and maintain data on people entering the US. Some people claim that over $15 billion have spent so far on this program, but it has only managed to catch a few suspicious people trying to enter the US. This certainly sounds like a lot of money.

It turns out that the $15 billion estimate probably isn't very accurate. The Government Accountability Office report on the US-VISIT program shows (Table 2 on page 13 of this document) the budget of the US-VISIT program to be this:

Budget ($ million)	2003	2004	2005	2006	2007
US-VISIT	$362	$328	$340	$337	$362

There's nowhere near $15 billion in the budget that the GAO lists. This means that either the GAO is off by roughly a factor of 10 or the news report that mentions the costs of the US-VISIT program is. The DHS certainly spends lots of money on security technology, and some of it is probably spent on things that aren't as useful as they could be.

The US-VISIT program installed new technology at 170 border crossings, and if the $15 billion figure was accurate, that would be roughly $88 million in equipment of each crossing point. Even with the high prices that the government pays for some technology, you'd probably be hard pressed to put that much high-tech gear at a border crossing. I'm betting that the GAO got this one right.

Quantum cryptography

Our intuition is developed by watching the world around us. This intuition works fairly well in situations that we usually encounter, but when we stray too far from these, our intuition fails miserably, and we have to fall back on mathematics and physics to understand how things work. In particular, if things get too big or go too fast, then our intuition fails, and we need to fall back on Einstein's theories of relativity to explain things accurately. GPS satellites, for example, are affected by both of these possibilities. They are near a big, heavy object (the Earth) and they go fairly fast in their orbits. Relativity tells us that time gets distorted in either of these cases, and we find that we need the framework of relativity if we want to make GPS satellites accurate enough to be useful. Without the use of relativity to correct for the slight time distortions that these satellites experience, position errors in a GPS system would accumulate at a rate of roughly 6 miles per day.

Our intuition also fails when things get very small. This is the realm of quantum mechanics, and the models that predict things accurately on this scale are nothing like what we see in our daily lives. In particular, quantum mechanics tells us that quantum systems exist in all possible states at once, and that measuring such a system collapses it into one of the possible states, losing information about the other states when it does this. So while a classical bit is either a logical 0 or a logical 1, a quantum bit can be both 0 and 1 at the same time, and if we measure its state it will turn into either a 0 or a 1, losing all of the information about the other state. This means that any information that we encode as quantum states has very different properties than the information that we encode using classical bits and bytes. It also provides the basis for three interesting technologies: quantum cryptography, quantum computing, and counterfactual computing. The most mature of these is quantum cryptography.

The term "quantum cryptography" is a bit misleading. The term describes a technology that is used to distribute cryptographic keys that are encoded as quantum information, so "quantum key distribution" is a more descriptive name for it.

An adversary who intercepts a transmission protected with quantum cryptography will destroy some of this quantum information when he tries to determine the state of what he intercepted. When this happens, he will be unable to make exact copies of the information, so he will be unable to retransmit an exact copy of what he received. Because of this, the intended receiver will be able to tell that this transmission was intercepted, and decide to not use the key that was observed by the eavesdropper. So quantum cryptography cannot stop an adversary from eavesdropping, but it can detect when such eavesdropping has happened. The first quantum cryptography protocol was invented by 1984 by Charles Bennett and François Brassard, and is commonly called the BB84 protocol.

In the BB84 protocol, for each bit that the sender needs to transmit, he needs to pick a coordinate system with which to encode the bit. This defines what states the quantum information contains. He can use coordinates based on the familiar binary 0 and 1, or he can use other sets of coordinates. He then encodes the bits using the appropriate coordinate system and transmits them. After this, he sends a list of the coordinate systems that he used for each bit. The recipient needs both the encoded bits and the coordinate systems that were used to encode them to recover the information that was sent in this way.

An eavesdropper who intercepts the encoded bits will destroy some of the quantum information in them when he checks their state. This loss of information will cause errors that will be detected by the recipient – some errors usually happen in any transmission, but too many errors indicates that eavesdropping has occurred. An eavesdropper can also intercept the list of coordinate systems that is sent, but without the information that was encoded with them, knowing the coordinate systems is useless.

Information protected by quantum cryptography needs to be encoded in quantum states, and existing implementations use individual photons that are then transmitted over a fiber-optic link. Because any hardware in a communication channel that boosts the fading signal strength needs to interact with the signal, existing quantum cryptography technologies are limited to a single fiber-optic link. Repeaters act just an eavesdropper, and destroy quantum information when they interact with it.

Quantum cryptography is an established and proven technology. There have been commercially-available quantum cryptography products since 1999, and there are now two vendors from which the technology is available. On the other hand, while the problems of key distribution and key management are indeed difficult, they have not become so difficult that quantum cryptography is an attractive alternative for most commercial deployments. So although the technology has been available for quite a while, it has not yet become a commercial success. Maybe we'll be seeing more of it in the future.

Insecure bank web sites

There's an interesting paper from the 2008 Symposium on Usable Privacy and Security by Laura Falk, Atul Prakash and Kevin Borders about the security design flaws that are present in web sites. According to this paper, 76 percent of banking web sites have some sort of design flaw that affects their security. Here's a breakdown into the five categories of design flaws that the paper looked at:

Design flaw	Fraction of web sites having the flaw
Break in the chain of trust	30%
Presenting secure login options on insecure pages	47%
Contact information/security advice on insecure pages	55%
Inadequate policies for user ids and passwords	28%
E-mailing security sensitive information insecurely	31%

I find the last line of this table is the most difficult to interpret. I don't know how many banking web sites e-mail password resets to their users, but it certainly seems like almost all of them don't encrypt the new password. Probably more that 31 percent of them do this, so I'm not sure exactly what's being counted here and what's not.

If roughly one-third of banking web sites will e-mail statements to their users, that's certainly not good. Encryption of e-mail is not much cheaper and easier now that it was even a few years ago. Fve years ago, the argument that it wasn't worth the hassle and expense to encrypt e-mail was probably valid. Today, however, it's certainly not the case, and there's now really no good reason not to encrypt e-mail that contains sensitive information.

The marginal utility of information

"Cuius rei demonstrationem mirabilem sane detexi. Hanc marginis exiguitas non caperet."

Pierre de Fermat

Many organizations do not implement simple countermeasures that could easily reduce the loss of sensitive information despite the apparent high value of some of this information. In one recent survey, laptop users estimated that the information on their laptops was worth approximately $1 million. Industry analysts estimate that there is roughly a 10 per cent chance of a laptop being stolen in a given year, so that we might expect a typical organization to be willing to spend up to 10 percent of $1 million, or $100,000, to protect the information on their laptops, perhaps through the use of full-disk encryption.

If the information on an average laptop is really worth $1 million and the information is not encrypted, we would expect laptop theft to be rampant. But because we see neither the widespread use of laptop encryption nor the rampant theft of laptops, we might suspect that either the information on an average laptop is actually worth much less than $1 million or that the chance of a laptop being stolen is actually much less than 10 per cent per year. The theory of marginal utility may provide another explanation.

The theory of marginal utility tells us that the value of something is defined by its least important use. Eugen von Böhm-Bawerk’s discussion of the marginal utility of corn to a farmer in The Positive theory of Capital demonstrates this.

Suppose that a farmer ends up with five sacks of corn from his harvest. The first sack he needs to survive until the next harvest. Eating the second sack will keep him healthy, but he has no further need for corn as food for himself past these first two sacks. He might then use a third sack to feed poultry to provide variety in his diet, use a fourth sack to create liquor and a fifth sack to feed to his pet parrot.

In these circumstances, the value of the fifth sack of corn is quite low to the farmer. If he loses one sack of corn out of five, he will not scale back each of his uses for the corn by one-fifth. Instead, he will decide to stop the use that provides him with the least value – feeding his parrot. On the other hand, if he has only a single sack of corn, the value to him is extremely high, for losing the final sack of corn may mean that he starves to death.

In general, the more corn that the farmer has, the less value an additional sack of corn has to him. Economists describe the phenomenon as the law of diminishing marginal utility: the additional benefit provided by an additional unit of a good tends to decrease as the total amount of the good increases.

Applying the law of diminishing marginal utility to information may provide some useful insights into the behavior of corporate security departments and let us predict some future trends. The information age has caused an explosion of information, and we should expect a diminishing marginal utility for this information as the total amount of it increases, particularly because this ever-increasing amount of information is often close to indistinguishable.

It is currently unfeasible to classify information to any significant granularity; data classification projects that try to classify data based on more than the source of the data usually fail. So current technology might require the same handling of any information that comes from an ERP system, or it might require the same handling of any information that is processed by an email system, for example. Within such broad categories, information is essentially handled in a common way.

The law of diminishing marginal utility tells us that the marginal utility of such information is defined by its least important use, and we should expect corporate information security organizations to protect their information as if this were the case.

So although there may some information in email that is of high value, we should expect email to be protected as if it were of low value. And because it is currently impractical to classify data according to the actual value of the data, we should expect to often see high-value data remain unprotected. This means that the slow adoption of security technologies like whole-disk encryption or email encryption can be explained by to the low value of some corporate data, and thus to the low marginal value for all data on an enterprise-wide basis.

There has also been a steady trend towards outsourcing core business functions, including those that involve extremely sensitive data. Not many years ago, it was unheard of for any business to outsource functions like accounting or payroll, while today these functions are routinely outsourced. More recently, there has been at least one successful business that provides a service that outsources the management of sales data, information that is extremely sensitive and potentially valuable to competitors. Even information security functions are starting to be outsourced.

The trend to outsource more and more critical business functions has coincided with the explosion of information, and the theory of marginal utility tells us that we should expect information to have a decreasing marginal utility to businesses as the total amount of information increases. Thus the trend towards outsourcing is certainly predicted by marginal utility theory.

If a business has a relatively small amount of information, the marginal value of the information is relatively high, and outsourcing is viewed unfavorably because it provides a chance for the loss of valuable information. But when a business has a relatively large amount of information, the marginal value of the information is relatively low, and objections to outsourcing disappear.

Currently it is often unfeasible to classify data beyond the source of the data, but technology is now being developed that may produce better solutions in the not-too-distant future that will allow businesses to classify data according to its actual value. In this case, marginal utility theory predicts that this will create a boom in products that will be able to provide strong protection to the high-value data that future data classification products will identify.

So in the future, data will still be protected according to its least important use, but the ability to separate data into different categories will make it possible to more narrowly define these categories. In this case, even the least important uses of valuable data will justify the use of encryption to protect it.

Thus better data classification products may soon create an increased demand for encryption technology; whole-disk encryption and email encryption products that have so far experienced fairly slow adoption rates may become more widely deployed as it becomes easy to identify exactly what data should be encrypted.