Superconductor

Lessons Learned from a Data Breach

I recently read "Heartland Payment Systems: Lessons Learned from a Data Breach." (PDF) This is a discussion paper published by the Philadephia branch of the US Federal Reserve. Bob Carr, the CEO of Heartland Payment Systems, was invited by the Fed's Payment Cards Center to give a talk about the lessons that Heartland had learned from the data breach that affected their systems back in 2008 to 2009. 

Carr talked about the relative strengths and weaknesses of various approaches to protecting sensitive credit card information and  how Heartland decided to use end-to-end encryption to ensure that their systems couldn't suffer another breach like they had just been through. If you're interested in protecting credit card data, you'll probably find Carr's discussion of the various technologies interesting. 

INTERPOL takes on cyber-crime

According to a story on the Washington Post web site, INTERPOL is setting up a facility in Singapore, the INTERPOL Global Complex for Innovation, and a big part of the mission of this new facility is to help fight cyber-crime. I'm not quite sure what to think about this. Lots of government officials are making statements about what an important step this is, but that may or may not actually be the case. I seem to recall INTERPOL being mentioned in The Saint, the '60s TV show that starred Roger Moore as Simon Templar. It might have been mentioned in one of more James Bond movies, and I seem to recall seeing warnings at the beginning of movies on DVD that tell you that they've expressed concern about copyright infringement. 

But it's not clear to me exactly what INTERPOL does and how their new Global Facility will actually help fight cyber-crime. Interpol's job, after all, is fairly vague: to help coordinate the actions of the world's police forces. And that may not be very useful in the case of cyber-crime because the cyber criminals are fairly clever. 

I've heard stories, for example, of how Russian hackers are very careful to not hack the credit card numbers of other Russians and to not target businesses in Russia. This makes going after them a very low priority for Russian law enforcement agencies who almost always have more pressing issues competing for their attention. And this doesn't seem like the sort of thing that an INTERPOL cyber-crime division would really have much of an effect on.

But here's what INTERPOL wants from their new facility:

The four main components of the Global Complex are as follows:

Innovation, research and digital security

• Boosting cybersecurity and countering  cybercrime;
• A forensic laboratory to support digital crime investigations;
• Research to test protocols, tools and services and to analyse trends of cyber-attacks;
• Development of practical solutions in collaboration with police, research laboratories, academia and the public and private sectors;
• Addressing issues such as Internet security governance.

Capacity building and training

• Research into training and methodology and the transfer of this research into police activities on the ground;
• Classroom, field and online training programmes for  National Central Bureaus; 
• Anti-corruption training, particularly in sport;
• Quality standards and accreditation.

Operational and investigative support

• Identifying and addressing emerging crime threats, for example,  intellectual property crime, environmental crime and  Asian Organized crime;
• A platform for  disaster victim identification;
• A  Command and Coordination Centre operations room; 
• Incident response and major events support.

International partnerships and development 

• Global partnerships with international organizations, governments, public and private sectors;
• Generation of revenue, donations and fundraising.

And although it's probably a good thing that there's an international effort supporting the law enforcement agencies targeting cyber-criminals, I really don't expect this particular organization to actually have much of an effect. That seems to be the nature of most government organizations. But in this case I'd love for the people at INTERPOL to prove me wrong. 

Why the TSA acts the way they do

There's an interesting interview with Kip Hawley, the former head of the much-reviled Transportation Security Administration, on the IEEE Spectrum web site. In this interview Hawley tries to justify some of the things that the TSA has done in recent years as being the legitimate result of reasonable risk management decision making. And although I'd really like to believe Hawley, my experience working for the US government leads me to believe that other explanations of the TSA's actions are probably more likely. But it's definitely an interesting interview that's probably worth reading (or listening to - there's also a podcast of the interview available, the link to which isn't working with this blog for some reason). 

Questionable HR practices yet again

It's probably appropriate that Munch's famous painting The Scream was just sold at action for a record $119.9 million. It's a good representation of how this article about how HR people are fighting a "war for talent" for information security professionals made me feel. The HR people are essentially saying that they want people who are top-notch experts in their fields with lots of deep knowledge. But then they also want those people to be flexible enough to do other things if they're needed. 

Things really don't seem to work very well that way.

HR people tried this back in the dot-com era, and what we learned was that almost all of your top-notch experts will quickly lose interest in working for you if you put them on projects that aren't related to their world-class expertise. So even if you somehow manage to attract good people, you'll probably be extremely unlikely to retain them for long if you do this. Maybe things have changed a lot from the dot-com era and your best people are very different from the ones back then, but I doubt that that's true.  

Best buffer overflow tutorial

Since I mentioned that I revisited buffer overflows on our recent Jack Bauer Day, I've been asked several times what the best way to learn about these is. My rule of thumb is that if I get asked the same thing often enough that I notice it that it's worth doing a blog post about it, so here are my thought on this.

I'd say that the single best way to learn about buffer [that used to say "beffer," which the spell checker oddly didn't flag as being an error] overflows is to worth through the dot-com era tutorial by the hacker that went by the name "Aleph One."

This tutorial is available here as well as other places on the Internet. And although I'd say that this tutorial's description of the segmented architecture of Intel x86 processors isn't as good as it could be, the rest of the tutorial is excellent. It walks you through exactly how buffer overflows work as well as the thought process involved in creating an attack that exploits one.

It doesn't really take that long to walk through the complete tutorial - maybe a couple of hours or so. And if you actually walk through it you'll almost certainly learn something interesting.

The last time I did this it turned out that you had to actually think a bit about what was going on. The tutorial gives detailed examples of code to write and tells you exactly what commands to use to compile it, but what I got wasn't exactly what the tutorial showed. This was actually good because it forced me to look at the differences. This forced me to actually pay attention to the material as I worked through it and this meant that I learned more than I would have otherwise. 

So if you have more that a very casual interest in buffer overflows and have an hour or two of spare time, I'd definitely recommend that you work through this material. You'll probably be surprised at how much you'll learn and how easy it really is to understand.  

Thinking about data-centric security

A breach in which hackers steal 1 million credit card numbers really isn't very exciting these days. You can actually expect to see at least one a month. It's only the very big ones that are newsworthy any more, and there's absolutely no reason to believe that this steady stream of big breaches is going to stop any time soon. 

The good news is that there is a way to solve this problem. The bad news is that doing it isn't easy. It will require a very different approach to protecting data and different IT architectures than are common today. But it's not impossible.

It's going to be hard work, but the alternative is to accept the fact that essentially all sensitive data is going to end up in the hands of hackers. 

The fundamental problem is that security is never perfect. Technology isn't perfect and the people who use it and administer it aren't perfect. The combination means that mistakes happen and that hackers sometimes get sensitive data. But the way in which technology has changed over the past couple of decades has probably made it worse for the security of sensitive information and the direction that technology is moving is probably going to continue this trend.

In the pre-dot-com days, business data typically lived its entire life in a mainframe that protected the data fairly well. The dot-com era saw this data move out of the mainframes and out to where it was more readily accessible by web-enabled applications. Now we’re seeing data move even farther out – into the cloud. At each of these steps there have been huge gains in efficiency, but it also seems that the data has become more vulnerable as this has happened.

And it’s not clear to me that we’ve made the choice to move sensitive data to more and more vulnerable locations fully realizing that when problems with either people or technology happen that more data tends to get lost from the vulnerable locations.

It’s unlikely that we’ll be able to get people to revert to the earlier, more secure architectures, so we need to find a way to work with the existing architectures to protect sensitive data better. The best way to do this is probably with so-called data-centric security, and if I can get people to stop asking me questions about GTH pants, I’ll say more about that tomorrow.

What type of pants?

After yesterday's post I've been asked several times what "GTH pants" are. I believe that the "GTH" stands for some phrase in French. I was terrible in languages in school, so I can't quite figure out what it stands for, but GTH pants are usually very brightly colored and often have unusual designs embroidered on them. Things like bright pink whales or neon green crabs. They're the sort of pants that most people wouldn't even think about wearing.

Unless you're in a contest with your friends to see who has the nerve to wear the most brightly colored and distinctive pants in public.

These contests were never actually openly discussed in any way, but it was clear when a contest was underway, who the eventual winner was and that the winner had earned the respect of his peers in a way that would be hard to do otherwise. There were also other unwritten rules to these contests. Even though your pants might be extremely outlandish, for example, everything else that you wore with them would easily have passed your school's dress code. And any comments on other people's pants had to be restricted to "Nice pants."

Some people seem to like GTH pants. (Note that these people seem to be exclusively men. Women seem to have the good sense to not do these sorts of things.) Lots of the people that I knew when I was younger fell into this category. Maybe it was peer pressure. Maybe it was just the lack of wisdom that often comes with being young. But in any event, I used to wear GTH pants from time to time and even enjoyed doing it.

Come to think of it, this may be why I don't mind talking about cryptography and elliptic curves these days, topics that most people find as annoying as bright green pants with pink whales embroidered on them.

So maybe the lesson to be learned here is that nothing that you do when you're young really ends up being wasted - it just ends up being useful in unexpected ways.

Has PKI met its Waterloo?

[In a previous post, I mentioned how blatantly reusing old blog posts is an idea that I should consider. A person with whom I was recently talking happened to mention how he didn't think that I would actually do that. This person clearly didn't know me when I was in younger, when I was a serious contender in the GTH pants contests that people that I knew would engage in now and then. So without any more explanation, here's a recycled post. Just cut and pasted to ensure that any spelling and grammar errors are also duplicated.

And, no, I don't still have any of my old GTH pants. The closest that I come these days is a pair of orange cargo pants, but they're nowhere near as, uh, distinctive as the madras and embroidered pants that we used to wear in the summer. And to fraternity parties.]

Public key infrastructure (PKI) has found few uses outside national governments due the high costs caused by its high complexity. Has PKI met its Waterloo in these factors?

As Napoleon's defeated Grand Armée withdrew from the battlefield at Waterloo, its retreat was covered by units from the Old Guard, part of a small, elite unit under Napoleon's direct control. The soldiers in this unit were the best and bravest veterans from his previous battles and the most feared soldiers in Europe. In addition to being Napoleon's personal bodyguards, these units were his weapon of last resort, and they were rarely committed in battle.

Eventually the Old Guard's position became untenable, and the English General Charles Colville offered them a chance to surrender and avoid being inevitably destroyed by the vastly superior numbers of Wellington's victorious army. The French General Pierre Cambronne is said to have replied to Colville's request, "La Garde meurt, elle ne se rend pas!" - "The Guard dies, it does not surrender!" Cambronne later denied having said this, but that didn't stop the words from being inscribed on the statue of him that was eventually erected in his home town of Nantes, and they have become one of the legends that are commonly associated with the battle of Waterloo.

Despite its questionable authenticity, Cambronne's defiant reply may be good inspiration for a rallying cry that may be appropriate for supporters of public-key infrastructure (PKI) technology: "PKI dies, it does not surrender!" Except for the single use in implementing SSL, the use of encryption to identify servers and encrypt connections to them, PKI has proven to be extremely difficult to use and expensive to support, but its dedicated advocates insist on continuing its use to the bitter end. They even continue to support its use in the face of many newer technologies that are just as secure, more user-friendly and have a much lower total cost of ownership. PKI's proponents seem to never surrender.

PKI technology was a very innovative idea when it was first introduced. The digital certificates that PKI creates and manages provide the basis for authentication, digital signatures and encryption, all of which are very useful to have. Unfortunately, PKI also turned out to have many practical problems that its inventors didn't anticipate. These practical difficulties made it too difficult for the average user to use, which then resulted in very high support costs. So although it had an extremely promising start, it turned out to be unsuitable for most uses, the most notable exception being the use by government organizations.

Governments have an entirely different set of priorities than commercial enterprises: while businesses need to be profitable in order to survive in the long run, government organizations do not. Staying profitable is of utmost concern to businesses; spending their budgets and keeping people employed are among the highest priorities of most governments, and costs are relatively less important. So while most businesses found PKI to be unsuitable for widespread use, governments didn't find its high costs objectionable. This has led to almost all of the use of PKI being confined to governments and government contractors. And despite the high costs of doing it, governments have continued to move ahead with expensive PKI projects, with the American government alone having spent over $1 billion on the technology to date.

But while this has happened, many recent innovations have made it possible to provide the same benefits that PKI once promised, but at greatly reduced costs. Encrypted e-mail, for example, has recently become fairly popular due to heightened regulatory compliance concerns, so it has become much more widely deployed than it was just a few years ago. But if you look at the secure e-mail solutions that are commonly used today, you'll find that they're usually not based on PKI. Even if a solution does support PKI, that mode of operation is rarely used by customers. But while the use of newer technologies for encrypted e-mail has boomed, governments have continued to use PKI to provide this capability, perhaps attracted by its superior ability to help them spend their budgets and keep additional people employed.

Cost that is caused by unnecessary complexity may turn out to be the battle that PKI can't win when it's pitted against these newer technologies, so it may be somewhat appropriate to refer to it as "PKI's Waterloo." But because its supporters seem to have been inspired by Cambronne's legendary reply to Colville, it may be quite a while until we finally see them admit defeat.

On the other hand, Waterloo was the first time that Napoleon's Guards retreated without being ordered to do so. Perhaps supporters of PKI will suffer a similar change of heart, realize that their battle can't be won, and give governments a chance to realize the same cost savings that the commercial world has already experienced. We can only hope that this happens soon. Although it's often attributed to him, American senator Everett McKinley Dirkson never actually said, "A billion here, a billion there, and pretty soon you're talking real money," but it gives an idea of the savings that this might allow.

Much Ado About Nothing

Attention developers: You must set your text editor to tabs-at-8. There's nothing to discuss, no other viable option, and anything else would be crazy. If you're writing in Python, that is. I only recently discovered this fact, and it adds a new wrinkle to the mostly decomposed dead horse known as The Tab Wars.

For those who aren't familiar with Python, it has the distinction of being one of very few computer languages in which white space is significant. Specifically, the indentation of the code determines the block structure; there are no braces or other clutter to tell the interpreter otherwise. It takes a little getting used to, but is actually quite nice. "Then what about tab characters?" I wondered one day.

For the indentation to be unambiguous, the language must know how to interpret a tab character relative to spaces, and Guido chose tabs-at-8, presumably because that was and remains the most common defacto standard (more on that later).

So let's say you are working on some python code that looks like this:

You're thinking, Cool, I get a bonus! But, as it turns out, you have your tabstop setting at 4, not 8, and moreover the code has been edited by a variety of people with a mix of tabs and spaces and what you really have before you is this mess:

Now we know that python uses tabs-at-8, by definition, so what it will actually see and do is:

and you will only get your bonus when pigs fly. And that's just not right.

How did we come to this? What can we do about it? Sit right down and you'll hear a tale.

Eons ago, early hominids produced text on mechanical devices known affectionately as "type writers". Many of these devices included a cool feature whereby little "tabs" of metal could be put at various places along the "carriage" (the thing holding the "paper") and a special key (the "tab key") would cause the carriage to advance to the next such tab. This was very handy for lining things up vertically (itself an ancient artform, largely forgotten, but I digress).

About one eon ago, things got all modern and teleprinters (literally: print from afar) appeared, followed a few short decades later by the American Standard Code for Information Interchange—the ASCII we know and love to this day. Among it's 128 codes are 33 control codes, most of which are far too powerful and unstable to be used anymore, but Code 9 is our friend The Tab. More properly, the Horizontal Tab (HT), because there is also a Vertical Tab (VT) but let's just pretend there isn't. The tab codes were intended, and in fact used, to cause printing to jump to various places that had been manually configured into the teleprinter for a particular print run—so things would line up nicely on pre-printed forms and whatnot. But people soon realized that this was insane and they stopped doing it.

Now while they could have just used ASCII 32's (space characters) to position things, apparently it seemed too much of a shame to abandon an entire control code, so instead they decided, not entirely formally, that henceforth Code 9 would mean "advance to the next multiple of 8 characters on the current line"—what we today call "tabs-at-8". This also provided a nice 8:1 compression of whitespace, something that mattered to people when each bit cost several dollars to store or send.

"Why 8?" you ask. Interestingly enough, the value 8 was chosen by Satan, because he could see the future and likes to cause trouble.^{[citation needed]} So tabs-at-8 was then widely adopted and for years and years countless computer programs (and programmers) simply assumed that that was what ASCII Code 9 had always meant and would always mean.

Then things got even more modern and someone realized that it wasn't terribly difficult for a given text program to let a User (we've heard about them) select how many characters a tabstop should be! Whoohoo, by simply changing an editor setting you could instantly... mess up all your indentation? Make your text file randomly incompatible with various other programs? I really don't know what the point was but someone thought it was really cool, and the idea spread like hoola hoops in the late 50's.

Despite the Tabs Liberation movement, tabs-at-8 remained a standard of sorts, including things like HTML, CSS, most Unix utilities, and of course the Python programming language, to name a few. So of course the idea of user-selectable-tabstops faded away, everyone settled into living with the devil's tabs-at-8, ASCII files were universally unambiguous, and there's nothing more to say, the end. Well, not quite.

As it happen, we can still find one small exception to the de facto standard interpretation of ASCII code 9 as a horizontal tab aligned on 8-character boundaries. In the mid 1970's a young college dropout started a company called Microsoft. This company has since risen to some prominence in the field of software development, and the designers of its flagship development environment—Visual Studio—were struck by the overwhelming beauty of tabstops matching indentation. Additionally they could see the sheer godliness of indentation at 4 chars. Ergo: Visual Studio comes preconfigured with tabs and indentation set to 4 for all languages. All but XML, that is, which is set to 2 and 2 for reasons I don't want to try to imagine. And of course for Python it... oh, there's no Python configuration for the VS editor. Never mind.

Of course it's very easy to just say "Microsoft is evil". It's also fun, try it! But MS evil or no, countless hoards of developers work in Visual Studio daily, using the editor defaults, because, after all, where are these settings anyway? And they enjoy the sheer godly beauty and perfection of tabs at 4, indentation 4 and all is Good and Just in their sight. If only Everyone could share in this beauty and join with them in peace and harmony oh what a wonderful ASCII world it would be.

Sadly, for some reason there remain hundreds—perhaps thousands—of developers who simply refuse to do all their work in Visual Studio. I can't explain why this is, but they are stubborn. More often than not these troublemakers will not have tabstop set to 4 in their editors. They might even have their editor set to insert spaces instead of ASCII 9 codes. In fact, it turns out in the end that the only way to defeat Satan's tabs-at-8 trickery is precisely to set your editor to never allow those Code 9's into your file in the first place! Replace tabs with spaces. ASCII code 32 is, after all, the only unambiguous sort of space you can put in an ASCII file.

Still, if you are fortunate enough to live in a 100% Pure Microsoft Development World—and you never code in Python—then you are free to retain the myriad and compelling benefits of tabs 4, indent 4, keep tabs. It's a lovely thing and I'm sure you will be very happy, and you'll barely hear Satan laughing if you plug your ears and go la-la-la-la-la. Everyone else will have to suffer with Insert spaces, with whatever indentation is suitable for the language they happen to be using. And they'll survive. Just please be nice and don't touch their code.

One final thing. It pains me but I am compelled to mention the one glaring exception to the No ASCII Code 9's rule: Makefiles. The Makefile grammar specifically requires that ASCII Code 9 be used to introduce command lines; spaces simply will not do. As wikipedia gently states: "This aspect of the syntax of makefiles is often subject to criticism." Count me in on that.

Maybe later we can explore end-of-line characters.

Buffer overflows are cool

Yesterday was one of our quarterly Jack Bauer days, where for 24 hours we (at Voltage) can do anything that we want to do. This time I decided to look at buffer overflows again, and this turned out to be lots of fun.

It had been so long since I had taken a close look at these attacks and how to implement them that I had forgotten most of the tricks, so I got to relearn and rediscover all sorts of interesting things yesterday. I suppose that's one of the big benefits of having an imperfect memory - you get to experience the fun of learning new and exciting things more than once.

So the big lesson of the day (for me) was that although bow ties may be cool, buffer overflows are even cooler.

Implementing cryptographic pairings: a magma tutorial

If you're trying to understand how to implement a pairing, like you might need to do if you're going to implement one of the many forms of pairing-based cryptography, there's a good tutorial available here that's based on the 2009 paper Implementing cryptographic pairings: a magma tutorial by Luis J Dominguez Perez, Ezekiel J Kachisa, and Michael Scott that's available here.  This tutorial seems to do a reasonable job of walking you through the basics of how to calculate a pairing and includes lots of useful magma implementations, but it seems a bit inaccessible to someone who doesn't already know how to implement a pairing.

But that's a very hard problem to overcome. There's a big learning curve involved with getting up to the point where you understand pairings well enough to implement one. I'd say that it probably took me 80 hours or so of reading and thinking about what I had just read before I could comfortably implement the Tate pairing. (Other, smarter people can probably do it in less.) The tutorials that are available on-line these days (like this one) can probably reduce the effort involved in doing this, but I'd still expect it to be a fairly difficult step to make for the first time.

Returning to our regularly-scheduled programming

As many people have noticed, I haven't actually posted anything in a while. Here's roughly why.

Last year at one of meetings of my sons' Boy Scout troop, while shaking hands to say hello to the other fathers who were there, I noticed that the men who had white-collar jobs had extremely weak grips compared to the men who had blue-collar jobs. Being one of the relative weaklings, I decided to see what I could do to get a stronger grip and eventually came across Ironmind's Captains of Crush grippers. The strongest of these, the Number 4, is so hard to close that only a few people have ever done it, and they're the sort of people who win the World's Strongest Man competition. 

I decided on a more modest goal: working up to closing the Captains of Crush Number 1 gripper. After a month or two of training on hand grippers things were going quite well. I couldn't crush a potato in my hand yet. You need the strength needed to close the Number 4 for that. But I was getting noticeably stronger.

But then the ligaments in my hand decided that they had had enough and to let me know that they felt this way they caused extreme pain in my hands if I made common motions with them. Including typing.

Having participated in all sorts of sports as a kid (and being stunningly mediocre in all of them) I've had lots of sports-related injuries over the years, but I was still quite surprised by the reaction of my hands' ligaments, and it ended up being much more painful and taking much longer to recover from than I would have expected. 

But now that the hands are back in reasonable condition, I'll be able to start blogging again. I even have lots of hand-written notes of things that I need to blog about, so we'll be able to return the regular blogging schedule soon. Next Monday at the latest. 

Cryptography for Mere Mortals #4

An occasional feature, Cryptography for Mere Mortals attempts to provide clear, accessible answers to questions about cryptography for those who are not cryptographers or mathematicians.

Q: What do people mean by “Data Masking”?

A: This is a confusing term, because it can mean at least two different things, both related to data protection/privacy:

1. Encrypted or tokenized data that is converted back to plaintext, but with some of the characters “masked” by characters such as “x” or “*”;for example, a Social Security number of “999-88-1234” might be returned as “XXX-XX-1234”

2. Production data that is obscured or obfuscated for testing or development

In the first case, the goal is to avoid any possibility that end-users will be able to access the entire value—typically a credit card number, Social Security number, or other account number—while allowing them to see enough of the value to verify it. For example, credit card customer service representatives typically ask the customer to provide the last four digits of their credit card number to make sure they are viewing the correct account record. Those customer service representatives do not need to see the entire number; indeed, allowing them to do so exposes the company to risk, because an unscrupulous employee could conceivably copy those numbers down and use or sell them. The Voltage SecureData Web Services Server provides this form of masking, as does the Voltage SecureData z/Protect product for z/OS.

The second case relates to the fact that while test and development systems need some realistic-looking data to operate upon, such systems are typically somewhat less secure than production systems, so the risk of a data breach is higher. In addition, as with the customer service representative example above, developers and testers should not have access to live data, and for the same reasons.

Further complicating the picture, there are multiple techniques for performing this obfuscation:

1. Shuffling

2. Tables & rules

3. Random values

4. Format-Preserving Encryption

Each of these has different strengths and weaknesses.

Shuffling: Shuffling is useful when the set of possible values is known—for example, the 50 US states. Existing values are replaced by other values from within the set, hopefully in a consistent fashion to maintain referential integrity (e.g., NY always becomes CA). This can be appealing because the obfuscated data strongly resembles real data: names are still recognizable names, etc.

The problem with this approach is, of course, that the set of possible values is often not predictable (addresses, for example). A tool can analyze the existing data and shuffle the current set of values, but when a new value is added, there is then no good way to shuffle it, since the existing set is already mapped.

Tables and rules: This approach is similar to shuffling, in that tables of values are built and rules are applied to replace production values. Again, the resulting data looks and feels “real”, and integrity is maintained since a given value is always replaced by the same value. Because tables of values are maintained, “extra” values can be pre-allocated, solving the “new value” problem.

The downside is that maintaining these tables is cumbersome, requiring configuration decisions, databases, and maintenance as the data grows. It is also worth noting that for audit purposes, test data that is not instantly identifiable as being obfuscated can require additional effort to prove to an auditor that the obfuscation is, in fact, in place and working.

Random values: This involves replacing the production data with randomly generated values. The resulting data is easily identified as not being “real” because values are gibberish (names are not normal names, etc.).

However, unless a database of mappings is built, referred to, and maintained, referential integrity will be lost because a given value in one obfuscated data source will not produce the same value in another obfuscated source. This will cause many or most applications attempting to use that data to fail when they attempt to combine data from disparate sources. If such a database is built, it adds significant ongoing management cost.

Format-Preserving Encryption: Using Format-Preserving Encryption to obfuscate data avoids most of the pitfalls of the other approaches. It guarantees referential integrity, requires no ongoing management of an ever-growing database and ruleset, and provides test data real enough for most or all applications, yet easily identified as not being live data that was somehow leaked into test.

FPE allows continued use of existing index fields, and can be applied to many different data types. Care should be taken to apply appropriate policy in some cases, such as birth/death dates—to avoid, for example, death preceding birth. This restriction applies to all masking technologies, and there are ways to handle such situations.

Obviously Voltage Security, Inc., with the SecureData product family providing Format-Preserving Encryption, believes that this is the best choice for most data obfuscation projects. It is usually trivial to add an encryption step to an existing process that copies data from production to test. In cases where no such process exists, using tables often seems appealing, but it is important to recognize the ongoing management that such a solution requires.

Nanny Cam

Back in 2009 I wrote a blog post about locking doors. I proposed that there are some people who think it is an indictment on the door locker to lock doors. Some people are proud that they live in a place where they don't need to lock doors. But some of them seem to go too far, by not only being proud of not locking doors, but by casting aspersions onto people who do lock doors.

While listening to NPR recently, I heard a woman make a statement that reminded me of this. She said something to the effect of, "If you have a nanny, and feel you need a nanny cam, then you have the wrong nanny."

She was saying you don't need to employ security measures (nanny cam) if you hire the right nanny. It seems to me that the unspoken assertion here is that if you hire the wrong nanny, it's your own fault. That sounds like blaming the victim.

Suppose someone wants to come into your home and steal your valuables (let's say you're very wealthy). Suppose that someone applies to be your nanny. Now let's suppose this individual is very charming and does a great job in the interview, they say all the right things and really wow you over. You check out the references, do a web search, maybe even get a background check. Everything looks good, so you hire that person.

One day you come home to find the nanny and possessions valued at over $100,000 gone.

The woman interviewed by NPR would probably say, "You hired the wrong nanny." To which you reply, "Well of course we did! But how were we to know we hired the wrong nanny?"

Maybe if you had had a nanny cam, this could have been avoided.

The problem with the attitude that says, "If you feel there's a need to employ security measures, then there's something wrong with you," is that there are plenty of bad people out there and you just want to protect yourself from them. There are dishonest people who will victimize you if they can. Maybe they are professionals and maybe can get through the most extensive security measures. But not all dishonest people are seasoned professionals. Employing security measures might not prevent all loss, but it can prevent some. And employing no security will only prevent loss if no one tries anything.

Ultimately, the woman interviewed on NPR is advocating some security measures, namely screen the nanny. She was simply scornful of those who want to use more. But the point of more security is that there are more bad things you can prevent because some crooks will get by some of the security. In other words, maybe measure A alone or measure B alone won't stop all crooks, but put the two together and you have something much more powerful. There comes a point where there's not much more you can do, and there are professional crooks who could get by any measures you do employ. But if it is within your power to apply some more security, it is not inherently a bad thing to do.

I think it is wrong to impugn the character of those who wish to employ more or different security than you think is appropriate.

Happy FIPS day

Happy FIPS day! If you work at Voltage, you'll know what that means.