E-Business Commentary -- Getting Down With Dirty Data

In the software world they call it "dirty data," but in the straight-talking world of manufacturing it's simply called bad information. No matter what you call it, incorrect data affects all industries -- manufacturing, high tech, retail, and financial services alike. Sometimes the "dirt" takes the form of a simple keystroke error. In other cases there's something more insidious at the root: a business practice, for instance, that has persisted for years without being exposed to either the harsh sunlight of a tough manager's skepticism, or, ironically, the rigid requirements of a new piece of software. Whatever its cause, bad information costs business billions of dollars each year. For manufacturers, these added costs usually turn up in the form of wasted inventories that must be written off, shipping costs on inaccurate orders, and return costs, to name a few. The problem is anything but trivial. The potential for dirty data to gum up the works in business is everywhere. One of the most egregious instances occurred May 14 when a Lehman Brothers Holdings Inc. employee in London accidentally added a zero to a customer's order. The customer wanted to sell $43.8 million worth of stocks, but when the order was executed, it dumped $438 million worth of equities on the London securities market. The order went through near the close of a light trading day, sending the FTSE 100 index down sharply; the index finished the day off 3.5%. The following day Lehman was able to execute trades that reversed the positions mistakenly taken, and the index recovered most of its losses. London Stock Exchange officials indicated their belief that the mistake had been inadvertent, but the potential for manipulation of the market (or any stock market in the world, for that matter) was laid bare for all to see. This is what computers and the brain that drives them -- software -- can do. On the one hand, they combine to bring us huge efficiencies in nearly every walk of life, personal and commercial. They magnify our work efforts, automating processes on the plant floor and in the office. Unfortunately, the flip side is that their potential to magnify our mistakes, unintentional or otherwise, always looms large. A case in point is the situation faced by National Steel Corp. a few years ago (Putting Out Supply-Chain Fires). In a revamp of its order-entry system, the integrated steelmaker discovered that four out of five customer orders had errors. Too many incorrect orders can distort demand, causing a manufacturer to make more or less product than is needed. In National Steel's case, it took a team of people assigned to validate the accuracy of all orders to straighten out the problem. Even so, when the company later went to install complex production-scheduling software, once again the bad-data issue bobbed to the surface like an underwater mine waiting to explode. Incorrect information was the culprit, the steel firm's managers found. The new software refused to work with data that was not perfectly "clean." As it turned out, the problems were caused by an old way of doing things, the practice of making as much as 25% more steel than the customer -- in this case the auto industry -- ordered, as a means to make up for steel produced with defects. The problems arose when the customer rejected more or less than that amount, creating either a surplus or a shortage of steel. National Steel has since instituted a number of amendments to the production-scheduling system to reflect these idiosyncrasies in its ordering and production process. In a similar but much more disastrous example, Nike Inc. sent flawed data to its suppliers in Asia, causing them to produce and ship more than five million pairs of the wrong shoes -- shoes for which there were no orders. As a result, the shoe and apparel manufacturer reported a plunge in earnings of about one-third during the quarter ending in February. What's important here is the computer-geek adage, "garbage in, garbage out." You put in an error, you get out an error, although the saying could be amended to "small garbage in, big garbage out." Until manufacturers have software that can sniff out mistakes and correct them, incorrect information will remain the bugaboo lurking in their IT closets.