This is an investigative note, and only nominally part of the Becoming Data Driven in Business series. A fair warning — this essay is not intended to be as useful as prior essays in the series.
One of the things I’ve been doing while writing this series on data is that I’ve been spelunking in the broader W. Edwards Deming literature, above and beyond the data tools and statistical methods of Statistical Process Control (SPC). Deming, as many of you probably know, was one of the fathers of SPC. But the tricky thing about his work is that it goes far beyond the data tools he helped introduce to the practice of business — and in many cases, seem in direct opposition to the ideas I’ve talked about in the past couple of essays.
At this point I’ve read enough to be sufficiently convinced that Deming’s system is a lot more ... profound than just the data tools alone, and in fact represents a fundamentally different view of operating a business. It’s also a little kooky: Deming’s name for his system is ‘A System of Profound Knowledge’; various writers have called Deming — amongst other things — a genius, a ‘business guru’ and a ‘moral philosopher’, which strikes me as both cultish and odd and naturally makes me more suspicious of his work. To be fair, it does seem like Deming was rather humble, though I continue to wonder at the kind of person who names their business philosophy ‘profound’.
The Deming Paradox
I’m writing about Deming’s overall philosophy here because of a fundamental tension I’ve noticed the more I’ve dug into his work: the data tools and statistical methods he popularised can and have been used to produce amazing business outcomes with horrible human costs. But the man himself espouses a philosophy that is deeply humanistic in nature. You could argue that his methods are used to produce the former outcomes mostly by ignoring his latter ideas. But I’m not sure how true this is, and I’ll get to that in a bit.
Let’s tackle the first part of that claim. What kind of horrible human costs? Well, consider the response that many of my Big Tech friends have had when I explained that I was digging into the source methods for Amazon’s operational rigour: “That’s nice.” they’d retort, “And cool if you get it to work for you. But Amazon has a funny way of sucking the soul out of everyone I know who has worked there.”
I’ve heard this enough times, from enough different sources that I mostly accept it as true. And from anecdotal evidence I also know that Western companies with sufficient operational rigour tend to be rather … unpleasant places to work. “Is that not the price of excellence?” a friend said, after I told him of this conundrum. “What, you expect such operational excellence to come for free?”
An obvious point in retrospect; I think about that exchange a lot.
(I exclude Eastern or specifically Japanese companies from the statement above because I don’t know enough about them to comment; Deming’s influence is largest in Japan. Perhaps they’re operationally rigorous, wonderful places to work? I don’t know enough to say, but I intend to find out.)
Here’s the tension at the core of Deming’s work. Arguably, Deming’s biggest and most lasting contribution might be the data techniques that he created for business. We’ve taken a look at some of those data techniques over the course of this series — the bulk of them focused on differentiating between routine and exceptional variation for the purposes of process improvement. Those same data techniques can and have led to incredible operational rigour. But then the systems that results from these tools sometimes result in crushing pressure for the individuals executing within the system.
What perplexes me is that so much of Deming’s overall business approach is not like this; it is in fact deeply humanistic in nature. Ed Barker writes, in A Symphony of Profound Knowledge:
Many of the myths Deming saw in management practice were based on false premises and beliefs. In his writings, Deming sometimes used the term “folklore” or “common sense” instead of myth. He wrote that common sense tells us to rank children, employees, students, and teams; to reward the “best” and punish the “worst”; to have quotas and numerical goals for individuals or groups; to assume that a problem always is caused by the people doing the work instead of the system in which they operate. He worked hard to help people to see the irrelevance, limitations, and the cost of these major myths entrenched in society. He wrote, “It is wrong to suppose that if you can’t measure it, you can’t manage it—a costly myth.”
Does this sound like a man who created many of the data tools that enabled human-crushing business practices? It sure doesn’t, to me. But Deming argues that many of his ideas — those same ideas that we’ve discussed in this series — lead quite naturally to his human-centric conclusions.
Take, for instance, his argument that understanding variation is the beginning of ‘knowledge’. If we accept that routine variation is present in every natural process, and that setting a target does not in itself help you achieve that target (you need to ‘listen to the voice of the process’, not the ‘voice of the specification or the target’) then you get conclusions like the following (again, from Ed Baker’s book):
Grading was developed more than 150 years ago as a method for the teacher to have a private conversation with the student’s parents. It has been replaced by what could be called the myth of defect detection. Rating and grading is analogous to inspection of parts in the factory. Inspection is after the fact, after the parts are produced. Therefore it does not improve the process that produced the parts; it does not prevent defects from occurring (emphasis mine). Likewise, many administrative practices in schools do not help to improve the system of education, practices such as improvement goals without a method to achieve them and grading and comparing schools to provide pressures to be above average, as if more than half of any group ever could be above average in any measurement.
Deming reported the case of an elementary school student who received two successive test scores below average. The teacher took that as a diagnosis of the need for remedial training and informed the parents. Most people understand that you can flip a coin and get two or three, even four successive heads or tails just by chance. This happened to the daughter of a colleague of mine, a consultant to organizations to help them learn and apply Deming’s System of Profound Knowledge, especially his theory of variation. When he was notified by the teacher of the “problem,” he provided her with some unsolicited consultation. Obviously, the teacher did not intend to hurt the child. She thought she was doing the right thing, but if she did send the child to a remedial class, it could have communicated to the child that she was not smart and begin a downward spiral of “Why bother to study since I can’t learn?” In this case, the teacher and the administration could have used remedial education to gain some knowledge of variation.
I can assure you that I had not thought to apply Deming’s ideas on variation to something as far-flung as school-level grading. But Deming had.
There are likely other implications of Deming’s ideas that I’ve not thought through.
How Effective is this Deming Stuff, Really?
I have a couple of big, unanswered questions at this point. The biggest is whether any company has successfully implemented the entirety of Deming’s business philosophy — both the data driven, operationally rigorous parts, as well as the humanistic, “treat all workers with joy and respect” part, and gotten it all to work. Since Deming’s ideas have found the most acceptance and implementation in Japan, I figure I’ll have to dig into the histories of various Japanese auto companies, which I don’t know a great deal about.
(If you’re reading this and have good book recommendations for Deming’s impact on Japanese industry, I’m all ears!)
But it so happens that there is a famous Western company that has adopted Deming’s business philosophy wholesale. That company is Koch Industries, and reading about Deming in Christopher Leonard’s Kochland was perhaps the first indication to me that this entire Deming thing wasn’t some odd management cult from an obscure business guru. (Also: discovering that Amazon inherited from SPC and Deming was one of the founders of SPC was a second signal, and what finally tipped me over to digging into his work). Now, it’s not clear to me if Koch Industries is a pleasant place to work — Koch is notoriously private, and I have no privileged insight into a company this far removed from my own industry. You also can’t say that Koch has been net good for the world — some people think its leadership positively evil (even as they — like most people in the world — use its products). But what you can say is that it has been ridiculously successful at business, and its goal to be the best counterparty of choice with its customers, partners and suppliers have mostly held it in good stead. Deming’s system has had no small part to play in that.
Koch, Honda, Toyota and Sony makes for more than three instances where Deming’s ideas have led to operationally excellent businesses. So, yes, I do believe that Deming’s ideas work. And I’d like for you to get a taste of what that means with an excerpt from Chapter 6 of Kochland below.
The entirety of Deming’s influence on Koch is documented in Chapter 6 of the book. It opens with the following story:
Koch and Deming (Excerpt from Kochland)
(...) in the early 1980s, Charles Koch’s philosophy was just beginning to be incubated. He didn’t have a fully formed set of guidelines to pass on to his managers. Instead, Charles Koch focused on the fully formed set of guidelines shaped by another man, a consultant named W. Edwards Deming. Charles Koch became fixated on Deming, and he set out to apply Deming’s methods across Koch Industries.
W. Edwards Deming was not simply a business consultant. He was more like a guru. For many years, he remained an obscure thinker within the United States, but he had become a major figure in Japan, where Deming helped Japanese automakers improve their factory production and build some of the strongest manufacturing companies in the world. Like Charles Koch, Deming stood apart from the mainstream thinking of America’s business community, and he wasn’t afraid to speak his mind about it.
Deming’s passion was for making better products, or, more accurately, for creating a system that could make better products,” the journalist David Halberstam wrote. Deming wanted to overhaul American management using mathematics. He was a quality control engineer at heart, and he thought that the manufacturing process could be improved only by using hard statistics. Deming taught companies to measure what they were doing, to analyze it, and then to improve it. (Cedric’s note: is this really true? I’m not sure).
Deming’s concept of continuous improvement was applied throughout Koch Industries, and the results were dramatic. One of the most successful students of continuous improvement was Phil Dubose, the oil gauger in Louisiana who had already mastered the Koch method of measuring oil. Dubose would eagerly absorb the lessons of Koch University. In doing so, he would see firsthand why Koch Industries became one of the largest companies in America even when most people had never heard of it.
*
After rising through the company ranks over the years, Dubose was promoted in 1982 to oversee Koch’s marine operations around the Gulf of Mexico. This put Dubose in charge of a fleet of barges that went from terminal to terminal, collecting crude oil and then shipping it to refineries in Texas. Some of his barges even traveled north on the Mississippi River to Koch’s refinery in Pine Bend, Minnesota.
Dubose was terrified by the promotion. There had been two previous managers of the marine unit, and both of them had failed to turn a profit. “If it failed again, I was going to go down with it,” he recalled. Dubose was determined to make the shipping barges turn a profit. He knew he had one tool to help him do this: the charts of W. Edwards Deming.
The fleet Dubose oversaw initially consisted of five large barges. They each carried about 8,500 barrels of oil. Each barge had a skipper and crew who lived on the craft while it traveled from port to port. The first matter of business that Dubose focused on was keeping costs down. Fuel was the largest cost the barges incurred. Rather than let the skippers fuel up the ships when they wanted to, Dubose required them to call his office when they were running low on gas. Then he would call the local ports and find the best price for gas, sending the skipper to the best location. This helped cut costs right away.
The tools from Deming helped Dubose go even further. Of all the charts he learned to make, he found that by far the most useful was called a run chart. Even decades later, he’d talk about run charts as if he were discussing a cherished family pet. “The best chart out of all of them . . . is that old-fashioned run chart. It’ll tell you where you’ve been and where you’re going,” he said.
A run chart broke down all the costs that a barge would incur. It had a separate category for each cost: groceries, fuel, maintenance, ship damage, and supplies. The run chart allowed you to track these costs as they shifted from month to month, letting you see “where you’ve been and where you’re going.” Dubose was taught to look for cost spikes. The reason was simple: you figured out what caused costs to spike, and you avoided it. Then you figured out what caused costs to fall, and you replicated it.
(Note: Run charts are a simpler form of the process behaviour charts we explored in How to Become Data Driven. The goal of run charts are the same as process behaviour charts: they display time series that help operators differentiate routine variation from exceptional variation. Unlike XmR charts, though, they do so through a simple set of four rules. Read more about run charts here.)
The critical part came next. Dubose printed run charts for each vessel and posted them in the skippers’ cabins. Each skipper could then see for themselves where they were running up costs and where they were saving money. Dubose turned each skipper into his own manager. Skippers were free to make their own decisions based on the run chart. Then Dubose went further. He started tracking the profits and losses for each barge. This made each skipper a small-business owner and each barge a small business. The skipper had all the information he needed to boost profits and the freedom to act on that information. And Dubose had total visibility into his fleet; he knew which ships were losing money and which were making it.
“It got to the point where the boats were competing against each other. I was just sitting back like a big old Cheshire cat in a tree,” Dubose said. Using data to drive changes at the level of each barge, Dubose boosted profits in the marine unit overall. His profit margin reached 33 percent. The trucking division, by contrast, was lucky to see a profit margin of 8 percent or 9 percent. As he boosted profits, Dubose was given more freedom and more resources. He added more ships, buying larger barges that could ship forty thousand barrels of oil at a time.
All the while, he was in contact with managers from Wichita. They helped him prepare his run charts, and they taught him other tricks from Deming. As he talked with more managers, Dubose learned that not everyone embraced the Deming formula. A lot of managers were accustomed to making decisions based on gut instinct. They thought the charts were just a gimmick. But as many Koch Industries employees would learn over the years, Charles Koch did not consider his guidance to be a gimmick. And following his guidance was not optional.
“Some of these poor rascals just couldn’t embrace [Deming’s] thing. They couldn’t get their arms around it. . . . They’d just zigzag a line across with a bunch of numbers. The people who couldn’t support that, well, most of them were let go,” Dubose recalled.
Leonard documents how Koch’s dedication to Deming’s ideas eventually led the company into several sticky situations, not the least being targeted in a Senate Select Committee investigation for oil theft in 1988, a direct result of immense internal pressure on employees as part of its continuous improvement program. Charles Koch later changed the company’s operating playbook to have 100% regulatory compliance, 100% of the time, to ensure there were no repeats of this and other incidents throughout the 90s. But I bring this up to be completely transparent with you: I expected this series to be about becoming data driven, in the service of rigorous business operations. I did not expect it to eventually tie into systems and people and incentives in broader society the way it has. Deming’s ideas are ultimately tools, and tools can be misused. I guess I shouldn’t be so surprised.
More soon.
This is nominally part of the Becoming Data Driven in Business series. The next part is Part 5, here: Process Behaviour Charts: More Than You Need To Know.
Originally published , last updated .
This article is part of the Operations topic cluster, which belongs to the Business Expertise Triad. Read more from this topic here→
