What does it mean to be data driven in business? Most people think being data driven means looking at charts on a daily basis. Or they attempt to use data in their orgs, and then fall into one of the many traps that come with the territory. How do you actually get good at using data for your operations? And how do you build the mindset and the organisational capabilities necessary to do it?
In 2022, I did some work for Colin Bryar and Bill Carr, former Amazon execs and authors of Working Backwards.
The project I did for them was about explicating Amazon’s Weekly Business Review (WBR) for a broader audience. I took on the project for two reasons: I wanted to implement the WBR in businesses I was involved in, and I wanted to learn what it meant to be data driven, from folk who were present when Amazon laid the foundation for its operational rigour today.
What I didn’t expect to learn: many of the ideas behind Amazon’s metrics rigour seem to come from the field of Statistical Process Control (SPC) — a subdomain of operations research created by Walter Shewhart in the 30s and popularised in the post war era by W. Edwards Deming and his contemporaries. For whatever reason, these ideas seemed to have entered and then spread throughout Amazon — perhaps as a result of Amazon’s strong operational hires during the late 90s.
I thought SPC’s source ideas were valuable enough to dig into, given the influence they've had on Amazon’s approach to business.
This series is about those ideas, and the various instantiations of the ideas in Amazon, in manufacturing, and in other businesses. As it turns out, many of the principles necessary to become data driven in business were worked out over the course of the 30s through the 80s by Shewhart, Deming, and others. They simply aren’t as well-known.
These series of articles are somewhat related to Statistical Process Control, if only because the ideas come from the founders of the field.
- You Aren't Learning if You Don't Close the Loops — An article about the Plan-Do-Study-Act cycle, sometimes called the ‘Shewhart cycle’ or ‘Deming cycle’.
- Process Improvement is Trickier Than You Think (members only) — The organisational and political dynamics you’ll face when you attempt to do process improvement.
- The Disaffected PhD Skunksworks: A Story About Process Improvement — What it's actually like to do process improvement when the organisational dynamics work against you.
- Goodhart’s Law Isn’t As Useful As You Might Think — Goodhart’s Law says “when a measure becomes a target, it ceases to be a good measure.” It turns out that the field of Statistical Process Control have long developed methods for protecting against this effect. We look at those ideas, and then examine a particular instantiation of the approach in the context of the Amazon Weekly Business Review.
- How to Become Data Driven (members only) — What does SPC have to say about becoming data driven? The central idea it uses is deceptively simple: in order to become data driven, SPC practitioners argue, you need to understand variation. We talk about why that is.
- Operational Excellence is the Pursuit of Knowledge (members only) — Knowledge is defined as ‘theories or models that allow you to predict better’. It turns out that the mindset necessary for operational rigour stems from the pursuit of ‘knowledge’. The beginning of ‘knowledge’, in their view, is a deep understanding of variation. This piece covers the practical implications of this mindset.
- There is No Truth in Business, Only ‘Knowledge’ — We look at some of the more philosophical implications of pursuing ‘knowledge’ over truth, particularly in business. As it turns out, assuming that there are no stable truths in business is a pretty good hack.
- Process Behaviour Charts: More Than You Need To Know — Process behaviour charts are the SPC practitioner's secret weapon to understanding variation. It also happens to be the simplest way to differentiate signal from noise when looking at a measure. We talk about everything you’ll need to know to be able to use them well.
- What to Think When Looking at a Chart — In a business context, what should you think when presented with a time series? This seems to be a really dumb question that nobody talks about — but that we now have interesting things to say, given all that we've examined over the previous five parts in this series.
- The Deming Paradox: Operationally Rigorous Companies Aren't Very Nice Places to Work — This is only nominally part of the series, and was written as a reflection on Deming’s body of work between Parts 4 and 5 above. Includes a case study of Deming’s management philosophy as applied in Koch Industries.