Why are process improvement initiatives so difficult to do?
There’s a famous 2001 paper by Nelson Repenning and John Sterman titled Nobody Ever Gets Credit for Fixing Problems That Never Happened, which explains how process improvement initiatives fail. The paper describes a fairly common organisational dynamic that becomes hard to unsee once you realise it exists.
The organisational dynamic goes like this:
There are, broadly speaking, two ways you can improve productivity: you work harder, or you spend time improving your process. Colloquially we all know this — we call this ‘working harder’ vs ‘working smarter’, and the dynamic is similar.
So now let’s set the stage: everything is fine and dandy when you’re productive and able to meet all task demands. In venture funded, early stage startups, you might not be particularly productive (and the work may be overwhelming) but often you can keep up by just hiring more bodies to do the work, so things remain manageable.
The problem starts when you can’t throw more resources at the problem. Say there’s a hiring freeze, or you simply can’t hire at the same rate that work expands. Now what do you do?
Managers can ask their folk to work harder, or they can accept lousier results — unfinished tasks, subpar work — while they spend time improving the processes. In other words you can pick a ‘better before worse’ option or a ‘worse before better’ option.
Most managers pick the former. And of course they do — if you have a critical contract coming up and one of your production lines go down, your first instinct isn’t to stop the other lines in order to perform preventative maintenance — it is to increase production output or to run lines through the night to meet the contract! In other words, most managers choose the ‘work harder’ option because it produces short term results.
Meanwhile, the process improvement option gets sidelined because a) process improvement isn’t guaranteed (experiments fail; new tools turn out to have unexpected problems, etc), plus b) there’s often a delay before process improvement produces results.
Now things get trickier. One wicked property of this dynamic is that whichever option you pick gets reinforced. Working long hours works for awhile, but then workers burn out or machinery run themselves down. As a result, managers pile on the pressure, since working harder has worked in the past (and now things are worse; you didn’t have the time nor the spare resources to dedicate to process improvement in the first place, so why would you do so this time around?) On the other hand, managers that somehow get on the process improvement loop experience a virtuous cycle where process improvements bear fruit, leading to increased capability and therefore excess time that can be dedicated to more process improvement activities.
(But of course there’s always this tempting desire to ‘harvest’ the excess time from productivity gains and just work harder to produce more. At which point the entire thing tips over and becomes a negative downward-spiral loop again — the authors write: “even when improvement programs yielded initial results, cost and schedule pressures soon tempted many organisations into downsizing or higher performance goals that drained resources away from improvement, weakening the reinvestment loop and causing capability to stall or even fall.”)
Finally, as far as self-reinforcing dynamics go, it’s actually more likely for the negative loop to take hold because people tend to use ‘shortcuts’ when the pressure is on to deliver. For instance, a factory cuts down on maintenance time, or a software engineer skimps on documentation during crunch periods. Both activities provide a short term boost in productivity (less work, yay!) but then turn out to be forms of debt that come due later. This means that the negative effects of the ‘work harder’ loop are particularly vicious, and kick in with a vengeance once you start down that path.
The solution, according to Repenning and Sterman, is to change the culture. You must get company leadership to understand the ’worse before better’ dynamic that comes with process improvement. You must get leaders to accept marginally worse outcomes today, in order to get the continuous improvement flywheel going.
(Or, in another scenario, you must get managers to dedicate new resources to process improvement, even with the knowledge that throwing these new resources at current work will likely result in short-term positive gains.)
Either way, you need some form of management cover.
The following series of cases demonstrate what that looks like.