The Five Sources of Uncertainty

I’ve mentioned multiple times on this blog that I’ve really enjoyed Gary Klein’s The Power of Intuition, because it provides practical methods for those of us who desire to build expertise in our chosen domains.

I like it so much, in fact, that I’m now on my third reread of the book — even though I’ve already summarised the book, written copious notes, and put various components of Klein’s research into practice in my life.

One of the most intriguing chapters in The Power of Intuition is Chapter 8, titled ‘How to Manage Uncertainty’. Klein uses this chapter to detail the work he’s done for the U.S. Marine Corps, when he was asked to make sense of the types of uncertainty that Marines faced in the field.

(If you’re new to this blog, note that Klein’s work is for decision-makers in ‘regular domains’ — that is, domains like fire-fighting, software engineering, operations, and management, where expertise is possible. Klein’s work does not apply to ‘irregular domains’ like stock picking, venture capital, and political forecasting, where expertise is difficult and intuition should not be trusted. Read more about that difference here).

We all deal with uncertainty in our careers. Think about the kinds of decisions that you face at work:

• Your boss was seen talking to the manager of a separate division in the company — could it be that she is moving away? What should you do if you knew?
• Sales for this quarter don’t seem to be doing well. What’s going on?
• You’re tasked to do pre-sales development for a big client. Your boss warns you that this might be a tricky deployment, and wants to know if it’s worth the time cost. How do you find out if the risk is worth taking on?

The problem with reading up on solutions for uncertainty, however, is that the popular methods we read about very often come from the realm of finance, where professionals have had to deal with uncertainty for far longer and at a much larger scale than is usually the case for the individual career. Such methods include actively adjusting for one’s cognitive biases, updating beliefs using a Bayesian method, and calibrating one’s beliefs by considering the big questions.

I’m not knocking on such methods — they are incredibly useful if you have to grapple with uncertainty in an irregular domain in your career. But it is also useful to consider the methods that expert decision-makers use to deal with uncertainty … in domains where expertise is possible.

The Three Factors

Klein notes that across the studies he’s done, three factors repeatedly come into play when Marines operate under uncertainty — factors that he argues generalise well beyond the battlefield:

1. The source of the uncertainty. There are more different sources of uncertainty than you might think.
2. The types of tactics available for handling the uncertainty. Most people don’t appreciate the large repertoire of tactics available to them.
3. The decision maker’s personal tolerance for ambiguity. People have different personal styles and are sometimes surprised that others don’t share their feelings about uncertainty.

In Chapter 8, Klein deals with all three factors. He lists the five sources of uncertainty that he’s identified over the course of multiple studies for the Marine Corp. He lists the many, many tactics available for handling uncertainty, tracked over a few years in his research. Finally he includes the Budner scale (Budner, 1962), which gives you a method to assess your personal approach towards uncertainty.

I should note that this elucidation of sources and techniques aren’t enough to make you proficient. That involves actually putting the methods I’ve summarised here to practice. But of all the techniques that Klein covers, this chapter lends itself most readily to a summary, and the creation of a checklist. My recommendation is that you write this down in a note-taking app of your choice, and refer to it when you next find yourself facing some uncertainty in your career.

(In fact, Klein recommends that you hold in your head a project or difficult decision you are facing as you read the chapter. I quite enjoyed that when I first read Power of Intuition — and I think you'll get more out of this post if you do that here. Hopefully, by the end of this post, you’ll have a better idea of what is really troubling you about your project, and what you may do about it.)

The Five Sources of Uncertainty

The word uncertainty is rather badly defined. In finance and economics, for instance, people occasionally substitute ‘uncertainty’ for ‘risk’, and what they mean by risk may be the volatility of a stock relative to the overall market, or it could mean an impending exchange rate change, or it could mean bad external macro conditions … you get the idea.

Klein is more pragmatic. He notes that the uncertainty in his domain of study really devolves to just five things. We may call all of them by the same name, but we should not treat them like they are the same thing.

The five sources are:

1. Missing information. We can be uncertain because we are missing important information. We may not have it, or we may not be able to locate it if it is buried in information overload. Either way, we cannot access the information when we need it.
2. Unreliable information. We can be uncertain because we aren’t able to trust the information, even if we have it. We may suspect it is erroneous, or outdated, or that we are receiving the same report from several different sources. Even if it turns out to be accurate, this distrust will affect our decision making.
3. Conflicting information. We may have the information and trust it, but it might be inconsistent with other information we have and trust. If this is the case, we are facing an anomaly.
4. Noisy information. We may have to sift through a lot of irrelevant information — noise — but if we can’t be sure if it really is noise, we have to take it seriously and that adds to our uncertainty.
5. Confusing information. We may have all the information we need, trust all of it, find that it is all consistent, find that it is all relevant, but we could still be uncertain if we cannot interpret it. This happens if the data were so complex we couldn’t build a coherent story for purposes of explanation. Or the data could allow more than one reasonable interpretation.

The reason it’s important to check against this list is because you should respond differently to different sources of uncertainty. For instance, responding to a lack of information is different from dealing with information you can’t interpret. This is easy to say, but usually difficult to do: often, we attempt to resolve the latter scenario with a search for more information — perhaps to avoid admitting to ourselves that we can’t interpret what we have at hand.

Tactics for Managing the Five Sources

So what tactics do we have for managing the five sources of uncertainty? The answer is that there are 12 broad categories. And many of them are intuitive to us.

Klein recommends that you take out a sheet of paper and write down the pieces of your problem that correspond to the five sources of uncertainty. For each of the sources that apply, list down the tactics you are currently using. Then, go through the following comprehensive list of possible actions. By taking stock of the tactics that are available, you might get some ideas about how to get what you want.

Klein notes that the following list was assembled over a number of years. To his knowledge, the list is exhaustive.

1. Delaying. Sometimes, not making a decision is the best decision you can make. In some cases, the crisis that got everyone worked up yesterday turns out not to have been a big deal today. Skilled decision makers have a good intuition for when a situation is likely to resolve itself, or when more information is likely to come. The danger to this tactic is that you may delay because you’re afraid to make a decision under uncertainty. The good news is that with experience, you will often be able to tell when to delay from making a decision.
2. Seeking more information. Demanding more information is the classical reaction to uncertainty. Humans do this naturally. Sometimes this makes sense, but people often use information seeking as a way to buy more time. They are essentially delaying while appearing to be busy. You have to make sure that you’re not using information seeking as an excuse, when you should be deciding. Skilled decision makers in regular domains have intuitions about where to seek more information. They are also able to gauge whether the information is sufficiently valuable, and whether it is sufficiently timely.
3. Increasing attention. Here, you change your stance to become more active in monitoring the situation. This is different from seeking more information because you’re not looking for a specific thing; instead, you’re just observing a developing environment so you can make your move at the right moment. The danger to this tactic is that a) you overdo it, or b) others in your org will interpret your ‘regular check-ins’ to be assessments of their performance (or some other bad interpretation). The resulting organisational dysfunction defeats the purpose of your monitoring.
4. Filling the gaps with assumptions. This is another natural response. Instead of gathering more information, you can reduce uncertainty by making assumptions about what the missing information is likely to be. In practice, we all need to make some assumptions, or we wouldn’t proceed very far. In certain analytical situations, people are cautioned to track all their assumptions, so that they may review their decision-making processes later. But Klein notes that this is often impractical due to the number of assumptions we generate; instead, Klein notes that expert decision makers use their intuition to tell them which assumptions are more tenuous, and which can be safely made without tracking.
5. Building an Interpretation. Once you have enough information, you will ‘sensemake’ — that is, you will attempt to paint a picture of the decision at hand. At this point, your brain is likely to generate a narrative that explains all the facts. This is the narrative heuristic in action. The danger here is believing in the first narrative you create, ignoring the facts that may explain other alternative takes. The trick? Generate multiple explanatory narratives at the same time.
6. Pressing On. Sometimes you can’t have all the information you need before making a tough decision. The solution is to just accept this, decide, and move on. Colin Powell’s comment about not needing more than 70% confidence demonstrates a readiness to live with uncertainty.
7. Shaking the Tree. One common option is to actively shape the environment. For instance, you could pre-emptively cut costs, in order to pressure a competitor to react to you. Klein notes that senior executives sometimes initiated a course of action simply to learn more about an issue: “we bought that company because we wanted to learn about that business”. (Whether this is always a good thing to do is an exercise for the alert reader.)
8. Designing Decision Scenarios. A decision scenario is a narrative that paints a picture of the future. This triggers the use of our facility for mental simulation. In an earlier book, Sources of Power, Klein describes how the Royal Dutch/Shell Group planning department used decision scenarios to influence top management. To quote: “… these scenarios are like mental models except that they are written down, charted out, and developed to change the way executives think. The problem with forecasts and conventional scenarios is that they try to provide answers. Decision scenarios, in contrast, are built to describe the forces that are operating so the executives could use their own judgment. Wack (1985) writes: ‘scenarios must help decision makers develop their own feel for the nature of the system, the forces at work within it, the uncertainties that underlie the alternate scenarios, and the concepts useful for interpreting key data.’” In essence, you narrate possible alternate futures. Klein notes the best results involve no more than three variables, and no more than six state transitions in the narrative. (e.g. ‘Oil prices remain high, but interest rates dive and our offshore unit doesn’t hit production targets. Here are a couple of scenarios that could happen …’)
9. Simplifying the Plan. Another way to reduce uncertainty is to simply reduce the complexity of the plan you are formulating. You could reduce the number of objectives. Or, if that is not possible, you could make plans more modular so that tasks have less dependencies on the successful completion of other tasks. That way if one set of tasks fail, other tasks are not affected, and you can work to find alternative solutions for the bits that have failed.
10. Preparing for the Worst. You can also plan for the worst, so that you aren’t affected if the most horrible outcome comes to pass. You can pre-emptively add more resources (e.g. more funding, and more manpower) to a plan in motion. Or you could prepare to survive the worst-possible scenario — for instance, you budget for a year of flat revenue so that your company can take it if the big project doesn’t pan out.
11. Using Incremental Decisions. One of the most common approaches to uncertainty is to take an incremental approach: you make a small investment, or do a set of small prototypes, to see if it pays off. The benefits: you get to learn, therefore reducing unknowns, you get quick feedback, and you are able to make improvements to a plan. But the drawbacks can also be severe: taking small steps will signal that you are not fully committed, and you may inadvertently reduce the enthusiasm of the team. The other danger is to be trapped by ‘sunk cost’ arguments. To fight against this, force yourself to view the initial investments as a ‘cost of doing business’, not a stake that has to be recouped.
12. Embracing the Uncertainty. Finally, there is an approach of embracing uncertainty as the mode of operation for an organisation. Klein notes that certain CEOs embrace uncertainty as a competitive advantage. He quotes one such leader in book: “If you fall in love with your plans, if you get frustrated when you have to deviate from your plans, you will find it difficult to adopt a flexible attitude.” In this view, it is better to be flexible and to see what options present themselves to you on a day-to-day basis, rather than to make long-term plans and execute against them.

I should note here that I struggled with Klein’s 12th tactic when I first read it. What did it mean to embrace uncertainty as a competitive advantage? How might that even work?

Recently, however, I feel like I’ve gained a better understanding of this approach after I finished The Outsiders by William Thorndike. Thorndike investigates eight CEOs who excelled at capital allocation, and notes that most of them acted opportunistically, rather than sticking to the ideology of a long term plan.

The quote that struck me the most came from legendary CEO Henry Singleton, who said (in reference to time management): “I know a lot of people have very strong and definite plans that they’ve worked out on all kinds of things, but we’re subject to a tremendous number of outside influences and the vast majority of them cannot be predicted. So my idea is to stay flexible (…) My only plan is to keep coming to work … I like to steer the boat each day rather than plan ahead way into the future.”

Keep in mind that Singleton is immensely believable: to this day, both Buffett and Munger call him one of the best CEOs to have ever walked the earth.

(Of interest to me, but perhaps not to many others, Singleton pioneered many of the capital allocation strategies that Buffett later copied with Berkshire Hathaway.)

I’m not entirely sure if Singleton and co’s ‘visionless, flexible, opportunistic’ approach to business is universally useful. But it is an approach to dealing with uncertainty, I’ll grant you that. Klein doesn’t make explicit recommendations for one tactic or another, after all — he merely documents what people do when faced with uncertainty.

Fin

So let’s recap.

There are three factors that go into dealing with uncertainty: the source of the uncertainty, the tactics available for dealing with the uncertainty, and the decision-maker’s own personal tolerance of uncertainty.

The sources of uncertainty are missing information, unreliable information, conflicting information, noisy information, and confusing information.

The tactics include: delaying, information seeking, increasing attention, filling the gaps with assumptions, building an interpretation, pressing on, shaking the tree, designing decision scenarios, simplifying the plan, preparing for the worst, using incremental decisions, and embracing uncertainty.

For the third factor, Klein recommends that you rate yourself on the Budner Scale to gain an idea of your personal tolerance for uncertainty. He closes that section with:

(Given your score) some of the tactics for dealing with uncertainty that were listed in this chapter must seem more reasonable to you than others. Does this fit with what you’ve learned about your tolerance for ambiguity?

This, too, is left as a literal exercise for the reader.

While most of Klein’s work deals with the search for and development of tacit knowledge, I’ve found this chapter on uncertainty to be remarkably good as a piece of explicit knowledge. The recommended procedure is to go through this information like you would a checklist, and apply it to every important decision you have to make under considerable uncertainty.

I hope this helps you as it much as it has helped me.

Originally published 02 July 2019, last updated 03 July 2019.