Five Takeaways on Predictions (For The New Decade)

One of the things that I didn’t expect when I finished my series on the Good Judgment Project last week (parts one, two, and three here) is that it would coincide with the end of the decade given the close of 2019. As it turns out, one of the things that people like to do at the end of a decade is to make forecasts about the coming decade. (They also like to write ‘decade in review’ posts, and marvel at all the predictions that we collectively got wrong).

Given that we’re seeing an uptick of predictions in our feeds over the next few days, I thought it would be a good reminder to list a few takeaways from the Good Judgment Project to keep in mind as we read predictions and look forward to the new year.

1. Predictions more than five years out are no better than chance.

One of the strongest results that Phillip Tetlock found — both from the four-year-long Good Judgment Project and his earlier, twenty-year-long, Expert Political Judgment forecasting tournament — is that forecasts for events more than five years out in the future are no better than chance. You’d get equally good results from a chimpanzee throwing darts at a board.

If you read a prediction in the next few days about something that can only be judged at the end of the 2020s, feel free to ignore it.

2. Beware vague verbiage predictions with no clear deadlines.

Associate Professor Yascha Mounk of the John Hopkins School of Advanced International Studies wrote the following prediction of the 2020s:

He does two things that allows him to weasel out of a proper assessment: he uses the word ‘may’, and he doesn’t set a deadline for his prediction. That way, if he gets it wrong, he gets to:

• absolve himself of the mistake (“I said may, which means I didn't believe it very much”)
• assert that his statement was a statement of hope.
• argue that “it hasn’t happened yet, but it will!”
• evaluate the outcome however he wants (“I’ll give myself a 50% on that, because Twitter kinda went away but Instagram turned out to be the election platform of the future  …”)

Serious forecasters make predictions on concrete statements with clear deadlines. They say “Twitter will be down 20% monthly active users by Q4 2022 when compared to Q4 2019” and not “social media will fade in importance”.

3. Remember that predictions are possible!

When Bill Gates wrote the legendary memo The Road Ahead, he said:

What I’ve said that turned out to be right will be considered obvious, and what was wrong will be humorous.

It is easier to believe that predicting the future is impossible and that all forecasts are bunk. Even experts who make predictions today preface their predictions with “oh, it's unwise to predict the future.”

But this isn't useful to us: we are required to make predictions in business and in life all the time. This makes believing in the impossibility of prediction a rather nihilistic view.

Tetlock and his collaborators show us that forecasting may be very difficult, but an above average level of performance can be attained. They demonstrate that the techniques of the best forecasters aren’t as magical as we might think; that superforecasters may be emulated and that their techniques are learnable. And they remind us that while we should be humble about the possibilities of fat tails and one-off events, that shouldn’t keep us from trying our best in the face of the fundamental uncertainty built into our world.

4. When it comes to predictions, things are less different from the past than analysts think.

Tetlock responded to Mounk’s Twitter thread of predictions with “Unfortunately, its first falsifiable forecast is false. People often over-predict change. One reason: we get more credit for correctly predicting change than the boring status quo.”

Nobody starts a prediction with “I predict the future will be the same as the past.” And yet it very often is. A New Yorker review of Tetlock’s work in Expert Political Judgment explains what he means by this:

When television pundits make predictions, the more ingenious their forecasts the greater their cachet. An arresting new prediction means that the expert has discovered a set of interlocking causes that no one else has spotted, and that could lead to an outcome that the conventional wisdom is ignoring. On shows like “The McLaughlin Group,” these experts never lose their reputations, or their jobs, because long shots are their business. More serious commentators differ from the pundits only in the degree of showmanship. These serious experts—the think tankers and area-studies professors—are not entirely out to entertain, but they are a little out to entertain, and both their status as experts and their appeal as performers require them to predict futures that are not obvious to the viewer. The producer of the show does not want you and me to sit there listening to an expert and thinking, I could have said that. The expert also suffers from knowing too much: the more facts an expert has, the more information is available to be enlisted in support of his or her pet theories, and the more chains of causation he or she can find beguiling. This helps explain why specialists fail to outguess non-specialists. The odds tend to be with the obvious.

Nassim Nicholas Taleb likes to rage at financiers and their use of value-at-risk models. He asserts that they deserved to be crushed, because the future is often unlike the past. Paradoxically however, for most of the time, most things keep on keeping on.

This fact is why prediction is both possible and so difficult at the same time.

5. Predictions are attractive because narratives are catnip for the human mind.

In that same New Yorker review of Expert Political Judgment, Louis Menand says it best:

Prediction is one of the pleasures of life. Conversation would wither without it. “It won’t last. She’ll dump him in a month.” If you’re wrong, no one will call you on it, because being right or wrong isn’t really the point. The point is that you think he’s not worthy of her, and the prediction is just a way of enhancing your judgment with a pleasant prevision of doom. Unless you’re putting money on it, nothing is at stake except your reputation for wisdom in matters of the heart. If a month goes by and they’re still together, the deadline can be extended without penalty. “She’ll leave him, trust me. It’s only a matter of time.” They get married: “Funny things happen. You never know.” You still weren’t wrong. Either the marriage is a bad one — you erred in the right direction — or you got beaten by a low-probability outcome.

In all honesty, I think people are writing and publishing predictions for the coming decade because it’s fun — because it’s such a pleasant thing to do this time of the year, curled up in sofas with eggnog and cake and good cheer. And it doesn’t hurt that predictions are attention-grabbing. Expert writers like making predictions and readers love reading them. The world is admirably arranged.

Perhaps Mounk was making predictions on Twitter because he was bored. Perhaps Tetlock was wrong to be poking at his thread, the same way that a serious film critic has no place near a Star Wars film. But this shouldn’t matter to us observers: as long as we recognise that we’re reading unrigorous predictions for entertainment and not for education, then all is well, and the world spins ever uninterrupted.

Have a happy new year, everyone. I’ll see you on the other side.

This post is part of The Forecasting Series. Read the rest of the posts here.

Originally published 31 December 2019, last updated 12 January 2020.