Data is Just an Added Sense

One of the biggest misconceptions about data is that it somehow stands in opposition to ‘gut’, or ‘intuition’.

One way this gets reinforced is that you will meet individuals who are slavishly beholden to the ‘data’ — they will make arguments and demand hard numbers to back a position, even when the numbers are difficult (or impossible) to get. Another variant of this is that these same folk will follow — blindly — wherever the data leads them — even when it goes against common sense.

The misconception can also be more subtle than this, though. Take this tweet by my friend David Weinstein:

The problem with increasingly using data to inform decision making is data can only ever capture a subset of what was, never what could have been that wasn’t, and it’s the latter that makes all the difference.

This is true, but not entirely true. But it’s the worldview that is revealing here, and therefore interesting: both the tweet and the earlier ‘data is everything’ stance stems from the same misperception.

I think I’ve figured out a common-sense frame to sidestep this problem. Applied intelligently, you can use it as an operating principle (or simply as a ‘memetic catch phrase’) to prevent the spread of the wrong perception.

Data is Simply An Added Sense

Humans have five senses: sight, touch, smell, sound and taste. These senses help us navigate the world around us. Feel hot? Get in the shade. Smell something bad? Perhaps ask before you eat it!

Similarly, business operators have senses that help them navigate the running of their businesses. Most business operators use only two senses:

1. Intuition — this is the gut feel that emerges from making lots of decisions — some good, others bad — over the course of the years of running the business. Good operators tend to ‘get things right’ — they have a good sense of what to do and what not to do.
2. Qualitative sensemaking — this is the “get out of the building and talk to customers” advice that you hear so often in startup circles. Good operators know that in order to make good decisions, they have to constantly touch base with reality. This may mean doing regular customer interviews, but it can also take other forms, such as reading Reddit threads and social media posts about your company, or randomly sampling support tickets, or scheduling factory visits, or listening to recordings of sales calls in your downtime.

These two senses can often be enough to build multi-million dollar businesses. My old boss and I built one such business without any use of data, after all. And of course there are operators who are better or worse at the two senses. Some folk develop good operating intuition faster. Others are better able to extract insight from qualitative data. The two senses are clearly interrelated: good qualitative sensemaking often leads to better intuition; good intuition leads you to do better customer interviews. But this much is obvious.

Data is just an added sense. It is not better or worse than the other two senses, in the same way that hearing is not necessarily superior to sight, or touch not superior to taste. It all depends on context. Every human sense has its place; your brain’s job is to piece together information from each sense in a way that creates a useful image of your surroundings. In the same way, data belongs on a bench with the two other operating senses you have available to you. You should use the whole bench. And — like all senses — using data simply has different tradeoffs.

Data can help capture things that you can’t otherwise detect. For instance, if a customer says that they love your app and use it every day, you might wonder if they’re telling the truth. Looking at customer engagement data will let you know things about their behaviours in a way that interviews might never do. (But also vice versa: interviews can help you find things out about your customers that data can never give you).

Data is also a great help when doing process improvement. If you rely on intuition or qualitative factors alone, you may wait a long time before you know that a process change has led to an improvement. But if an effect is measurable — a big if! — you may speed up your improvement loops because you can get to certainty much faster.

Data is a third sense. Most operators use only two of the three senses available to them. This isn’t smart. You don’t decide to go on a hike blindfolded if you are naturally sighted; you are doing yourself a disservice if you do so. With humans, the impairment of a sense has a specific connotation: it leads us to say that a person is ‘disabled’. In business, however, the loss of a sense is sometimes used as a badge of honour, such as when LinkedIn influencers crow that being ‘data-driven’ is bad, and that they’re better off being ‘data-informed’ instead.

(It’s been my experience that the overlap between those who have terminology debates e.g. about being ‘data-informed’ vs ‘data-driven’ and those who actually use data to drive decisions is vanishingly small).

The Misconception

The misconception I referenced earlier is to believe that data is somehow ‘better’ or ‘worse’ than the other two senses. But now that we’ve framed it as an ‘added sense’, both beliefs are clearly misframed. It doesn’t make sense to say that sight is better than hearing, or that smell is better than taste: if you have N senses, it makes sense to just use all of them, instead of debating which is better.

Data is not more or less objective than qualitative sampling, nor is it superior to the hard-won intuition that comes from business expertise. It simply captures different things about reality. Every data-driven operator I’ve met over the course of my investigating the Becoming Data Driven series seems to understand this.

Here are a couple of examples:

1. In the context of the Amazon Weekly Business Review, participants are taught that “when stories contradict the data, always investigate the stories, not the data.” This is because data is always a lossy compression of reality. If a story contradicts your metrics, you are likely measuring the wrong thing. You should investigate the story to see what your data misses.
2. I often model behaviour from people more skilful than me; one of the most sophisticated operators I know was always able to reach laterally — across all three tools — when evaluating a hypothesis. They were just as likely to say “ok, let’s go talk to some customers” as they would go “ok, let’s instrument this and run an experiment”. Each sensemaking mechanism has different costs, which they constantly evaluated against the potential value of the knowledge gained.
3. One story that I like from early Amazon is that when the executives were trying to figure out why growth was stalling, they broke the question down two ways: “of the people who have heard of Amazon, why aren’t they customers yet?” and “of the people who are customers, why don’t they buy more?” For the first question they hired a market research firm to find folks who had heard of Amazon but hadn’t yet bought anything from the company. And for the second question they instrumented the order flow and asked users a survey question at the end of a successful order or at the point of cart abandonment. Both approaches told execs something about their business reality, and spanned the sensemaking toolbox available to them. The answer to both questions led to the discovery of what Eugene Wei later named ‘an invisible asymptote’; I highly recommend reading that essay for a primer.

If you treat data as ‘just an added sense’, you would not treat its conclusions as necessarily true. This was why former Amazon exec Colin Bryar was always quick to say that “your use of data should always stem from a deep qualitative understanding of your customer” — something that initially I struggled to understand. I’ve since concluded that this statement hints at a particular approach to using data: it means that whatever you learn about your customer should make sense, and you have more than one tool available to you to understand your customer.

One final way that we may explain this is to say that a goal of data is to ‘build and verify intuition’. This implies that your use of data does not stand in opposition to your gut. Recall that in business, knowledge must have predictive validity. When you make a decision, you are in effect saying: “I predict that this action will lead to such-and-such outcome.” If your predictions no longer come true, you should recognise that your knowledge is no longer valid. You should update your intuitions. But intuition can be updated from both qualitative and quantitative sources — though not everyone seems to reach for both.

Why don’t they reach for both? One reason is that it’s harder to use data to sensemake — because nobody really teaches you how to read a wiggling chart. But then it’s not easy to make sense of customer interviews either. Sensemaking is a skill — and you can get better at the skill. There’s no excuse not to try.

Make This Actionable

How do we use this frame?

I’ve hinted at its use at the start of this piece: you can use “Data is an Added Sense” as an operating principle in your company. This was not my idea; I was explaining this view of data to the CEO of a venture-backed company a couple weeks back. “That’s a good way of putting it,” he said, “I think we should turn that into an operating principle at our company.”

“Oh,” I said, surprised. “Why?”

“I’m afraid that folk will use data to browbeat everyone else,” he said. We talked about how data tends to be extremely legible, and therefore always runs the risk of overtaking other forms of sensemaking. “We encode everything as principles in our company,” he said, “This seems like a good addition.”

I agreed with him then. I said that more people would likely benefit from this frame. And so now I’m telling you.

Originally published 13 August 2024, last updated 14 August 2024.