Follow Your Nose

I wrote a series on tacit knowledge in the middle of last year, where I discussed the forms of knowledge that ‘cannot be captured through words alone’. I framed that entire series with a lens that I think a lot of people are familiar with: imagine that you’re a junior employee, and you see a senior person in your company that is much better at some skill that you desire, and you say to yourself that you want to be like her. How do you get there?

The common answer that most people give is ‘oh, go do some deliberate practice’ — which is reasonable, because DP is famous. But what happens next is that you investigate DP, and then you try it out, and then you maybe get a little better, but most of the time you give up. Surely there is a better way of getting good? And the point of my series was that deliberate practice proponents mostly just repeat what they’ve read from some blog or book or YouTube video; they haven’t actually investigated the source literature. If they did, they would quickly learn that DP doesn’t apply to many skill domains, and that it has nothing to say about skills with little-to-no pedagogical development. I then said that when you want to learn from an expert, what you want to do is completely different — you’ll want to learn through simulation, and osmosis, not DP — and so the domain of knowledge that you want to dig into is a field called ‘Naturalistic Decision Making’, or ‘NDM’, which specialises in extracting tacit knowledge from the heads of experts. You’ll want to steal their methods for yourself.

That series did very well — it ended up on the front page of Hacker News and on Twitter, and in many newsletters. Whenever I talk to readers they tend to bring up the tacit knowledge articles, mostly because they are interested — as I was! — in getting better at their jobs.

But what I want to talk about in this essay is something much simpler: how did I find those answers? How did I notice this was a question worth answering? And how long did I take to look?

Ask Questions, Find Answers, Write Them Up

I wrote a half-serious tweet last week, saying:

I was being a little glib here, but the bones of the process are all there. (And I’m fairly certain that the two-to-three year timeline is real — good questions take years to answer!)

I thought that it would be useful to write up a few notes about that process, mostly because I had not known this was possible just four years ago. The gist of this is that if you:

1. Keep your eyes open for questions you want answered as you go through life (or your career!)
2. Be patient; investigate them when you have leads but also wait for the answers to come to you,
3. And then write the answers up as you uncover them …

Then you’ll have a process that is very similar to mine.

I’ll tell you the story of how I discovered the NDM literature, and then I’ll spend a little time speculating at why this process works. Finally, I’ll conclude with a handful of recommendations that would hopefully be of some use to you.

The Tacit Knowledge Question

Long term readers of Commonplace would know that I ran the Vietnamese software engineering office for a Singapore company in my previous job. I was responsible for everything: hiring, finding an office, getting the accounts done (and finding an accountant to get the accounts done), bank administration, and so on. As part of that experience, I learned to be a good manager; as part of learning to be a good manager, I created a standardised training program for all of our new hires.

One of the things that I discovered shortly after creating and debugging that training program was that I could teach all kinds of lower level software skills — code review, unit testing, deployment, familiarity with our codebase, Python best practices — but I couldn’t teach taste. And taste was important. By this I mean that our tech lead, Hieu, could sketch out the program structure for a particular project, and when we implemented his design, it would all work out fine; when anyone else — myself included — designed the structure for a project, it would inevitably lead to painful refactoring down the line.

This was a problem. It meant that the entire team had to bottleneck on Hieu’s approval. I thought to myself: “if I can learn this skill, maybe I can teach it!” and so I asked him, repeatedly, over three years, to explain to me how he did it, but all I got was “yah, I don’t know, it just felt right.”

I’ve written about this episode in my first tacit knowledge post, so you can read more about that there. And I remembered googling around for ‘learning science’, but not getting very far — at most I found pieces on spaced repetition, or dual coding, or deliberate practice, which I eventually investigated and found wanting. None of them gave me the tools I needed. So I stopped chasing down leads for a few years.

I started Commonplace in mid-2018. At the end of 2018, when I began putting together some resources for my Framework for Putting Mental Models to Practice series, I chanced upon a talk by Michael Mauboussin and Daniel Kahneman at the Santa Fe Institute.  In that talk, Kahneman mentioned — almost as an off-hand comment — that he enjoyed working with Gary Klein on a paper about expert intuition. I downloaded their paper and read it the same night. And as I read it, a thought occurred to me, quietly, as if spoken from afar: “What if ‘expert intuition’ was the keyword that I was looking for? What if Hieu’s ‘taste’ was really intuition, and what if this Klein person” — I did a quick Google on his background — “what if this Klein person had done research to unpack all the forms of expert intuition?”

I procured a copy of Klein’s first book, Sources of Power, and within the first two chapters realised that I had found exactly what I was looking for. Klein told the story of his research career — that he started out with a grant from the US military, who wanted to understand how soldiers made decisions under extreme stress and uncertainty. He discovered that many professionals in such situations ‘knew’ what to do; they didn’t really decide. He explained that expert intuition could be broken down into a model, and that this model, which he named the ‘recognition primed decision making model’, could be used to extract tacit mental models of expertise. He explained how his fellow researchers had invented techniques for extracting expertise from the heads of experts, and that they had also developed techniques to turn that expertise into training programs for the military. Most importantly, however, Klein explained that he had helped create a new branch of psychology, and that this field of psychology had a name: ‘Naturalistic Decision Making’ — the worst possible name (I thought!) for a field about expertise!

So no wonder that I couldn’t find anything about learning from experts — I was simply looking at the wrong places.

After this, it was easy. I started hunting for books and papers in the NDM community. I learnt that a large group of them had put together a new book in 2019, The Oxford Handbook of Expertise, which summarised the key findings of the field. And I learnt the names of the most prominent researchers, so that I could go and dig into their research.

I wrote my series on tacit knowledge a year and a half later.

Why It Works

So why did this work?

I think it worked for a number of reasons.

The first, most obvious, reason is that it is now the 2020s, and the internet makes it easier to find hidden silos of information.

The second reason is that, given that it is 2021, it is incredibly unlikely that you are the first person to have asked your question. Someone out there has likely already asked it before you, and someone has likely already investigated it. For things like learning science, this is most likely an academician (though the field of NDM is actually very applied!); if you’re investigating topics of finance or business, then what you want to find are historical accounts of businesses or market crashes, alongside more typical research papers.

The third reason this works, I think, is that I was perfectly willing to dive into primary sources of information. This was a happy accident — when I was in university I spent a fair amount of time reading research papers, because I hung around research-oriented friends. But by primary sources of information I mean more than just actual papers: I mean difficult tomes by primary authors, not popular science books; podcast interviews, not written accounts. The probability that you’d find the answers to your questions goes up rather remarkably when you are willing to do primary research. And the reason for this is simple: if you limit yourself to popular sources of information, you would find yourself at the mercy of middlemen. If no popular author or blogger is interested in your exact obsession, then you’re tough out of luck.

Wrapping Up

I think the important thing to say here is that I didn’t know I could discover the answers to my questions so easily. But I don’t think I’m very unusual here; it appears that I was not the first to have noticed this. In Chapter 9 of David Epstein’s Range, for instance, Epstein makes the case that much of modern science is siloed, and there are advantages to an individual researcher willing to hunt for ‘undiscovered public knowledge’. Epstein writes:

(Don Swanson) realized he could make discoveries by connecting information from scientific articles in subspecialty domains that never cited one another and that had no scientists who worked together. For example, by systematically cross-referencing databases of literature from different disciplines, he uncovered “eleven neglected connections” between magnesium deficiency and migraine research, and proposed that they be tested. All of the information he found was in the public domain; it had just never been connected. “Undiscovered public knowledge,” Swanson called it. In 2012, the American Headache Society and the American Academy of Neurology reviewed all the research on migraine prevention and concluded that magnesium should be considered as a common treatment. The evidence for magnesium was as strong as the evidence for the most common remedies, like ibuprofen.

Swanson wanted to show that areas of specialist literature that never normally overlapped were rife with hidden interdisciplinary treasures waiting to be connected. He created a computer system, Arrowsmith, that helped other users do what he did—devise searches that might turn up distant but relevant sets of scientific articles, and ignited a field of information science that grapples with connecting diverse areas of knowledge, as specialties that can inform one another that can drift apart.”

And in response to that, I wrote:

If I may add a possible corollary: if you are a blogger with an ability to read scientific papers at the edge of some field, you may be nearly as useful for the advancement of human knowledge as a scientist producing primary research in a lab; you merely have to read the right papers and ask the right questions and then write about what you’ve found.

But even if you aren’t a blogger, you can apply the same process to your life:

1. Ask questions — things that you might have noticed in your job, or in your past, that just don’t add up. Keep them in your head.
2. Look for answers — Google widely, read primary sources whenever possible, wait patiently when you’ve run out of leads. Because when you least expect it, you might run into things that set you on the right path.
3. And then write it up — if not for yourself, then perhaps for others who might come after you. But mostly for yourself.

In other words: follow your nose, and trust that it might lead you to someplace interesting. In 2021, the odds are pretty good that it will.

Note: Obviously, investigating answers sometimes requires putting things to practice. Related: a personal epistemology of practice, and the four theories of truth as a method for critical thinking.

Originally published 10 March 2021, last updated 11 March 2021.