This is Part 5 of the Learning in Ill-Structured Domains Series. Read the previous part here.
We’ve talked a fair bit about Cognitive Flexibility Theory, a theory of expertise in ill-structured domains. About three months ago I published How Note Taking Can Help You Become An Expert, and followed that up with The Principles Are Useless On Their Own and Ill-Structured Domains Aren’t Necessarily Wicked. This collection of blog posts represent the progression of my thinking on CFT, as I worked out the implications of the literature. I think I have spent enough time now, and given myself enough distance, to understand what it all means.
This piece is a collection of actionable rules that will sum up the series on learning in ill-structured domains. If you’re new to these ideas, you’ll want to start from the top. But if you’re a long-term reader, this will hopefully act as a reference sheet for the entire series.
Rule 1: Figure Out If You’re in an Ill-Structured Domain
An ill-structured domain is a domain where concept instantiations are highly variable. A well-structured domain is a domain where concepts always show up in exactly the same way.
Most real-world domains are a mix of ill-structured and well-structured. For instance, in software engineering, computer programming is well-structured (a for-loop always works the same way), but software design isn’t (architectural approaches may vary in implementation, depending on the details of the problem domain).
It’s important to figure out which type of domain you’re in because the two domains demand different approaches for mastery. If you’re in a well-structured domain, you should just study the underlying concepts or principles. But if you’re in an ill-structured domain, you’ll have to seek out cases in addition to the principles, since the way these principles show up are hugely variable.
Coincidentally, the higher up you go in most careers, the more ill-structured the problems you encounter will be.
Rule 2: When Reading Principles or Frameworks, Actively Search for Cases
If you’re in an ill-structured domain, it’s not enough to read principles or frameworks if you want to be effective. You’ll also need to know what they look like in the real world, in order to have a shot of using them.
Why is this true? Recall that ill-structured domains consist of highly variable concept instantiations, which mean that each instance of a concept will be somewhat novel to the practitioner.
As an example, the 7 Powers framework tells us that businesses must build a ‘durable competitive advantage’, or risk having their profits eroded by the inevitable entrance of new competitors. In reality, it is totally possible — depending on the context — for this to take about a decade to show up.
When you’re reading a business framework, or a set of principles from a believable practitioner in an ill-structured domain, it is important to seek out 10-20 cases to colour in the detail. These cases should be as different from each other as possible. Don’t stop at just the frameworks alone.
Rule 3: Read History for Fragments, Not Lessons
When reading business stories or biographies, it’s tempting to reduce the story you’re reading down to a set of pithy lessons.
This is not a good idea. Most stories are artefacts of the specific moment in time they occurred. Any lesson you draw from those stories are thus likely to overfit the unique circumstances of the time.
A more appropriate way to read biography is to say “ok, this part of the story is an instance of X”, where X is a concept in the domain you’re interested in. That isn’t to say that the concept might instantiate in exactly the same way in the near future — merely that it is one way that it has instantiated.
One of the two claims that CFT makes is that experts in ill-structured domains reason by combining fragments of prior cases. They reason by analogy.
Why do they do so? The answer that CFT offers up is that it is extremely difficult to reason backwards from case presentation to underlying principles in an ill-structured domain. Cases are often too context-dependent to generalise into universal lessons. At most, we can say “this is one way that such-and-such principle can show up” — and then we can poke at the structural factors that led to it looking like that, though we cannot fully know all the factors involved.
As a result, you’ll get far more traction treating biography as a way to build up your collection of concept instantiations. Don’t bother trying to distill it down to atomised takeaways. Have some faith that your brain will draw from the fragments of stories you’ve collected, when you need to, in the future.
Rule 4: Reasoning by Analogy is … OK!
Experts in ill-structured domains tend to reason by comparison to prior anecdotes. This tends to result in certain … interesting conversational turns. Charlie Munger, for instance, has a tendency to quote from business events in the distant past — “this reminds me of the ‘Cola Wars’ right before the Great Depression …” and so on.
It is tempting to dismiss such anecdotes as ‘about one-off events’, and that ‘this is not applicable to our current situation’ and ‘why is he quoting from something half a century ago?’ But such objections miss the point. If you’re dealing with an expert practitioner that presents you with an analogy, it’s not likely that he's arguing that events from half a century ago would replicate in the here and now. Rather, a more accurate read is that the anecdote serves as a concept instantiation that may apply to you, even if the mapping isn’t perfect. Bringing up the anecdote makes it possible to tease apart the deep structure embedded within, in order to better reason about the present situation.
Another way of talking about this is that it is meaningless to reason in the abstract about ‘universal principles’ in business, since the way these principles show up changes depending on the industry, the time period, the competitive dynamics and so on. No, a more realistic way to reason is by referencing concepts in multiple analogies, which brings along the full context of the concept in action.
So when meeting an experienced practitioner in an ill-structured domain, you shouldn’t be too surprised if she says something like “this reminds of the time when …” — and then proceeds to tell you an anecdote. And you shouldn’t dismiss the anecdote out of hand; it's usually more productive to search for the concept embedded within, and engage with the structure of the anecdote, than it is to stick to the fiction that ‘first principles thinking’ is all there is.
Rule 5: Focus on the Differences in Cases, Not Just the Similarities!
When studying cases, it’s natural to look for similarities in each case that you study, especially when you understand that the cases are instantiations of some set of concepts. But CFT tells us that experts also look for differences in ostensibly similar cases.
The reason it makes sense to do so is as follows: in ill-structured domains, cases tend to have emergent properties that are greater than the sum of their parts. Cases also tend to be context dependent, and lend themselves to multiple ways of knowledge representation. A framework or lens that might make sense when applied in one situation might not work as well (or might have weird quirks) when applied to another. So experts know to treat each case as its own thing. Focusing on the differences is an easy way to remind yourself to do that, since it fights against your natural bias towards seeing similarity.
Rule 6: Search for Stories That Are Relevant To The Framework, But Might Not Fit
In an ill-structured domain, no framework can perfectly capture all the potential novelty that may emerge during concept instantiation. So it’s not just that frameworks may show up in really weird ways when you look at the actual cases, it’s also likely that a framework might not work even most of the time. And that’s ok!
CFT tells us that experts in ill-structured domains “avoid rigidity in understanding, remaining open instead, with an appreciation for the sometimes limitless range of uses of knowledge in new combinations, for new purposes, in new situations.” That’s a complicated way of saying that not every framework or lens needs to work in every case, and not every case must be explainable with a simple principle, concept, or framework.
So, when you learn a new framework, find confirming cases (as per Rule 2) but also remember to find related cases that the lens can’t be easily applied to.
The world is complicated, so cases are complicated. And so it goes.
This is Part 5 of the Learning in Ill-Structured Domains Series.
Originally published , last updated .
This article is part of the Expertise Acceleration topic cluster. Read more from this topic here→
