The Principles are Useless On Their Own

This is Part 3 in a series on learning in ill-structured, novel domains.

Last week we took a look at Cognitive Flexibility Theory (CFT), a theory about adaptive expertise in ill-structured domains. This week we’ll talk about some of the second and third order implications of the theory.

What falls out when you try and take CFT seriously?

Dalio’s Principles

One of the first series I published on Commonplace was a sequence of essays on Ray Dalio’s 2017 book Principles. I mentioned that Principles had a huge impact on me when I first read it in 2011; in both versions of the book, Dalio argued that coming up with generalised principles to help you deal with repeated scenarios should make you more effective over time. In fact, he goes further:

Principles are fundamental truths that serve as the foundations for behavior that gets you what you want out of life. They can be applied again and again in similar situations to help you achieve your goals. (...) it’s very rare for people to write their principles down and share them. That is a shame. I would love to know what principles guided Albert Einstein, Steve Jobs, Winston Churchill, Leonardo da Vinci, and others so I could clearly understand what they were going after and how they achieved it and could compare their different approaches.

The telegraphed idea, of course, is that if you can get at the principles of other successful people, you’d be able to copy the bits that work for you or appeal to you, and become successful yourself.

I no longer believe this to be true. Or, to be more precise: after learning about CFT, I think you need to add another step for Dalio’s assertion to become true. Dalio’s argument goes something like:

• Let me tell you about my principles, and you can pick and choose what you like.
• You can then act in a principled manner, and achieve your goals.

But this hasn’t been true in my experience. What is more accurate is something like the following:

• You read someone else’s (in this case Dalio’s) principles
• You sit with one principle for a long time, looking for cases where it can be expressed.
• You work out enough of the application of that principle through trial and error or through reinterpretation of other people’s experiences, and can then act in a principled manner in order to achieve your goals.

Principles has made a huge impact on me, but it hasn’t resonated as much with those I’ve recommended the book to. And I’ve had identical experiences with other, similar, books — amongst them Andy Grove’s High Output Management, a short book of management ‘principles’, if you will, that I attribute much of my management skill to. I’ve possibly recommended HOM more than I’ve recommended Principles; like Principles, HOM doesn’t seem to result in demonstrably better outcomes for most of the people I recommend it to — at least, not to the degree that it did for me.

I’ve often wondered at this result. Over time, I’ve found myself recommending principled books less often, due to the seemingly ineffective outcomes of such recommendations. (Actionable books are different — usually people are able to pick up on the actionable bits, even if they struggle to get at the principles embedded within the recommendations).

I’ve mostly chalked up this effect to the nature of reading itself — most people are too busy to read books at any amount of serious depth, plus you can only learn what you are ready to learn.

But CFT gives me another, plausible explanation as to why most readings of HOM and Principles are so ineffective. Most people don’t realise that you have to sit with such books for a long period of time in order to use them well. By pure chance, I consumed both books with a lot of time baked in: I read and reread Principles over a period of 7 years; I took a month per chapter for HOM, over a period of about eight months, moving on from each chapter only when I was sure I had applied and worked out most of the implications of the preceding chapter. It has always struck me as weird that it was necessary to sit with these books for such an extended period of time — why should it take so long to go from principles to practice?

My typical answer has always been that ‘it takes some time to internalise principles’ … but I think CFT has a more coherent explanation. Both Principles and High Output Management are books about applying concepts in ill-structured domains. The problem is that they are books about abstract concepts, but usage of such concepts demands some familiarity with how the concepts are instantiated in real world cases. What CFT tells us is that it is very difficult to go top-down from abstract principles to effective action; I am convinced, at this point, that Dalio and Grove are able to articulate the concepts, they are able to explain the reasoning behind those concepts, but they do not actually operate from the explanation of the concept itself. If we believe CFT’s claims about experts in ill-structured domains, Dalio and Grove likely do something much simpler: they refer back to examples of the concept in action and reason by comparison to the set of such cases in their heads.

I happen to believe this because it’s also what I do today, after spending so many months working through HOM back in 2016. Grove’s core idea, stated in the first chapter of the book, is that the ‘principles of production’ may be applied to knowledge work in the same way that it is applicable to manufacturing; one way to read the book is that it consists of case instantiation after case instantiation of this principle in action when applied to management. At this point I can reel off any number of examples from my own personal practice that display this principle in action.

And so here is one implication of CFT: when you’re reading books of principles, you should schedule some time — perhaps a month or two — for each principle that you desire to use. Then, you should spend that time looking for real world cases that express the principle in action. These may come in the form of stories from friends, or stories in books, or introspection from personal action.

Another way of saying this is that you may ‘grok’ principles by reading about them, but the only way you can use them is when you’ve worked out all the ways that they’re expressed in the real world.

Sometimes this takes years.

Take a Simple Idea and Take It Seriously

Now that I say it like this, I suspect that there’s a similar dynamic going on with ‘take a simple idea and take it seriously’. Late last year I wrote about Nomad Investment Partnership’s Nick Sleep and Qais Zakaria’s collective careers, as viewed through the lens of Charlie Munger’s ‘take a simple idea and take it seriously’.

I marvelled at how Sleep and Zakaria took one insight and built an entire investment career out of it; they quit at age 45 after returning 921.1% to their clients over a period of 13 years.

Why did it take Sleep and Zakaria so long to work out all the implications of scaled-economies shared? The reason, I think, is likely what I argued above — that concepts are only useful when you know how they are instantiated in the real world. If you read the Nomad Partnership letters, you’d realise that Sleep and Zakaria spent most of it working out how the advantage expressed itself in real world scenarios. They did so from 2004 (the first mention of shared cost savings) to 2012 (when Sleep wrote, to his surprise, that many great historical businesses seem to share a similar philosophy around passing cost savings back to consumers). This meant that they spent a period of eight years working out all the implications, chasing down all the historical analogues, and digging into the actual execution of one particular subtype of competitive moat.

If it took them that long to wring all the actionable insight from one concept, how long might it take for you to chase down the instantiations for concepts in principled books? By comparison, my eight months with HOM doesn’t seem that long.

Go To The Cases, Don’t Reduce

As a reminder, CFT has two primary claims:

1. To deal with novelty, experts in ill-structured domains construct temporary schemas on the fly by combining fragments of previous cases.
2. They have something the authors call an ‘adaptive worldview’: meaning that they do not think there is one root cause or one framework or one model as explanation for a particular event that they observe in their domain.

We’ve talked a little about the implications of taking (1) seriously, so let’s turn to some of the implications of (2).

One obvious implication is to adopt the adaptive worldview — that is, to believe that in ill-structured domains, there is no one cause, framework, model, or explanation for any reasonably complex event. But to adopt this worldview is easier said than done — depending on the context, I find my mind going to the single most coherent explanation for the thing.

CFT tells us that experts don’t do this. They resist reduction to a single prototype or a single explanation by doing the following (my notes on application in the margins; I encourage you to ask yourself these questions as we go along):

• Paying attention to cases in all its rich glory and de-emphasising concepts — This is not very difficult for me.
• When learning, focus on the differences in each new case instead of observing (only) the similarity— Notice that I said when learning — as opposed to when acting. When acting in the domain, experts do the opposite, drawing from fragments of previous — presumably similar — cases.
• Looking at cases and experiences as wholes with emergent properties instead of reducing down to abstract forms with generalisable principles — I actually found this aspect of CFT a wonderful excuse to focus on the details of each case, instead of looking to draw abstract takeaways. I justify this by saying that the context matters — and that context will be something I have to think about when I use them in action later.
• Expect unpredictability, irregularity, contingency and indeterminateness in everything they touch — The most difficult bit of this for me, incidentally, is that last one: I find it very hard to reason about indeterminate causes and to accept that partial explanations are the best that I can get to in some situations. (For instance, consider the question “why did Trump win the election?” If an election is a national-level event, there is usually no one answer to such a question, since it expresses the aggregate preferences of many different people and many different communities. But I still find it difficult to believe that there is no one answer; my brain yearns for a single causative explanation.)
• Stress context dependency — Relatively easy once you’ve internalised that everything in history is path-dependent.
• Avoid rigidity in understanding, remaining open instead, with an appreciation for the sometimes limitless combinations of old knowledge in new combinations — This can be difficult for me in certain contexts; more on this in a bit.

I find it interesting that certain aspects of the adaptive worldview are easier for me to do than others. In an operating context, I’ve long made my peace with not knowing certain underlying causes. For instance, when debugging an organic marketing funnel, you often have to guess at why conversions are up a certain month and not others — sometimes you can point to a set of changes that you did weeks in the past, but you’re never really sure. I’ve gotten used to this kind of indeterminateness, which is usually necessary when applying new marketing techniques.

But I’m still terribly reductive when it comes to certain other practices. For instance, I was recently talking to my ex-boss about the practice of Objectives and Key Results (OKRs) — a goal-setting framework that originated from Andy Grove (who called it Management by Objectives) and that was later implemented at Google at the urging of board member John Doerr, so taken was he by Grove. From Google the practice spread to much of Silicon Valley. The apotheosis of the OKR movement was likely a 2018 book by Doerr titled Measure What Matters; my ex-boss read the book and liked it so much he thought he’d implement it.

I grimaced when he told me this.

“Why?” My old boss asked. I relayed the usual argument against OKRs, commonly articulated in many places but perhaps best expressed by Snowflake CEO Frank Slootman:

Another source of misalignment is management by objectives (MBO), which I have eliminated at every company I've joined in the last 20 years. MBO causes employees to act as if they are running their own show. Because they get compensated on their personal metrics, it's next to impossible to pull them off projects. They will start negotiating with you for relief. That's not alignment, that's every man for himself. If you need MBO to get people to do their job, you may have the wrong people, the wrong managers, or both.

A similar objection, this time from Marty Cagan, highlights a different set of potential problems:

After many years of being a very vocal advocate for the OKR (Objectives and Key Results) technique, in the majority of companies I meet, I have stopped recommending the practice. (…)

The OKR technique came from companies that had empowered product teams in their DNA. OKR’s are first and foremost an empowerment technique.

The main idea is to give product teams real problems to solve, and then to give the teams the space to solve them. (…) the telltale sign of mismatch out there is that companies think they can “check the empowerment box” by giving the team objectives, yet still continue to tell them the solutions they are supposed to deliver — nearly always in the form of a roadmap of features and projects with expected release dates.

(…) in so many companies, each manager – the manager of engineers, the manager of designers and the manager of product managers – creates their own organizational objectives, which are then cascaded down to their employees.

While this may sound reasonable, and is not necessarily a problem for other parts of the company, in practice this means that these employees – when working with their cross-functional colleagues in a product team – are all working on their own objectives, rather than working collaboratively on the team objectives.

(…) And finally, and really the root of the problem, is that in the vast majority of companies I see that are struggling to get any value out of OKR’s, the role of leadership is largely missing in action.

And in addition to these remarks, I’d heard enough horror stories from friends in Google and elsewhere to think that OKRs came with significant potential downside — enough to make me rethink the entire exercise.

My old boss pushed back. He argued that a) OKRs might not work when there is lack of clear leadership, but my old company was still small enough and focused enough that it might not present a problem, b) the incentive skew might be more manageable in a smaller company, and finally c) having some system might be better than not having a system at all.

To which I had all sorts of counter arguments, but I stopped myself mid-sentence. I was thinking about CFT, and I realised I didn’t really have much to say.

First: I realised that much of my opposition was gut-level; I’d internalised the disgust with OKR systems for so long that I couldn’t reason about the flaws of the system separately from the specifics of its implementation.

Second: I realised this was perhaps a perfect example of a concept in an ill-structured domain. After all, there is much variability in how an OKR system might be implemented, depending on the industry, size, product, culture, and leadership of each company. I simply didn’t have enough cases in my head to know when and where OKR implementations might have worked (and when it hasn’t) in the past. I also didn’t have a good experience base to draw fragments from.

And so I conceded to my old boss that perhaps it was worth trying. I realised — per CFT — that it wasn’t very productive to argue from reason or logic in this situation, given the sheer variability of OKR implementations; if we wanted to be more rigorous about this decision, what we should do is to go out and actively scout for detailed stories about OKR implementations. As it stands, the most that we can say right now is that a particular type of failure mode exists when implementing OKR systems; any implementation that my old boss attempts should keep that mode in mind.

Cases, Lots of Them, Preferably as Different from Each Other as Possible

We know from CFT’s pedagogical recommendations that so long as we are operating in an ill-structured domain, it’s a good idea to seek out as many cases as possible whenever we learn a new concept. Preferably, each case we collect should be as different from the previous ones that we currently have.

But there are two complications that I’ve been thinking about:

First, how do you present concepts in writing, given the primacy of cases? I think about this in the context of this blog, where I write a huge deal about making decisions in messy, real-world contexts. How do you talk about concepts when you know that it’s the cases that are truly important?

Currently, Commonplace blog posts take on the loose form of:

• Assertion
• Reasoning
• Illustrative example
• Another illustrative example

CFT suggests that the structure should be something like:

• Assertion of concept
• Reasoning
• Illustrative example
• Second illustrative example that’s very different from the first
• Third illustrative example that’s very different from the first and second
• Fourth illustrative example …

I know this might not be relevant to most of you reading — certainly the set of people who must write about concepts or whose job it is to teach concepts are rather small. But it’s something that’s worth thinking about if it’s a component of your job.

Second, constructing a CFT learning system is limited by the number of cases we can look at. In certain domains, articulating a case is very difficult. Software engineering comes to mind here — most consequential software design decisions are made in the context of a large, sprawling codebase, embedded within a complex business context. How do you capture the series of events and decisions that led to the current state of the system? How do you present the software lessons in a readable format to a novice with no familiarity with that system? I kicked this idea around with a programmer friend, but the best we came up with was ‘do a CFT case library with The Architecture of Open Source Applications’.

It does seem that certain domains lend themselves more easily to CFT than others. In medicine, there is an existing practice of publishing case studies. In business, there is a large selection of business biographies and histories, written by business journalists who are motivated by book sales and speaking gigs.

But other domains appear more hostile to the case study — for programmers it seems like skill is limited by the number of large systems a programmer is exposed to over the length of their careers.

I don’t have good answers for this one. If you have ideas, let me know.

This is Part 3 of the Learning in Ill-Structured Domains Series. Read the next part here.

Originally published 15 February 2022, last updated 23 November 2023.