This is part of the Expertise Acceleration topic cluster.

An Extracted Tacit Mental Model of Business Expertise

Fan of great business?

Join 8,000+ sharp investors and operators like yourself, and we'll send you a collection of Commoncog's best articles right away:

This is Part 7 in a series on tacit knowledge. Read Part 6 here. This is also the start of a new series on tacit knowledge in business.

If you’re a long time reader of Commonplace, you’re probably familiar with my obsession with tacit knowledge — the idea that certain types of expertise is locked up in the heads of experts, that they can’t really put such expertise into words, and that you should pursue techniques to acquire tacit expertise — methods like emulation, scenario training, and apprenticeship.

In my first piece in the series, I wrote:

It is not reasonable to wait for an expert systems revival, nor is it fruitful to expect CDM to be applied to your field, or for a Boyd-like genius to appear. We should act as if tacit knowledge were a fact, because it is more useful to think about ways to gain that tacit knowledge directly, instead of hoping for some breakthrough to make tacit knowledge explicit.

Over the course of writing this series, however, I’ve come around to the idea that it might be possible to extract tacit knowledge from the heads of experts around you. The turning point was my discovery of the ACTA paper, which teaches a radically simplified method for cognitive task analysis. (Cognitive task analysis, or CTA, is the set of techniques that Naturalistic Decision Making (NDM) researchers use to extract tacit mental models of expertise from the heads of subject matter experts).

That said, it’s nearly always more fruitful to mine the literature for already-extracted mental models of expertise. A few weeks ago, I learnt of Lia DiBello, an NDM researcher who specialises in the extraction of tacit mental models of business expertise.

DiBello’s findings are extremely compelling. She writes, in The Oxford Handbook of Expertise:

After several years using different methods, we noticed that highly talented business performers are very similar to each other. That is, they have decoded the domain of business much like chess masters had decoded chess. This led us to realise that —as a domain—business itself is an orderly closed system of relations between principles, and that so-called intuitive experts in business have an implicit grasp of this. If this is the case, it would follow that all highly skilled business experts should recognise each other and share a common mental model of these principles, although they may manifest differently in different industries. Since business has been evolving in recent years, this model would have to be very robust, running deeper than recent societal changes. (emphasis mine)

These claims sound ridiculous. A closed system of relations between principles? A shared common mental model of those principles? No way.

As a result of that chapter, I dove into DiBello’s publication history. This turned out to be relatively easy — I quickly learnt that DiBello has published a relatively small number of papers, especially when compared against the research output of some expertise researchers. Initially, I found this frustrating, but then I realised that she has spent the bulk of the past 20 years consulting for big companies, applying her research to the real world. (During her interview on the NDM podcast, she says (54:04) somewhat ruefully, “Because of how I wanted to get a lot of data, I ended up selling my soul to big business, and I made a lot of rich people richer. But it was how I got to prove that what we do works.”). She spends more time today attempting to adapt her training methods for broader uses, to make them available to businesspeople who might not be able to afford a full consultation. To my mind, this makes her more, not less, believable.

And what a body of work she has.

This piece is a high-level summary of what I’ve found. It marks the start of a new series on tacit business expertise; I intend to dive deep into some of the more subtle implications of her work in the next handful of posts. I’ll also include a full list of citations and links to papers, along with a scan I made of her chapter in The Oxford Handbook of Expertise.

Let’s start out with what DiBello found.

The Oxford Handbook is a remarkably chonky book.

DiBello's Main Ideas

DiBello’s work is remarkable because she gets business. In her NDM podcast interview she explains that she came from a business family; naturally, she gravitated to the study of business experts after her PhD.

The bulk of her work follows from these three big ideas:

Business expertise consists of a handful of properties: (a) it is team-based — meaning that business expertise is actually a form of distributed cognition, with expertise spread amongst an executive team; (b) it consists of two things: domain-specific mental models of the business and cognitive agility; and (c) there is a deeper mental model that underpins those domain-specific mental models, which explains how good business people are able to recognise other good business people, even if they are from different industries.
She argues that general problem solving ability does not matter so much in business so much as a property she describes as cognitive agility — which is the ability to update mental models in light of new information.
Her primary innovation is an intervention known as the FutureView Profiler. The Profiler is able to evaluate the shared mental models of an executive team: it is able to (a) identify blind spots in the executive team’s model of the business, (b) able to evaluate cognitive agility, and (c) able to measure expertise in business primarily because it asks business people to do year-by-year predictions of similar companies within their own industries.

I’ve highlighted the most immediately useful elements of her work above, but I want to note that the ideas sit on top of a few other contributions that are novel from an expertise research perspective:

DiBello’s insight that domain-specific prediction tasks are a useful way to evaluate business skill is a huge contribution to the literature on expertise. She spends a few paragraphs in her 2010 paper [2] discussing why the Profiler works, and why other assessments of business skill do not. (The gist of the argument is that you want to evaluate how a business person’s mental models update in response to a dynamic context; you do not want to evaluate what currently exists in their heads).
Her company, Workplace Technology Research Inc (WTRI), has spent years implementing company training simulations in virtual worlds. Apart from the obvious technical effort (she explains [3] that they decided to code a full world of their own, on a game engine of their choice, mostly because existing companies they collaborated with kept getting acquired or shut down) — she has novel insights on the pedagogical effectiveness of using virtual game worlds to teach business simulations.
Last, but perhaps most importantly, DiBello argues that business expertise is primarily a matter of organisation. She writes, in the Oxford Handbook: “Like experts in other domains, business experts rely not on greater analysis or greater information, but better ways of structuring or organising their knowledge. Further, a business expert differs in the manner in which he or she looks at the business landscape, particularly with regard to what is linked with what and what is important to manipulate.” As a result, the bulk of DiBello’s training methods aim to do what is called ‘cognitive reorganisation’ — that is, WTRI does not seek to teach business people new concepts about their domain per se, but to instead reorganise existing domain-specific mental models in their heads.

These ideas — in particular the observation about cognitive reorganisation — are conceptually interesting, and important if you want to implement DiBello’s training methods in your company. But they aren’t necessary for our current discussion, and they are difficult to grok. (In fact, it appears to be an open question if one is able to recreate WTRI’s training methods; DiBello has said that many have tried — even with assistance! — and failed).

I’ll set these ideas aside for now, and spend the rest of this piece examining the tacit mental model of business expertise that DiBello successfully extracted two decades ago.

The Mental Model of Business

In the Oxford Handbook, DiBello writes:

In a study funded by the National Science Foundation conducted over four years (NSF Award ENG 9548631) we found that those who show considerable and consistent talent in business have such a mental model, shown in predictable ways of making use of information. Leveraging our relationships with companies and clients, we had unusual personal access to study a large number of highly placed leaders. Doing in-depth studies of talented business leaders who were repeat successes—even in very challenging markets—and who had risen to very high positions (such as chairmen of large corporations) and maintained that level of position even as business itself has grown much more complex, we discovered a shared mental model among all of them. [1]

That shared mental model is essentially a triad:

Our research revealed that people who have achieved a high level of business expertise have a deep understanding of the following three core areas: (1) factors involved in effective operations, (2) forces influencing the market, and (3) those driving business finance and economic climates. Consistently successful business leaders have been shown to intuitively understand these areas and their impact on each other, and to pay attention to this fundamental triad in a uniquely dynamic way within a guiding context of business strategy. For example, these experts are able to sense that market conditions change based on environmental indications that others may fail to notice. Further, they foresee the consequences to their business operations and finances, and thus make necessary adjustments proactively. Unlike most business professionals, they are attuned to the early indicators of widespread change. Beyond this, they are expert at keeping the triad in balance, or shifting the balance when external conditions are conducive to do so (e.g., focusing on marketing during favourable economic times). (emphasis added) [2]

Throughout the paper, DiBello describes the triad using different names. One way of describing it is ‘supply/demand/capital’. Another is ‘leadership/strategy/finance’. To summarise:

Supply, or leadership — These represent operational concerns related to the business, what DiBello calls ‘factors involved in effective operations’. Naturally, how the operational concerns are expressed depends on the industry (e.g. you have to be good at product development if you’re in consumer software, and you must have competent manufacturing if you’re Chobani). Think expansively when it comes to each category of the triad — this category includes things like org design, incentive structures, execution cadence, and even the ability to get things done within the company (e.g. execs must have good mental models of the org and its people, and the ability to push through plans given internal politics).
Demand, or strategy — These represent the exec’s understanding of the market, what DiBello calls ‘forces influencing the market’. Of course, market dynamics is a very broad category, and includes everything from market shape, competitive analysis, positioning, changing consumer demand, and the ‘path to power’ that we’ve talked about in our discussion of 7 Powers.
Capital, or finance — At the basic level this represents the exec’s understanding of basic financial concepts like cash flow lockup, return on invested capital, margins, and their relationship with the other two categories (what DiBello calls ‘factors driving business finance and economic climates’). The factors here may be equally expansive — depending on the business, the capital leg of the triad may include expertise with raising capital (e.g. equity financing, debt) or the ability to understand the implications of changes in the capital environment (e.g. John Malone was amongst the first in the cable industry to take advantage of junk bonds in order to finance expansion; a more contemporary example might be startup founders who understand how to manipulate extremely liquid private markets to their advantage).

As DiBello mentions, the key property of business expertise is in understanding how a change in one leg of the triad affects the other two legs, at least within the context of one’s specific industry. Experts are able to notice cues in the environment long before novices are able to, and are then able to forward-simulate what must be done to the other two legs of the triad. So, for example, if there is a change in the capital environment, an expert businessperson might ask how that affects competition (this is the market — the demand leg of the triad). And they might immediately ask what changes the company must make in response to those market shifts? (This is the supply leg of the triad).

This mental model of business is incredibly high-level, but it tells us a lot.

For instance, it tells us that expertise may be evaluated according to the facility an exec has with each of the three categories. DiBello writes:

In contrast, competent managers (one level down from experts) tend to be very talented in only one or two of these areas; however, they often do not understand the dynamics between these areas as well as the “superstars” do. Competent managers are likely to be very successful when larger market or economic trends are favourable to their specific skills. (emphasis added) [2]

The implication is that if you want to get really good at business, you should systematically acquire skills in each of the three categories, specific to your particular business, and then — more importantly — grok the relationships between the three categories.

It also explains another observation that DiBello makes:

Experts in the same domain easily identify each other. Even if they don’t agree with another expert’s choices, they easily recognise that they share a common perceptual experience within their area of expertise. [2]

With the triad model, we may begin to understand how an experienced businessperson is able to recognise another experienced businessperson from a different industry, despite having no idea of the specifics of the other expert’s business or market. The answer to this puzzle is that they are able to recognise the intuitive facility with which the other person reasons about supply, demand, and capital in their own businesses! When I read DiBello’s articulation of the model, I thought back to certain businesspeople who have told me that they can tell if someone ‘gets it’, even if they can’t articulate how; what I now think is happening is that their brains must be attuned to the existence of a triad in the other person’s head.

Do I buy this model of business expertise? Yes, I do. Nearly everything I know about business lines up with the model. To my mind, this explains a handful of unrelated phenomena that I’ve long puzzled over. For instance:

Why are fresh MBA grads unable to grok the implications of, say, cash flow in a given business, despite learning it in their courses? The answer is obvious: they have yet to internalise the relationships between a) market conditions and b) operational realities with c) the financial concepts they learnt in school. Every business is a system, and developing intuition for a system requires you to watch that system in action. Business students who have no real world experience of a business do not usually have such a mental model. DiBello writes, as part of a consulting engagement: ‘As a process check, we administered the same instrument to an equal of number of business-knowledgeable controls (BS or MBA business students who were not yet functioning managers or consultants and had not ever actually run a company of any size). Their performance was much lower. This, too, was an expected result.’ [2]
How many different ways are there to build Power in business? I used to think that companies with better products would win, and then I believed that companies with better distribution would win, and then I ran a business for awhile and realised that businesses have won from differentiated product or differentiated distribution or differentiated access to capital; the possibilities were more varied than I had originally imagined. With the triad, it’s clearer to me that temporary competitive advantage may emerge from any of the three categories: you could gain a temporary edge from operational excellence (supply/leadership), or from exploiting some market opportunity (through product innovation or repositioning into a new category — this is demand/strategy), or even from exploiting some change in the capital environment that others in your industry do not realise exists (e.g. Malone with debt-funded, tax-sheltered expansion — this is capital/finance).
Buffett’s quote “I am a better investor because I am a businessman, and a better businessman because I am an investor” makes more sense when seen through the lens of the triad — it's likely that exposure to equity investing or actual business operations would shore up one or more of the three categories. As a trivial example, a businessperson without a competent understanding of capital markets is missing one leg of the triad; an investor without a sufficiently realistic understanding of business operations (supply) is missing another.
Everything in An MBA for Business Operators may be organised neatly into one of these three categories. In fact, if you look at Permanent Equity’s checklist through the lens of the triad, you’d realise that it is a mix of supply and capital considerations … but leaves out demand. This makes sense — to a private equity fund like Permanent Equity, the demand/strategy side of things is the bit that is most specific to the individual business, and cannot be generalised.
Can management consultants or former VCs become good business operators? With the triad in mind, we can be more specific about what must happen: management consultants are likely better equipped in the demand leg of the triad, with some understanding of the finance leg; VCs are better equipped in the finance leg, and somewhat skilled in the demand leg. How well they do is how quickly they level up on the supply leg of the triad, and how well they internalise the relationships between all three.

To reiterate the most important point again: DiBello’s explication of the triad mental model tells us that it is the relationships between the three that we need to pay attention to; expertise in each leg of the triad is more commonplace when we rise through the stages of business ability.

Cognitive Agility

DiBello also points out business expertise is made up of one other quality — something she calls cognitive agility:

Given the volatility of markets and the complexity of team decision making in large organisations, we need not be as concerned with the general cognitive ability of today’s executive so much as his or her domain-specific expertise and cognitive agility, which is often associated with the kind of intuitive expertise developed through experience. Cognitive agility is what allows executives to rapidly respond to changing situations and revise guiding mental models to meet performance demands. (emphasis added) In sum, these insights require us to shift away from thinking of the cognitive ability of business leaders as fixed and static, and instead to focus on the way in which expertise evolves over time in response to dynamic environments. It also requires us to discard attempts to locate decision-making expertise as a fixed capacity that resides within the individual decision maker. [2]

Building on theories of emergence, adaptation, and flexibility from complexity science and NDM, we use the term cognitive agility to refer to the extent to which an individual revises his or her evaluation of a situation in response to data indicating that conditions have changed. The converse is cognitive rigidity, where the person is impervious to new data, being dominated by a rigid framework or paradigm that acts to filter out new, possibly relevant, information, creating blind spots. [2]

When I read this section of DiBello’s 2010 paper, my eyes basically popped out of my head.

For several years now, I’ve tried to talk about the limits of frameworks, as experienced from the perspective of a business operator. I’ve noticed that when a market changes, or when you’re operating in a local environment, no framework can fully capture everything that you’re seeing as you brush against reality. I’ve attempted to capture this idea by writing blog posts such as Reality Without Frameworks, and How First Principle Thinking Fails and Good Synthesis is the Start of Good Sensemaking. More importantly, I’ve attempted to articulate DiBello's notion of cognitive rigidity, with a trait I invented that I named ‘Dismissively Stubborn’. I wrote that piece to explain the type of people I’ve learnt to avoid in startup environments.

But it turns out that DiBello had thought of this, and more, a full decade before I did. I was also surprised — and delighted! — to learn that senior business leaders have thought about this particular aspect of business expertise:

(We present) a case in which we used the Profiler to assess the senior management team of a medical device company. Our relationship with the company began when the chairman and acting CEO asked us to evaluate his senior staff for succession planning purposes. In particular, he wished to decide who, among the division presidents, would succeed him as the CEO. At the time, the company was growing very rapidly. During the four months we worked with the company between 2005 and 2006, we saw them grow from $300 million to $400 million. A pending acquisition was intended to increase the company’s size to $600 million by 2008.

The CEO of the company was aware that there is a qualitative difference between competent managers, who do very well in interviews or on personality tests, and experts, who can perform under challenging conditions. He wanted to protect himself from (mere) competence at a time when his industry was in a high growth phase. He was afraid that a favourable market might be masking a lack of skill—and cognitive agility in particular—among his senior staff. (emphasis added).

So now we must ask: how do DiBello and colleagues evaluate domain-specific expertise and cognitive agility?

This is probably a good time to talk about their Profiler.

The FutureView Profiler

In their 2010 paper, DiBello, Lehmann and Missildine laid out the requirements of a good instrument to assess business expertise:

We believe that an instrument that effectively assesses business expertise must (1) be able to draw on the intuitive expertise of organisational members; (2) tap into the specific mental models individuals use to approach problems, rather than basic problem solving or generalised decision making skills; (3) locate individuals within the distributed cognition of an organisation, that is, specify the individual’s role in the organisation; (4) be able to identify strengths and blind spots of the organisation’s transactive memory system and, in so doing, highlight dimensions of expertise that contribute to high-level decision making; and lastly, (5) measure not only cognitive ability, but also cognitive agility, that is, the capacity to rapidly revise one’s mental model in the face of dynamic feedback.

The Profiler requires the user (1) to examine and analyse the same material used by actual experts (e.g., annual reports, 10K’s, analyst’s reports, etc., from actual companies) to make business decisions, (2) to make predictions about the company with respect to a number of domains (e.g., revenues), and (3) to judge various aspects of the company (e.g., evaluate the management team).

Specifically, the Profiler asks users to answer several questions about the company to predict how its finances will develop over the next five years. Users’ answers are evaluated in terms of Dreyfus’s five-stage model of expertise to determine their business acumen and level of expertise within the domain of business strategy. The Profiler also helps us identify blind spots in users’ thinking; that is, we can discern areas where users seem to miss aspects critical to the company’s performance. Moreover, for each prediction or judgment users must indicate the specific materials they relied on, such as discrepancies in annual report statements or the general state of the company’s finances. This aspect of the Profiler taps into the heuristics that are guiding users’ decision making. [2]

How are users evaluated? DiBello et al note that the triad (the extracted mental model of business) isn’t directly addressed in the Profiler itself — even though it informs the design of the assessment:

These three core areas are not directly addressed in the questions we ask in the Profiler. Rather, an individual’s answers are rated and compared to those of the ideal expert. The questions are based on tangible outcomes and concern the company’s performance. The questions ask users to evaluate their company with respect to its strategy, leadership, and finances. These are areas that senior executives would be expected to make valid predictions about and are good ways to reveal their underlying business expertise. For example, a question about finances would be: As the senior manager for the business, which aspect of financial performance most urgently requires your attention? This question would then be followed by a series of options, such as “cost of goods sold,” “fixed assets,” “receivables,” or “R&D.” To answer this question—that is, to know what is troubling or of concern in the total context of the company at that time—the user must understand operations, finance, and market trends. Answer options reflect levels of expertise, 1 through 5, corresponding to the five stages in the Dreyfus model. The “correct” answer is empirically determined. Because these are actual cases, the outcomes are known and the root causes of problems have been identified.

In addition to these Dreyfus questions tapping their business expertise, users are asked to provide three predictions that refer to quantitative outcomes in the company’s performance. These questions can vary slightly, as the instrument is customised according to a company’s needs; however, they will ask about issues, such as profits and revenues, that are indicative of how well a user can synthesise information about the company to predict real-world outcomes. For example, we may ask: What is your prediction of the company’s profits for the next 12 months? Users then indicate their answers along a 5-point Likert scale whose points are labelled appropriately, for instance, in the preceding example, options may range from “down 20% or greater” to “up 20% or greater.”

After this initial set of questions, DiBello steps forward the simulation and presents the results of their first year of predictions. This is the primary way that they are able to assess cognitive agility:

Users then go on to Year 2. After reading through the Year 2 materials, they are able to see whether or not their predictions were validated by the company’s performance. In other words, they receive feedback as to the accuracy of the previous year’s predictions. They repeat this exercise for five years of material, making predictions and getting feedback as to whether or not their predictions have been validated. This iterative process taps into users’ cognitive agility, allowing us to assess the degree to which users are able to revise their thinking in the face of feedback about company performance and their own predictions. Cognitively agile individuals learn from their inaccurate predictions or judgments and improve as they go along. (emphasis added)

Furthermore, we can see persistent blind spots that may exist and occur consistently across all five years. Thus, the Profiler can provide windows into the acumen, blind spots, mental models, and agility with which users approach complex decision making for a real company. It is important to keep in mind that all of the data provided are from real companies. We do not determine the outcomes, as they are taken from actual annual reports and actual finances over a given period of time.

In addition to evaluating users’ direct responses to the Dreyfus and Likert questions, we also ask users to point to sections of the material they reviewed (including statements in the annual reports and financial summaries) that informed their decision on a particular question. For example, the annual report might mention a recent acquisition. This may reflect a strategic turning point for the company. As such, after users respond to the question on strategy, they must indicate what sections of the material influenced their decision. Users are allowed to choose up to five reasons for their decision. In this case, users should be aware that the recent acquisition plays a major role in the company’s strategy going forward, even if this is not immediately obvious. As such, when users answer this particular strategy question, they should be pointing to the acquisition as a basis for their decision. This not only allows us to evaluate the expertise and accuracy of their responses, but also gives us a window into the kinds of information they use to approach the decision-making process. Furthermore, this aspect of the Profiler allows us to determine whether users are utilising information based on the three core areas listed above: supply, demand, and finance. Specifically, we can see whether users approach the questions in an expert manner by evaluating the degree to which they are paying attention to these core areas as they make decisions.

In her 2019 chapter in the Oxford Handbook, DiBello presents a second application of the Profiler, this time in service of a financial services company that was recovering from the repercussions of the 2007 financial crisis:

The assessment used here was a version of our online business simulation in which each participant must solve a business mystery by predicting what will happen over the course of a business’s life in one-year intervals after being given a year’s worth of data at a time and judge the actions of the executives in the case.

We chose an actual case that was similar to this company’s but which encountered its challenges in a different time of history. In contrast to their company, the company in the case had not made the same mistakes. After removing all identifying information—including dates—we presented each person with five years of material, one year at a time, and asked them to examine the company’s data, decisions, plans, and declarations and predict the outcomes of the following year and judge the actions of the executives. We use a scoring scheme designed to evaluate the present or absence of our model of an ideal business expert. We also ask the participants to choose the clues in the material that drove their thinking.

Each clue is like DNA or fingerprints at a crime scene. An expert will not only see that it is significant, but what it signifies. Therefore, if the participants predict that the executives in the case do not see that a market downturn is coming and choose the clues that in fact are an early warning sign of that fall, we can see that they are attending to the right clues in the market portion of the model. Each question and each clue is coded using an association matrix that links back to the model. From this we can generate charts of a person’s mental model compared to an ideal expert, and a heat map showing by colour where good clues were identified and where there are blind spots (black) where clues were overlooked.

Figure 1 shows some material that a participant would examine. The question and clue selection screens are all online and involve easy clicks. The participant can stop mid- stream, save their work, and come back to it later. The idea is to replicate the actual decision process of busy business executives.

In real life, the banking executives were having a particularly difficult crisis as a team. They were having serious disagreements with their CEO about how to recover. Some were working behind the scenes to have him removed by the board of directors. A unique feature of our instrument is that it can show how a team is thinking. When we mapped the executive team’s results—as a team— on the dimensions of expertise critical to their recovery, we found that they were at risk for making the same mistake again due to a shared blind spot in the area of finance and risk management. The only person on the team who could mentally simulate the eventualities accurately was the CEO with whom they were violently disagreeing and whom they were trying to depose.

This study is interesting on two fronts. It shows how easily a team can develop a common theory of the crime over time, even when it is not working, perhaps explaining how whole companies can sometimes implode. But it also shows how complex business has become and how the insights of one person can be difficult to communicate. Fortunately, using these data, the executives’ advisors were able to convince the executive committee that—as a team—they needed to re-distribute responsibility and put those with the deepest insight into the capital risk environment on the front line during the recovery. As a committee, they could function as an expert but only if they knew where the different elements of their expertise resided.This company did recover, enjoying a doubled stock price in less than a year, but it was by developing a shared understanding of how they were going to divide up running the business and yet keep all the parts linked in the way they ran the committee meetings. In the end, they followed the insights of the one lone wolf in one area, abandoning their preferred notions of what would work while they came up to speed on what he was seeing.

I believe DiBello when she says that the Profiler is able to detect blindspots in an exec team, that it is able to evaluate cognitive agility, and that you may use it to assess the expertise of business decision makers. I’m just not sure how this may be used, if one isn’t able to afford DiBello’s services.

(In theory, a generic Profiler may be used, though you can’t expect to score very highly if you don’t have relevant expertise in that particular business or industry.)

As of today, DiBello’s methods — both Profiler and training interventions — have been used by over 7000 individuals; a description of her work reads:

Studies of more than 7000 people at all levels exposed to DiBello’s methods indicate that learning was accelerated by several months in all cases. Other studies indicate that unprecedented performance improvements and significant competitive advantages were achieved by companies using DiBello’s methods. The method is now used globally in four continents in industries as diverse as mining, transportation, financial services, information technology (IT) implementation, manufacturing, pharmaceuticals as well as others [4]

And in her NDM podcast episode, she mentions, almost in passing (32:22):

A lot of senior leaders will look only at the financial data, and they won’t look at the market data. And maybe the market data will tell them that they’re making the wrong decision. So the heat map will be blacked out there. And we can tell that they have a bias that prevented them from being successful. (…) When we’ve done it with some large companies, we usually do it with a control group — which is some people lower down in the organisation, outside the C-suite. And we sometimes find some very gifted people, who are in under-represented groups, who would normally not be promoted to high level positions. And they are promoted as a result.

Ultimately, the most useful thing that I got out of DiBello’s published work was the fact that this research even exists — that it has been applied and has generated results for many companies; that there is a shared mental model of business expertise that exists despite the differences of many business domains.

More importantly, the work that DiBello has done is believable by my practical standards for truth. She's applied her work in multiple company interventions across multiple industries over a two decade period, and many interventions have resulted in measurable million-dollar outcomes. A predictive model that works when tested against reality, by a person with skin in the game, over a multi-decade period, is pretty much the strongest result you can ask for in a messy domain like business.

At the end of her chapter in The Oxford Handbook, DiBello writes:

Businesses are more alike than different, and scalable educational applications in the form of generic rehearsals may have the same expertise accelerating benefit as our labour-intensive and expensive face-to-face rehearsals. (emphasis mine)

If you’d had told me a few months ago that this was possible, and that business was ‘more alike than different’, I would have told you to stop joking. And yet here we are.

Read Part 2: The Importance of Cognitive Agility →