Not all experiences create skilled intuitive judgement with time
What are the activities in which skilled intuitive judgment develops with experience? What are the activities in which experience is more likely to produce overconfidence than genuine skill?
Intuition definition
The situation has provided a cue: This cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition.
Conditions to hone intuitions
In particular, we explore two necessary conditions for the development of skill: high-validity environments and an adequate opportunity to learn them.
Environment to develop skilled intuition
A crucial conclusion emerges: Skilled intuitions will only develop in an environment of sufficient regularity, which provides valid cues to the situation.
Hone your intuition using early indicators
For example, it is very likely that there are early indications that a building is about to collapse in a fire or that an infant will soon show obvious symptoms of infection. On the other hand, it is unlikely that there is publicly available information that could be used to predict how well a particular stock will do—if such valid information existed, the price of the stock would already reflect it. Thus, we have more reason to trust the intuition of an experienced fireground commander about the stability of a building, or the intuitions of a nurse about an infant, than to trust the intuitions of a trader about a stock.
Why any kind of forecasting or predicting job will not be a skilled intuition
The depressing consistency of the experts’ failure to outdo the novices in this task suggests that the problem is in the environment: Long-term forecasting must fail because large-scale historical developments are too complex to be forecast. The task is simply impossible.
Developing wrong intuitions even with regularities
He confirmed his intuitions by palpating these patients’ tongues, but because he did not wash his hands the intuitions were disastrously self-fulfilling. High validity does not imply the absence of uncertainty, and the regularities that are to be discovered are sometimes statistical.
How to develop intuition
These include the type of practice people employ, their level of engagement and motivation, and the self-regulatory processes they use.
Talent
Extraordinary players such as Fischer and Kasparov were able to recognize patterns that other grand masters could not see on their own — although the weaker players could recognize the validity of the star’s intuition when led through it.
Creativity
Widely shared patterns of associations exist, which everyone can recognize although few can find them without prompting.
Incorrect attribution to intuition
A common genre of business literature celebrates successful leaders who made strategic decisions on the basis of gut feelings and intuitions that they did not adequately check, but many of these successes owe more to luck than to genius.
Difficulty in realizing skilled vs unskilled intuition
Of course, the mechanisms that produce incorrect intuitions will only operate in the absence of skill. If people have a skilled response to the task with which they are charged, they will apply their skill. But even in the absence of skill an intuitive response may come to their minds. The difficulty is that people have no way to know where their intuitions came from. There is no subjective marker that distinguishes correct intuitions from intuitions that are produced by highly imperfect heuristics.
Professions with expertise:
Poor performance among experts:
Conditions for experts to do well
The factors that we identified—the predictability of outcomes, the amount of experience, and the availability of good feedback — were included in his list.
Some professions have fractioned expertise
Three professions— nurses, physicians, and auditors —appeared on both of Shanteau’s (1992) lists. These professionals exhibited genuine expertise in some of their activities but not in others. We refer to such mixed grades for professionals as “fractionated expertise,” and we believe that the fractionation of expertise is the rule, not an exception.
Unpredictability in most domains
There are a few activities, such as chess, in which a master will not encounter challenges that are genuinely new. In most domains, however, professionals will occasionally have to deal with situations and tasks that they have not had an opportunity to master.
When feedback on longterm judgments are vague, it becomes ripe for overconfidence
Finance professionals, psychotherapists, and intelligence analysts may know a great deal about a particular company, patient, or international conflict, and they may have received ample feedback supporting their confidence in the performance of some tasks—typically those that deal with the short term—but the feedback they receive from their failures in long-term judgments is delayed, sparse, and ambiguous. The experience of the professionals that DK has thought about is therefore conducive to overconfidence.
What true experts don’t do…
Weather forecasters, engineers, and logistics specialists typically resist requests to make judgments about matters that fall outside their area of competence. People in professions marked by standard methods, clear feedback, and direct consequences for error appear to appreciate the boundaries of their expertise. These experts know more knowledgeable experts exist.
Real reason is low-validity
Findings in which the performance of human judges is inferior to that of simple algorithms are often cited as evidence of cognitive ineptitude, but this conclusion is unwarranted. The correct conclusion is that people perform significantly more poorly than algorithms in low-validity environments.
When is a statistical approach better?
A statistical approach has two crucial advantages over human judgment when available cues are weak and uncertain: Statistical analysis is more likely to identify weakly valid cues, and a prediction algorithm will maintain above-chance accuracy by using such cues consistently.
Example of using algorithms
The evaluation and approval of personal loans by loan officers is an example of a situation in which algorithms should be used to replace human judgment. Identifying the relatively small number of defaulting loans is a low-validity task because of the low base rate of the critical outcome. Algorithms have largely replaced human judges in this task, using as inputs objective demographic and personal data rather than subjective impression of reliability. The result is an unequivocal improvement: We have fairer loan judgments (i.e., judgments that are not improperly influenced by gender or race), faster decisions, and reduced expenses.
When is using an algorithm better?
Our analysis suggests that algorithms significantly outperform humans under two quite different conditions: (a) when validity is so low that human difficulties in detecting weak regularities and in maintaining consistency of judgment are critical and (b) when validity is very high, in highly predictable environments, where ceiling effects are encountered and occasional lapses of attention can cause humans to fail.
Conditions necessary for the construction and use of an algorithm:
Challenges after implementing algorithms - automation bias
We also agree that algorithms that substitute for human judgment must remain under human supervision, to provide continuous monitoring of their performance and of relevant changes in the environment. Maintaining adequate supervision of algorithms can be difficult, because there is evidence that human operators become more passive and less vigilant when algorithms are in charge—a phenomenon that has been labeled “automation bias”.
Pre-mortem reduces overconfidence and improves decisions
An example is the premortem method for reducing overconfidence and improving decisions. Project teams using this method start by describing their plan. Next they imagine that their plan has failed and the project has been a disaster. Their task is to write down, in two minutes, all the reasons why the project failed.
Pre-mortem also reduces suppression of dissenting opinions
It also offers a solution to one of the major problems of decision making within organizations: the gradual suppression of dissenting opinions, doubts, and objections, which is typically observed as an organization commits itself to a major plan.
Awareness of intuitions
Skilled judges are often unaware of the cues that guide them, and individuals whose intuitions are not skilled are even less likely to know where their judgments come from.
Confidence is unreliable indication of skilled intuition
True experts, it is said, know when they don’t know. However, nonexperts (whether or not they think they are) certainly do not know when they don’t know. Subjective confidence is therefore an unreliable indication of the validity of intuitive judgments and decisions.
Illusion of skill breeds overconfidence
Although true skill cannot develop in irregular or unpredictable environments, individuals will sometimes make judgments and decisions that are successful by chance. These “lucky” individuals will be susceptible to an illusion of skill and to overconfidence.
Be careful of calling in “experts”
Professionals who have expertise in some tasks are sometimes called upon to make judgments in areas in which they have no real skill.
Challenge in determining true expertise
It is difficult both for the professionals and for those who observe them to determine the boundaries of their true expertise.