Oh dear. And yet, I’m constantly asked by friends outside the discipline, ‘Ken, what do you think is happening in the Ukraine?’ What we have here is an illusion of knowledge, or an argument from authority. Give someone a title, an office in an ivory tower, some knowledge of history, maybe some language skills, and you create an illusion of understanding. A confident manner, bold, snappy, authoritative statements, all boost credibility.
So much for prediction, what about understanding? Surely expertise helps us here? Plenty of tenured posts depend on it.
But I think not. ‘Hindsight bias’ describes our tendency to find causal relationships when looking backwards into the past. My students do so frequently when asked to judge whether or not someone was a good general. With the exception of German panzer generals (for reasons I don’t quite fathom), good generals tend to be those that won – the outcome explaining the quality of their generalship, quite aside from the myriad other factors that might actually have been involved.
Experts are often no better, I am sure. One thing we are very good at is storytelling – finding meaning, pattern and causal relationships amidst the clutter of complex social affairs. We underplay the random and blind chance.
You think then that we’d be more modest about it all, but then one suspects that ‘experts’ are as susceptible to a final bias as the rest of us – the optimism bias: assuming that we ourselves perform better than everyone else.
Nassim Nicholas Taleb writes very engagingly about this in The Black Swan. The gist of his point–not sure if its originally his–is that experts are blinded by detail. By following every event in a place/relating to a subject in minute detail they tend to miss the big waves. The ‘trick’ then is to pay less frequent attention, sampling a wider range of subjects at greater intervals. Exactly how I do ‘research’! :)
More seriously, I reckon experts on foreign affairs ought to study more history and more philosophy and practice the phrase ‘I don’t know what will happen in X. Ask me again in a hundred years.’
Hi David,
Taleb borrowed this expression of the Problem of Induction from Karl Popper, The Logic of Scientific Discovery (London: Routledge 2002), 4: …no matter how many instances of white swans we may have observed, this does not justify the conclusion that all swans are white. This work was first published in 1935.
Popper in turn, was influenced by the thoughts on causation by David Hume in his A Treatise of Human Nature, first published in 1739 – see his Book I: Part III, Section IV: Of the component parts of our reasonings concerning causes and effects.
Jason Matheny of IARPA is doing some extremely smart work on figuring out who makes the best predictions by running a series of tournaments in which different teams attempt to predict political events in South America. I seem to recall that a team led by Phillip Tetlock was winning, well ahead of the various teams of regional experts – I think Metheny ascribed this to Tetlock’s understanding of the limits of detail and to the team’s ability to cultivate and stick by certain principles of prediction.
Also, the book I’m working on is all about the advantages and limitations of specialists vs generalists in various fields. One of my arguments is that amateurs often bring a useful mixture of confidence in their judgements and humility in their caveats. (Norman Dixon’s critiques of hidebound, detail-obsessed military professionals will certainly make an appearance, if only because he’s so endlessly quotable).
I will take an “expert” who has been trained to consider his answers across a broad spectrum of ideas and facts over the run of the mill bloviatier.
Part of the problem is less that expertise values little in such areas, but rather that there is too much self-promotion of expertise. Considering the world of policy commentarists, in a market that bean-counts page hits, favourites, followers and re-tweets as the measures of quality there is too much room for charlatanism to intervene to boost one’s qualifications.
What a really good expert should be able to do is not tell you the right answer – there is none, events have not finished unfolding – but offer you the spectrum of correct answers. It is the job of the policy maker to decide amongst those, depending on what he decides is most important at that moment to the government or organization for which he works. Perhaps part of the problem is that policy makers have become lazy and have changed how they used the input of experts, taking what should be informed guidance and turning it into a recipe.
Jill’s point is useful. We might distinguish between ‘experts in facts’ and ‘experts in thinking’. The former should be on hand to lay out the basic situation, raise objections to unworkable solutions and offer context, while the latter cultivates skill in thinking without bias and in applying methodological techniques.
I think one of the strange things in asking ‘experts’ to comment on complex social phenomena (such as foreign or military affairs) is that the decisions being made are not being done by ‘experts’. They are made by people (often amateurs, other than by dint of their current job title). Decisions by those in government are not necessarily done through a process of rational consideration. Ego, partial information, bias, internal log rolling, stupidity, face, fear, lack of procedure: all these real world ‘frictions’ get in the way. Sometimes the ‘expert’, with his or her data, org charts and historical understanding, misses the point. It is not necessarily detail that clouds expert judgement, but lack of appreciation for the messy, venal, arrogant, idiotic and sometimes brilliant way real decision makers work.
This is what one gets from the work of Graham Allison and Bob Woodward, for instance. Ideas are floated, become reified, get forgotten, are mistranslated. Sacred cows, White Elephants, and iron rice bowls are paraded about. And then sometimes things are sloppily executed. Some work, some don’t.
The truth is foreign policy/military operations are what happens when human beings are busy making plans.
Of course, economists are the quintessential experts: brilliant at predicting the past. And sometimes they even manage to get that wrong too!
The legendary physicist & Nobel Laureate Richard Feynman said it best: Science is the belief in the ignorance of experts.
Feynman achieved public fame by dropping a piece of rubber o-ring into a glass of iced water before a nationally televised hearing of the Rogers Commision into the Space Shuttle Challenger “accident” which killed the entire crew. That the rubber o-ring’s became rigid and would cease to function well in cold was known but experts mesmerised one-another with Powerpoint presentations. Unfortunately, the day of the doomed flight was freezing – the flight should have been postponed.
Edward Tufte actually (tongue in cheek, but true never the less) attributes the “accident” to Powerpoint. His satiric slide titled the “Cognitive style of Powerpoint” is a classic.
For someone unfamiliar with Feynman – one of the brightest minds of the last century – a good place to start is here:
http://en.wikiquote.org/wiki/Richard_Feynman
James Gleick’s enthralling biography of Feynman is aptly titled: “Genius”
It occurs to me that most people with expert knowledge in a functional domain fairly quickly realise the limits of their abilities. The problem occurs in the opposite direction; if you consult two experts, and one scratches his beard and says, “well, it’s complicated…”, and the other gives a confident answer, who’s going to be called back?