An old saw has it that an expert is a drip under pressure. Nevertheless our modern world loves experts--demands them, in fact. The modern expert has taken the place of the ancient soothsayer, or rune reader. The most worthy and vital contribution that experts are required to make is predict the future. This is why they function in our Unbelieving culture as the modern equivalent of augurs.
Most of their predictions are stupid superstitions, appealing to the more base human emotions of fear and greed. The more lurid the prophecy, the more valued. The status of "expert" buoys up its certainty and credibility. As G. K. Chesterton astutely observed, when men stop believing in the Living God, they do not believe in nothing; they begin to believe in everything. Credulity characterises an Unbelieving culture, such as now dominates the West.
There is an irony in this. The West prides itself on its knowledge, its scientific rigour, its evidence-based governments. In reality it has become suborned to superstitious speculations gilded with a veneer of academic rigour and research. The role of the expert in our world is to provide the gilding.
In a recent article in The National, Robert Matthews takes a much needed sceptical look at experts and their auguries. It turns out that the more famous the expert, the worse his predictive abilities.
The more famous the expert, the worse his predictions
Last Updated: May 8, 2011
As predictions go, it was truly disturbing - made all the more so by the authority of the source.
In 1989, Dr Mustafa Tolba, the head of the UN Environment Programme (Unep), warned that over the coming years as many as 50 million refugees would be wandering the globe to escape the ravages of climate change.
By 2005, Unep felt confident enough to say the 50 million mark would be reached "by 2010". Other experts agreed, among them the celebrated environmentalist Professor Norman Myers of Oxford University.
So where are they? In a word, nowhere. A recent study by the International Institute for Environment and Development (IIED) in England found no evidence of any mass migration caused by climate change.
On the contrary, it suggested that - unsurprisingly - people prefer to stay in their own country in the face of environmental upheaval.
Unep has tried to disown the prediction, but has succeeded only in sparking a media furore after being caught removing graphics clearly stating "50 million climate refugees by 2010" from its website.
It looks like the agency is learning the truth of the ancient Chinese proverb that "prediction is very difficult - especially of the future".
It's unlikely to give up making predictions, though: after all, that's what expert groups are supposed to do. But as a fascinating new survey of the prediction business shows, we should all be much more sceptical about forecasts - especially those made by experts.
As the journalist Dan Gardner points out in Futurebabble: Why Expert Predictions Fail And Why We Believe Them Anyway, experts have been getting predictions wrong for centuries, for all kinds of reasons.
In 1789, the English economist Thomas Malthus showed with almost mathematical certainty that the world was condemned to mass starvation by the obvious fact that populations increase exponentially, inevitably outstripping food supplies.
Two centuries on, UN Food and Agriculture Organisation figures show that even the least developed nations are enjoying rising food levels. Clearly, Malthus had not banked on the ingenuity of agriculturalists to feed the world.
Everyone makes mistakes, of course, but as Gardner shows, experts are quite often prone to making them.
He cites the results of a pioneering study begun in the 1980s by Professor Philip Tetlock, a psychologist at the University of California who brought together hundreds of experts in political science and economics, and asked them to predict what the future might hold.
The result was a collection of more than 27,000 forecasts whose veracity was then checked over the following years. The outcome, published in 2005, was salutary. It showed that the typical expert did not perform significantly better than random guessing.
But Prof Tetlock went further, trying to identify why some experts were so much worse than others. He found that political beliefs or levels of optimism made no difference: a cheery right-winger was just as likely to do badly as a miserable Marxist. Qualifications or access to confidential information did not matter, either.
Far more important, he found, was the mindset that the experts brought to making predictions.
Those who did badly did not like getting bogged down in complexities, or weighing up the evidence from a variety of sources. Instead, they had a habit of making predictions that complied with some grand, overarching thesis. And having made their predictions, they were - ironically enough - strikingly confident about them.
A grand thesis, simple views, confidence ... as Gardner points out, that's pretty much a thumbnail sketch of the perfect media pundit.
Yet according to Prof Tetlock's research, those are precisely the characteristics of experts whose predictions are worse than random guessing. And that, in turn, suggests that the very fact a pundit makes regular media appearances means we can ignore his or her predictions.
Gardner reports how Prof Tetlock put this simple rule to the test using Google hits as a simple way of measuring the "celebrity" of each of the 284 experts who took part in his study. Sure enough, the more famous the expert, the worse his performance. So if we can't trust the pundits foisted on us by the media, whom can we trust?
According to Gardner, we should look for experts who do not start from the assumption that some Big Idea (often their Big Idea) is correct. The future has a habit of making a mockery of grand theses. Instead, the starting point should be information gleaned from a wide variety of sources. Once some broad-brush conclusions have been made, the most reliable forecasters tend to analyse where their conclusions came from. Explaining them to others often helps to reveal assumptions and leaps of faith that just can't be justified.
The final characteristic to look out for, says Gardner, is simple humility. Anyone with total confidence in their prediction of the future should be treated with suspicion. Paradoxically, those who say merely that there is a "high chance" of some event happening are more likely to be right than pundits who simply declare "it will". And those who make precise, long-term predictions - like, say, 50 million environmental refugees by 2010 - are best ignored completely.
Yet the real problem with forecasting is not so much dodgy pundits, as those who demand to hear them. Most of us are looking for some certainty in this uncertain world, and we crave the kind of certainty touted by "experts". Until we wean ourselves off this irrational desire, the law of supply and demand means we will continue to get the pundits we deserve. And that's a prediction you can totally rely on.
Robert Matthews is visiting reader in science at Aston University, Birmingham, England
No comments:
Post a Comment