The experts' calculations
Here's how I would answer these questions. Shares are risky, so they must offer a higher expected return to tempt us to own them. Let's say this expected return is 5%. This gives us a level of the Dow Jones industrials average next year of just over 12,000. Given that traders now think the annualised volatility of United States equities is around 17%, this suggests a one-in-six chance of the Dow being 17% below 12,000, and a similar chance of it being 17% above it. Getting slightly technical, there's a two-thirds chance of the Dow ending 2011 between 10,000 and 14,100 – a range of more than 4,000 points. And there's a one-in-three chance of it being outside this range. If this strikes you as a very wide range, that's the point. It's common to under-estimate uncertainty. One reason for this – other than wishful thinking – is that we are overconfident. We often exaggerate the precision of our judgment.
Overconfidence can cost individuals
This is not only a widespread error, but an expensive one. Economists at the University of California Davis have shown that individual investors lose fortunes by trading shares too often – because they are over-confident about their ability to spot good shares. And Norwegian research has found that investors own lots of shares in firms in industries they work in, and yet these investments actually under-perform the market. People think they know about shares' prospects because they work in the industry. But they are wrong.
Companies are not immune from this wishful thinking
Overconfidence is also costly for firms. Academics Ulrike Malmendier and Geoffrey Tate found that overconfident chief executives are prone to making bad investment and takeover decisions. Overconfidence makes us set lofty goals. The lesson here, then, is that we should be more humble about our judgments, and recognise our fallibility, right? Wrong. Overconfidence isn't always a bad thing. In fact, it can be the platform for success.
One reason for this is that it can motivate us to try hard. When asked recently whether he believed United Kingdom football team Arsenal could win a trophy this year, player Robin van Persie replied: "We have to. Everything starts with belief. If you don't have belief, you'd better stop playing football." The footballer who thinks they can become a winner, or the musician who thinks they can become a star, is more likely to put in the practice and training. They are therefore more likely to succeed than their more rational counterparts, who might get disheartened and de-motivated.
We believe our businesses will thrive
What's true of football is also true of business. Official UK figures show that around 10% of firms "die" every year. This implies that there's only roughly a one-in-three chance of a business lasting 10 years. Faced with such horrible odds, few people would become entrepreneurs. They do so because they are overconfident, and believe they can overcome these odds. What economic dynamism we have, we owe to overconfidence.
And others overestimate us
There is, though, another reason why overconfidence works. People mistake other people's overconfidence for actual ability. An experiment by researchers at the University of California Berkeley showed that if someone of middling ability thought they were better than 80% of people, their partner in the experiment ranked them as better than 60% of people. The research says overconfident people send out "competence cues" – they speak louder and more fluently, and have more emphatic body language – and these cues are wrongly interpreted as signs of genuine ability.
The two-edged sword
This suggests that overconfident people are more likely to win promotion than their more rational and realistic rivals – so they are more likely to become those chief executives who make expensive errors. Overconfidence, then, is a two-edged sword. It's the cause not only of failure, but also of success. Some irrationalities, then, can sometimes help us – which is why they are so persistent and widespread.