What is... | April 16, 2014

What is base rate neglect?

Have you seen one of the stores that proudly advertises how someone once bought a winning lottery ticket there? They sometimes have a photograph on the wall of the smiling winner, oversized cheque (bearing lots of zeroes) in hand.

Stories of a win can encourages others to buy lottery tickets. When the win occurs at a particular shop, others are likely to go there, hoping a bit of luck will have rubbed off. But what’s missing is the odds of winning. Is it one in 10,000? One in 100,000? That's far too ambitious. In reality the chances of winning a UK lottery are around one in 14 million. You’d need to buy 100 tickets a day for almost 400 years to get every possible combination of numbers.

Overestimating the chances of winning the lottery is a classic example of what academics call “base rate neglect”. Put simply, base rate neglect is a failure to take into account enough background information when estimating how likely an event is to happen.

Should I buy that insurance?
A quick Google search can tell you the true probability of winning the lottery but in many areas of life it’s not so easy to figure out the odds. For example, many low-cost airlines offer you the chance to buy travel insurance while booking a flight. How likely is it really that you’ll have an accident on holiday? One in 150? One in 2,000? You almost certainly don’t have enough information to calculate something like that, or you might not have the time to think it through. Also, your decision-making might be affected by a “hot state”, like being excited about your trip. For some people, it might just be easier to accept the insurance, even if they could get a better deal elsewhere.

A similar dynamic might exist in an electronics store, when the sales representative offers an extended warranty and mentions that three similar products were returned last week. The question that should spring to mind is three out of how many. This isn’t just a matter of smarts. A famous 1978 study found that even doctors at Harvard medical school can fall prey to base rate neglect when given a tricky diagnosis problem.

Jumping to conclusions
Closely related to base rate neglect is the representativeness heuristic, a common mental shortcut where people evaluate how likely something is based on something like a “first glance” impression. An example might be assuming a nerdy-looking young person wearing glasses is a student. Using this heuristic isn’t necessarily a problem - because it often works.

However, the base-rate being underweighted here is the fact that there are thousands of slightly nerdy looking people in glasses, of a similar age, who aren’t students. The representativeness heuristic can have serious implications for financial decisions. Someone might, for example, use the shortcut and decide to invest with a fund manager partly because they seem very self-assured, wear an expensive suit and have a nice office.

The base rate being ignored is the true skill level of fund managers in general at picking investments. While fund managers tend to be very confident, few of them are successful – suggested by this study that found up to 75% of 2,000 actively managed US funds returned no more than what should be expected by taking increased financial risk.

Make smarter short cuts
It’s not just amateur investors who ignore the base rate. Research published in 2010 examining business decisions shows that professionals who underweight statistical data in favour of anecdotes make worse business decisions. Similarly, investors often overestimate the likelihood of success of a dramatic event such as a merger between two well-known companies without taking into account the fact that the majority of mergers fail, detailed here.

Lastly, one of the most famous papers in behavioural finance, by Werner De Bondt and Richard Thaler, argues that share markets frequently overreact to dramatic news events. Individual circumstances will vary but based on this research, a sensible investing strategy might be to focus on broad, long-term trends which take into account all the background “base rate” information. When it comes to personal finance, these long-term trends are perhaps best summarised in University of Chicago professor Harold Pollack’s index card, reproduced on the Washington Post, which recommends smart thinking shortcuts (saving 20% of income, paying off credit cards in full and choosing funds based on criteria such as being passively managed, diversified and with low fees).

Don’t forget the losers
A lesson here is to minimise base rate neglect by maintaining a sense of perspective. Remember that store with the picture of a smiling lottery winner? If that tempts you to buy a ticket, pause a moment and fill in the base rate chance of winning by imagining the store covered with pictures of people who didn’t buy winning tickets. In your mind’s eye you should see one small picture of the happy winner surrounded by millions of pictures of glum people who’ve wasted their money.

InvestingPersonal financeBehaviourMistakesLottery

Mark Egan

Behavioural science consultant