… but not very well.
That was the the headline from some ground breaking research released last week by Income Lab, a US-based software platform. However, the detail of the research is particularly useful for those of us who have an interest in retirement planning (including anybody who has retired, or is about to!).
One of the things we do when advising people about retirement is to run a stress test. This tells us how likely it is that you will run out of money during your lifetime, based on your spending plans and the retirement resources you have. This is based on the assumptions and the model which we use, which considers things like investment returns, inflation and how much these things vary.
What the research has done is to compare how well the various ways of creating the assumptions and the models have done at predicting the real world. What it shows is that using historical data is one of the better ways of predicting the future, and this is the approach which we use. We use a system called Timeline, which takes data for investment returns and inflation, going back to 1915, and works out the likelihood that you will run out of money during your lifetime, based on your personal plans for retirement. The research identified that the models were quite good at predicting the future in some periods in the last 50 years or so. But less good in other periods. The worst period of performance for the models was the 1980s, when economies were booming.
Weather forecasters tend to prefer a “wet” bias in their predictions – this means that they prefer to be wrong by telling you it will rain when it doesn’t, rather than being wrong by predicting dry weather when it actually rains. Weather forecasters know that they can’t be 100% accurate about rain, so they prefer to use models which err on the side of rain – a “wet bias”.
The researchers from Income Lab looked at whether the stress testing models available have a wet or dry bias – i.e. if they were wrong, did they predict success instead of failure, or failure instead of success. The historical model which we use has a “wet” bias – it has been pessimistic. That’s good in a way as “not running out of money” when you thought you might, is not a bad outcome really!
One important conclusion from the research is that we still can’t predict the future well enough to set up a retirement plan and just leave it to run its course for 30 years or so. This probably won’t ever be possible! However, the method which we use to assess the likelihood of the success of your retirement plan is one of the best methods available and we can adjust for its limitations. That will help us to tell you if you have enough put aside to allow you to stop work, and to help you to work out if you are spending too much or too little.
If you are interested in the research itself, follow this link to the Nerd’s Eye View blog https://www.kitces.com/blog/monte-carlo-models-simulation-forecast-error-brier-score-retirement-planning/
Philip Wise | email@example.com
Managing Director and Chartered Financial Planner