Have you heard an economist claim forecasting accuracy of 98% or better? They’re out there. It’s a bold statement in light of our Federal Reserve acknowledging that their own forecasts of one year out or longer are no more accurate than a statistically generated guess. Bold claims of forecasting accuracy may not be as they first appear.
I’ll use the U.S. employment report as my example to illustrate how misleading claims of accuracy can be. The same though is true for nearly any economic indicator as well as many investment related measures.
The employment report is one of the most watched and studied of all economic indicators. Dozens of economists provide “expert” forecasts about what can be expected in each monthly report. They support their forecasts with analysis and explanation that is intelligent and thoughtful but, as we will see, you could blindly pull a number out of a hat and be just as accurate.
The Shocking February Employment Report
The February employment report was a shock to many, with numbers coming in far below the average forecast, providing a great example of how wrong forecasts can be.
The economy has been adding roughly 200,000 new jobs per month for the past few years. The easy and obvious prediction would be for another 200,000 new jobs. The consensus, or average, estimate for February was 180,000. On CNBC just minutes before the February report was released, six experts confirmed their estimates, all in the range of 150,000 – 250,000 or so, and provided their rational for their “expert” views.
The report was a surprise showing just 20,000 jobs added for the month and only minor revisions higher for prior months. The “experts” were surprised…or more like shocked!
Once the initial blow of the report was absorbed and the reality of their horribly inaccurate forecast set in, these same experts then proceeded to give their explanation as to why the report was so weak. Think about this…the same economists who just minutes before were explaining why they thought the economy would add 180,000 jobs are now the same economists explaining why it only added 20,000 jobs! Clearly anything can be explained with the benefit of hindsight!
The Illusion of Accuracy
Some economists have devised ways to make it appear to you that their ability to forecast is far better than it actually is. The trick is to forecast one thing when you may think it is something different. For example, a forecast of 180,000 new jobs in February compared to the actual number of 20,000 was off by 160,000 jobs or an 88.9% miss. This is a forecasting accuracy of just 11.1% (100.0 – 88.9). But what if it wasn’t new jobs they claim to forecast but instead a related but much easier number to get very close to?
What’s being forecast?
Let’s get it clear what people talk about as it pertains to the employment report. Below are some headlines from stories covering the February employment report from leading news organizations:
Furthermore, the first sentence of the actual employment report from the Bureau of Labor Statistics states “Total nonfarm payroll employment changed little in February (+20,000)…”.
This all makes clear that the number everybody is interested in is the number of new jobs created or lost. In the case of the February report, it’s all about the estimate of 180,000 new jobs created. But what if an economist instead was forecasting a different number, TOTAL employment, and not the number of jobs added or lost? The accuracy rate is very different!
At the end of January 2019, total nonfarm employment was 150,587,000. In my above example, the forecast is for total employment to grow to 150,767,000 or 180,000 new jobs created. The point is that the forecast is for total employment and not on jobs created even though it is the number of jobs created that gets reported and all of the attention.
As it turns out there were only 20,000 new jobs created and total employment grew to 150,607,000. Again, the forecast of new jobs created was off by 88.9% resulting in an accuracy of just 11.1%. But, as the math works out due to the large size of the numbers, the forecast of total employment was off by just 0.1% and the economist claims an accuracy rate of 99.9%! Based upon this math, an economist could be wrong by millions of jobs created and still claim a very high forecasting accuracy.
Anybody (you, me, my mom, your kids, etc.!) can forecast total employment within a few million jobs. We could all claim a very high forecasting accuracy. It’s simply useless.
Economists’ inability to forecast the number of jobs created or lost is well documented. It’s not just forecasting the change in employment that cannot be done well but also the overall economy, the stocks market, and many similar things. We all so badly want to believe someone can tells us something reliable about the future of the economy and stock market but every bit of evidence suggests these forecasts are unreliable at best.
Be suspicious of any economists or market expert suggesting they have and can accurately forecast. And certainly think long and hard before acting on such forecasts.