Friday , August 17 2018

On thin predictions

chris by
chris
My articles My books
Follow on:
Summary:
Back in 2016 I wrote that “immigration controls will hurt decent people…because they are the softest targets.” Paul Cotterill interprets this as meaning that I saw the Windrush scandal coming. I think he’s being too kind, but this illustrates an under-appreciated point – that good predictions can be “thin” in the sense of containing little data. I’ll give you two other examples from my day job. One is that back in 2007 I pointed to record foreign buying of US shares as a warning that the All-share index would fall sharply over the following 12 months. Which it did. Then last February I forecast that sterling would rise. Which it has – by 15% against the US$. Now, I don’t say this to toot my own horn.  (There's a reason why these examples are ten years apart!) I do so to point to

Topics:
chris considers the following as important:

This could be interesting, too:

chris writes For automatic stabilizers

chris writes Over- and Under-reactions in politics

chris writes Our broken politics

chris writes Labour’s Bank of England problem

Back in 2016 I wrote that “immigration controls will hurt decent people…because they are the softest targets.” Paul Cotterill interprets this as meaning that I saw the Windrush scandal coming. I think he’s being too kind, but this illustrates an under-appreciated point – that good predictions can be “thin” in the sense of containing little data.

I’ll give you two other examples from my day job.

One is that back in 2007 I pointed to record foreign buying of US shares as a warning that the All-share index would fall sharply over the following 12 months. Which it did.

Then last February I forecast that sterling would rise. Which it has – by 15% against the US$.

Now, I don’t say this to toot my own horn.  (There's a reason why these examples are ten years apart!) I do so to point to something these calls have in common. Both are lacking in description. The forecast that share prices would fall in 2008 makes no reference to the financial crisis. And my forecast for sterling ignored most of the things that are usually thought (perhaps wrongly!) to determine exchange rates such as growth or interest rates.  

 Instead, both these calls rested upon single facts and single theories – that foreign buying of US equities was a measure of irrational exuberance (it still is) ; and that exchange rates overshoot (pdf).

These are not isolated examples. A single simple fact – is the yield curve inverted or not? – has done a better (pdf) job of predicting recessions than the more data-intensive forecasts of macroeconomists, most of whom have missed (pdf) almost all recessions.

What this tells us is that sometimes we don’t need big models to predict things – be they micro-founded or not. One strong fact sometimes beats lots of weak ones*. In these cases, we have lead indicators, but not any detailed explanation.

In saying this I am echoing Jon Elster:

Sometimes we can explain without being able to predict, and sometimes predict without being able to explain. True, in many cases one and the same theory will enable us to do both, but I believe that in the social sciences this is the exception rather than the rule. (Nuts and Bolts for the Social Sciences, p8)

You might think that this contradicts the message of Tetlock and Gardner’s Superforecasting. They say that foxes (who know many things) make better forecasters than hedgehogs, who know one thing.

I’m not sure there’s a contradiction here. For one thing, many of the cases they consider are one-off political events – sometimes quite obscure ones – where success requires careful data-gathering. In the cases I’m considering, however, data is abundant and the trick is to distinguish between signal and noise, to decide what to ignore. Occasionally, one fact sends a stronger signal than many. Which corroborates Tetlock and Gardner’s point, that “in most cases, statistical algorithms beat subjective judgment.”

This is not, of course, to say that simple lead indicators always work. Obviously, they don’t. My point is merely that forecasting is not the same as modelling, nor the same as telling a good story.

* Certainly, rules of thumb often beat attempts to forecast particular events, but that's a different story.

About chris
chris

Biography data hidden due GDPR Data Protection. Author consent pending.
(Economic Blogs is not responsible for linked external content)

Leave a Reply

Your email address will not be published. Required fields are marked *