Understanding Regression Towards The Mean | ‘The Nassim Taleb & Incerto Podcast’

Regression towards the mean attempts to dampen our hyperbolic reaction to both outliers and those significant but rare exceptions.

Regression towards the mean explains how a random variable which lies outside of the norm will tend* to eventually return to the norm. * = because it’s worthwhile to caveat that nothing is perfect and so while the great majority of cases will reliably tend towards the norm it is not a universal principle. Exceptional cases exist where perhaps an outlier variable will not over time regress towards the average.

A classic example of regression towards the mean is a very familiar one to my countrymen of Australia. How regression towards the mean applies to those who are ‘good‘ at pokies (slots, for my dear international reader).

A Classic Example Of Regression Towards The Mean

Your odds are the same every time you ‘go for a slap’ (play slots). The time of day, type of machine, how long the other guy has been losing on it, the venue you go to, beers you have consumed – these all seem like relevant variables which you can exert over the game to increase the odds in your favour.

But, like so many ways we think we can exert our influence on the world, they are insignificant and subject to folly.

Every time we sit down to a cathartic session on the pokies, our odds of winning on that machine are always the same (within our control). You might hit a ‘hot streak’ and strike feature after feature. But while you may have stumbled across an against the odds victory on this session, play the machine long enough, and random luck will regress to the mean and return you to normal, ending up losing after all.

This is the cause of so much lamentation from your mates about the necessity to re-create the conditions of the night you struck feature gold!

See my article about survivorship bias, taken from my Nassim Taleb Podcast to see about ‘beginners luck’ and how this is simply regression to the mean explained in another fashion.

πŸŽ™οΈ Listen To This On The ‘Nassim Taleb & Incerto’ Podcast

On Apple Here
Spotify Here
Everywhere Else

Consequences Of Bad Sampling

Poor sampling, or asymmetric sampling, can return you a disproportionate insight into the focus of your study.

By not taking a wide enough, or long enough sample – you expose yourself to having only sampled those events or inputs which, unaware to you, exist at the tail. Therefore you fool yourself into results abnormally high or low over time, and so will ultimately and begrudgingly experience regression towards the mean.

Regression Towards The Mean In Real Life

Regression towards the mean is best understood over longer time horizons.

Think about how regression to the mean could be understood through the prism of financial markets.

Buffet insists that should you just buy the S&P500 and hold for a lifetime and your return is likely to be significantly better than many iterations of short term speculation. Now, this statement only really makes sense when you have the time patience and earned wisdom of 90+ years on this earth.

Because, had you been given the advice to simply buy the most boring middle of the road index fund during the years leading up to the tech bubble of the early 2000s, then remaining on the sidelines through the 100%-200%-500% returns of some of the tech stocks would have made you look like a fool. The same can be said for the market bubble pre-2008 GFC, or the crypto run of 2020. But overall, more people lost everything overall more than they returned a 500% bagger.

If you were trying to measure average market returns and sampled the tech stocks of the early 2000s then you would have experienced a very said and disheartening regression towards the mean in the following years.

Your sampling off average stock market returns would have been in the tail end years.

Furthermore On The Financials

Returns can be very unstable in the short run but very stable in the long run.

More quantitatively, it is one in which the standard deviation of average annual returns declines faster than the inverse of the holding period, implying that the process is not a random walk, but instead, periods of lower returns are systematically followed by compensating periods of higher returns, as is the case in many seasonal businesses.

What if you sampled the husky sledding income of my mate Uno’s business in Γ–stersund during the month of July? You’d be under the impression that the business was in fact much smaller than is the reality. If you returned to sample again once a month for the following 24 then you would realise a very satisfying regression towards the mean in understanding the true value of Uno’s dogs.

Implications Of Regression Towards The Mean

This is a statistical phenomenon very important to scientists and researchers who are trying to design and implement experiments. For these types, it is extremely important to sample correctly, for as we have seen, bad sampling can lead to regression towards the mean that skews completely the findings of any specific study.

Daniel Kahneman – an extraordinarily famous psychologist and subject and author of many books, including the brilliant ‘Undoing Project’ plus, is someone Nassim Taleb praises very enthusiastically, discovered how regression towards the mean might also apply to certain behaviour.

Daniel Kahneman Regression Towards The Mean Example

Kahneman was testing how Israeli fighter pilots responded to different types of feedback. Some he would encourage and some he would discourage, he would then measure the differences in their subsequent performances.

To make the story short I will simply explain the findings as follows. When one makes a severe mistake, their performance will later, usually return to their average level anyway. This will seem like an improvement and as “proof” of a belief that it is better to criticize than to praise. In the contrary situation, when one happens to perform high above average, their performance will also tend to return to their average level later on. The change will be perceived as deterioration and any initial praise following the first performance as a cause of that deterioration.

Just because criticizing or praising precedes the regression toward the mean, the act of criticizing or praising is falsely attributed causality. 

Always Be Sceptical Of Loaded Dice Or A Loaded Coin

So, while it is possible to see 100 heads in a row, as you flip a coin and bet on the outcome, the occurrence of such an outcome is so unbelievable that it is you are likely being taken for a ride.

The same can be said of a diceman who can seemingly throw exactly what he needs every time.

Don’t let an outlier prove a rule, instead beware of regression towards the mean and understand that over a long enough timeline, assuming you don’t meet 0, most things returns to an average distribution.

πŸŽ™οΈ Listen To This On The ‘Nassim Taleb & Incerto’ Podcast

On Apple Here
Spotify Here
Everywhere Else

Subscribe To My Newsletter – πŸ—’οΈ A Curious Worldview!

Cheers for reading through to the end you legend!

Every month (or couple of weeks) I write a short, popular email.

In it, you will have your worldview expanded upon. It will either challenge your beliefs or open your eyes to something new. Never boring. Sometimes in will include things you don’t want to see. Sometimes it will be exactly what you were looking for.

It is free and the best thing you can do for me. If you are keen, chuck your email in below and then CHECK YOUR SPAM!

MAKE SURE YOU CHECK IF THE EMAIL GOES TO THE WRONG EMAIL FOLDER – PLEASE DRAG EMAIL INTO PRIMARY FOLDER

Leave a Reply

Your email address will not be published. Required fields are marked *