Monday, January 4, 2010

Regression to the mean


So it’s 2010 and plenty of people out there are making New Year’s resolutions. Personally I’m not one for picking a specific date to commence a goal, once I’ve set it I’d rather get going immediately. I do however appreciate that everyone is different and that the important thing is finding something that works for you. I will however point out why I believe New Year’s and most other resolutions fail miserably.

We are all typically the average (or mean) of our environment. If you want to reduce your golf handicap, you don’t play with people who you constantly beat, you play with people who are better than you. Even though you may lose, your game will improve because it is moving towards a higher average. The same theory applies in many other walks of life if you stop and think about it. Maybe the reason you aren’t a billionaire is that you don’t hang out with billionaires, instead you typically hang out with people in your own socio-economic demographic (i.e. those who make a similar amount if money as your do).

Now let’s say that you’ve made a resolution to change. Typically this change is quite different from your current situation (or mean/average position). One of the most common New Year’s resolutions is to lose weight. People who probably do very little exercise suddenly decide to do 10 times the exercise they would normally do. If they could maintain that 10 times regime then everything would be great however I’ll content that the chances of being able to maintain any deviation from the mean actually decrease the further the deviation is from the mean. In this case I would say that the chances of maintaining the 10 times resolution are no better than one in 10 i.e. 10%.

Instead, if they made a resolution to do just a little bit more exercise (at least initially), to twice their current average or mean say, I’d bet their chances of succeeding are now one in two i.e. 50%. The issue is that people fail to appreciate the inertia that the mean or average effect brings. In short, it is very difficult to change overnight, because what you are ultimately looking to achieve is an improvement of your mean performance, whatever that is and one sudden massive change doesn’t stand much chance.

If you plan to make a life changing resolution then to be successful you need to look at it like walking down a road as well. In such a situation you either walk down the left or the right hand sides. Walking down the centre is simply madness because sooner or later you will lose an argument with a vehicle. So before you start out you need to come to a decision of either walking on the left hand side (no I won’t do this) or the right hand side (yes, I will do this) but never down the middle of the road (maybe I’ll do this).

As simple and obvious as all this sounds most people typically over look these things when making their resolutions. They figure the more radical the change the better without understanding that the chances of success decrease with the distance from the current mean or average. True success lies in improving the mean value of your resolution and the easiest method achieving this is via small but regular increments. This regularity requires a discipline to make the desired change - that is, walking down the right hand side of the road rather than the middle.

How many road kills are we likely to see in the next few days as people walk down the centre of the road vacillating with all the best intentions of making a change but dooming themselves to failure because they failed to understand the overriding trend of the regression to the mean? It’s all simple math really.