I recently learnt that a number of people think that the more the number of variables you use in your model, the better your model is! What has surprised me is that I’ve met a lot of people who think so, and recommendations for simple models haven’t been taken too kindly. The conversation usually goes…
Black Box Models
A few years ago, Felix Salmon wrote this article in Wired called “The Formula That Killed Wall Street“. It was about a formula called “Gaussian Copula”, which was a formula for estimating the joint probability of a set of events happening, if you knew the individual probabilities. It was a mathematical breakthrough. Unfortunately, it fell…
On finding the right signal
It is not necessary that every problem yields a “signal”. It is well possible that sometimes you try to solve a problem using data and you are simply unable to find any signal. This does not mean that you have failed in your quest – the fact that you have found the absence of a…
Missed opportunities in cross-selling
Talk to any analytics or “business intelligence” provider – be it a large commoditized outsourcing firm or a rather niche consultant – and one thing they all claim to advise their clients on is strategies for “cross sell”. However, my personal experience suggests that implementation of cross-sell strategies among retailers I encounter is extremely poor.…
Nate Silver Interview At HBR Blogs
HBR Blogs has interviewed Nate Silver on analytics, building a career in analytics and how organizations should manage analytics. I agree with his views on pretty much everything. Some money quotes: HBR: Say an organization brings in a bunch of ‘stat heads’ to use your terminology. Do you silo them in their own department that serves…
Hedgehogs and foxes: Or, a day in the life of a quant
I must state at the outset that this post is inspired by the second chapter of Nate Silver’s book The Signal and the Noise. In that chapter, which is about election forecasting, Silver draws upon the old Russian parable of the hedgehog and the fox. According to that story, the fox knows several tricks while the…
Why standard deviation is not a good measure of volatility
Most finance textbooks, at least the ones that are popular in Business Schools, use standard deviation as a measure of volatility of a stock price. In this post, we will examine why it is not a great idea. To put it in one line, the use of standard deviation loses information on the ordering of…
Anscombe’s Quartet and Analytics
Many “analytics professionals” or “quants” I know or have worked with have no hesitation in diving straight into a statistical model when they are faced with a problem, rather than trying to understand the data. However, that is not the way I work. Whenever I set out solving a new problem, I start with spending…
Exponential increase
“Increasing exponentially” is a common phrase used by analysts and commentators to describe a certain kind of growth. However, more often than not, this phrase is misused and any fast growth is termed as “exponential”. If f(x) is a function of x, f(x) is said to be exponential if and only if it can be…
Should you have an analytics team?
In an earlier post, I had talked about the importance of business people knowing numbers and numbers people knowing business, and had put in a small advertisement for my consulting services by mentioning that I know both business and numbers and work at their cusp. In this post, I take that further and analyze if it…