The Pit, The Fall and The Reichenbach Retreival

The world is abuzz with words like Big Data, Cloud, Efficiency, Real-Time, Analytics, Power, Complex, etc. Most companies are being implored to think about these words. By organizations that are building “capabilities” around it.

Somehow, I think, we are missing the point.

The problem can never be smaller than the tool that solves it. And that to me, has been the bane of the world of analytics/big data/ <fit the new key word> professionals. They create a world where the tool becomes bigger than the end goal it serves. Right from poorly implemented data warehouse solutions, to point-in-time  dashboards that answer a question relevant 5 years back. Do you need to solve your “Big Data” problem? Or, do you need to solve for your business issues?

• Nevertheless, over close to a decade of trying to be an analytics professional, I have seen more examples of organizations trying to latch on to a fad rather than focus on what they have. Social media is just an example.

You’re walking towards a pit, and you should do it only if you’re fond of them and the treks and the views they offer. Not to fall.

If you are just about venturing into the web world, a free plugin of google analytics can reveal a lot to you, to get you started. Instead of a million dollar investment in sophisticated tools and dashboards. Secondly, a good looking dashboard does not always reveal something additional. It reveals in a palatable way. If you’re focused on what you want, that is, if you’ve figured out your business problem, you don’t always need that sexy solution.

Reminds me of that debate we had about two attractive girls – the difference between the two was that the first was sexier, but the second more marriageable. The second had more suitors while the first evoked desire a lot more often.

• A lot of these giant ideas fail. You do fall in those pits. Investments that don’t seem to be worth it. Models which stop being predictive unless fresh blood is pumped into them at regular intervals. Technology that becomes obsolete faster than your ability to eat French fries. You will make mistakes, and I guess that’s not that bad an idea, but the least you need to do is be aware of the costs.

For instance, internet was the in-thing. It still is. But the world is already looking at mobile as the next big thing. Right from iOS to Android enabled devices to extremely interesting content delivered in real-time. For every dotcom that succeeded, there are many that got bust because the fundamental idea itself was not thought through.

Microsoft Excel could handle 65k rows of data with great difficulty at some point. Today, it can handle a million rows with lesser amount of difficulty! And people have started talking about billions of data points. However, a lot of beautiful insights are driven sometimes out one unbiased and strong validated hypotheses. Which often, smartly done, does not take a million rows.

• Last point – it’s difficult to recover from a large investment gone wrong. And I do not always mean monetary investment. Most firms fail to look at executive time investment on half-baked investments as a loss. The opportunity cost of such falls may be significant.

EndNote: The environment today has given us more power in our own hands than most of us are capable of handling. It overwhelms us. We run hither tither and grope around to latch on to the keywords that supposedly enlightened souls are spewing at venomous speeds. No-one is wrong. Yet, the right question for you needs to be the same as it has always been – Is this what the business needs? No. Not wants!

In the Indian mythology, the tale of Bhasmasur is fairly popular. Bhasmasur was a demon who practiced austerity of several years in extreme conditions to please the gods. Lord Shiva appeared in front of him and agreed to grant him a boon in return for his perseverance and dedication. Bhasmasur requested for a power that allowed him to burn anything down to ashes (bhasma) that he’d put his hands on. As the powers, and hence, the tyranny of Bhasmasur reached its pinnacle, the other Gods implored Lord Vishnu to save them. Vishnu took the form of a beautiful danseuse (Mohini Avatar), and tricked Bhasmasur into copying his (or her) dance steps. However, as Mohini put her hand on her own  head while dancing, so did Bhasmasur on his head. And thus, Bhasmasur burnt himself down.  Bhasmasur forgot the reason why he aspired for that powerful hand. The hand that burnt him down.