The Broken World of Marketing Data Sciences


Do you know why the world of marketing has a response rate of 2-3% and is happy with it? Even lesser? It is because the world of marketing data and marketing data sciences is broken. And marketers don’t know a way around it.

Here’s what the data looks like typically-

  • Market Research & Panel data – For decades, marketers have been happy dealing with panel data that fulfills what they believe in. Like the huge TG for their product and innovation. So much so, that it takes years of business disasters in a row for a Nokia to understand how things have changed. But Panel Data is at best a coarse approximation. The sampling is flawed 9 out of 10 times. Flawed. I am not even talking about biases. Or, fraudulent data entry.
  • Focus Groups – Less said the better. The FGDs are not representative (by design), the insights are more often than not an accident, and there is a serious dearth of good marketers capable of running half-decent and probing focus groups. What you can usually depend on here, though, is something which is unanimously panned or applauded. Like the Kala Bandar (Black Monkey) of Delhi 6 (the movie).
  • Test & Control Groups – Several practitioners have lost their voices screaming about the need for quality T&CG setup. The only real deal here, to some extent, was the world of database marketing. Unfortunately, DBMs believed a little too much in their potential, and focused less on their improvement areas, a problem that often accompanies success.
  • POS/Warehouse Data – Logged in/ loyalty customers is what we are capable of tracking, they’d tell you. And can they track you across your properties? Their properties? Households? Don’t they have the data already? And what about the other data elements? How good are they? What about that graph model thing?
  • Big Data – Lets mine social media, twitter sentiments, facebook posts, and let us use key opinion leaders and influencers, and viral content. And we will have machine learning for real time recommendations and offers. And… I know. I believe in the possibilities. But do you really understand it? Where does it even begin?

What this has, sequentially, led to, is –

  • Lack of trust in the data – an obvious side effect, won’t you think?
  • Rise of Witch Doctors –People with creative halos, learned expressions, snob behavior, and/or a “heightened sense” of what the consumers want. The primary reason why you’d trust them – because they tell you so. And, because you have no alternate version.
  • A Kodak Moment – All traditional data providers are getting destroyed by Google, Facebook, Inmobi and the likes. These companies have brought “attribution” and measurement to the center of the discussion.

There has been, therefore, a dearth of high quality and effective use of marketing data. Take for instance, Amazon. It’s a data-science influenced company. Yet, what you get to see on the landing page tells you how broken the listings/ recommendation algorithms are. And they’ve been playing with it for a while. Flipkart fares much worse. The others are not even talking about it or worth talking about. Offline retailers have little good data. Brand managers work with incorrect data most of the time (or trade data). Yet, conversations about analytics and big data makes for good headlines.

Nevertheless, there are few philosophical departures, none too easy, to keep pressing forward.

  • Invest in data – Most companies need a multichannel data play, common denominators across platforms, and investments in “data collection”. It’s long, tough and they don’t have the right focus or resources to do it.
  • Improve data quality – Data needs to become a constant companion in every discussion. Across HR, Finance, Legal, Admin, Marketing, Sales, Strategy. Any investment in that direction would, today, mean creating hundreds of cross-walks, resolving inconsistencies, requiring layers of data cleansing, and finally, standardizing the definitions. It would be, for most companies, a journey of at least 12 months if they are smart, ambitious and ruthless. Ruthlessness is the key.
  • Data in the front office – One recurring funny moment in every consultants’ and analytics professionals’ life is walking into a cross-functional meeting, with at least 40% of the attendees reacting at some point– “I’ve never seen this data before” OR “this data is wrong, because I checked last week and it was something else”. That’s because data and analytics sits in the back-office, and reacts to distributed demands and definitions. If it were in the front office, like revenue and profits, it would be reported correctly 8 out of 10 times.
  • CEO’s Mandate – All professionals, in time, will have basic understanding of data, like they (should) have of English (as a proxy for language of the business) and Mathematics. Till then, however, it needs to be driven like a CEO’s mandate. Will that happen?
  • Data Monetization – All of this pain becomes easier if start monetizing the data itself. Data and Analytics are a cost center. They can be a P&L. Especially, if companies stop being scared of how priceless their data is. It is not. The only thing that makes your data priceless is how it is used. Governance around the usage of data might help companies more than sitting on it and hatching it.

You know one thing that would really help? Knowing when you need help and being candid about it! And – I’ll not get into the psychological reasons (I know better than data), infrastructural reason (our technology and analytics team is not ready for it) or business reasons (it is not a priority at this point) why marketers don’t ask for help.

<First published on linkedin>



Analytics Thought For The Day: The Power of Grunt Work

<My first post on LinkedIn after the invitation to publish>


The other day, I was watching a fascinating soccer match (English Premier League). Somewhere towards the end of the game, I realized that the poor fellas had been running for about 94 minutes to end with a 1-0 scoreline. Somewhere, in those 94 minutes, there were more than 12 genuine attempts at scoring, several high quality defense maneuvers, many strategic plays, but just one that hit the bulls-eye. Was it worth it? Yes. Were the remaining 93 minutes 30 seconds irrelevant? No.

To me, those 93.5 minutes, and the quality of play therein, decides the quality of the team, the real value of that 1-0 scoreline. That grunt work decides whether it was all worth it in the end. Agreed, in the end, a 0-0 scoreline is not even half as awesome as a 1-0 scoreline (especially when my (preferred) team wins). But would I rather just watch the highlights (and that one goal) along with the match ending analysis? Hell, no!


Fortunately for my clients, and unfortunately for me (and maybe some of my team members), at a very early stage of my analytics career, I graduated from being an individual contributor to being a team and engagement manager. The clients never had to realize how sub-par I was at being an individual contributor (one deficiency less!) while my teams had to dumb things down for me at times. However, a common emerging theme through the years and projects has been the power of grunt work – the act of doing the supposedly less interesting tedious work over and over again. We tend to discount the tedious work for the glory of the big moments.Having to clean and reorganize the data a few times over in a project to ensure that the business objectives are still central, and not the data scientists’ need to build a super-complex model or deploy a super-efficient algorithm. The highlights look better focusing on the buzzer beaters than on the continuous court runs required to be at the right place at the right time for that buzzer beater.


What do I mean by that?

In any form of sports, coaches insist on getting the basics right. The stance, the form, the shot, the power, the follow through, the stable head position, and so on. And to get these right, a good player (and not even a great player) has to keep doing it again and again. Borrowing an analogy from the game of Cricket, I have always been fascinated by how after getting beaten by a particularly beautiful delivery, even great batsmen like Sachin Tendulkar or Rahul Dravid or Ricky Ponting play out the same delivery in their head and in their shadow practice a few times. One can see them going through the motions of replaying the shot they had just played and a calculation of sorts on what they would do differently the next time.

Having reached the level of proficiency they had already, they shouldn’t be bothered with it, right? Wrong! You still have to go out there and get the basics right. Every time. All the time.

Likewise, in any analytics project that we undertake, one of the most critical things is to get the baseline right. The problem that we are solving, the exact analytical problem definition, the data that we’re going to look at, the process of reformatting and re-purposing the data to our needs, the need to do the univariates and bivariates before we get to the multivariates and the models, and so on.

And it is here that one of the most glaring needs of the analytics and data sciences industry lies, in terms of talent creation and talent management. The way we are growing our data scientists of the future, we are hoping that most of them would be able to build a model without touching and feeling the data, or find AI/NN solutions without caring about the basic business hypotheses.


And be honest here! How many of us find this dealing with the data business one of the most tedious things to do? Something that we would happily outsource to someone else? Something that we would happily let someone else take the accountability for? Or blame for?

Are you satisfied with your analytics training?

Lets assume that you’re in a third party analytics services/ consulting organization. I assume that your organization started with a core of like minded people with the relevant analytics credentials. And I would like to assume one of the following two things – that you’ve grown somewhat, or are planning to scale soon.


Source: The Cartoon Guide to Statistics (from Avinash Kaushik’s blog)

You must have hired or are planning to hire fresh talent. Recent undergrads and graduates. When you hire them, do you put them on a project on day 1? If not, do you train them? What is the nature of this training? The duration? What did you assume about their knowledge base when you started training them? Do you conduct the training internally using consultants’ time? Or, do you bring in trainers? What about the continuing education – understanding Digital in 2010, for instance?

Now, some more assumptions. You assumed that the quants have the knowledge or appreciation of strong technical and statistical principles behind analytic problems. You also assumed that the engineers are good programmers and problem solvers. And lastly, you assumed that the MBAs will be able to manage projects, learn enough technical stuff, support pre-sales and sales activities, etc. Somewhere, you also assumed that each of these groups will acquire the skill-sets of the other groups.

Pan the camera towards India analytics talent now. There are three distinct talent sources right now. Quants (Masters in Statistics/ Economics/ Computational Mathematics), Engineers/ Problem Solvers (just smart B.Techs), and Managers (MBAs). Think about the questions  for these three groups separately.

More questions now.

  • Did you train them on a mix of SAS/R, Excel, Powerpoint, Access, SQL? Something about messaging, charts, dashboards?
  • Did you spend 2-4+ weeks training these smart professionals?
  • After you trained them, were you satisfied (completely) with where they are? Were they really ready to become independent analytics professionals?
  • Did you question your training methods? Assuming, they are not delivering over and over?
  • Did you ever question the foundational assumptions made? That a quant is not really appreciative of what goes into all this? Or, a manager who is not quant enough to start with, can never really manage?

Here is what I am thinking –

  1. Can the same level of satisfaction with training be achieved in just 3-5 days? Can the remaining training duration be used for cutting edge?
  2. What if you have two different sets of training? One that equips you for delivering projects, and another for turning you into a data scientist? Is there a preference for one over the other?

Chances are that you are too busy or have too many things going on. Based on several senior folks in the industry that I have met recerntly, and over the years, this is an industry wide problem. An analyst with three years into the industry may not know anything beyond basic programming methods – not analytic thinking, not methodology depth, and not the ability to have an engaging conversation. By itself, it may seem OK, since you’re focusing on the core programming skills in the earlier years. However, what if you’ve damaged the raw material that you started with? That out of 100 people that you hired, in the hope of creating those data scientists, only about 1 or 2 really set on the path of becoming one?  What I fear for is the quintessential mediocrity that we bring to a particular field of work. IT is a great example of how we’ve treated talent. Politics is another one.

Like in almost everything else, the dependence on the aptitude of the new hire, and their given interest in the field of work is what everyone is depending on. It’s the model that has been used by strategy consulting. And by being so myopic towards project delivery, the industry is missing the knowledge component of this knowledge economy. The recent curriculum innovation in certain universities, such as NYU Stern, Northwestern, UNC, etc. gives me hope. India needs to move on that track too.

P.S.  a question – have you every wondered why the core team’s talent/ skillset is a lot higher/superior than the people you hire when you’re scaling? Is it OK to be like that?

Note: The reason I did not talk about the in-house analytics teams – they rarely have a culture of training their analytics professionals. They usually just deploy them. While I am a fan of BYOL (bring your own learning), I don’t think its applicable everywhere.

Analytics in Healthcare: 5 Initiatives

(Image source:

Quite a few of the projects I have done in the last three years have been around the practical use of analytics in healthcare. One could thank the Obama administration and the healthcare reforms for it. But these projects were interesting. They ranged from market expansion strategies for pharma clients to operational effectiveness for healthcare providers to process and risk assessment for payors. And while going through the drills of business development and engagement delivery, there are three things I noticed –

  • Data assets– Healthcare, more often than not, has the best and the worst data. Best because of the nature of trail left behind for everything. Drug trials have data. Your medical history is captured in some shape or form. Medical transcription started some years back, adding more backbone to data. Claims data is analyzed significantly. Your test reports are getting digitized. At the same time, under the impression that most of it is “art”, its not organized data in some very significant parts. The movement from ICD9 to ICD10 – from analysis standpoint, is in progress. That being said, someone who can create a better, robust and standardized taxonomy stands to gain a lot here. Most of what I saw was trash. And I did see a bit.
  • Application – The application of analytics has widened from the traditional clinical trials and drug discovery cycles to everyday business of healthcare. It is one of the better and more expansive adoptor right now. People may remember the Heritage Health competition with good bounty for analytics problem solvers. Which, by the way, followed the ubercool Netflix challenge.
  • Simultaneous change across all entities. Payors, Providers and Pharma guys – All of them are adopting analytics in a wider way almost at the same time. Within banking, for instance, risk was one of the first, database marketing the second. I don’t think operations use analytics as much still. Within retail, companies are still struggling with POS analytics. A lot of the brand guys across industries still think of the art as the driver of sales, while discounting behavioral economics and analytics.

So, what are my top 5 picks for analytics investments in healthcare –

  • Market Restructuring – Does this mean that there should upstream and downstream investments made by the players, yes. Like, Intermountain becoming an integrated payer and provider. Or, maybe, a Pfizer payer-provider-pharma integrated business. or Kaiser? At the least, the payor provider integration is imminent. And when that happens, how will you maximize your benefits, customer benefits, and social benefits, is where analytics should focus. This is where most strategy firms will find opportunities. If someone can productize the approach, awesome!
  • Sales Force Redesign – As an effect of PPACA and Healthcare Act, there has been some consolidation in the industry, and the subsequent standardization of protocols and medical care. Hedging your risks seems to be one of the priorities that hospitals, individual practitioners have had to focus on. There has been an increase in network and institutional affiliation over the last couple of years. Hence, there is going to be a changing buyer design as well. Instead of the traditional sales rep model, newer sales strategies that will focus on horses for courses – such as account management model for network penetration, risk sharing model for adoption and share of wallet, and price war for competitive categories. Pharma companies will need to rethink their sales models, that’s a given. Analytics can do that. ZS traditionally has done that well. I believe Deloitte and PwC will be other companies to watch out for here.
  • Plan design and administration – with the change in customer portfolio , and the commitment to medicare/Medicaid segments,  payors will need to revisit their plans, their pricing, and their claim analytics business. Organizations with strong risk and pricing analytics, underwriters, and actuaries, should customize their play to make some money on this one.
  • From human to inhuman – A fun moment in an engagement is to convince a young analyst why, in certain cases, creating a disincentive for traveling to the hospital (which is a higher cost destination) and opting for low cost channels (telephonic care, medical vans on route, etc.), is not an inhuman way of looking at healthcare. But I guess, finding these more efficient, non-human interventions can help bring down healthcare costs. This will require working very closely with nurses, administrators, doctors. Non-human should become inhuman.
  • From patient to household – Almost all healthcare analysis is conducted at a patient level. For payers, a part of the analysis is subscriber and/or plan level. However, for modular plan designs, better cross-sell, better service administration, the three groups need to evolve to the household level too.


And while on the subject, where is a good healthcare bureau data? There are some that are evolving rapidly and provide a good starting base (SKA, HIMSS, AHD, etc.), but lots of work is needed.



Should Data Analytics Be Outsourced?

I was following this discussion on linkedin, and as expected most of the responses fall in the extremely simple to understand category of – “it depends”.  Because, fact of the matter is, like everything else, it depends.

So, I am trying to move from my usual “it depends” to the hopefully helpful “it depends on”.

  1. Is Analytics at the core of your business? (Like it is at Amazon or Capital One. Think about how you make decisions in your strategic and tactical reviews. Think about how most important discussions take place in your organization. )
  2. Do you have a leadership role equivalent of Chief Data Scientist in your organization? (This could be Chief Analytics Office, or Head of Risk Analytics, or Head of Consumer Insights or something like that. And hopefully, this is not just a figure head designation. )
  3. Do you have a well-defined analytics function? (How big and organized is the analytics team of your organization? Do you have one large team? Or many small teams?)
  4. Has your analytics function grown rapidly in the last 2-3 years? (Think about the number of people or the variety of projects handled by your analytics team)
  5. Do you have a set of senior people who can scope and define analytics projects? (Managers/Senior Managers who can add business layer to analytics problems (art) while having enough understanding and appreciation of the tools and techniques (science))
  6. Would you say that the organization has mature data assets? (Do you think almost all your potential ideas and hypotheses in the recent times have been addressed without there being significant data gaps or assumptions to be taken care of? Are these optimized databases? Are consistent and quality dashboards available across the organization?)
  7. Do you have any existing outsourcing relationships already? (Say,  data warehouse? Or, HRMS? Or some MIS reporting?)


The more the number of yeses that you score between questions 2 through 7, the higher your chances of being able to extract good value out of a outsourced data analytics. For 1, if your answer is yes, you should NOT outsource your data analytics.  And for the discussion here, outsource references to long term outsourcing contracts, and not  one-off analytics projects given to external vendors/ consultants.


1. Is Analytics at the core of your business?

Like most things core to your business, if analytics is core to your business, then you should not outsource (you can still offshore it to your own captive center, though that would be a separate long debate). Risks include exposing your core to 33 other through offshore employee churn, significant management bandwidth wastage in protecting core IP, and not getting enough business input from the outsourced relationship. (Most risk analytics firms in the market have their founders coming from strong risk backgrounds at banks and investment firms. Did they violate some NDA or IP act when they set up these analytics firms? There is a very thin line to be de drawn here between what exists in one’s head as knowledge and what exists on a piece of paper as protected IP). Outsourcing as a process serves you well when you focus on repeatable non-core activities where efficiency/ costs/ speed/ organizational focus, etc.


2. Do you have a leadership role equivalent of Chief Data Scientist

This is clear outcome of your focus on analytics as a differentiation capability and your ability to dedicate senior bandwidth to an outsourced analytics relationship. Most technology outsourcing contracts have had senior CIOs/CTOs paying close. So, if you want to make these work, better have a senior guy look at it. Not the brand management fella (no offence meant).


3. Do you have a well-defined analytics function?

Ah. So, you don’t have an analytics function at all. Which more often than not implies that you haven’t thought through what exactly is the analytics you need or what for do you need this analytics. In this case, I recommend that you give one off projects to someone through your IT team, which most likely owns the data at this stage.


4. Has your analytics function grown rapidly

Usually, this would imply that the buyer base has grown/diversified. It gives you an opportunity to organize, consolidate and focus on the more strategic projects being taken care of internally or through one off projects, and a lot of ongoing reports/ standardized work being executed through an outsourced relationship. This is the time where you start measuring efficiencies in your current activities.


5. Do you have a set of senior people who can scope

In most cases, the project scope can vary, and data definitions need to evolve on a constant basis to meet the needs of the business. Unless there is someone senior enough who is paying attention to these, the ball does get eventually dropped and you hear comments like – “but the last time…” or “but I thought…”. Such involvement also ensures that the outputs are not devoid of business relevance.  A committed senior person ensures relevant utilization, better output and better value for the organization.


6. Would you say that the organization has mature data assets?

Unless your data is reasonably mature, there is no point pursuing analytics, leave alone the idea of outsourcing analytics. Any step to your organization using analytics starts with using data, which starts with cleaning data. This was a fun question to catch the slackers!


7. Do you have any existing outsourcing relationships?

Unless you know how to handle outsourcing relationships in general, you should not think of analytics outsourcing. Howsoever standardized the analytics might become, its more knowledge intensive and artsy than system intensive. Hence, you should rather focus on system or process intensive work getting outsourced first, understand engagement and communication protocols, SLAs, dealing with outsourced cultural conflicts, etc. before you outsource knowledge intensive activities.  And an embedded assumption here is that standardize reporting contracts are “process intensive” and not knowledge intensive.


The answer to the above questions usually gets you closer to “CAN analytics be “SUCCESSFULLY OUTSOURCED”. Whether it SHOULD, is still going to be a wider debate, which includes cost benefit analysis, availability of talent for captive organization, scale efficiencies, IP protection, doing it offshore with your own unit vs. doing it outside, etc.


Summary Of the LinkedIn Comments –

The arguments for outsourcing included – skill set gaps in the organization, cost arbitrage, focusing resources on core activities.

Arguments against include – always worthwhile to build own analytics capabilities, QA issues, turnover in vendor and subsequent knowledge retention and transition, engagement protocols and batch mode vs. interactive real time mode of working, and lack of business knowledge/context knowledge.


Some of the comments (in summary, not verbatim) are included here –

  • Gary S – No one knows business better than company employees.  If they have the analytical skills, then analytics should not be outsourced.
  • Simon G – It’s a worthwhile investment to build in house capabilities, even if it takes time. .. Sometimes a particularly specialist and repeatable requirement comes along, which is suitable for outsourcing.
  • Peter W – Sometimes an outsider can spot something missed internally.
  • Michael Mout – Depends on the size of the company. Smaller companies cannot afford a full time analytical team.
  • G. Jack Theurer – Difference between outsourcing and offshoring (maybe to an internal unit). Outsourcing lcaolly has benefit of specialized knowledge and talent, and occasionally budget. Offshoring – difficult to run in batch mode. DA is an art and a science. Science is the same across most organizations. Art isn’t.
  • Nagesh PSkillset gap can be filled by the external team
  • Kapil M – is cost arbitrage still there? Cost vs. Value?  80:20 model (extended team vs core team) – meets the business knowledge needs as well
  • Duane S– You can train your staff rather than trying to outsource.
  • Jon Jian-An L – business analytical problems are always undefined and do not have closed form solutions. Issue of high analyst turnover – retention of informal knowledge
  • Mario Segal – Turnover. Challenge of smooth transition,  strong on analytical techniques but weak on products or markets.
  • Krishna Agarwal  – Is the core product/service based on analytics? If so, keep it inhouse. Most organizations do not have senior mgmt bandwidth/ devote resources for creation of the ecosystem needed to realize the full potential, ROIof in house team is genrerall y poor.
  • Imran Ahmad Rana –Studied Analytics in Quality. Orgs that have it inhouse are more effective than those who are outsourcing.


Data Democratization To Analytics Democratization

One of the more recent buzzword in the world of analytics is data democratization, used in the context of public data, govt. data or just organizational data. In layman terms, it means that everyone in the organization should have access to consistent (and preferably correct and comprehensive) information presented to them in easily consumable formats.

I am a huge fan of the possibilities that data democratization would lead to. However, the real tipping point of the world moving to an analytics led marketplace is analytics democratization, and not just data democratization.

Questions before I go further –

  • Do you think it takes a lot of time for your team to get a particular report/analysis/data slice that you need for, say, an urgent meeting/review?
  • Do you think that there are far too many people  involved in this process of getting the data to you? DW/ BI/ Analytics/ Product/ BU/…?
  • If you are an analyst, do you think you get too many ad-hoc requests in a day? Are some of these what you’d call a re-hash of a previous request? And are you scared of telling the requester that you’ve already given it to them albeit in a different format/ it can be derived from the earlier data dump?
  • If you are a business manager -Do you always get that feeling that you do not have enough information? Do you feel that you need something more? Do you also get frustrated with how long it takes to get the analysis that should have been  a cakewalk? Do you also feel that the analyst has just dumped a whole lot of data without stopping to think about the insights?

If your answer to most of the questions is yes – then your organization is most likely miles away from reaping the benefits of data democratization.

Barrier to adoption

Merely by giving access to data, access and consumption of data cannot be guaranteed. For instance, open source and freely available Linux got the appreciation of a lot of people, but remained the geek’s preferred OS. Masses stuck to their user friendly iOS and Windows systems. Even when Linux became more user friendly, the half-baked OS democratization had already created more barriers for the commoners. And the whole idea of migrating from their existing “warranty led OS” to an open source OS is something that masses (and democracy) were not ready for.

The barrier is not data, but understanding

Most stakeholders who do not advocate data driven thinking, and hence analytics, are the ones who have a limited view of what analytics is. Providing data and reports to them never solves the issue of analytics adoption. A closer look at the real leaders in this industry, e.g. CapOne, P&G, Amazon, Walmart, etc. would tell you that “analytics thinking” is ingrained in the way these organizations function and think. In these organizations, analytics has been democratized.

Data by itself is only half the value

Data is almost like a blunt weapon that can be used to club the enemy, or carved into a knife for the kitchen, or a sword for the battlefield. It can also be converted into a home equipment like needle or farm equipment like spade. Without questioning the merit of democratizing data, the real value of data can only be unleashed by making customizable adoption formats. Most end users would still look for flexible thin applications that help them understand data as per their need, or provide enough reports/dashboards that makes the need to be creative redundant. The latter will lead to the generation and maintenance of hundreds of reports that never get used, or whose existence is never ROI justified.

A report is as the reporter does

bad_dash1The number of dashboards and reports that I have seen in my career is an insignificant number. The significant aspect though is the “narrow’ application of most of these reports. Or, the demand-supply gap. Demand of the user, and supply from the report designer. Also, a lot of these reports are so rigid that a change in them requires raising a service request of sorts that goes through three layers of approvals and few weeks of analyst time. The number of consulting engagements that get delivered on the basis of inconsistent reports that different parts of the organizations use is a different story altogether. I believe that once analytics thinking is democratized, the products/reports/dashboards will start evolving to the needs of the masses, have flexibility to adapt on tap. And that’d be an interesting place!


So, even if you’ve democratized data sufficiently, you need people to understand the tools and skills that enable their conversation with data. I have seen too many “smart” people get scared when you throw too much data at them at a high speed. And whoever tells you that you cannot train the whole world what multicollinearity and mixed models are is likely trying to fool you. They need to tell you that you don’t need to.

Let’s aim for a culture where a quant(?) can communicate in a language that business can understand and the business can communicate in a language the quants can understand. And to that extent, organizations need to invest not just in building Big and Bigger Data capabilities, but also an analytics thinking mindset second.

And that’s an area where I see a significant gap. There are few people capable of educating analytics teams, and fewer capable of educating executives.

Image Credits:
1. Infolytics, here

2. Here

3. Here

The Great Analyst Series– The 9-pointers

What makes a great analyst? It’s a question that plagues every manager and every analyst who’s working in third party analytics organizations/teams. Less so in the case of captive teams, because the model works more on the lines of individual contributors rather than large managed teams, hence dramatically modifying the equations. Though some of these apply universally.

The adjectives I have heard fall in these categories –

  1. Keep it error free. The first and the biggest. This one’s a killer. An analyst falls from being great to not so great in that one short span where a client or the manager’s detects an error. Especially, if its a client. Akin it to your confidence in betting your salary on your analysis. An interesting practice would be – Keep aside 100 bucks for every time an error is found on a file or an email that leaves your inbox/machine. Even if you did not create the file. Two errors is 200 bucks. Even if they are on the same email/file attachment. At the end of the week/month, see how much you’ve put in the kitty. Donate that money to a charity. Or, if you’re not keen on social stuff, then order sandwiches and samosas for the office. If you’re a good tracker, you might see yourself make more mistakes at 5:30 in the evening to start with (because you sent out an analysis too close to a client call or to the end of the day).
  2. Plan and communicate. Managers struggle with the effort estimation done by their analysts. They’re never too sure if the buffer they’ve budgeted for is appropriate. So, it’s a trial and error process. You can make it easier for them and for yourself by planning, adding buffer, communicating the plan to the manager, and if you see a risk, communicating upfront about not just the delay, but also the reason for it and the planned remediation. It’s good to know that you’ve got it in control, even if there is a problem. My love for analysts with a solid personal project management method might be a bias, but I have always been wary of geniuses who do everything in their head. Maybe because I am not smart enough to reside in their head. But if I had to take a bet, I’d bet on most managers being as dumb as me.
  3. Engage. An analyst that takes a task as given, and comes back with the output against that task is not a bad analyst. After all, the baseline expectations are met. But, when you’re discussing the top tier analysts, you’re focusing on the “exceeds expectations” kinds. And they engage more. With the engagement, with the team, with the problem and with the analysis. They come back with a checklist of having delivered on the task, and some more, and a few questions or ideas to develop it further. Sometimes, they come back with questions and ideas about why the earlier discussions/paths were not the best.
  4. Question, but with a pause. Sometimes, you come across a glaring “situation”. And you want to question it. And you question it rightly. No problems. The point where it becomes a potential problem is when you question it too long. Sometimes, and rather most of the times, it’s a smarter move to demonstrate the problem you have with something tangible. So, if the analysis is flawed and the manager ain’t gettin’ it, then go ahead and do it the manager’s way first, show the reconciliation/ triangulation errors, propose an alternate, do it the alternate way, and show why your method is better. It’s a much easier conversation than telling the manager that they’ve got a few bolts missing. And you make your point, earn your brownie points (assuming you don’t offend the manager with your in-the-moment superiority complex). What definitely never works out (I have seen it true for at least some of the analysts I’ve worked with) is – I told you so!
  5. Be visually delightful. Don’t dance, please! An analyst that transcends the analysis and can communicate it effectively always gets more brownie points. So, if you have two analysts where one is a better thinker, and the other is a better presenter (assuming baseline attributes are not at risk), the chances of the better presented succeeding more often in their professional career is higher. And when I say communication – it’s all forms – the written, the oral and the visual. And the visual has two components again – the visual aesthetics of the output, and your own visual aesthetics. A shabby looking person with some irritable personal habits can find it difficult to be role model, despite their technical brilliance. I feel flimsy saying this, but sadly, this is true.
  6. Adapt. Be flexible. Managerial styles are different. There are micro-managers and there are delegators. And there are indifferents. I know it’s a lot to expect the youngest one in the team to be the most mature, but never hurts to know, right? So, monitor the manager carefully in the first couple of days of an engagement, and alter your style to have the most fruitful relationship. The best of the managers can teach you a lot. The worst of the managers can teach you how they became a manager in the first place (which might be a career goal of sorts)!
  7. Provide leverage! A good analyst provides good leverage to his/her manager. In common language, I translate it to the ability to preempt and take up several mundane tasks that your manager is spending time on, without compromising on the quality of the work you’re expected to be doing. Doing the manager’s job so that the manager can focus on their next set of priorities. It’s an attitude that I cannot over-emphasize the importance of.
  8. Don’t just earn your salary. Not everyday. When you look around, you will notice them. They are prone to saying – this much money means this much work, or “I can work harder if there is more money to be made”, or “whats the point, I am not going to make more because of that”, or “today was a productive day and I managed to get my work done”, or some version thereof. They think that “earning your salary” equivalents to being meaningfully busy for 8-9 hours a day. When you’re just trying to be meaningfully busy, you forget trying to be the best you can be. Make the system feel bad about paying you only your salary. Make the system think about how best it can value you. By better projects, better career, better opportunities, and in the process, hopefully, better salary.
  9. Read. A. Lot. I have experientially seen it to be true. I am not sure other managers are aligned with me on this one. In my view, a well-read analyst always finds more dimensions to analyze in a project than a walk-in-walk-out analyst. The well read could nerdy (e.g. reading about the cutting edge work like Watson or In-memory databases) or business savvy (e.g. mobile payments and NFCs) or socially aware (literary works that provide insight into how people and societies think and act). But it’s a strong hypotheses in my experiential learning that a good reader usually becomes a good thinker.


Why a 9 pointer? Because a perfect ten should be all this customized to your own inimitable style, as and when you find it.

Daily Dose of Analytics: Indian Politics

543 loksabha seats. 28 states (and 7 union territories), 2 major parties and several small to mid-sized ones. Reflect on 2009 elections and what you may remember is tonnes of analysis and political commentary that the front pages and editorial pages of a newspaper or the jarring voice of Arnab Goswami would’ve shove down our throat and mind. And in there, not many real recommendations. Lots of critique and counter critique. Almost passively taking sides. Yeah, part of the problem is that media can’t be seen taking sides too blatantly.

The limiting thing about the political commentary that we read everyday in newspapers and tabloids is that its too political and too top-down. Worse still, the best political analysts are sitting by the sidelines working for media houses and focusing on “what is happening?”, and in some cases trying to reinforce the biases that some of the media house leaders may have in favor or against some parties. The “hypotheses” get passed around as insights. If the result lines up, we are prone to saying “I said so”. And if it does not, we are smart enough to reengineer the explanation.

Where am I headed with all this? Recently, people have been really excited by the use of analytics in predicting the results of US Presidential Elections 2012. However, the next game changer could be using analytics to drive political results. Transforming the business of politics using analytics.

Why not approach the whole scene ground up? What does BJP need to do to win a seat in Kerala? Can it? The answer is always a yes, right? Given the resources, costs and commitments required, should it? Maybe not.

For instance, let’s take BJP’s predicament. A party which has been the second fiddle for too long now, and had a go at power once. Has a strong national recall, but low/moderate national appeal. A strong brand which stands for something, which probably the party isn’t playing to. Or is afraid of playing to. Strong foothold in a few states, swinging foothold in some, and no foothold in many.

The question to ask – Is there a way to become the party of choice for at least 60% of the Lok Sabha seats? (I have selected seats and not population. Because the eventual result talks about seats, and not the percentage of people who voted for you, or the voter turnout or some such metric).

Political analysts look at the problem in totality. Or, in complete isolation. None of them has ever tried to or would most likely be able to put together a draft success/growth strategy for BJP. If it were a consulting gig, there are far too many frameworks (opportunity assessment, market entry, investment planning , blah blah) that consultants would reuse/create. But then, most consulting gigs are also top down. That’s where analytics could score by being bottom up in such scenarios. Analytics is special in its ability to not lose much by switching from being top down to being bottom up. Though you know that the effort is much higher for bottom up recon in this case.

In your first series of interviews, you’re bound to encounter significant amount of experiential and tribal knowledge– “This is how it happens. I know it” or “That’s how that community has always been!”, “It’s a strong Dalit foothold”, “BJP needs to get away from its non-secular image”, “… find a strong young leader”, etc.
But, once you’re done with these discussions (and they are important for understanding the issues and perceptions and hypotheses), you will need to understand the voters, what may make them change their existing decision in favor of BJP or what they may like their next MP/MLA to do, etc. The answers, not surprisingly, will still be simple and basic. Some practical, some impractical. And a lot of data already exists to support most hypotheses including this one simple hypotheses – most elections fought on the back of strong infrastructural or social development go favorably for the incumbent. But there are many more triggers that influence consumer choice. And like in business, in politics too, customer can be the king.

This is the point where you’d ask me to shut up because I don’t know jackshit about all this. And of course, my political awareness is not top of the charts. Like that first quiz by Rambo at IIMB, you’d start looking for my name from the bottom of the list.

Maybe, you are right. Or, maybe, I don’t care. Because my final question remains – doing what you’re doing right now, what hope do you really have of changing the game in the coming elections? And focusing on what you’re focusing on right now, do you think you will get 272? And lastly, if not, then would you rather lose the next one too instead of focusing on something that can get you there, rather than hope for more idiocy from Congress leaders? For Congress, on the other hand, the question is, how long are you going to keep hoping that you are the best amongst a confederacy of dunces?

Daily Dose of Analytics : Coffee Shops And The Personal Touch

I have always found coffee shops to be a shared yet extremely personal space. Swarming with people, but you always get your quiet space, or the space to discuss the biggest and the most profound of topics.

Much before I started working, as a student with an insignificant pocket money, a friend and I would save just about enough money in a week or a month to the go have a cup of cappuccino at the newly opened Barista at Vasant Place market in Delhi. It was an aspirational act for us. Back then, a cup of coffee costing 30 bucks was a luxury that middle class students like me could not afford every day. I survived a week on bus passes and about 100 bucks. With chole bhature in college canteen costing 5 bucks, it wasn’t too difficult, in case you are wondering. However, But for the coffee shop manager, I was somewhat of a regular.

During my MBA days, I welcomed the opening of the Café Coffee Day inside the campus. While the poor guys had stiff competition from the legacy Nescafe machine serving super sweet desi coffee for 5 bucks or so, there were loyalists who would go to the café regularly. I would do that sporadically (continuing financial constraints). Yet, while at the café, it was a personal experience. Reason – the fellow at the counter knew me well enough by my third visit, and my order as well.

The phenomenon continued with me and Tushar playing “Jaadu Hai Nasha Hai” on the jukebox of the CCD at Ispahani Center in Chennai, or the string of coffees (mostly with biwi, TG, Shumeet, Shilpa, NehaG, Sulabh, Aziz and/or several others at Inductis) at the CCD at Solitaire Plaza on MG road. At these places, the old age touch of the coffee shop team knowing you, smiling, understanding what you’re going to order, and gradually establishing a personal connect with you was a part of the reason why I would go to the same coffee shop over and over again, even as the very cup of coffee became a standard output from one outlet to another. And more outlets, maybe closer to where I was, popped up at regular intervals. Ajay (at CCD Solitaire) even invited us for his marriage, even though he was really confused about who’s dating who for a very long time, given the NC2 combinations of coffee-ing!

Over the last few months, there are two coffee shops that I have frequented with great regularity. The Di Bella at BKC, and Gloria Jeans at Powai. However, these two are regulars because they are convenient. Whenever I am in BKC (was the norm when I was still working and continues even now with the people I meet there), it’s the only half decent option. CCD’s coffee quality has become despairingly bad in the last year and a half. GJC in Powai is also close to home, half decent coffee, has power connector points for me to work uninterrupted for some time, and enough quick bite options close by. And is open till about 1AM.

Now, in both these cases, I don’t think the folks would recognize me from one visit to another. I would probably need to strip and dance before they’d start recognizing me on my subsequent visits. Like the coffee, the customer is becoming a standardized product, is it?

The answer is no, in all likelihood. And that’s where Retail/POS Analytics should help do the job that the friendly neighborhood stores were doing so effortlessly. All the nearby stores would know me and my parents, back in the days, because of several factors – smaller/closer communities, repeat visits, continuity of the people who managed the same store over years, and lastly, a general culture of taking interest (which the modern world can called nosy as well). Retail Analytics can make this very easy for most.

An example that comes to mind – the small touches that Amex customer care often adds. For instance, last year, I called them for a query in February and they knew that my birthday and my anniversary are around the corner. How? B’day is easy. But the year before, I had some purchases around those dates, the address on my file and my wife’s file are the same, and lastly, my wife had purchases around the same date. Someone inferred it to be an important date. Not exactly what, but most likely, the flag of an important went up. The customer care executive promptly asked me if I had any vacation plans and if I needed any help. To the extent of suggesting that I could redeem some points against some of the travel options because I had a very healthy point balance.

Earlier, a lot of these required manual effort. Like that branch manager at your bank, or the store owner at the nearby store, or Ajay continuing at the same CCD for two years on the go. Now, data quality (better organized and cleaned data being made available in large volumes), and simple analysis can make it very easy for the POS person. If nothing else, a simple name to call out for and the last four digits of the credit card being swiped could start establishing the relationships, right? Next swipe, bring out that 10% discount coupon for registering – bingo – name and address collected. Follow it up with Coffee Clubs/ Loyalty – wonderful retention. Especially, for GJC, in an area where there are approx 5 or 6 coffee shops close to each other. Will occasional errors happen? Yes. But as long as the touchpoint is consciously used as a positive reinforcement, the impact cannot be negative. Analytics should take care of the machine so that the touchpoint can continue to be more human.

On that note, why hasn’t any coffee chain thought about organizing coffee evenings for groups of friends? Movies, Coffee and Sandwiches. ☺


[The Daily Dose series could evolve into a series of stray thoughts on analytics in daily life)

Betting Big on Big Data?

Are you in a hurry to catch up with all the Big Data news and how it’s going to affect your organization? Are you worried that you’ve missed the social media bus? Or, is someone telling you to move to the cloud? Hadoop? Terabytes and Petabytes of information that you need to process? Real time systems and dashboards? Enterprise mobility solutions? iPads? Micro-segmentation? Platform enabled solutions? 1-to-1 solutions?

If you’re close to a coffee shop, I’d recommend you walk in, get yourself a cup of coffee and sit by the window and relax. Some of us want you to over-react and buy technology, services, analytics, cloud, or something else because that’s what we always do. And that’s what you always do with your customers. Once you’ve identified a buzz word, you want everyone to catch on to it. We want you to geel that you’re missing out on something groundbreaking.

Even though the fundamentals of what is being consumed hasn’t shifted significantly. What do I mean by that?

  • Are you selling a different product?
  • Are you selling it to a new customer?
  • Have the underlying economics changed? Of creating/delivering the product experience?
  • Has the channel changed? Are all your customers shifting online?


And several such basic questions. What exactly are you expecting this investment to deliver for you? And while you’re still noodling over big data and analytics, think about this –

  • Big Data is a very contextual thing. For my mother, big data would mean that all three of us siblings start talking at once. For the HR department of a small organization, it would be sifting through the paperwork required to get everyone a work-permit in the various countries where our team members might be required to work. For the marketing team of another organization it could be the buzz that each of their campaigns is generating across channels, and whether its effectively being tracked
  • Data has always been big, in relative context terms. What has been a challenge is your ability to process this data. Microsoft excel moved from 65k rows to a million plus and continues integrating it with other Microsoft database tools to add more functionality. Likewise, programming interfaces started developing intuitive UIs for tech-incapacitated analysts carry on with their analysis. The tools will evolve to support the needs of the hour. Your need is to evaluate your game and what lies ahead, and not get caught up in what the critics are saying all the time (not to say that you should never listen to them). Don’t always look in the rear view mirror. And don’t always listen to the back-seat driver. Sometime’s you have to deal with the cockpits.
  • The basic rules of engagement have not changed. Analytics should focus on the business. Business should always start with the basics. One of the best managers I worked with had this habit of never recommending anything analytically complex to start with, but rather focusing on a few questions. Consulting firms take a lot of pride in their hypotheses driven approach to problem solving. The same applies here. The analysis/analytics/modeling etc. is a tool to answer the needs of the business. It is not the answer itself. I think it was Einstein who said that if I have finite time to solve a really difficult question, then I’d spend 95% of the time thinking about the right questions to ask, because asking the right question invariably gets you to the right answer.
  • Don’t let it go the IT way. Remember the large scale technology investments in your data warehouses, organization systems, POS implementations, etc. Remember how you realized every three months that something was not being captured by that system? Or, not accurately enough. Almost every client that we have worked with, and this includes the ace financial services firms, insurance firms, retail giants, etc. using analytics heavily, the quality of data has been suspect. For three reasons – the difference between legacy systems and incremental value added infrastructure for specific needs. B) No clear owner. C) Constant back and forth between business and technology on what is required vs what is possible. Analytics is at the same cross road, and combined with the mistakes made on the data quality front, you will soon find yourself repeating your mistakes in a more real time manner.
  • Differentiate between analytical capabilities, technological capabilities and business capabilities. Technology will help you process big data, but you need analysis capabilities to question the changing dimensions of your business. And if any analysis that is not tied to the business it’s impacting, it might as well stay in the analyst’s laptop.

It’s not to say that you should not invest. It’s time you start running a fact based business, if nothing else. Or, as consultants are blamed for – stop pulling insights out of your backside. It’s time you developed the capabilities to do backward and forward looking analysis backed by strong business cases and communicated through effective visualization and quality dashboards. At the same time, don’t get swayed with this large wave of discussions. Its time you pulled out the rocking chair in the attic, sat back, and thought about how to do more with less, and how to get the basic fence in place. Because, it’s also the time when confusing information will hit you at ever increasing speeds.

%d bloggers like this: