I had an interesting conversation on Twitter earlier with a friend of mine, Jon (who I should say is one of the brainiest guys I have ever met... and quick to learn!). His questions were centered on ROI and whether it should be the only thing that mattered when one designs a campaign.
Now, I really wouldn't venture into getting into a debate with him - since he's a PhD in Marketing. But I thought his questions were interesting.
Photo from heytherespaceman on Flickr
If we didn't measure ROI, then what's the point of spending?
We've always believed that marketing - like any other corporate funds - are investments. They are being 'spent' to create value for the shareholder. If a certain line on the corporate expenditure accounts didn't serve any purpose, then it shouldn't be there, right? We might as well save it for something else.
Returns of marketing investments should therefore be measured.
But should ROI, then dictate the plan and its executions?
That's a deeper question, really - which brings us back to my previous post on whether communications and media planning were an art or a science. My answer is, yes... but.
And it's a big but.
We have to clarify exactly what we mean by returns. Are we talking about returns on investments in the form of savings and additional value? Are we talking about returns in the form of being talked about in digital communities and vox-pops and forums? Are we talking about returns in awareness and genuine affinity and likeability towards the brand? Are we talking about products being purchased off the shelves? Are we talking about users of the competitive-brands moving to ours? Or our competitors falling in love more deeply with our brand?
Because ROI is on the verge of becoming another 'buzz-word' - like "engagement".
Everybody wants ROI. But everybody has got a different view of what ROI is and should be. And therefore should be clarified.
Brand managers have their own ROIs - salary increments and perhaps, that promotion to AGM position. CMOs have their own KPIs - perhaps, that golden parachute and more stock options or discounted stocks, and eventual accession to become the CEO. CEOs have their own ROIs - stock prices, analyst ratings, and well, a positive disposition from the board.
Because ROI is 'bandied' around - and is almost always mentioned in every and any meeting between client ang agencies - it is bound to lose its meaning.
(An aside: When I was working on a fast-moving consumer good which shall go unnamed, their KPI measurement for their planning agency was "savings generated" - the same KPI for their buying agency because "that is ROI for all media investments"!)
So ROI - or ROMI or ROAI or iROI or iROMI or iROAI or ROMA or iROMA - needs be clear.
And every one needs to be aligned on what it is.
Lest it becomes another buzzword - "important but meaningless".
But ROI is only a number; how many ideas and campaigns have been killed because its ROI cannot be ensured because pretest scores are bad?
It is indeed only a number - but it is a number that should inspire, encourage, and motivate - not restrain and constrain.
We have this habit of looking at numbers as if they were a manifestation of God (with all due respect to physicists and mathematicians - and well, gnostics). But I see numbers as guideposts.
People don't like the ad during the pretest? Rather than dwell on the score, ask why. What is it about it that makes it so... wrong? Why is it unlikeable to this particular person? Why is it unlikeable to this particular group?
Most pretests are run on groups and/or on people. And usually in the form on unfinished TVCs. Surely, they cannot appreciate the beauty of the idea in a rough form. And the fact that you have convened them into a room (however natural, comfortable, open that room is...) have put them in a different mode altogether.
The more important thing about pretest scores is "Why is this not working on this paticular person or this particular group?" Any self-respecting pre-test moderator would want to explore that!
Turning the question around: How many ideas and campaigns have been killed because they have not been tested and their ROI potentials unmeasured?
Which - I believe - is also a valid question: Just how many ideas, campaigns, thoughts, executions have been killed because they have not been invested into properly because they were not measured, they were not subjected to some form of indicative pretest?
It is again back to "numbers versus gut" - "art versus science".
So what now?
In statistics and econometrics, we have a concept called the "error". Whilst statisticians and econometricians think of the "error term" differently, the term is there in models to remind us - constantly - that we cannot ensnare every single thing with numbers.
In econometrics, "error terms" are sometimes called "innovations". (Let's not get into the technical details for now...)
And that could exactly be that: we can derive all the numbers and formulas that we want, create stringent benchmarks and tests and questionnaires and surveys... but we will never really capture reality - the audiences' reality.
It doesn't mean the stats and the formulas are wrong: it's just that, well, it's incomplete. And it can never be complete.
Because that's the nature of numbers.
ROI? Yes. Testing? Yes. Measuring post-campaign effects? Yes.
But they are your tools - you're not theirs.
Very true - tools offer a means to an end, not an end in themselves.
Too often, we think that research results are the answer, instead of using them to inform our decisions.
Democracy usually delivers the lowest common denominator, and mediocrity is always forgettable: http://eskimon.wordpress.com/2009/04/03/death-by-democracy/
Posted by: eskimon | 19 May 2009 at 23:06
Great Discussion.
Used to be that getting lift on your creative tests, great response rates, or a positive return on investment (ROI) on a specific campaign was good enough for marketing to declare success.
This doesnt seem to be the case anymore with more and more metric tools surfacing.
I have spoken to many Marketing Directors of late and what they are telling me is "YES, ROI is important when it comes to Marketing" but the question i pose them is for every campaign, you might be utilising different media platforms from print to online to broadcast and some are simply just not measurable, how do you know which of the platform is successful and which isnt?
The main point is, Marketers are looking at chain effects of their marketing dollars. If they spent 20 dollars with ABC, how is the 20 dollars going to affect the remaining 80 dollars? Whats the rolling effect? Are the objectives of each media platform purely to generate sales or is it a leadup to larger things?
Every media platform has its strength and hopefully a unique purpose. Its all about integration and instead of purely looking at ROI, we should look at ROO (Return on Objectives) on each of these platforms and clearly map out how they can affect the Customer Journey.
Art or Science? Both! Integration! If its all about science, we should be in finance :)
Posted by: William Chin | 20 May 2009 at 23:12
Great write-up!
This has been a topic that have been 'bugging' me of late. Its amazing that time and time again, people talk about ROI and only ROI. It seems that it is the holy grail of all measures in campaigns.
This is something I said in twitter to 2 great guys "hmm..could ROI be just a "phantom number" that does not reflect the true net profit of the campaign?"
Could the above hold water? Really could it just a phantom number? What is really success? success in the short term or success in the long term? please define success (opps the theorist in me is surfacing :) )
eskimon is spot on about them being tools that offers a means to an end and William also made a valid point about RO (Objectives), this could very well be the measure one undertakes and it could work better. Maybe, maybe not, I don't know but at least it clearly gives a measureable response to the firm's stated (and identified) objective.
Perhaps my two cents worth is the following approach.
1. Get to know the targeted customers more intimately (know how they interact in the online and offline space) - what makes they tick (how and why)
2. What is your organization's objective? is it to increase brand recall, to sell more products? to get this customers to be your walking advertisement?
- Then perhaps use a RO (objective) to measure your objective.
3. Strategize, think of the end point first and then work towards it.
Last but not least, then
4. Engage tools to execute/enable the strategy and fulfill the said objective
- bringing back to the point eskimon said tools are just a means to an end.
William - as of whether it is being an art or science, well could this question shed some light?
Q. Do you marry for love or do you marry for money?
A. (From me anyways) - I reckon the question is flawed to begin with, it should read as "to what extent do you marry for love and to what extent do you marry for money".
Apply that to marketing and (also ROI):)
Best,
Jon
Posted by: Jon W Chin | 20 May 2009 at 23:41
Tools must are just that: tools. But we are often caught up with bells-and-whistles, which turn out to be only smokes-and-mirrors. I say there is no one tool that can be a panacea to all our questions in marketing nor in advertising.
Not even research can capture everything - no matter how intensive and extensive such research may be. We have not gone beyond the point of actually understanding what truly moves and motivates our target audiences.
Not even the current flavors of the moment, "ethnography" and "social network analysis", can provide the answer.
Anyway, thanks for the comment. ;)
Posted by: Philip Tiongson | 24 May 2009 at 21:42
Thanks, William, for the comment.
I guess one of the problems that we face could very well be that we are trying to (mis-)apply a concept from finance unto marketing, presumably with the belief that finance is an "exact science" since it deals with numbers AND dollars AND real losses/gains.
As the last year has demonstrated, finance is not entirely "that exact". And it is not entirely "that scientific". Even the best of the investors mix numbers with gut - and some would even go to the extent of just saying "I was plain lucky".
Our problem in marketing ROI is we want to measure it with "exactitude" - with 'perfection'. We just can't.
But that doesn't mean we shouldn't.
Your suggestion is indeed a good way of starting: Knowing what each medium or channel stands for and is for in the mix and fully justifying it is the first step.
Thanks!
Posted by: Philip Tiongson | 24 May 2009 at 21:48
"hmm..could ROI be just a "phantom number" that does not reflect the true net profit of the campaign?"
ROI could potentially be a phantom number if it is not defined strictly. Which is why I would go for a definition that goes beyond revenues, market shares, or awareness - but rather profits.
The problem with that however, is "how does a client share her profit-based ROI goal with her agency-team?" More basic perhaps is "how do you calculate a profit-based ROI goal?"
===The 4-stage approach that you have mentioned above is a good start - I think most companies (and strategists) would have such a process. (Even Kenichi Ohmae [The Mind of a Strategist] thinks the same way, though his has #3 rather more complicated.)
The problem lies however in the quantification of the objectives: even if we had ROO, if the goal of the objectives (I know it's kinda redundant) is not strictly-defined, then how?
Also, let's say your goal is an ROI of 20% (and that you have found a way to calculate that on an aggregate basis), how do you break that down across different channels?
======Each medium could deliver something - and each could have a role and an objective. But could a campaign still fail even if each and every medium played their roles and delivered against their objectives?
I think so.
In econometric/regression modeling, a model could have a very significant "F" and non-chance probability level suggesting that the 3-5 variables you have in the model explains the predicted variables very well in sum.
HOWEVER, it is possible that individually, each of these 3-5 variables could very well be statistically insignificant on their own!
So again. Another problem on ROI. :)
======On "marry for love or money"? I am not sure. Perhaps for the former - because "love makes the world go 'round" and "love will keep us alive" :)
Seriously: the golden mean between art and science - which may or may not lie in the center because art and science may not necessarily be at opposing ends.
Hmmm.
Too many questions on ROI, no?
Posted by: Philip Tiongson | 24 May 2009 at 22:04
Thanks for the response, Phil. Very interesting...
My thoughts...here you go.
1. re: "profit-based ROI goal" - how to calculate profit ROI - i dont know (but I'm keen to investigate), that aside...
On a slightly different note, but what I can foresee it might be rather tough - involving time-series and longitudinal approach - not so much from a technical point of view i stress but more from company's point of view - time is a factor? most often then not, people in the position of authority (at that point in time) would not be in that position long enough to "wait". oh well. I might be wrong.
2. re: "quantification of objectives" - yeah i would recommend companies do just that - quantify their objective and define it explicitly - they need to know exactly what their objectives are. Which brings us back to the eskimon's tools point, companies sometimes pay too much attention to the 'tools' and forget the fundamentals. Get your basis right and know exactly what you seek to achieve :)
3. Re: "let's say your goal is an ROI of 20% (and that you have found a way to calculate that on an aggregate basis), how do you break that down across different channels?"
That's a good one, in my head i was saying "can one calculate that?" haha. I will look at the weightage. Most often then not, alot of companies are rather short sighted (though they say they think long term, ok not all Ive seen some who walk the talk) :) Some campaigns can only see their TRUE effect after a period of time imo.
4. re: "Each medium could deliver something - and each could have a role and an objective. But could a campaign still fail even if each and every medium played their roles and delivered against their objectives?"
well strictly speaking, if you had crafted out the objectives and met those objectives, technically (against those objectives) you have succeeded but I do see your point. :) having those objectives gives the company an idea of their start and end point and hopefully garner some invaluable insights.
Across the mediums, there will be cases of one medium performing better than the other but hey that's the job of the consultant (or agency or even the client) to know (based on research or perhaps even gut feel) - business is risk-taking after all :)
5. re: "art and science" - interesting that you pointed out that art and science may not necessary be at the opposing ends but literature (and verbal) depicts that very picture (unfortunately). It has always positioned art and science as opposing ends and as such, it has naturally 'conditioned' us to think that way in my view.
Indeed, a fair number of questions on ROI, but hey its fun! and very real. Its the number ONE term that pops up in meet-ups (actually all) :)
Best,
Jon
Posted by: Jon W Chin | 24 May 2009 at 23:15
No worries Philip,
I totally agree with the statement "Finance is not entirely that exact".
ROI is a good indication of success but cannot/should not be used to determine success over a short period of time. We need to decide if the particular campaigns objective is to increase revenue within 2 months or increase loyalty over 2 years.
Other branding activities such as rebranding is an exercise which might not be necessarily quantifiable.
An example, we didnt have that many metrics in place back in the 80s recessions. (what has the world turned into)
Businesses which chose to maintain or raise their level of advertising expenditures during the 1981 and 1982 recession had significantly higher sales after the economy recovered. Specifically,
companies that advertised aggressively during the recession had sales 256% higher than those that did not continue to advertise.
Thus, its is critical that companies continue to develop a strong brand through a healthy consumer-brand relationship.
Brand loyalty is vital for greater sales results which might not be measurable at that direct moment.
Brand loyalty requires all sorts of promotional/advertising campaigns and even if 1 of the promotional activity fails, the overall end objective may be achieved.
Thus, science or art? its in accordance to how strict the CEOs are.
I dont think any marketer would say that marketing must be measured before they can spend the dollars because establishing emotional linkage to a brand might take months to years to happen. It doesnt mean they will have to stop branding themselves.
Will
Posted by: William Chin | 26 May 2009 at 02:26