A few days ago, WARC saw fit to publish this opinion piece from Vince Usher and Adam Ferrier titled, "Why creativity always loses the battle against eight indifferent consumers." Yes, I know it was published on April 1st. The trouble is, while the article is a joke, I worry some people might take it seriously. In fact, judging by the comments to this post
, some did.
Salesmanship or well-backed argument?
I'll skip the precis, you can read the article from this link if you must, but the first of Usher and Ferrier's conclusions is that,
"Many market research companies laud the fact that their benchmarks show no difference between a finished ad and a storyboard concept, but our research suggests that this is much more likely salesmanship than a well-backed argument."
So why is this funny? Given how ad agencies often feel about pre-testing, it is all too easy to take the statement at face value. How about we take a little look at just how ridiculous their research was?
Creativity alone does not guarantee effectiveness
As noted in the article, the "Monty the Penguin"
ad for John Lewis was incredibly successful as part of a long-running campaign for a familiar brand in the UK. So, our two brave thinkers took the ad, turned it into a storyboard and tested the storyboard and the finished film in Australia, where the brand has a negligible presence.
Now that must be a joke, right? Because no agency worth their salt would propose using "Monty the Penguin" to advertise John Lewis in Australia, would they? A UK audience knows what John Lewis is, why they might shop there, and many eagerly anticipate the brand's Christmas ad campaign. In the UK the ad needs to do little more than entertain and remind people of the brand's existence (although I seem to remember it also sold a lot of toy penguins).
The fascinating thing about advertising is that its success or failure is inherently bound up with a brand's history and the local culture, and yet, I have noted in the past, agencies often seem to think that the same idea will work irrespective of either brand or country. Witness the rash of copycat ads that pop up around the world following the success of campaigns like "Get a Mac."
Even the best ads can fall flat when the context changes. As a case in point, you might remember the Cadbury's Gorilla ad from 2007
. The ad was a huge success in the UK, and subject to a whole bunch of bs about how the ad failed in pre-testing, which was simply not true
…in the UK. I may be mistaken, but I am pretty sure someone told me the same ad was tested in South Africa and it bombed.
Given that backdrop, is it any wonder that "Monty the Penguin" lacked relevance to the Australian audience? Usher and Ferrier conclude,
"Without context consumers will reject, or at least view sceptically, the very best of advertising."
A not so indifferent audience
I believe the focus group attendees might have been paid to give their opinion. So, they were not indifferent, were they? They felt they had to do something to earn their money. If one of the questions asked was, "What do you think of this ad?", I suspect they would offer an opinion. And given the lack of context in which the ad was shown, the reaction was mixed at best.
The problem is that people do not think about ads, they respond to them. And part of the challenge for a TV ad is to overcome the natural indifference of the audience. A focus group audience is not indifferent in the same way as people watching a TV or an online ad.
A skilled moderator, one that knows the biases of the focus group environment and understands how ads work in real life, could probably figure out that the content was likely to be engaging. However, the moderator probably would end up questioning whether the ad was likely to be effective, because as one respondent pointed out, it would be helpful if the ad highlighted which brand was being advertised, never mind how it might be relevant to them.
To predict behavior, test appropriately
Honestly, I am not a big fan of qualitative ad testing, particularly when focus groups are used. If you want to test a storyboard, then do so one-on-one with the moderator explaining what is being shown and said. That at least will overcome "group think" and boring voiceovers. Use the findings to guide the subsequent development process, do not use them to try to predict effectiveness.
In the WARC article, Usher and Ferrier question the ability of pre-testing to predict positive in-market behaviors (disingenuously questioning the value of quantitative pre-testing alongside focus groups). If there is one thing I can agree on, no way, no how should you test an ad's potential motivational power based on two focus groups and a storyboard.
If you want to test an ad's likely impact on behavior, then do so quantitatively. Use a market research company that has done its homework and has uses questions that anticipate in-market behavior. And do not test a storyboard. Test an animatic, rip-o-matic or steal-o-matic, that gives people half a chance to figure out what the finished film might look like. I can vouch for the fact that provided the agency has done their job in representing their vision correctly, the results will give a good idea of how the ad is likely to perform in-market.
So, you think that is funny?
I have never understood the industry fixation with April Fool's jokes, and, sadly, I do not find the article funny. I guess I am jaded from fending off too many people arguing a similar case against pre-testing based on even less evidence. But what do you think? Care to offer an opinion?