Angela Jeffrey, Vice President Brand Manager, ABX Advertising Benchmark Index

Angela Jeffrey, Vice President Brand Manager at ABX Advertising Benchmark Index, shares in this post her views on how advertisers measure creative today. She shares some examples of the pitfalls of depending on Likability in choosing creative, comments on how does PR/news impact advertising’s success and explains why all mediums are equally effective.

Q:  Please describe what ABX is and what it does. 

AJ: ABX Advertising Benchmark Index is a syndicated ad effectiveness service with the largest normative database (100,000+ ads) in the world.  ABX tests all ads for clients and their competitive sets across all media such as television, radio, print, digital, outdoor, in-store signage, and more. Results are delivered in 24 hours, enabling clients to adjust creative and thereby optimize media spend. Every ad is measured across 14 key performance metrics including awareness, messaging, reputation, relevance, likability and calls to action. ABX data has been shown to improve market mix models by up to 15%, and is a fraction of the cost of traditional ad testing.

Q: How are advertisers measuring creative today? What are the implications of testing only 10% of your creative?

AJ: Most advertisers today hire research firms to copy test 10-20% of their creative while it is still in development.  Other media types don’t have the same level of concentrated spending as TV, and for most companies only TV is copy tested. The reasons more ads aren’t tested are threefold: money, time, and philosophy. Money: the typical cost of pre-testing is $20,000 to $30,000 per ad and the budget is limited; Time: the quickest testing firms report back in a week, but the great majority take 3-6 weeks before results are provided. Often there is simply not enough time to copy-test; and Philosophy: some creative teams resist copy testing believing creativity can’t be measured and they believe they know what works.

Timing is often the biggest obstacle and even if an ad is copy tested, by the time results are ready, the team is under pressure to get the ad into the marketplace. Changes may be made to the ad, but advertisers don’t have the luxury of sending the “fixed” ad back for further testing. At some point, it’s a matter of faith that changes made resulted in an improved ad.

But we know from evaluating over 100,000 ads across all media that the majority of ads underperform, so the current methodology does not result in predictably good creative outcomes. This chart shows ABX ad testing results for the last seven days with each ad depicted by a dot. The yellow line is the “average” for effectiveness.  You can see that outside of TV, more than half the ads in the other media categories are well below the yellow line and many TV ads also underperform. These ads represent millions of wasted media dollars, not to mention wasted production costs.

Q: What are some examples of the pitfalls of depending on Likability in choosing creative? How does PR/news impact advertising’s success?

AJ: Many advertisers depend heavily on “Likability” to determine which ads will be produced and run – and often this depends on their own internal opinions. But ABX research has found that Likability is neither predictive of creative effectiveness nor highly correlated against other key metrics. We most often rely on whether we like an ad rather than on more predictive metrics, and the result is a majority of ads that underperform with millions of dollars in wasted media spend.

ABX research has shown:

• How much we like an ad has little impact on our association of the advertised brand with the ad (Brand Linkage);
• How much we like an ad has little impact on how well we understand the Message being delivered;
• Likeability does have a positive impact on feelings toward the brand (Reputation), but may have no impact on a more immediate Call-to-Action.

If you find this hard to believe, see this summary of a recent Forbes story on Super Bowl ads for 2016. ABX measured 61 ads against 14 Key Performance Indicators and contrasted Likeability scores as measured by USA Today against the ABX Index (the overall measure of ad effectiveness). Forbes quoted ABX President Gary Getto in summarizing the results, “Likability was wrong 54% of the time.” In fact, many of the ads that scored poorly on Likability were still effective.

As for PR and news impact on ad effectiveness, ABX is now working on a study in conjunction with the Institute for Public Relations (IPR) to generate more precise algorithms to apply to the impact of earned media on paid advertising and vice versa. The goal is a more holistic understanding of the impact of integrated communications on business outcomes.

Q: Are all mediums equally effective?

AJ: With the great migration to all-things-digital, one might assume the other types of media are no longer valid.  This couldn’t be farther from the truth, however. ABX testing  has shown that all mediums work almost equally as well depending on the specific needs of a campaign.  But most important, they depend on the brilliance of the creative used.

In comparing top-scored ads across all media, ABX found that all showed similar patterns for Awareness, Messaging and Any Action.  However, Online Video, TV and Radio scored best in Reputation, and Outdoor had the biggest impact on Purchase. Of all mediums, radio seems to offer the biggest opportunity being least expensive and having the least powerful creative.  A “great radio” creative team could have an enormous impact!

About Angela Jeffrey
Angela Jeffrey is a national award-winning veteran of PR, advertising and marketing with JCPenney, Jeffrey Communications and national agencies. She later became an expert in PR measurement after developing, and selling, one of the first DIY measurement tools in the PR industry, and consequently became interested in the intersection of paid and non-paid media effectiveness.

Today, Angela is Vice President Brand Manager for ABX Advertising Benchmark Index, an advertising-effectiveness firm that tests creative in ALL media in virtual real-time. ABX is in the process of structuring a test of PR effectiveness based on panel response to advertising before and after a PR or social media event.

Angela is a recognized evangelist, thought leader, writer and speaker for PR measurement and evaluation with an emphasis on correlating share of voice to business outcomes. She also serves as a member of the Institute for Public Relations Measurement Commission. You can reach Angela via email, follow her on Twitter or learn more on LinkedIn.