In a new study of advertising content that combined both neuro-physiological measures and traditional copy test measures, G&R has found that the two assessment approaches can yield different guidance when evaluating copy effectiveness. Commercials with strong emotional activation as measured by facial electromyography (fEMG) may not have high recall or persuasion, and visa-versa. In other studies, we have found positive correlations between fEMG and recall or fEMG and persuasion. Continue Reading →
Tag Archives | research
A new study on the validity of fMRI results has found significant errors in the software that is standardly used in fMRI analysis. One of the bugs, which has since been fixed, had existed for 15 years, potentially making flawed much of the current thinking about how the brain works. Continue Reading →
Problem: Settling for Engagement.
As media choices grow, engagement has received interest as a metric for elevating cross-media assessment. All eyeballs are not equal and some media channels, particularly digital ones, assert they are better than others at delivering eyeballs that matter more to a particular brand. Engagement focuses on the qualitative side of communication and deepens traditional reach and frequency discussions.
But engagement is only a partial measure of total advertising effect. Engagement is loosely defined as producing a positive brand experience. It is often operationalized through such measures as views, clicks, likes, shares and comments. Knowing whether an ad experience has been engaging is more actionable than knowing where and when an ad ran, but it is not as actionable as knowing whether an ad influenced buying behavior. Achieving sustained performance improvement and brand building requires more than engagement. Continue Reading →
Thoughts on the Use of Online Opinion Polling by a Gray Lady and a Peacock
To qualify candidates for the recent 2015 GOP debate, Fox News used results from polls conducted by Bloomberg, CBS News, Fox News, Monmouth University and Quinnipiac University. The fact that all five studies were conducted via telephone is not a coincidence.
Although web-based surveys are prevalent within the market research industry, they have not been widely embraced by the opinion-research field, and news outlets generally avoid reporting on findings based on online panels. The reason, as ABC News explains in its current polling standards:
Methodologically, in all or nearly all cases we require a probability-based sample, with high levels of coverage of a credible sampling frame. Non-probability, self-selected or so-called “convenience” samples, including internet opt-in, e-mail, “blast fax,” call-in, street intercept and non-probability mail-in samples do not meet our standards for validity and reliability, and we recommend against reporting them.
Problem: Using Short-Term Measures to Determine Advertising Performance.
Marketing ROI calculations often measure activities such as sales, inquires, and clicks to gauge the effectiveness of a program. The strength is that these measure what someone actually does, as opposed to what he or she says they will do. However, activities, even ones as concrete as sales, are short-term behaviors; what someone does today is not always predictive of what he or she will do tomorrow. As a result, analyses that are based on activities tend to support programs that produce short-term effects at the expense of programs that also have material long-term benefits. For example, using sales data alone, couponing generally outperforms advertising, but this conclusion would not make for a sound marketing practice. Continue Reading →
Super Bowl watching is a social experience. We watch the game eagerly for its pomp and competitiveness, but also to assemble with friends and family, be part of a community, and engage with others as the show and the advertising unfold. We share food and drink. We talk before during and after the game. And, increasingly, we use social media to broaden our connectedness. The propensity to engage in social media activity influences the Super Bowl viewing experience and how people consume advertising messages. According to new research from G&R, the more actively involved a person is in social media, the more commercials they pay attention to and the more favorable their reaction to them is. Continue Reading →
Ad research pioneers George Gallup and Horace Schwerin had mixed opinions about the value of norms. They understood the theoretical importance of benchmarking, but they also understood the knowable and unknowable error that all benchmarks have. Indeed, Horace Schwerin once commented that he believed copy tests should be conducted without norms. It was his view that norms could be a crutch that helped marketers avoid thinking in absolute terms about what their advertisement should achieve.
Although an unconventional view today, it is still an interesting and important proposition. Even in the early days of copy testing, when the development of norms was a relatively straightforward undertaking, norms were not always the useful benchmarks they seem. A main idea communication norm, for example, is likely to be based on both advertisements that have a single key selling message, as well as ones that have more complex, multi-faceted communications. Moreover, the norm will usually be based on what the respondents of each test perceive the selling message to be, which may or may not have been the same as the communication objective. Or it may include ideas that are inherently easier or more difficult than yours to convey. Depending on your own communication objectives, it can be preferable to set an absolute criterion that the message must be played back by a certain proportion of respondents rather than compare to a norm based on commercials with message content and strategies.
But questions about norms are even more relevant today. Today’s advertising world is more complex than in Gallup and Schwerin’s time, when advertisers had a mass market orientation. Copy testing companies simply provided extensive, broadly-based, category-specific norms based on methodologies that could accumulate extensive databases based upon samples of men and/or women between the ages of 18 and 49 or 65. Since then, media has evolved from providing mass appeal programming and publications toward a diverse industry replete with a vast landscape of special interest offerings. Meanwhile marketers, armed with segmentation research that facilitate honing communications in on target markets with the greatest leveragability, take advantage of the expanding media opportunities with initiatives targeted to more specifically designated audiences. Thus the advertiser’s need for norms has changed with implications for the quality of norm that is often available.
- Today, advertisers often specify that samples meet specific target definitions designed to match their demographic or psychographic targets. Competitors within the same category may specify different sampling targets or more have different screening criteria for identifying like samples. A norm for travel and leisure services, as an example, may consequently be comprised of a variety of sample definitions, some based on advertising targeted to the premium traveler, others to the budget minded; some to family vacationers, others to business travelers; some to the youthful and adventurous, others to older vacationers with a more relaxed mindset. As a result, when a category norm may exist, it may not reflect the specific target in which the marketer is interested.
- The increased use of pretesting has given advertisers greater flexibility and agencies greater creative freedom. A downside to this positive development, though, is that tested concepts are often included into a normative database even though not all will have been taken into final production nor received media support. When a norm is based on finished commercials, as were typical erstwhile syndicated services, it represents, by definition, all executions that were judged “airworthy.” If pretested concepts are included, today’s normative databases may present a lower than desirable hurdle if they include concepts that failed to pass muster.
- Even highly effective advertising will not have been developed to be effective on all performance measures. Some commercials are developed to be strong in some aspects of communication and other commercials are developed to “move the needle” in other areas. For example, sometimes ad liking will be an important criterion and sometimes it is not. The same can be said, at times, about recall, persuasion and ad advocacy. Since most ads are tested within a standardized system for the company, they are tested against measures that are not always relevant to the objectives of the advertising. Since norms are typically compiled based on complete test results, they may be based on cases where the measure itself was not relevant to the test commercial’s communication objectives.
- Advertisers also have a wide variety of techniques and vendors from which to choose. Consequently, any given vendor may have experience in a category but only with a limited repertoire of clientele. Unless a program is in place to add non-client readings to their database, a prospective new client may be comparing their results to a “norm” based upon a non-representative sampling of category advertising.
The consequence of all this is that any particular norm today is based on a data set that is likely shallower and less comprehensive than they have been, making their use less straightforward than in earlier already problematic times. Schwerin’s proposition may even begin to have greater appeal under these circumstances, but solutions exist for ensuring that useful benchmarks are also considered. Knowledgeable firms will usually have addressed these issues and be in a position to provide workable answers through means like the following:
- Using their experience and breadth of testing to craft a norm specific to your target audience;
- Establishing that performance within subgroups does not to show much variation; and/or
- Devising a test design that will provide its own benchmarks.
As a result you can be more confident that you are comparing the specific target audience reactions of your advertising to that of your competition, giving you a strong perspective for assessing the likely effectiveness of your messages.
It was mentioned above that it has been customary since copy testing’s early days for most companies to evaluate their advertising against category-specific norms. However, there is a school of thought that all advertising norms provide a better benchmark because each ad must compete against the totality of all advertising in order to break through. While there is merit to maintaining perspective on how all advertising tends to perform in a given testing system, losing sight of how well your advertising is doing relative to your competition among your target audience is risky business. A company in a high interest category that accepts an ad that is above the all advertising average without determining that it may nevertheless be below industry norms will be at a disadvantage.
As with other aspects of research, experienced professionals at your vendor will be able to provide guidance on how to best establish action standards for your test and will be best able to use these capabilities to get the most out of your results and provide strong direction for your communications.