Top

Archive | Research Methods

Say It Ain’t So, Chuck

Thoughts on the Use of Online Opinion Polling by a Gray Lady and a Peacock

To qualify candidates for the recent 2015 GOP debate, Fox News used results from polls conducted by Bloomberg, CBS News, Fox News, Monmouth University and Quinnipiac University. The fact that all five studies were conducted via telephone is not a coincidence.

Although web-based surveys are prevalent within the market research industry, they have not been widely embraced by the opinion-research field, and news outlets generally avoid reporting on findings based on online panels. The reason, as ABC News explains in its current polling standards:

Methodologically, in all or nearly all cases we require a probability-based sample, with high levels of coverage of a credible sampling frame. Non-probability, self-selected or so-called “convenience” samples, including internet opt-in, e-mail, “blast fax,” call-in, street intercept and non-probability mail-in samples do not meet our standards for validity and reliability, and we recommend against reporting them.

Continue Reading →

Framing Research for a Pharmaceutical Messaging Campaign

A leading biopharmaceutical company was preparing to launch a new product into the U.S. market. Concerned about the fallout from potential negative news reports about the product, they developed a consumer directed messaging program to help answer potential consumer questions and concerns about the product. They enlisted G&R to pre-test the communications program. G&R recommended a two test cell plus control design that framed the different mindsets that consumers might bring to the communications by stimulating exposure to either a negative or neutral news article about the product, before exposing the messaging strategy. To evaluate the effectiveness of the communications, G&R measured levels of consumer concern on key dimensions before and after each framing cell of respondents was shown the messaging.

The research showed that the communications were successful in reducing overall concern levels by over 20%and answering the prevailing consumer question about the product by over half in both cells. The research also found areas where the messaging program did not adequately address consumer questions and suggested additional content strategies to help alleviate those concerns.

If you would like to learn about G&R’s Framing Research, please Read More >

What’s in a Norm? Food for Thought

Ad research pioneers George Gallup and Horace Schwerin had mixed opinions about the value of norms. They understood the theoretical importance of benchmarking, but they also understood the knowable and unknowable error that all benchmarks have. Indeed, Horace Schwerin once commented that he believed copy tests should be conducted without norms. It was his view that norms could be a crutch that helped marketers avoid thinking in absolute terms about what their advertisement should achieve.

What is in these norms?Although an unconventional view today, it is still an interesting and important proposition. Even in the early days of copy testing, when the development of norms was a relatively straightforward undertaking, norms were not always the useful benchmarks they seem. A main idea communication norm, for example, is likely to be based on both advertisements that have a single key selling message, as well as ones that have more complex, multi-faceted communications. Moreover, the norm will usually be based on what the respondents of each test perceive the selling message to be, which may or may not have been the same as the communication objective. Or it may include ideas that are inherently easier or more difficult than yours to convey. Depending on your own communication objectives, it can be preferable to set an absolute criterion that the message must be played back by a certain proportion of respondents rather than compare to a norm based on commercials with message content and strategies.

But questions about norms are even more relevant today. Today’s advertising world is more complex than in Gallup and Schwerin’s time, when advertisers had a mass market orientation. Copy testing companies simply provided extensive, broadly-based, category-specific norms based on methodologies that could accumulate extensive databases based upon samples of men and/or women between the ages of 18 and 49 or 65. Since then, media has evolved from providing mass appeal programming and publications toward a diverse industry replete with a vast landscape of special interest offerings. Meanwhile marketers, armed with segmentation research that facilitate honing communications in on target markets with the greatest leveragability, take advantage of the expanding media opportunities with initiatives targeted to more specifically designated audiences. Thus the advertiser’s need for norms has changed with implications for the quality of norm that is often available.

  • Today, advertisers often specify that samples meet specific target definitions designed to match their demographic or psychographic targets. Competitors within the same category may specify different sampling targets or more have different screening criteria for identifying like samples. A norm for travel and leisure services, as an example, may consequently be comprised of a variety of sample definitions, some based on advertising targeted to the premium traveler, others to the budget minded; some to family vacationers, others to business travelers; some to the youthful and adventurous, others to older vacationers with a more relaxed mindset. As a result, when a category norm may exist, it may not reflect the specific target in which the marketer is interested.
  • The increased use of pretesting has given advertisers greater flexibility and agencies greater creative freedom. A downside to this positive development, though, is that tested concepts are often included into a normative database even though not all will have been taken into final production nor received media support. When a norm is based on finished commercials, as were typical erstwhile syndicated services, it represents, by definition, all executions that were judged “airworthy.” If pretested concepts are included, today’s normative databases may present a lower than desirable hurdle if they include concepts that failed to pass muster.
  • Even highly effective advertising will not have been developed to be effective on all performance measures. Some commercials are developed to be strong in some aspects of communication and other commercials are developed to “move the needle” in other areas. For example, sometimes ad liking will be an important criterion and sometimes it is not. The same can be said, at times, about recall, persuasion and ad advocacy. Since most ads are tested within a standardized system for the company, they are tested against measures that are not always relevant to the objectives of the advertising. Since norms are typically compiled based on complete test results, they may be based on cases where the measure itself was not relevant to the test commercial’s communication objectives.
  • Advertisers also have a wide variety of techniques and vendors from which to choose. Consequently, any given vendor may have experience in a category but only with a limited repertoire of clientele. Unless a program is in place to add non-client readings to their database, a prospective new client may be comparing their results to a “norm” based upon a non-representative sampling of category advertising.

The consequence of all this is that any particular norm today is based on a data set that is likely shallower and less comprehensive than they have been, making their use less straightforward than in earlier already problematic times. Schwerin’s proposition may even begin to have greater appeal under these circumstances, but solutions exist for ensuring that useful benchmarks are also considered. Knowledgeable firms will usually have addressed these issues and be in a position to provide workable answers through means like the following:

  1. Using their experience and breadth of testing to craft a norm specific to your target audience;
  2. Establishing that performance within subgroups does not to show much variation; and/or
  3. Devising a test design that will provide its own benchmarks.

As a result you can be more confident that you are comparing the specific target audience reactions of your advertising to that of your competition, giving you a strong perspective for assessing the likely effectiveness of your messages.

It was mentioned above that it has been customary since copy testing’s early days for most companies to evaluate their advertising against category-specific norms. However, there is a school of thought that all advertising norms provide a better benchmark because each ad must compete against the totality of all advertising in order to break through. While there is merit to maintaining perspective on how all advertising tends to perform in a given testing system, losing sight of how well your advertising is doing relative to your competition among your target audience is risky business. A company in a high interest category that accepts an ad that is above the all advertising average without determining that it may nevertheless be below industry norms will be at a disadvantage.

As with other aspects of research, experienced professionals at your vendor will be able to provide guidance on how to best establish action standards for your test and will be best able to use these capabilities to get the most out of your results and provide strong direction for your communications.

Positioning Research for a Specialty Retail Chain

Successful advertisers have one thing in common: their messaging distinguishes their brand in the minds of their target audience. It establishes who they are and what they stand for. It differentiates the brand from its competitors in a way that rings true with loyal brand users and provides distinctive appeals to non-users.

Strategic research is a valuable tool for developing this kind of successful advertising. It provides companies with insight into the triggers behind consumer brand selection – the core values and beliefs that people hold when deciding between marketplace alternatives. Well-designed strategic research unveils positioning opportunities for the brand. It identifies what is important to the buyers of your brand and the buyers of your competitor’s brands as well as how successfully each brand delivers on these expectations. Insights developed from this research reveal positioning opportunities for the brand to drive successful creative that resonates with your user base and attacks competitive vulnerabilities.

G&R was recently enlisted to evaluate the position that a leading company occupies in the crowded retail space and investigate the effectiveness of the company’s communications. Using a stimulus-aided questionnaire and advanced analytics and data displays, the research identified the company’s position relative to its key competitors, spotlighted a positioning opportunity for the brand that capitalizes on brand strengths important to both brand shoppers and competitive shoppers where the competition underperforms expectations. Armed with this insight our client can now exploit competitive weaknesses with a positioning built on brand strengths that both customers and competitive shoppers value.

If you would like to know our positioning capabilities, please contact us.

Content Marketing Research for a Leading Hospital

Many companies use Content Marketing to supplement traditional marcom channels and communicate new information via a variety of alternative media contexts. Recently, a prominent hospital launched a custom magazine publication in an effort to grow awareness in the healthcare market, communicate with leading medical, academic, and business influencers, and position the hospital as the definitive resource and leader in the medical and biotechnology fields. The company engaged Gallup & Robinson to design and conduct research to help the hospital better understand the magazine’s effectiveness, including the characteristics and needs of its audience, their attitudes towards the custom magazine, and the publication’s influence on perceptions about the hospital. The research provided benchmarks for quantifying performance and revealed how the audience thought and felt about the magazine. It resulted in new insights about how to improve lower-rated elements, adjust delivery methods and strengthen engagement moving forward. Learn More >

Cause Marketing Research for an Adolescent Health Initiative

Raising awareness or encouraging behavior modification for a social or public health issue is quite a different communications challenge than persuading consumers to purchase commercial products. Campaign designers and funding sources face two underlying questions: 1) is a campaign worth the investment of time and resources; and 2) how does one increase the chances that the campaign will be effective?

G&R was recently enlisted to evaluate a campaign which sought to establish a comprehensive community health model aimed at promoting healthy adolescent development and preventing dating violence. The initiative’s co-sponsors wanted to understand the target audience reactions to 8 concept executions and to determine which of the executions might be considered stronger or weaker candidates for the campaign. The research identified the strongest campaign ads based on overall stopping power, behavioral disposition, and emotional engagement. Select executions were recommended for strategic positioning in order to reach target subgroups and ways to optimize campaign communication were presented.

If you would like to learn about G&R’s Cause Marketing capabilities, please Read More >

Claims Substantiation Study for Major International Manufacturer

In a competitive marketplace, many advertisers make claims against similar companies and brands to influence customers to choose their product over those of their competitors. At the foundation of this communication is the requirement that the claims be true. Survey research is often used to provide the basis of the claim and to challenge the basis of a competitor’s claim. However, between January 2006 and June 2011, the National Advertising Division (NAD) of the Better Business Bureaus found that 71 percent of the consumer perception surveys introduced by parties to an NAD proceeding were unreliable and, therefore, had little or no impact on the final outcome of case. Claims research needs to be carefully constructed and executed if it is to have much value for the advertiser.

Recently, a major international manufacturer hired G&R to test several claims made in a competitor’s advertising campaign, alleging that the competitor’s products were widely preferred by consumers on a number of criteria. In order to test these clams, G&R launched an exacting and rigorous four-stage study where participants were asked to use and evaluate comparable products from different manufacturers and rate the products on a range of qualities. Stimulus exposure and questionnaire administration were carefully controlled. The study showed that the competitor’s claims were not reliable and was used as part of a successful challenge to the NAD.

If you would like to learn about G&R’s Claims Substantiation capabilities, please Read More >

Why Your Persuasion Measure is Out of Date

This morning when I got up and went into the bathroom, there were 4 brands of soap, 2 brands of shampoo and 2 brands of conditioner from which to choose. After showering and dressing, I went into the kitchen and selected breakfast from 15 different boxes of cold cereal that sit on our pantry shelves. When I packaged my lunch, I had three brands of yogurt in the refrigerator from which to choose. Fortunately my wife and I are empty nesters; were our daughters still living at home, our product arrays would doubtless be even broader.

Such proliferation of brands within our household repertoire might suggest to many marketers that we do not represent particularly brand loyal consumers when in fact the opposite is true. Most of the cereals, yogurts, soaps, shampoos and conditioners are brands we have purchased repeatedly and even advocated to our family and friends. But our loyalty does not take the form of loyalty to a single brand within any given category and we are hardly alone.

Andrew Ehrenberg, late of the London Business School, when talking about brand loyalty, said that consumers select from a “portfolio of brands” often simply because they want a change. This observation has significant implication for brand managers. Building brand usership levels is not merely a matter of winning dedicated customers over to your brand. It is a far more complicated process as brand users switch between a favored set of brands for a variety of reasons. The classic tagline from the famous Peter Paul commercials for their Mounds and Almond Joy brands, “Sometimes you feel like a nut, sometimes you don’t,” is indicative of consumer spontaneous decision making that takes place in a wide array of product and service categories, not just impulse purchase categories like candy bars. Brand managers in the new marketing landscape are challenged with the creation of advertising that breaks through, reminds consumers of the availability of their brand, and primes it in their minds. But advertising must also reinforce brand traits that keep the brand within the consumer’s portfolio, and informs them of news about the brand, such as product improvements and line extensions in order to invigorate the brand and broaden its appeal over other portfolio brands. Both of these ends, priming and meaning reinforcement, contribute to creating preference and passion, which deliver substantial monetary value through loyalty and advocacy.

However, as marketing communications have become more sophisticated in their appeals to consumers, encouraging them to engage more deeply with the brand, persuasion measures used to gauge the effectiveness of those communications have not kept pace and remain rooted in the 1970s. Standard persuasion measures employed by many of the more popular copy testing methods are pre/post attitudinal change measures. Diagnostic tests rely on even more basic one-dimensional rating scales.

Pre/post attitudinal change measures grew in popularity in the 1970s as researchers began to believe that recall-based systems needed an additional measurement to provide insight into how well an advertisement helped develop a preference in the consumer’s mind for the advertised brands. The marketing model in vogue in those days was still focused on winning customers to the brand in the belief that, once there, they would become exclusively loyal users. Consequently, these persuasion measures saw the world in simple, brand-switching terms.

In a standard pre/post persuasion measure, respondents participating in an ad test are asked to indicate the brand they prefer in the test category as well as a number of masking categories prior to seeing the program or reading the magazine and accompanying advertising. Typical question wording instructs the consumer to “select the brand you truly want.” After exposure to advertising, respondents are asked again to select their preferred brand in each category using identical question wording. “Persuasion” occurs when a respondent who did not select the test brand initially selects the brand after ad exposure. This model intends to mimic the circumstance of a consumer with no prior preference for a given brand being “persuaded” by advertising to “truly” want the brand to the exclusion of other category brand choices. As such, it purports to measure the extent that consumers who formerly selected a competing category brand are persuaded to “switch” to the advertised test brand.

However, a “brand switching” mechanism of this type provides little information about how persuasive the advertisement has been in the realistic, far more complex world wherein consumers accustomed to preferring a portfolio of brands may have been influenced. In a “brand switching” mechanism, respondents who indicate preference for the test brand before seeing the advertising are not investigated further; their preference is taken for granted.

By the same token, many very basic diagnostic-style copy systems use a unidimensional scale, which asks respondents to simply indicate how likely they are to buy the brand after seeing the advertisement. For consumers with the brand already in their portfolio, intention levels are high and greater insight into the effect of the advertisement on brand purchase intentions cannot be derived with this measure.

In order to provide greater insight into this complex persuasion dynamic missing from standard persuasion measures, Gallup & Robinson developed Advanced Persuasion, a multi-dimensional metric grounded in engagement theory. Advanced Persuasion incorporates the entire persuasion dynamic, investigating ad effect among both consumers who already include the brand in their portfolio as well as those persuaded to the brand for the first time. Research has shown that standard persuasion measures can miss as much as 82% of the persuasive effect of an ad by failing to consider consumers with the brand already in their portfolios. These are the consumers standard persuasion measures fail to investigate and for many established brands, they represent the foundation of brand profitability, returning to the brand again and again.

Moreover, these consumers may be influenced by an ad along any of a number of dimensions, consequently Advanced Persuasion incorporates multiple measures in order to capture changes in consumer attitudes toward the brand along a number of key fronts where response has proven to be both sensitive to ad messaging and indicative of subsequent purchasing.

For those brands whose advertising needs to be about more than breakthrough, buzz, and liking, Advanced Persuasion provides an important new measurement option that will help lead to better, more informed advertising copy decisions.

Why Your Persuasion Measure is Out of Date

This morning when I got up and went into the bathroom, there were 4 brands of soap, 2 brands of shampoo and 2 brands of conditioner from which to choose. After showering and dressing, I went into the kitchen and selected breakfast from 15 different boxes of cold cereal that sit on our pantry shelves. When I packaged my lunch, I had three brands of yogurt in the refrigerator from which to choose. Fortunately my wife and I are empty nesters; were our daughters still living at home, our product arrays would doubtless be even broader.

Such proliferation of brands within our household repertoire might suggest to many marketers that we do not represent particularly brand loyal consumers when in fact the opposite is true. Most of the cereals, yogurts, soaps, shampoos and conditioners are brands we have purchased repeatedly and even advocated to our family and friends. But our loyalty does not take the form of loyalty to a single brand within any given category and we are hardly alone.

Andrew Ehrenberg, late of the London Business School, when talking about brand loyalty, said that consumers select from a “portfolio of brands” often simply because they want a change. This observation has significant implication for brand managers. Building brand usership levels is not merely a matter of winning dedicated customers over to your brand. It is a far more complicated process as brand users switch between a favored set of brands for a variety of reasons. The classic tagline from the famous Peter Paul commercials for their Mounds and Almond Joy brands, “Sometimes you feel like a nut, sometimes you don’t,” is indicative of consumer spontaneous decision making that takes place in a wide array of product and service categories, not just impulse purchase categories like candy bars. Brand managers in the new marketing landscape are challenged with the creation of advertising that breaks through, reminds consumers of the availability of their brand, and primes it in their minds. But advertising must also reinforce brand traits that keep the brand within the consumer’s portfolio, and informs them of news about the brand, such as product improvements and line extensions in order to invigorate the brand and broaden its appeal over other portfolio brands. Both of these ends, priming and meaning reinforcement, contribute to creating preference and passion, which deliver substantial monetary value through loyalty and advocacy.

However, as marketing communications have become more sophisticated in their appeals to consumers, encouraging them to engage more deeply with the brand, persuasion measures used to gauge the effectiveness of those communications have not kept pace and remain rooted in the 1970s. Standard persuasion measures employed by many of the more popular copy testing methods are pre/post attitudinal change measures. Diagnostic tests rely on even more basic one-dimensional rating scales.

Pre/post attitudinal change measures grew in popularity in the 1970s as researchers began to believe that recall-based systems needed an additional measurement to provide insight into how well an advertisement helped develop a preference in the consumer’s mind for the advertised brands. The marketing model in vogue in those days was still focused on winning customers to the brand in the belief that, once there, they would become exclusively loyal users. Consequently, these persuasion measures saw the world in simple, brand-switching terms.

In a standard pre/post persuasion measure, respondents participating in an ad test are asked to indicate the brand they prefer in the test category as well as a number of masking categories prior to seeing the program or reading the magazine and accompanying advertising. Typical question wording instructs the consumer to “select the brand you truly want.” After exposure to advertising, respondents are asked again to select their preferred brand in each category using identical question wording. “Persuasion” occurs when a respondent who did not select the test brand initially selects the brand after ad exposure. This model intends to mimic the circumstance of a consumer with no prior preference for a given brand being “persuaded” by advertising to “truly” want the brand to the exclusion of other category brand choices. As such, it purports to measure the extent that consumers who formerly selected a competing category brand are persuaded to “switch” to the advertised test brand.

However, a “brand switching” mechanism of this type provides little information about how persuasive the advertisement has been in the realistic, far more complex world wherein consumers accustomed to preferring a portfolio of brands may have been influenced. In a “brand switching” mechanism, respondents who indicate preference for the test brand before seeing the advertising are not investigated further; their preference is taken for granted.

By the same token, many very basic diagnostic-style copy systems use a unidimensional scale, which asks respondents to simply indicate how likely they are to buy the brand after seeing the advertisement. For consumers with the brand already in their portfolio, intention levels are high and greater insight into the effect of the advertisement on brand purchase intentions cannot be derived with this measure.

In order to provide greater insight into this complex persuasion dynamic missing from standard persuasion measures, Gallup & Robinson developed Advanced Persuasion, a multi-dimensional metric grounded in engagement theory. Advanced Persuasion incorporates the entire persuasion dynamic, investigating ad effect among both consumers who already include the brand in their portfolio as well as those persuaded to the brand for the first time. Research has shown that standard persuasion measures can miss as much as 82% of the persuasive effect of an ad by failing to consider consumers with the brand already in their portfolios. These are the consumers standard persuasion measures fail to investigate and for many established brands, they represent the foundation of brand profitability, returning to the brand again and again.

Moreover, these consumers may be influenced by an ad along any of a number of dimensions, consequently Advanced Persuasion incorporates multiple measures in order to capture changes in consumer attitudes toward the brand along a number of key fronts where response has proven to be both sensitive to ad messaging and indicative of subsequent purchasing.

For those brands whose advertising needs to be about more than breakthrough, buzz, and liking, Advanced Persuasion provides an important new measurement option that will help lead to better, more informed advertising copy decisions.

Belief Elicitation Research for Animal Health Company

A leading international biopharmaceutical company asked Gallup & Robinson to identify and understand behaviors and attitudes associated with pet ownership. G&R developed a framework to distinguish differences among segments of the population. We identified clearly differentiated demographic, regional and attitudinal segments. Differences in belief systems of each owner population segment explain variances in care behaviors, leading to unique messaging strategies which can be used in campaigns aimed at helping owner caregivers to ensure their companion animals live longer, healthier lives. Read More on Belief Elicitation >