Marketing Surveys: Be Careful What You Ask For

Gaining a deep understanding of consumer needs is the mantra of many companies, who spend hundreds of millions of dollars searching for rich insights and to better understand competitive position.    Sadly, I often cringe when looking at the results of some of this product research. All too often what appears to be an objective market survey – those multi page questionnaires deployed in the mail, online or in person – ends up yielding  bias-ridden and overly pessimistic results with limited value. 

Misleading or inaccurate feedback has serious implications for B2C and B2B firms who, in part, base investment decisions and marketing strategies &  tactics on consumer and prospect feedback.  Obviously, well-intentioned researchers do not intend to generate poor survey results, so why does it happen? According to research produced at Stanford and other universities, weak survey results arise primarily from two factors:  1) the role that advanced survey notification plays on what consumers say in their feedback and; 2) an innate bias and artificiality common in many questionnaires.

The Stanford study concluded that notifying consumers to anticipate an upcoming survey will very often lead to unexpected, and generally negative, results.   In essence, warning respondents in advance that there will be questions on a product will lead them to believe they need to focus on negative aspects about the product, thereby consciously or unconsciously skewing the feedback. Most consumers want to be honest, and even helpful, when they participate in a survey. However, for many consumers, being helpful means being (constructively) critical or at least offering suggestions for improvement, as opposed to simply stating their opinions, good or bad. 

Moreover, other research undertaken by Stanford and Rice Universities found that asking consumers to compare two or more brands yielded more negative brand feedback than consumers who undertook themselves to compare brands.  Consumers who were asked to compare became unusually cautious and less objective. As one of the researchers said, “The mere fact that we had asked them to make a comparison caused them to fear that they were being tricked in some way.”

Furthermore, many consumer surveys ask respondents to do and provide explanations for things they would never do in real life.  As such, these surveys create an artificial reality that lacks relevancy and the ability to really understand what drives behavior. For example, how many customers go into stores or visit a web site and then evaluate it based on 10 variables?  Moreover, how many consumers can peer into their sub-conscious to understand why they act in certain pre-wired ways?

Not surprisingly, the Stanford research is at odds with conventional marketing wisdom and prior academic work which found that querying consumers just prior to shopping or encouraging buyers to compare products on the shelves was likely to yield the best quality feedback.

By no means am I suggesting that businesses should not listen to consumers or completely do away with marketing surveys.  Rather, listening to consumers must be tempered with healthy skepticism and supported by a variety of feedback sources.  Some strategies could include-

1.  Improve survey design and execution by removing pre-warnings, minimizing the number of variables to measure and eliminating “artificial” and leading questions;

2.  Utilize multiple research tools to gain richer, more relevant insights.  These tools should balance quantitative and qualitative (e.g., focus groups, 1:1 interviews) approaches;

3.  Overlay surveys with ethnography (e.g., “day in the life” studies) and anthropology to explore sub-conscious drivers of behavior.

For further information on our services and work, please visit the Quanta Consulting Inc. web site.


No comments yet

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: