Lately, I’ve been doing a lot of research that involves new product concepts, web pages, packaging mockups and other types of stimuli. As a researcher, I’ve always been willing to provide feedback to clients on the stimuli they create, and they’re generally receptive to my suggestions. With that in mind, here are two immutable market research principles that we should all keep in mind when creating research stimuli.

1 – Research participants can only respond to something that’s in front of them. It’s difficult for most people to form an opinion on something entirely hypothetical. If you need to understand what flavors consumers might be interested in for your sports drink, don’t expect them to tell you; they’re going to need a list. If you want to know what menu options users would like on your web page, your mockup needs an example for them to react to. And if you want to know how much they might be willing to pay for that sports drink, your concept will need to include some pricing information—there’s no sense in asking ‘how much would you pay?’ In other words, people can’t generally tell you what they want—you need to show them something you think they might want and see if they agree.

2 – Research participants cannot help but respond to everything that’s in front of them. Reactions to a stimulus will be shaped by the entirety of that stimulus. And there isn’t any way to tease apart the elements that drove that reaction—you can’t unscramble an egg. So, if you don’t want to talk about price, don’t include one in your concept. If you’re not interested in discussing a new brand logo, don’t include it on your web page wireframe. In other words, stimuli must focus tightly on research objectives, and not include any extraneous information. The more your stimulus contains, the more likely it is that respondents will be distracted by issues that aren’t relevant. Also, don’t include more information in your stimulus than could realistically be found in real-world marketing executions. If there’s only room for one claim on your package, don’t put five in your concept.

Remember – a stimulus is just another way to ask a question. As with every question asked in qualitative research, it should be focused on your research objectives and must contain the right amount of detail.

We’ve all been in that uncomfortable situation that comes up periodically during qualitative research – DEAD SILENCE.

The conversation just stalls, and you have no idea what to ask next. So, what to do?  Here are a few solutions that will help to break the ice and get things jump-started.

First, Don’t Panic. This is an opportunity to reset the conversation and steer things in a new, perhaps unexpected, direction. If you don’t want to do that, go to the next topic. Unless you’ve covered every topic in your guide – including those ‘time-permitting ones’ – there should be more to discuss. So just move on.

Don’t Fear the Pause. This is hard, but silence can often be beneficial. Participants may feel a need to say something, and that could lead to an unexpected insight.

Take The Pause Head-On.  Say something like … “uh-oh – why has the room just gone quiet?”

Make The Topic of Conversation the Conversation Itself.  It’s okay to encourage a bit of introspection to get things rolling again.  You can ask:

“How is this conversation making you feel?”

“Has anything said in this conversation surprised you?”

“Have your thoughts and feelings about this topic changed in the course of this conversation?”

“Is there something we’re not talking about here that we should?”

“Is there something unsaid lurking beneath the surface of this conversation?”

Have A Few ‘Back Pocket’ Questions. Be prepared for that sudden lull. It’s a good idea to have some ‘questions for all occasions’ to ask if you’re stuck. I recently heard Stephen Colbert call this the – ‘In case of emergency, break glass, remove question’ question. Here are mine:

“What do I not seem to understand about this topic?”

“What don’t people understand about this topic?”

“Who do you think this product/brand/category is for?”

“You used the term ______ a minute ago.  What did you mean by that?”

So, don’t fear awkward silence.  It happens naturally during many conversations. Use it to your advantage and possibly to gain insights you never anticipated. Awkward silence can be your friend – if you’ll let it.

My thanks to master moderators Michelle Ogren Jeffrys and Pam Goldfarb Liss for their input on this topic.

Trial and error are often the path to success.

Sometimes you just have to try something and learn as you go. A few years ago, I conducted research on concepts for a new healthcare-related service. Most of these ideas were quite well developed. However, while we were designing the research, the client mentioned one idea they had that was so ‘out there’ that they didn’t quite know how to describe it. In fact, they even struggled when discussing it amongst themselves. That being the case, the client felt we should hold off on testing this idea.

I pushed back a little, pointing out that squeezing in one more concept would not be a big deal. I said, “look— just write something up. It doesn’t matter how bad it is. We’ll show it in the first group, see what people say, adjust accordingly, and then show it again to the next group. By the time we get to the end of the research, maybe we’ll have something.”

The reaction to the concept in the first group was primarily confusion. But, after some discussion, the participants began to understand the idea and made suggestions as to how to better describe it. We took this feedback, edited the concept, and showed a revised one in the second group where it did a little better.

As the research went on, we continued to adjust. By the final group the participants understood the idea fairly well, but still didn’t quite know what they thought about it. Afterwards, the client felt they had a much better understanding of what their idea was, and what they needed to do internally to continue to develop it. About a year later we showed fully developed descriptions which performed very well among individuals in certain consumer segments.

Just taking your best shot and adjusting along the way can be nerve-wracking, but it’s also effective— iteration is often our greatest teacher. We live in a staggeringly intricate world, and many systems and situations are so complex that they simply can’t be intuitively understood. It’s naïve to think you can design a perfect approach up front. Don’t’ fall into the trap of thinking that knowledge and experience alone will lead you to an optimal solution. This principle applies to all aspects of our lives, and certainly to marketing and market research.

Here are some examples of how a trial-and-error approach can be used in marketing and market research:

• Some of my clients routinely optimize marketing tactics like service bundles, promotions and banner ads by introducing something in a limited geography or for a very short period. They see how it perform, tinker with it a bit, and then broadly introduce the optimized tactic.

• When I write a discussion guide, I make a point of including multiple possible ways to ask questions, and several back-pocket exercises to be used if the primary exercises don’t work as hoped. It’s not realistic to think that everything I put into the guide is going to elicit the desired information, and it’s important to be ready to turn on a dime.

• When screening research participants, give yourself plenty of time for recruiting. It’s likely you’ll learn some things during the first few days of screening that will make you realize you need to make some changes to either the research specifications or the questionnaire.

• When designing research, consider breaking it into multiple phases. Depending on your objectives it might be valuable to have time to step back, think, adjust and do some more research. A phased approach can allow that.

I’ll leave you with one final thought. To learn from trial and error, you need to be willing to fail. So, it’s important to manage expectations. Also, remember to not let yourself get too discouraged when your first try at something goes very badly. Remind yourself that it’s all part of the process. Early failures are the price of ultimate success.

Once, as a brand manager, I was involved in qualitative testing of four advertising directions for an upcoming campaign to identify the strongest one for full-up production. Unfortunately, no clear winner emerged. After the final focus group, we agonized about what to do. Finally, the moderator piped up: “You people are crazy. What did you expect? Did you really think such a small number of consumers could provide that kind of guidance?” He went on to point out that one ad was clearly a dog and one was mediocre, leaving us with two that delivered on all aspects of the creative strategy. He told us, “You’ve got two good options. Just make a decision. As a group, you’re fully qualified to make the call.”

This was a good lesson for me, one that I have taken to heart ever since. Instead of trying to crown a champion, eliminate the losers and select a ‘winner’ from what remains.

I conduct a lot of qualitative research that involves showing creative stimuli. These include things like package designs, advertisements, and new product concepts. It’s common for clients to set identifying the strongest execution as a goal for this type of research. This is an understandable objective, but often not realistic.

Picking the single best performer among several possibilities requires both precision and accuracy to have confidence in your results. Qualitative research, much as I love it, isn’t great for either of those things. Not only are the samples too small, but the research approach doesn’t lend itself well to these goals. The sample is not designed to be balanced and projectable. And the data gathering methods, which are informal and exploratory, don’t produce the kind of consistent data needed for accuracy and precision.

As we all know, qualitative research shines when your goal is to explore foundational issues and develop hypotheses for further testing. It is that additional testing, particularly if it is properly designed quantitative research, that will provide accurate, precise, statistically reliable data.

While qualitative research usually isn’t up to the task of identifying the single strongest option, it’s more than adequate for eliminating the weaker ones. In addition, it can provide detailed and nuanced insights about each execution to inform the client’s process for making a final decision. That way, clients should feel confident in their ability to choose their winner based on the research.

If clients feel they must select a winner, include such evaluative criteria in the discussion guide as relatability, sense of urgency and main message playback. Also, have clients identify any non-research-based criteria that can also be considered. These could include things like strategic considerations, competitive environment, consistency with past marketing activities, etc. You might also want to consider a hybrid approach in which the qual informs more in-depth quantitative insights.

Regardless of your approach, it’s a good idea to agree – beforehand – on what the tiebreaker criteria will be for final selection. And be sure everybody understands the limitations of qualitative research in picking winners when testing marketing concepts and creative stimuli.

My kids were always making promises when they were young. Some they kept, some they didn’t. But they never broke a promise sealed with a pinkie swear.

That’s partly because they wouldn’t pinkie swear unless the promise was one they were pretty much positive they would keep. But the ritual of the pinkie swear also clearly pulled a lot of weight.

And that’s the thing about rituals. They have a deceptive power. Marketers tend not to focus on them, and that’s a shame—they’re missing a big opportunity. As a qualitative researcher, I’ve always made a point of looking out for rituals, because they’re often a clue to things of hidden importance.

As a case in point, a few years ago I was conducting research among men being treated for a medical condition with a multi-drug therapy. During the interviews it came out that quite a few of the patients said a prayer when they took one of the three drugs. After a lot of probing as well as supplemental interviews with the doctors who treated them, we learned that the patients were aware that this drug was the least effective of the three. And, of the three drugs, this one had the highest rate of compliance. Patients almost always took it. Had we not noticed the ritual around that one pill, we wouldn’t have discovered some pivotal facts about the patient experience.

So, whether you’re developing marketing strategies or conducting qualitative research, it’s crucial to be aware of rituals. They’re precious gifts from research participants that should be honored and studied closely.

There are four key things to know about rituals.

Rituals are more than mere habits. Habits exist for their own sake – I always lock the door when I leave my house because I want the door locked. Rituals are behaviors that have a higher purpose; my wife and I always share a gin and tonic on the first day of spring because we want to celebrate that winter has ended and we got through it together. So, a ritual elevates a behavior from something commonplace to something profound, even sacred.

Rituals are signals. They are clues that something important is happening. Sometimes a ritual is a sign that something seemingly insignificant is actually profound. Or, sometimes, it’s a way people transform something ordinary into something special. Whether they signal significance or create it, rituals tell us that something’s going on, and that we need to pay attention. They can also be indicators of what people believe and value—there’s no point in adding ceremony to something you don’t care about.

Rituals mark transitions.  When a ritual occurs, something essential has shifted, such as:

Physical location

Time period

Social or economic circumstances

Type of occasion

State of mind

Life stage

So, in marking change, rituals create structure and provide transitions with importance. Furthermore, because transitions can be scary, sometimes the role of a ritual can be to provide comfort.

Rituals create a sense of community. They are often shared experiences that bind us together. And they can provide valuable information regarding relationships among individuals—people who share rituals may be more than just casual acquaintances.

So, if marketers and researchers want to uncover what’s actually important, they must look for rituals. Here are some ways to discover and explore rituals:

Train yourself to watch and listen for them. Use the discussion guide to remind you to be on the lookout. This is particularly important when working out in the world, whether you’re in a store, a home, on a street corner, wherever. In fact, one of the most important reasons to conduct research in such places is that you can see rituals right before your eyes.

Don’t be shy about openly asking about rituals.

“What are some rituals you follow on this sort of occasion?”

“Do you have any habitual behaviors associated with this situation?”

“Are there certain practices you always follow with respect to this?”

Look for practices, objects or locations that seem to serve no obvious practical purpose. If, when asked about them, participants have difficulty articulating their reason for being, that’s a pretty good clue you may have uncovered a ritual. Remember—rituals are often private things that carry emotional weight, so probe with sensitivity. It is essential to treat rituals we discover with respect and show our appreciation to those sharing them.

Create process maps. An exhaustive diagram of an occasion that clearly details all its component behaviors enables us to distinguish the elements that have a practical function from those that serve as rituals.

I’ll leave you with one final thought. So many cherished rituals have been taken from us this past year—graduations and proms, religious observances, funerals and weddings, cultural events. This has created a sense of dislocation. We feel untethered without these rituals to provide structure and mark time and space. And so, we’ve been creating rituals, such as 5 o’clock cocktails and zoom yoga. I know people who, for the first time in their lives, have started saying grace at meals. In the absence of old rituals, we’ve created new ones. Human beings are unable to function without rituals in their lives. And that’s why marketers need to seek them out.

My thanks to my friend and colleague Alicia Menanteau, PhD for her input on this post.