One of the 20^th century’s great philosophers, Mike Tyson, said that. And while I definitely wouldn’t argue with Mike, I’m a big advocate of planning data analysis. If you know me, you’ve probably figured out that I don’t think analysis gets enough attention when it comes to qualitative research. We spend a lot of time planning how we will recruit and conduct research, but then take a seat-of-the-pants attitude towards analyzing our data. And this is a problem, because not planning your analysis can profoundly compromise the value of your research.

Qualitative analysis can be divided into five stages:

• Planning

• Debriefing

• Consolidation

• Unstructured musing

• Structured analysis

I’ll provide my thinking on some of these analysis stages in future newsletters. I’ve already written a piece on consolidation but in this one I’m going to drill down on Planning.

An analysis plan should be a part of any comprehensive research design, qualitative or quantitative. In fact, the more effort you put into planning your analysis while you’re designing your research, the better your analysis will be. Planning has become even more important in recent years, as timelines get shorter, and as researchers are increasingly expected to analyze data from a variety of sources. With all that in mind, here are eight key best practices to follow when planning your analysis:

Consider research objectives and resultant decisions

Every study is (or, at least, should be) driven by business issues. Typically, clients have one or more decisions they will make on the basis of the research findings. Your design and analysis plan should directly address each of these questions and decisions. If it doesn’t show clearly how the data you gather will answer those questions and guide those decisions, something’s missing.

Set action standards

Action standards tell you what you need to see in your data in order to trigger a specific decision. They often take the form of thresholds for metrics like brand awareness, purchase intent, preference, and so forth. It’s crucial that action standards be established BEFORE the research—it’s not OK to eyeball the data after the fact and settle upon action standards then.

Provide structure to your data

Qualitative research is different from quantitative in that the data is, by its nature, less structured. Nevertheless, there are ways to provide some structure, and the more organization you can give to your data, the easier your analysis is going to be. Some ways to provide structure for in-person and webcam research include:

• Written reaction exercises

• Respondent worksheets and polling questions to capture reactions to stimuli

• Respondent markups of research stimuli

• Waiting room questionnaires

• Note-taking worksheets or easel pads for observers and moderators

For online bulletin boards, structure is even more important, as they yield so much data (I conducted a board late last year that generated a 1,200 page transcript and nearly 700 images and videos). Some ways to organize that data include:

• Tagging participants prior to the research on the basis of demographics, attitudes and behaviors (such as ‘three or more kids in home,’ ‘concerned with environmental sustainability,’ or ‘online category shopper.’)

• Tagging responses as they come in, such as ‘concept positive,’ ‘brand negative,’ ‘concerned about cost,’ etc.

• A.I. enabled textual analytics can be invaluable for surfacing trends from the data that you otherwise might never pick up.

Furthermore, how you divide up the sample for your research isn’t just important for group dynamics. Splitting up your groups or boards by demographics or behaviors can make your data better organized, as you’ll have more coherent and focused conversations.

Identify possible tools

It’s important to apply analytical models and frameworks to your data. Thinking in advance about what tools might be relevant could influence how you design and conduct your research. Exactly what tools you’ll end up using might change once you’ve gathered your data, but considering tools up front will give you the opportunity to plan conversations and exercises that will lead to fruitful analysis.

Focus your stimuli

The mark of a good research stimulus, be it a product concept, an advertising storyboard, a packaging prototype, whatever, is that it will generate readable responses. This means focus. Concepts should focus on a single benefit, story boards should focus on one clear selling proposition, and so forth.  Focused stimuli will yield focused, easily analyzable responses. 

Create a deliverable outline

I’m a big believer in drafting a final report outline while designing the study. This is a good way to set expectations as to what the deliverable will cover, and also provides a roadmap for your analysis plan. This outline represents the blanks you’re going to have to fill in, and knowing that in advance is a good way to make sure your research is focused on actionable findings, insights and recommendations.

Think in advance about the role of multiple data sources

The research studies I’m involved with increasingly generate data from a variety of disparate sources.  These can include:

• Conversation

• Creative and projective techniques

• Narratology

• Textual analysis

• Biometrics

• Quantitative survey data

• Syndicated data

• Big data analytics

If you’re going to successfully weave a story together from these sources you must plan for that. This means you’ll need to think about all of the research objectives and resultant decisions, and plan out which of them will be addressed by which data source.

Prepare to be flexible

Dwight Eisenhower once wrote, “I have always found that plans are useless, but planning is indispensable.” Ike’s point was that, while plans nearly always change in the face of new information, the thinking that went into them is still useful. All that creation of structure and identification of tools, etc. still counts, even if things don’t pan out exactly as expected.

As they say in the military, if you fail to plan, you plan to fail. Carefully planning your analysis before you conduct your research will increase the chances of uncovering that brand-redefining insight, and reduce the possibility of a failed study.

For the last several weeks, nearly everybody I know has been spending an awful lot of time video conferencing.  And, in other news, all of a sudden, most of the qualitative research I’m doing these days is taking the form of webcam interviews or focus groups.  This isn’t a big deal for me—I’ve been conducting webcam research for many years now.  The webcam is a great tool, one that solves a lot of logistical problems, and that can provide highly insightful findings.  But I’d like to take this opportunity to remind you of the existence of one of the oldest of the old-school qualitative research tools: the telephone interview.  There are a lot of things about phone interviews that make me love them.

They’re expedient!

Logistically, phone interviews are simpler to schedule than any other approach—nearly everybody has access to a phone pretty much anytime, and just about everybody knows how to use their phone – even my mother.  What’s more, lots of people have access to multiple phones, decreasing the possibility of scheduling and technical problems.

They’re agile!

If you’re looking to conduct research quickly and flexibly, look no further than the telephone interview.   Phone interviews can often be scheduled on the spot.  I recently did a study among medical practitioners. Nearly half of them, upon completing screening, said that they were available right away.  We were able to pair just about all of them with an interviewer immediately, increasing our recruiting yield and shortening our project timing.  So, if ‘agile’ research is something you seek, the phone interview can be a great way to achieve that.

They’re foolproof!

Compared with most online platforms, a low-tech phone call offers far fewer things that can go wrong.  Aside from a battery going dead or somebody losing their signal, there aren’t too many bad things that can happen.  And, as I mention above, respondents frequently have access to another phone—that’s a nice insurance policy.

They’re cheap!

Platform costs can range from free to $1.00/minute at the most.  Compared to many online market research platforms, this is a bargain.  And recording is a snap.  Many conference calling platforms offer free, high quality archived recording.

They’re profound!

The phone interview’s most important virtue may be depth of insight. Think about your own experiences. Have you ever had a long, deep, intimate phone conversation with a friend? Of course you have! In fact, you’ve probably had lots of those. That’s because a phone call strips away distractions and anxieties. There’s just the sound of two people’s voices. Nobody has to worry about all the aspects of how they look (and let’s face it – everybody’s hair looks terrible these days), and nobody gets distracted by the appearance of the other person or any other visual elements.

Conducting research on the phone lowers the stakes, reduces the sense of expectation, and makes people feel less inhibited. In addition, there’s no anxiety due to unfamiliar technology.  This can make telephone interviews the best option for emotionally-loaded topics: sensitive healthcare issues, politics and personal finance to name just a few. It’s also good to bear in mind that phone interviews are perfect for studies that don’t require you to expose complex stimuli.  In other words, if what you primarily need to do is engage in deep conversation with your research subjects, a voice-only platform might meet your needs better than any other approach, including face-to-face interviewing.

So, this time of social distancing might offer a valuable opportunity to reacquaint yourself (or get acquainted in the first place) with this venerable research technique. Consider phone interviews for studies in which:

You need to compress timing

Money is tight

You don’t need to present complex stimuli

You’re faced with a low incidence recruit

You’re researching sensitive, emotionally loaded issues

With all that in mind, here are a few tips and best practices for getting the most out of telephone interviews:

Who calls whom? Participants don’t always want researchers to have their phone numbers (and vice versa). In addition, this might constitute personal information, which can place researchers and clients at risk. So, I typically have everybody dial into a conference calling service.  While most respondents can dial-in at no additional cost to themselves, it’s important to be sensitive to this issue. So, I always make sure there is a toll-free dial-in option available to the participant. And, for studies involving lower-income participants, I’ll just make toll-free the default option.

Anybody listening in? Do your clients want to monitor some or all of your interviews? If so, you’ll need to inform your participant that this is going on. In addition, having a way for your clients to communicate with you during the call might be a good idea.  If you’re using a conference calling platform— such as freeconferencecall.com or Zoom (with the webcams turned off)—this capability may be built in. If not, text messaging might work well.

Any distractions present? Remind your participants at the beginning of the call to take a minute to eliminate any possible distractions from their environment. This could mean closing the door to the room they’re in, and closing any distracting browser tabs and applications. In addition, it’s important that recruiters confirm when scheduling calls that the participant will be able to conduct the interview undisturbed.

How reliable is your phone? Whenever possible, I conduct phone interviews from a landline. I know this isn’t as cool as my smartphone, but the sound quality is much better, and the possibility of the call being dropped on my end is pretty much nil. I also encourage the respondent to do the same. If using my smartphone, I’ll make sure it’s fully charged, that all sounds and alerts are muted, and that the internet connection is disabled to the greatest extent possible.

Do you need to show stimuli? As mentioned above, if you need to share complex stimuli, phone interviews are probably not your best research tool. But it’s easy to show simple stim, such as written concepts, print ads and 2D packaging renderings. The cleanest solution is emailing them to the participant, if that’s feasible. If not, it’s not difficult to set up a secure file sharing link through the cloud services offered by Google, Microsoft, Dropbox and others. Teleconferencing services might also present a workaround for this problem.

Nobody can see you. So, remember that no meaning or context will be conveyed by either you or the participant through facial expressions, body language or hand gestures. As a result, such subtleties as humor, irony, sarcasm and sadness might not come across. This can create some risk of misunderstandings, so it’s important to regularly repeat back to your respondents your understanding of what they have said for verification.

So, don’t forget.  The telephone interview is your friend. Used wisely, it can enable research that is agile, cost-effective and – most importantly – insightful.  It’s easy to forget about the phone, now that we have so many powerful, high tech, feature-rich solutions available to us.  But, sometimes, it’s still the best tool for the job.

Here’s a fundamental principle of market research: the less your participants know about the true purpose of your research, the better the quality of the data you’re going to collect.

Why?  When participants think they understand the business issues and information objectives underlying your research, they’re likely to take that as a license to think like marketers, copywriters or managers – instead of just being themselves.  What’s more, sometimes knowing what the study is about will lead them to try to tell you what they think you want to hear.  They might also try to ‘game’ the research in order to achieve some purpose of their own (such as lower prices), or engage in ‘virtue signaling,’ such as waxing rhapsodic about their vegan lifestyle because they think that’s what they’re supposed to say.  In additions, if research participants don’t know the topic of the research, it allows that topic to come up on its own—or not.  If, when, and how key discussion areas enter the conversation organically can be a key source of insight.

This is not to say that I’m opposed to transparency.  On the contrary, I believe that researchers and marketers have an obligation to be fully transparent with research participants, users and consumers.  It’s a question of timing.  While it’s important that participants eventually understand a research study’s purpose and who’s behind it, they don’t need to have that information from the very beginning.  That being the case, while it’s necessary to begin with your purpose obscured, it’s also important to end in a place of transparency.

So, at least at the beginning of your research, treat your participants like mushrooms; keep them in the dark.  With that in mind, here are some thoughts and best practices on how to do that.

• It all starts with recruiting.  When you’re screening for research – whether qualitative or quantitative – make sure that your questions won’t tip off the participants.  This will almost certainly make your screening questionnaire longer, but it’s worth it.  If you’re going to screen out respondents on the basis of attitudes towards the client’s brand, bury that brand in a list of brands.  If you’re screening on the basis of category usage, include the client category in a list of categories.  I will often have someone who is not familiar with the research review the screener to see if she can correctly identify the category, client or topic. Another step to consider – if possible, tell the recruiter as little as possible about who the client is and the topic of the research.  What they don’t know they can’t spill.

• Brief the facility or platform provider.  It’s very easy for somebody at the focus group facility or online research platform you’re using to let the cat out of the bag unintentionally.  If they understand how imperative it is to keep the client and topic confidential, they’ll be less likely to do this.  And – as with recruiters – if you can avoid sharing this information, that might be a good idea.

• When conducting qualitative, how you phrase questions is crucial.  One of my favorite questioning techniques is what I call the ‘Bankshot Question.’  I use it all the time for a variety of reasons.  The idea is that sometimes the direct approach isn’t the best; depending on the situation, a direct question can feel rather confrontational.  An oblique style is helpful when dealing with sensitive topics, or for situations where people might not want to share their honest opinions.  And it’s also a great way to avoid revealing the true purpose of the research.  So, rather than asking the actual question you have in mind, ask a closely related question and see if you get the answer to what you really wanted to ask.  For example, instead of asking moms what they currently feed their kids, ask what they – themselves – are eating these days.  Or instead of asking what participants think about the Affordable Care Act, ask them what they like and dislike about their current health insurance situations.  One bankshot technique I find to be particularly effective is asking for stories.  So, instead of having a participant explain how he purchases power tools, I’ll ask him if he has a favorite story about a home improvement project.  If you’re going to try this approach, be aware that it’s an iterative one that takes some patience.  You might need to attempt more than one bankshot before you put the ball in the pocket.  You might also need to let respondents ramble a bit so that they have the time to get to what you really want to hear about.  This is where silence is your friend.  Just sit quietly, smiling and nodding.  Your participants will feel the need to fill the silence, and might give you the information you seek.

• Be prepared to deflect questions.  There has been a general societal trend toward greater expectations of transparency on the part of governments, companies and other institutions.  As such, I find that participants are more willing than they used to be to ask me directly who my client is, and about the specific purpose of the research.  While it may have once been acceptable to ignore these sorts of questions or provide a vague response, that’s no longer OK.  Now, when I get questions like this, I reply that I’m happy to answer them – but not just yet.  However, I always to make sure that I provide those answers before the group or interview is over.

So remember, don’t let the cat out of the bag before you have to.  The longer you can keep your participants from figuring out what your research is about, the better the accuracy of the data you’re going to gather, and greater the quality of the insights it’s going to reveal.

I’ve been encountering the term ‘self-care’ a lot lately. People seem especially preoccupied these days with seeing to their own needs. This concern encompasses numerous dimensions: physical, emotional, spiritual. There’s nothing wrong with this. These are trying times, and who better than you to make sure your needs are being met? I’m certainly part of this trend—just ask my wife. She’ll tell you that I’ve been indulging myself a bit more than usual recently. That said, a focus on taking care of yourself has its limits. While attending to your own needs is certainly important, it’s only half of the task. Because, if you really want to fully care for yourself, you must orient your efforts towards others.

The good news is that, right now, there’s no shortage of people who desperately need your help. So many are isolated, lonely and anxious. So many more are in financial distress. If you direct time, attention and other resources towards their challenges, not only will it help them, it will help you as well. Some specific things you could do:

• Contribute to your local food bank.

• Check in on your neighbors who are particularly vulnerable. Do they need groceries? Is their recycling brought to the curb? Are prescriptions they need being picked up?

• How about the small businesses in your town? In addition to patronizing them to the extent you can, would they benefit from your expertise as a marketer or researcher?

• And don’t underestimate the value of just writing a check. A lot of service organizations really need that more than anything else.

John Donne was correct nearly 400 years ago – none of us is an island. We’re all connected, all a part of a greater whole. Another person’s happiness or suffering is also yours. The true path to well-being lies in focusing yourself on the needs of your fellow humans. Because there’s more to who you are than just you.  Fully caring for yourself requires taking care of others.

I’m going to tell you story.  It’s about qualitative research, but it might not seem like that immediately, so bear with me.  It’s a story about my friends Betsy and Missy.  I’ve known them both for over 25 years.  But they’ve known each other a lot longer.  Because, at the start of their first day of kindergarten – right after Betsy sat down at her desk – Missy, who was a perfect stranger, turned around and asked Betsy, “can we be friends?”  Betsy said, “yes.”  And, all these years later, they’re still close friends.

Why am I telling this story, aside from the desire to tug at your heartstrings? Because it illustrates the boundless potential of each moment we experience.  Who would have expected a simple question posed by one kindergartner to another would result in a decades-long, thick-and-thin friendship?

Qualitative research differs fundamentally from any other kind of research (quant, syndicated, big data analysis, etc.) in that it is a string of moments we experience directly and personally; it is really just a fancy term for conversation.  Think about your own experiences; just about everybody can recall life-changing conversations they’ve had at various points in their lives.  Conversation is – by its nature – unstructured and unpredictable.  While you might know where a conversation will begin, you have no idea where it’s going to end up.

And so, every moment in qualitative research carries unlimited promise.  Every instant has the potential to create something game-changing.  Whether it’s a participant describing her relationship to a brand in a way no member of the brand team has ever considered, or an anecdote related by a shopper that finally makes it clear why shelf movement has dropped so sharply in the past six months, conversation leads to unexpected places that reveal things that are entirely new.

This means, that if we design with this characteristic of unlimited possibility in mind, any qualitative study can reveal unexpected, extraordinary insights.  But this doesn’t happen by itself.

So, when designing qualitative, it’s important to make sure that your design allows for this possibility.  Here are 6 principles and techniques that can increase the chance of having one of these moments:

1. Stay away from a box-checking mindset. Qualitative research isn’t about methodically working your way through a list of discussion topics and making sure you ask all the predetermined questions.  It’s about feeling your way through a dark, unfamiliar landscape and discovering what’s there.

2. Ask open ended questions. Closed ended questions are for quant.  Or, if you’re trying to establish some sort of objective baseline, ask closed ended questions as part of the homework, in the waiting room, or put them in your screener.

3. Get diversity in your participants. If your participants are recruited to be alike, they’re probably going to give you a limited range of points of view.  So diversity is a virtue: economic, ethnic, attitudinal, demographic, experiential, the list goes on and on.

4. Get diversity in your data. In addition to conversation, projective, creative and story collection techniques can provide unexpected insights.  And, if you’re looking to push the envelope a bit more, consider tools like biometrics and textual analysis.  I also like to diversify my data by mixing interaction methods; group discussions, IDIs, online bulletin boards, intercept interviews and social media listening can all peacefully coexist in one study.

5. Allow time for tangents and discussion of new issues. If you set too many guardrails around the course of discussion and don’t permit participants to talk about what they want to talk about, you’ll never wind up someplace new.  So let people ramble a bit.

6. Prepare for unexpected insights to be uncovered after the fact. Whether you’re observing or moderating research, you don’t always realize in the moment that something important just happened.  That’s one reason to make sure the research is fully documented and recorded.  Transcripts and good-quality videography can go a long way towards ensuring your post-analysis is as successful as it could be.  And relying on a single person to analyze the findings and create the final deliverable may not be the best idea.  Analysis and reporting – at its best – is a team sport.  The more eyes, ears and brains you have reviewing the data, the better the chance that somebody will spot something remarkable.

Mr. Richter – my high school biology teacher – used to tell us incessantly that Louis Pasteur wrote ‘fortune favors the prepared mind’.  In order to have that lucky, blinding flash of insight, you have to lay the groundwork for it.