One of the 20th century’s great philosophers, Mike Tyson, said that. And while I definitely wouldn’t argue with Mike, I’m a big advocate of planning data analysis. If you know me, you’ve probably figured out that I don’t think analysis gets enough attention when it comes to qualitative research. We spend a lot of time planning how we will recruit and conduct research, but then take a seat-of-the-pants attitude towards analyzing our data. And this is a problem, because not planning your analysis can profoundly compromise the value of your research.
Qualitative analysis can be divided into five stages:
-
Planning
-
Debriefing
-
Consolidation
-
Unstructured musing
-
Structured analysis
I’ll provide my thinking on some of these analysis stages in future newsletters. I’ve already written a piece on consolidation but in this one I’m going to drill down on Planning.
An analysis plan should be a part of any comprehensive research design, qualitative or quantitative. In fact, the more effort you put into planning your analysis while you’re designing your research, the better your analysis will be. Planning has become even more important in recent years, as timelines get shorter, and as researchers are increasingly expected to analyze data from a variety of sources. With all that in mind, here are eight key best practices to follow when planning your analysis:
Consider research objectives and resultant decisions
Every study is (or, at least, should be) driven by business issues. Typically, clients have one or more decisions they will make on the basis of the research findings. Your design and analysis plan should directly address each of these questions and decisions. If it doesn’t show clearly how the data you gather will answer those questions and guide those decisions, something’s missing.
Set action standards
Action standards tell you what you need to see in your data in order to trigger a specific decision. They often take the form of thresholds for metrics like brand awareness, purchase intent, preference, and so forth. It’s crucial that action standards be established BEFORE the research—it’s not OK to eyeball the data after the fact and settle upon action standards then.
Provide structure to your data
Qualitative research is different from quantitative in that the data is, by its nature, less structured. Nevertheless, there are ways to provide some structure, and the more organization you can give to your data, the easier your analysis is going to be. Some ways to provide structure for in-person and webcam research include:
-
Written reaction exercises
-
Respondent worksheets and polling questions to capture reactions to stimuli
-
Respondent markups of research stimuli
-
Waiting room questionnaires
-
Note-taking worksheets or easel pads for observers and moderators
For online bulletin boards, structure is even more important, as they yield so much data (I conducted a board late last year that generated a 1,200 page transcript and nearly 700 images and videos). Some ways to organize that data include:
-
Tagging participants prior to the research on the basis of demographics, attitudes and behaviors (such as ‘three or more kids in home,’ ‘concerned with environmental sustainability,’ or ‘online category shopper.’)
-
Tagging responses as they come in, such as ‘concept positive,’ ‘brand negative,’ ‘concerned about cost,’ etc.
-
A.I. enabled textual analytics can be invaluable for surfacing trends from the data that you otherwise might never pick up.
Furthermore, how you divide up the sample for your research isn’t just important for group dynamics. Splitting up your groups or boards by demographics or behaviors can make your data better organized, as you’ll have more coherent and focused conversations.
Identify possible tools
It’s important to apply analytical models and frameworks to your data. Thinking in advance about what tools might be relevant could influence how you design and conduct your research. Exactly what tools you’ll end up using might change once you’ve gathered your data, but considering tools up front will give you the opportunity to plan conversations and exercises that will lead to fruitful analysis.
Focus your stimuli
The mark of a good research stimulus, be it a product concept, an advertising storyboard, a packaging prototype, whatever, is that it will generate readable responses. This means focus. Concepts should focus on a single benefit, story boards should focus on one clear selling proposition, and so forth. Focused stimuli will yield focused, easily analyzable responses.
Create a deliverable outline
I’m a big believer in drafting a final report outline while designing the study. This is a good way to set expectations as to what the deliverable will cover, and also provides a roadmap for your analysis plan. This outline represents the blanks you’re going to have to fill in, and knowing that in advance is a good way to make sure your research is focused on actionable findings, insights and recommendations.
Think in advance about the role of multiple data sources
The research studies I’m involved with increasingly generate data from a variety of disparate sources. These can include:
-
Conversation
-
Creative and projective techniques
-
Narratology
-
Textual analysis
-
Biometrics
-
Quantitative survey data
-
Syndicated data
-
Big data analytics