June 10, 2021
Given the proliferation of data mining and research tools and tactics in digital, we thought it worthwhile to break down a handful of the more popular methods for qualitative research currently used online, what they’re great for, what they’re not so great for, and some tips on how to get the most from each.
This is far from an exhaustive list, and there are further distinctions to be made between evaluative and generative research tactics (we’ll get more into this in a later post), but for now we want to touch on four.
Qualitative research helps to answer the “why?” and “how?” questions. For example, why are users finding your checkout process confusing? How do users decide to purchase candy online? Qualitative research contrasts with quantitative insights gained from analytics, surveys, or other passive data generation techniques. Quantitative techniques produce large sample sizes and numbers that help reveal the presence of a problem or the magnitude of it. On the other hand, qualitative research utilizes smaller sample sizes to uncover deep insights about your users, website, or product that you cannot get from looking at numbers alone.
Qualitative research allows you to get much richer insights, such as why a user did something. You can reveal specific errors, ask people what they were thinking while on a website, and get rich feedback beyond a rating scale.
Qualitative and quantitative research have strengths and weaknesses, so research that combines the two is best. For example, qualitative research can generate insights that can be validated by looking at quantitative data and determining how many users encountered this problem. Conversely, quantitative data such as analytics can show where to use a qualitative study to understand user behaviors and motivations.
Here are explanations of four qualitative methods, pros and cons, and recommendations for your organization.
Usability tests are the most popular qualitative method in UX/CX research. The amount of insight you get simply from watching a user use and react to your product or website is astounding. People can get bogged down in the design process trying to understand what a user needs and wants. A quick usability test with a small number of users can reveal critical problems with a product that you would never catch on your own.
Usability testing is simple but powerful. First, experimenters give users directions to interact with your product or website while thinking aloud. Then researchers watch and note as users attempt to complete the task. Their successes, failures, and thoughts are all recorded.
Moderated user tests include a researcher who directly interacts with the participant and asks questions, where unmoderated user tests are conducted without a human moderator using software such as Usertesting, Trymyui, or Lookback. Both offer unique strengths and weaknesses, so determine which works for you.
User interviews are often a great way to learn more about your users, their lives, and their thoughts about your products. Interviews work great at the beginning of a new project to develop empathy and drive design decisions. Talking to actual users keeps the process centered around human needs and wants rather than business or development goals.
To conduct user interviews, create a script of questions that you want to know more about your users. However, don’t be afraid to deviate from the script and pursue any interesting things they said. The true strength of an interview lies in delving deeper into answers and asking follow-up questions. If you ask “why” enough times, you will get beyond superficial answers and learn more about the actual problems that your users want to solve.
Competitive research encompasses any investigations that systematically evaluate competitors in relation to your product. Every new project should start with competitive research. You need to know what’s out there to make something truly innovative. One of the simplest and most effective ways to conduct competitive analysis is to create a matrix of relevant features that you and your team can fill out based on your observations. More intensive methods such as competitive usability tests or impressions tests can reveal deeper insights into how participants interact with your competitors. Use these findings to focus your next round of design development or research.
Surveys are traditionally a quantitative research method, but they can be used to gather qualitative insights as well. Open-ended questions and free-response allows users to speak their mind. With text surveys, you can leverage powerful text analysis tools to conduct sentiment analysis or extract themes. You can also easily weave in quantitative questions to get the best of both worlds.
The wording of questions is especially important so that you can get the correct data. You also need to realize that what people say on a survey and what they actually do can be very different. I find surveys very useful for gathering quick information about preferences, but less accurate about actual behavior. Surveys can be a source of good qualitative data, but the true strength of surveys is the ability to gather a large sample size and make inferences using statistical tests.
If you’re interested in learning more about how to put qualitative research to work in support of improving your firm’s customer experience, don’t hesitate to drop us a quick note at firstname.lastname@example.org.