By Nate Brucher

Given the proliferation of data mining and research tools and tactics in digital, we thought it worthwhile to break down a handful of the more popular methods for qualitative research currently used online, what they’re great for, what they’re not so great for, and some tips on how to get the most from each. 

This is far from an exhaustive list, and there are further distinctions to be made between evaluative and generative research tactics (we’ll get more into this in a later post), but for now we want to touch on four. 

What is qualitative research?

Qualitative research helps to answer the “why?” and “how?” questions. For example, why are users finding your checkout process confusing? How do users decide to purchase candy online? Qualitative research contrasts with quantitative insights gained from analytics, surveys, or other passive data generation techniques. Quantitative techniques produce large sample sizes and numbers that help reveal the presence of a problem or the magnitude of it. On the other hand, qualitative research utilizes smaller sample sizes to uncover deep insights about your users, website, or product that you cannot get from looking at numbers alone.

Qualitative research allows you to get much richer insights, such as why a user did something. You can reveal specific errors, ask people what they were thinking while on a website, and get rich feedback beyond a rating scale.

Qualitative and quantitative research have strengths and weaknesses, so research that combines the two is best. For example, qualitative research can generate insights that can be validated by looking at quantitative data and determining how many users encountered this problem. Conversely, quantitative data such as analytics can show where to use a qualitative study to understand user behaviors and motivations.  

Here are explanations of four qualitative methods, pros and cons, and recommendations for your organization. 

Usability Tests

Usability tests are the most popular qualitative method in UX/CX research. The amount of insight you get simply from watching a user use and react to your product or website is astounding. People can get bogged down in the design process trying to understand what a user needs and wants. A quick usability test with a small number of users can reveal critical problems with a product that you would never catch on your own.

Usability testing is simple but powerful. First, experimenters give users directions to interact with your product or website while thinking aloud. Then researchers watch and note as users attempt to complete the task. Their successes, failures, and thoughts are all recorded.

Moderated user tests include a researcher who directly interacts with the participant and asks questions, where unmoderated user tests are conducted without a human moderator using software such as Usertesting, Trymyui, or Lookback. Both offer unique strengths and weaknesses, so determine which works for you. 

Pros:

- Small sample sizes, but deep insights

- Observe actual behavior

- Moderated settings allow follow up questions

- Can be used at any stage of the design process, from prototypes to fully formed products

Cons:

- No quantitative insights, which means it is hard to generalize to a larger population of users

- Can take longer to analyze

Tips:

- Make the tasks as realistic as possible. For example, if you ask someone to purchase something, consider reimbursing them for actually sending the item to their address.

- Make sure users speak aloud so you can get an idea of what they’re thinking in addition to their behavior. 

User Interviews

User interviews are often a great way to learn more about your users, their lives, and their thoughts about your products. Interviews work great at the beginning of a new project to develop empathy and drive design decisions. Talking to actual users keeps the process centered around human needs and wants rather than business or development goals. 

To conduct user interviews, create a script of questions that you want to know more about your users. However, don’t be afraid to deviate from the script and pursue any interesting things they said. The true strength of an interview lies in delving deeper into answers and asking follow-up questions. If you ask “why” enough times, you will get beyond superficial answers and learn more about the actual problems that your users want to solve.

Pros: 

- Rich data from the mouths of users

- Opportunity to explore deep and ask follow up questions

- Great for exploring new opportunities or learning the background of your users (generative)

Cons:

-Can lack realism, it is best to provide concrete situations and examples to evoke context

- Requires more analysis than a simple survey

Tips:

- Always ask open-ended questions that allow the participant to elaborate and expand on their answers

- Provide visual, auditory or other contextual cues to help them remember things

Competitive Research

Competitive research encompasses any investigations that systematically evaluate competitors in relation to your product. Every new project should start with competitive research. You need to know what’s out there to make something truly innovative. One of the simplest and most effective ways to conduct competitive analysis is to create a matrix of relevant features that you and your team can fill out based on your observations. More intensive methods such as competitive usability tests or impressions tests can reveal deeper insights into how participants interact with your competitors. Use these findings to focus your next round of design development or research. 

Pros:

- Learn from others, good and bad

- Broaden your thinking and design

Cons:

- Can lead to copying for competitive parity, as opposed to actual innovation

Tips:

- Include a wide range of competitors (near and far), some of the best insights come from companies that don’t necessarily compete with you but provide a great customer experience

- Create a matrix to keep track of your insights and features to easily compare among competitors 

- Consider doing actual studies, such as usability testing, surveys, or impression tests to get more data on competitors and your own products

Surveys

Surveys are traditionally a quantitative research method, but they can be used to gather qualitative insights as well. Open-ended questions and free-response allows users to speak their mind. With text surveys, you can leverage powerful text analysis tools to conduct sentiment analysis or extract themes. You can also easily weave in quantitative questions to get the best of both worlds. 

The wording of questions is especially important so that you can get the correct data. You also need to realize that what people say on a survey and what they actually do can be very different. I find surveys very useful for gathering quick information about preferences, but less accurate about actual behavior. Surveys can be a source of good qualitative data, but the true strength of surveys is the ability to gather a large sample size and make inferences using statistical tests.

Pros

- Easy to create and collect data

- Can get a mix of qualitative and quantitative

- Analysis of text is powerful and easier than voice

Cons

- Lack of realism

- Does not measure behavior

Tips

- Always pay attention to the wording of questions, make sure they are clear, and not asking users to answer two things at once (double-barreled questions)

- Keep questions focused on preferences rather than behaviors

If you’re interested in learning more about how to put qualitative research to work in support of improving your firm’s customer experience, don’t hesitate to drop us a quick note at hello@cxperts.io.