04 Mixing Methods in UX Research

Learning with Lee Anne

I graduated with an HCI degree and a minor in design in 2018. I (naively) believed that I was fully equipped to understand UX in business, outside of school, but it turns out — I wasn’t! (Surprise!) Luckily, we never stop learning due to our peers, mentors, and random articles on the internet like this one. Since this is a time where my vulnerabilities and struggles are leading me to better understand research, I thought I’d document my learning process of how UX Research works in my everyday world of business!

A large part of UX Research is the methods that are used to conduct studies. There’s individual interviews (IDI), surveys, diary studies, ethnographies — just to name a few. Each of these methods have their own strengths, so when combined, it can be a very powerful way of getting a 360 view of the insights.

Here are some examples of mixed methods I’ve used before and why they were so useful together:

Diary Studies + Individual Interviews

Project example: There was a research need to figure out the likability and effectiveness of a particular stimuli that signaled information about Diabetes. Over the course of the week, stimuli was sent to participants through an app throughout the day. For each stimuli, they were required to answer a short survey about their current environment and reactions to the stimuli. Additionally, we also had 30 minute individual interviews with each of the participants before and after the three-day long diary study.

Why mixing methods was helpful:

• The diary study allowed us to receive feedback in a realistic setting

The strength of diary studies is that it allows us to gather feedback in real time and in real life. Because this stimuli would realistically play throughout a person’s day when they’re not paying attention, we wanted to emulate this circumstance in the study in order to accurately determine if the stimuli was good enough to do the job in a real world setting. Solely having individual interviews would not have reaped the same valuable feedback because replicating that experience in a static lab setting would not have been realistic.

• The individual interviews enhanced the “why”

The strength of individual interviews is that it allows for a conversation with the participant.Having participants explain things to you live as opposed to through a video or online forum is much more powerful. Being able to probe and clarify with the participant helps the team understand the “why” of participant responses.

This research started off with an obvious need for a diary study, but as we continued to understand the goals of the research, we found value in incorporating individual interviews. When planning the research design, be open to the idea of creatively achieving results, which sometimes means mixing methods. Overall, for us, this dynamic procedure produced a better understanding and reliability of the research results.

Surveys + Individual Interviews

Project example: There was a research need to figure out the usability of some screens as well as the value of the information on them. For this project, we had participants take a survey that included an unmoderated usability test and a few additional questions. They were required to take this survey before their interview time, which was a 45-minute session that went into depth about a few of the survey questions and also looked at a few more screens.

Why mixing methods was helpful:

• The survey allowed for more ground to be covered in the IDIs

One of the strengths of surveys is that it allows people to process questions and tasks on their own without the pressure of a moderator watching over them. Because there were lots of usability questions that required people to process the screens, it was a better idea to do the bulk of that in an unmoderated setting. This also allowed the IDIs to be shorter (an advantage in terms of keeping people’s attention) and more focused on the conversations of what people liked/ didn’t like/ and what they understood/ did not understand.

• The individual interviews enhanced the “why”

Just as in the example before, individual interviews helped enhance the reasons for the results. Additionally, people are generally less inclined to be more descriptive in open-ended comment boxes in surveys so betting on explanations in surveys alone can be ineffective. Knowing the reasons people were not able to complete certain tasks or why they answered the way they did was very important to understand and probe on for this project.

Using these methods in conjunction with one another made things more efficient for both the participants and the moderator since we were able to cover a lot of ground without taking up too much time. Sometimes with complicated timelines and schedules, having a mix of methods can help solve for that.