User Research can make a huge difference in the success of a product or service. When we understand the behaviors and goals of our users we are in a better place to server their needs. With so many methods and User Research tools out there, how does a person know which tool to use and when?
Have you ever found yourself trying to hang a picture, only to realize that you don’t know where your hammer is? You dig around in the junk drawer for a moment before realizing you left it out in the shed. Rather than put your shoes on and go get it (cuz, ugh, outdoors), you instead do what any right-thinking person would and grab the next hard object you can find, then spend 20 minutes smashing the nail into a wall with a wrench. This ultimately results in a bent nail, a dinged up wall, and you, resignedly walking out to the shed to go get the right tool.
Applying the wrong method when conducting user research is a lot like hammering a nail with a wrench. While a wrench is a great tool for certain tasks, it’s just not the right one for the job at hand. Similarly, you would not attempt to loosen a bolt with a hammer…unless you were like, the worst mechanic ever.
These days, most people have some idea of what user research is, its purpose, and the value behind it. They get that it helps mitigate risk, validates assumptions, guides the prioritization of features, increases the likelihood of product adoption, etc ..the list goes on and on. Unfortunately, the general perception of user research is often confined to a just a few strategies, such as surveys or product testing. While these tactics are not without merit, and certainly provide a level of both quantitative and qualitative value, limiting the scope of your research initiative to these few strategies will result in many metaphorically bent data-nails. Not all methodologies work for every app, user group, or situation. To successfully understand your users, you’ll need to employ right tool based on the research goals you are trying to achieve.
A new client recently asked us to send out a survey that would help them understand what their users were thinking while trying to accomplish a certain task. Up until that point, this particular client hadn’t engaged with their users very much and this initiative represented a step in a positive direction; a step towards user-centered design. Sounds great right? After all, any time you can talk to actual users is a win, so conducting a survey to understand their needs would probably be helpful. Well…sort of…
We certainly could have sent off a well-crafted survey, asked all the right open-ended questions, and vetted the thing left and right to rid it of any shred of bias. In the end though, it wouldn’t have mattered. Not because surveys are a bad way to gather data, but because in this case, they simply weren’t the right tool for the job.
User Research and Data Types
To understand why this was the case, we first need to understand the type of data we were looking for. According to Indi Young, user experience consultant, author, and founding partner at Adaptive Path, user research data can be grouped into three categories: Preference, Evaluative, and Generative data. Each one of these data types comes with different issues and require different techniques for extraction. Preference data refers to the opinions, likes, and desires of users, generative data relates to the mental environment in which users get things done, and evaluative data pertains to what is understood or accomplished with a tool.
The chart below covers each of these data categories, the best techniques for extracting that type of data, and their ideal uses.
Young, Indi (2008-02-01). Mental Models Rosenfeld Media. Kindle Edition.
What our clients really wanted was to understand how their users think, their philosophies, motivations, and environments; all examples of generative data. As you can see in the above chart, surveys are a great tool for determining user preferences, or providing demographic information, but they aren’t ideal for gaining an understanding of users’ mental environments.
Employing The Right Tool
After further discussion and collaboration with our client, we decided to conduct non-directed interviews with eight of their users.
During the interviews, we stepped back from the client’s current solution and instead asked users to talk about their roles, allowing the conversation to evolve naturally. The user led the discussion, we actively listened, and asked follow-up questions based on what we heard. We used their words, expressions, terms, and tools, being careful not to introduce our own references.
The results were very insightful. By having this open conversation, guided by the user, we were lead down paths we didn’t even know existed. Prior to the interviews, we had no real information about the users’ mental environments. Through our generative research however, we discovered that these users actually had several different roles, with distinct needs, and that they were approaching the problem space from a completely different direction than our client.
Using the right approach, we were not only able to find out how our client’s users organized material, but we also gained a bunch of other valuable business intelligence. After meeting with the client and presenting our findings, we were left with multiple action items we could pursue:
- Explore the distinct user roles
- Prioritize which role to focus on first
- Address our user’s pain point of lack of time
- Add data to our proto-persona and lay the groundwork for building our UX personas
- Use the correct terminology in future prototypes
All information we couldn’t have gotten through a survey.
Moral of the story: Take the time to put on your shoes, go out to the shed, and grab the right tool.
To learn more about UX methodology, check out UX Design: People Over Features, Outcomes Over Output.