Tag Archives: UX Design

Selecting the Right User Research Method

User Research can make a huge difference in the success of a product or service. When we understand the behaviors and goals of our users we are in a better place to server their needs. With so many methods and User Research tools out there, how does a person know which tool to use and when?

User Research tools

Have you ever found yourself trying to hang a picture, only to realize that you don’t know where your hammer is? You dig around in the junk drawer for a moment before realizing you left it out in the shed. Rather than put your shoes on and go get it (cuz, ugh, outdoors), you instead do what any right-thinking person would and grab the next hard object you can find, then spend 20 minutes smashing the nail into a wall with a wrench. This ultimately results in a bent nail, a dinged up wall, and you, resignedly walking out to the shed to go get the right tool.

Applying the wrong method when conducting user research is a lot like hammering a nail with a wrench. While a wrench is a great tool for certain tasks, it’s just not the right one for the job at hand. Similarly, you would not attempt to loosen a bolt with a hammer…unless you were like, the worst mechanic ever.

These days, most people have some idea of what user research is, its purpose, and the value behind it. They get that it helps mitigate risk, validates assumptions, guides the prioritization of features, increases the likelihood of product adoption, etc ..the list goes on and on. Unfortunately, the general perception of user research is often confined to a just a few strategies, such as surveys or product testing. While these tactics are not without merit, and certainly provide a level of both quantitative and qualitative value, limiting the scope of your research initiative to these few strategies will result in many metaphorically bent data-nails. Not all methodologies work for every app, user group, or situation. To successfully understand your users, you’ll need to employ right tool based on the research goals you are trying to achieve.

For Example…

A new client recently asked us to send out a survey that would help them understand what their users were thinking while trying to accomplish a certain task. Up until that point, this particular client hadn’t engaged with their users very much and this initiative represented a step in a positive direction; a step towards user-centered design. Sounds great right? After all, any time you can talk to actual users is a win, so conducting a survey to understand their needs would probably be helpful. Well…sort of…

We certainly could have sent off a well-crafted survey, asked all the right open-ended questions, and vetted the thing left and right to rid it of any shred of bias. In the end though, it wouldn’t have mattered. Not because surveys are a bad way to gather data, but because in this case, they simply weren’t the right tool for the job.

User Research and Data Types

To understand why this was the case, we first need to understand the type of data we were looking for. According to Indi Young, user experience consultant, author, and founding partner at Adaptive Path, user research data can be grouped into three categories: Preference, Evaluative, and Generative data. Each one of these data types comes with different issues and require different techniques for extraction. Preference data refers to the opinions, likes, and desires of users, generative data relates to the mental environment in which users get things done, and evaluative data pertains to what is understood or accomplished with a tool.

The chart below covers each of these data categories, the best techniques for extracting that type of data, and their ideal uses.

Indi Young Research Method Types

Young, Indi (2008-02-01). Mental Models Rosenfeld Media. Kindle Edition.

What our clients really wanted was to understand how their users think, their philosophies, motivations, and environments; all examples of generative data. As you can see in the above chart, surveys are a great tool for determining user preferences, or providing demographic information, but they aren’t ideal for gaining an understanding of users’ mental environments.

Employing The Right Tool

After further discussion and collaboration with our client, we decided to conduct non-directed interviews with eight of their users.

During the interviews, we stepped back from the client’s current solution and instead asked users to talk about their roles, allowing the conversation to evolve naturally. The user led the discussion, we actively listened, and asked follow-up questions based on what we heard. We used their words, expressions, terms, and tools, being careful not to introduce our own references.

The results were very insightful. By having this open conversation, guided by the user, we were lead down paths we didn’t even know existed. Prior to the interviews, we had no real information about the users’ mental environments. Through our generative research however, we discovered that these users actually had several different roles, with distinct needs, and that they were approaching the problem space from a completely different direction than our client.

Using the right approach, we were not only able to find out how our client’s users organized material, but we also gained a bunch of other valuable business intelligence. After meeting with the client and presenting our findings, we were left with multiple action items we could pursue:

  • Explore the distinct user roles
  • Prioritize which role to focus on first
  • Address our user’s pain point of lack of time
  • Add data to our proto-persona and lay the groundwork for building our UX personas
  • Use the correct terminology in future prototypes

All information we couldn’t have gotten through a survey.

Moral of the story: Take the time to put on your shoes, go out to the shed, and grab the right tool.

To learn more about UX methodology, check out UX Design: People Over Features, Outcomes Over Output.


UX Design: People over features, outcomes over output

Picture yourself in a meeting with a client who wants to build something new and innovative. There are a variety of stakeholders at the table and everyone’s arguing over features. Each person has their own idea of what to build and how to build it, but no one can agree where to start. Finally, someone turns to you and says “UX, what’s your professional opinion?”

This is the moment to pump the breaks and ask yourself, “Why are we arguing about solutions when we don’t understand the people, problem, or product yet?” Think about it this way: when you are sick, you start the process of getting better by going to the doctor and talking through your symptoms. Then, a series of tests may be ordered to get a clearer picture of the problem. Finally, the doctor builds a diagnoses and prescribes an appropriate treatment plan. Drafting a list of features before analyzing your users is like walking into the doctor’s office demanding a random cocktail of medications that may or may not solve any of your ailments...and could result in some interesting side effects.

“Designers can design a solution to a problem, but they can’t design a solution to a disagreement.”
-Mike Monteiro, Speaker, Owner of Mule Design Studio

It’s too early in the process to be arguing over features. How can you determine what features your users want without a clear idea of who they are? By its very nature, building something “new or innovative”  means that it has never been done before. You are blazing a new trail. The hypothesis is that somewhere on the other side a business objective exists, but at this stage, the goal, the users, and perceived problem(s) are all fuzzy.  Many people will have ideas of what the right direction looks like, but it is important to remember that those ideas are all hypothetical. At this point, you are guessing and guessing is risky. So how do you mitigate that risk?

State Your intent.

“Design is the rendering of intent”
-Jared Spool, Writer, Researcher, and expert on Usability

First things first, everyone needs to get on the same page about what this project is trying to achieve. What is the intended outcome, for whom, and why?

One way to gain a shared understanding is to have each stakeholder write out what they believe the goal(s) are.  Have them do this as individuals, rather than as a group will ensure that the ideas remain uninfluenced.  Then, bring them back together to review the results. You will often find that everyone has a completely different idea about what the goals and desired outcomes are. It is this lack of shared understanding that usually leads to contention over what to prioritize. Once everyone is one the same page, the next step is to gain a consensus around what will be tested. LeanDog’s downloadable Agile Discussion Guide covers a number of tools and techniques, such as Fist-To-Five and Six Thinking Hats, that can help facilitate this type of discussion.

Define Your Users.

Who do you believe will benefit from the product or service you are designing? What do you currently know about them? How will you craft experiments to test your hypotheses with actual target users?

“…your job is to minimize output, and maximize outcome and impact.”
– Jeff Patton, Agile, Lean, UX and Product Design Evangelist

The goal of building out personas first is to minimize unnecessary output. Output, in this case, refers to all the work put into a project. Building the wrong things due to a lack of quality data will certainly result a high amount of output, but the things that will be delivered are likely to be of low business value. In order to minimize output and maximize value, you have to first understand the behaviors and goals of your target users. This can be accomplished by conducting interviews and observations with enough users to provide the qualitative data necessary to draw accurate conclusions. So, how do you know you have interviewed the right users, or the even right amount? Kim Goodwin’s book, Designing for the Digital Age: How to Create Human-Centered Products and Services,  has some excellent guidelines on how to go about this:

  • First, identify likely user roles
  • Start with a minimum number based on how narrow the roles are
  • Multiply for the factors most likely to affect behavior
  • Look for ways to condense your schedule and incorporate other factors
  • Adjust for possible no-shows and poor interviews

Generally speaking, you will need 5-8 participants from each role. Once you have conducted your research, synthesized the result, and identified patterns, you should have what you need to form your personas.

What are personas good for?

Personas are a project’s guiding light. When questions arise about what to do, build, or test, the first place to start is the user personas. Will this thing you are creating maximize outcome and impact for your persona, or will it be perceived as irrelevant and useless by the very users it was intended for? It may be tempting to forge ahead on assumptions and educated guesses in the interest of getting the ball rolling, but the cost of starting over at or near the end of the development cycle will likely incur ten times the cost of taking a more thoughtful and deliberate approach.  Remember: the more risky the assumption, the more validation is needed. Innovation happens when there is a constant build, measure, learn loop (see Lean Startup)

Think back to your meeting at the beginning, but this time, imagine that you have done your homework and developed well-defined user personas – all with pains, needs, wants, and desires identified. Now, when stakeholder disagreements form over project direction, you can refer back to your research findings, validate assumptions, and prioritize highest value features.

Personas allow you to take the opinions and self-referential thinking out of the equation. This is not to say that the team’s input should be completely ignored. Rather, all input should be measured against the quantitative and qualitative data you have, to ensure that what you build supports the goals of your personas.

The more you can advocate for the needs and goals of users, backed with supportive quantitative and qualitative data, the better the resulting product will be. It is critical to remember that people are at the center of design. Visual design, code, marketing, and UX are all supporting elements that are ultimately focused on meeting the needs, wants, and desires of the user.

People over features, outcomes over output.

Want learn more about how we apply UX Design principles when building innovative solutions? Let’s talk! We’d love to hear from you!