Interviewing Users Blog

During our “Ask Me Anything” with Steve Portigal, author of Interviewing Users and Doorbells, Danger, and Dead Batteries, we touched on subjects ranging from how to handle difficult clients, Steve’s favorite band, and his recommended reading, to dealing with “heavy topics” in interviews and how to improve your skills. Read on for a recap of the session, and please join our Slack here to stay informed about when our next #rm-chat author AMA will be!

Q: What book except your own would you recommend to read for UX designer? -Natalia H.

A: There are so many things to learn about. For design, I think this recent book by Scott Berkun is a great examination of how design is everywhere and in everything. It’s a quick read, it’s fun, and it’s empowering for designers I think, telling us again about all that we do, we have done, we can do.

Q: I often deal with “heavy topics” such as life, disability, cancer etc in my line of work, so it is not unheard of that we have someone break down and cry in interviews. We try our best to mitigate and avoid unnecessary stress (participants health and safety is our primary concern and we tell them in advance about the topics that we will be discussing), but can you talk about some advice how to mitigate these situations or what to do when this happens? -Fabian B.

A: Of course the stress and the emotion is going to impact all the people involved, say the participant and the researcher. I think researchers need to keep in mind that they aren’t (in most cases) trained for this, and that they need to find ways to take care of themselves. Often we will emote less in the interview than we might want to, so we want to leave space for ourselves to have feelings, to have reactions, to have someone to talk to. Work out what that is going to be. Who are you going to be able to speak with? When will you be able to speak with them?

For the participant, I learned something new to me from Sarah Fathallah at the Advancing Research conference when she talked about referral paths, something that if you are doing academic research that has an IRB ethical review needs to be put in place (someone correct me if I have this wrong) – where you identify things that might come up, like if someone reveals they are being abused, or is going through addiction, or having suicidal thoughts, you already know what your action is going to be.

I don’t know that we need that level if we aren’t doing say specifically traumatic research, but it’s the idea of planning for that. I’m also not saying that we need to DO anything; I think well-intentioned but under-informed do-gooderism is potentially worse than doing nothing in certain situations.

The complexity here is you can’t come up with every possible thing that might come up. But you can come up with SOME. I am also intrigued by sort of a generational shift in how we see our role; from “we need to observe, listen, empathize, not judge” to “all those, but we must also help.”

Q: I suspect a lot of us always did at least some of our research remotely, but now with covid, we’re seeing way more remote/virtual research. What are a couple of your top best practices for doing remote research well and getting high-quality information from respondents? -Amy B.

A: I think it’s worth co-opting that thing I see tweeted all the time “you aren’t working from home, you are working from home amidst a global pandemic.”  Same for research, right? All the parties are living through an emergency. So yes, there may be dogs and kids, and construction noise, and someone may be in their garage so they can get some privacy. And as we’ve adapted our tolerance for informality in terms of focus, energy, duration, attention in work, we can apply those to these interactions with strangers. A guy I spoke with yesterday put us on pause as his daughter asked for the car keys, she just got her license the day before. It was fine. I didn’t get ruffled like I might have in another time (no that’s crazy I’m nothing if not entirely cool and rolling with the changes). I don’t have a good answer for how fully remote research locks us out of getting to certain people. I just had a brainstorm with a colleague who was thinking about how to understand how people were or will be using transit, given that we can’t go on transit right now, and that the people who he’d want to learn about are possibly not sitting at home with a room with a laptop and a webcam and a good internet connection that is available all the time for them to just get on a call with us over Zoom. So who are we excluding even more so now (as society is excluding people in those circumstances even now) ? I know people are trying hard to deal with this, but I don’t know?

Q: Which musical group has had an interesting impact on your life and how? -Corey B.

A: I’m going to say The Tragically Hip. Being an expat Canadian, when the Hip would come to town it was always an interesting gathering of the community, at least a bunch of fellow Canucks. You’d go to a show and see hockey jerseys and University leather jackets, and just stuff you don’t EVER see in the US. And as time passed, we all got older, the fans got older, the band got older, and how we expressed ourselves and our identity shifted, gradually. I haven’t really moved through an era of my life as clearly in common with other people, not close friends, just strangers, but to feel it so tangibly through ANOTHER experience has impacted me. And of course the ultimate end of aging is death and the loss of Gord Downie still impacts me every day pretty much. I listen to the band and my many bootlegs and think about them and him, constantly. Perpetually.

Q: Do you have any tested tips/tricks on working with difficult clients? I recently had a challenge of convincing the client to use our approach when they wanted to do something else. Curious to hear what you do in such situations. -Kama K.

A: I don’t know enough about the context, and the power dynamic, as I think it often comes down to that. Two big directions, though: Empathy/Walk away. For the first, and I don’t mean to tell you stuff you already probably know and do in all your relationships, but there is that weird power to detoxify some broken interactions when you just listen, when you ask more, when you acknowledge, etc. Probably this has some buzzword for leadership people, I don’t know what that is. It sets you up to say authentically “Yes, And, and not No, But.”

For the second, if someone is a client, then you aren’t in a full-time job with them. Of course as someone who also provides services to clients, I want to do a good job, I want to be appreciated, I want to bring value, I want to be employed, and brought back in, etc. So I don’t like that framing that design people in agencies sometimes like to espouse which is making the client the enemy. But we can walk away. And even if we just hold in our head that we have the OPTION of walking away can making a less desired choice perhaps more tolerable. I’m CHOOSING to do this which I don’t agree with and which distresses me but I could and may at some point choose NOT to do it.

Q: What are your most counter-intuitive insights about research as it propagates through the product development process? -Scott W.

A: I don’t know that I can claim counter intuitive as a goal, if you agree with me then it’s probably not THAT counter intuitive? I think there’s this desire to create models and visuals that say if you are here do this, if you are here do that. Whether that’s the double diamond model or (VERY HELPFUL) recommendations for methods based on a stage in a process like Christian Rohrer’s.

I just am not that organized, I am just not able to constrain myself to a discrete stage, maybe I’m trees and that’s the forest and I just can’t do the forest well enough. That all being said, I think there’s something about how these processes are a continuum, a gradient, and not stages. I know Software Development Lifecycle Methodologies are all about gates, and stages, and review cycles, and (ugh) sprints, but I think in terms of where we are in terms of certainty, of belief, of ideas, it’s a much more creative process, I can’t control what thoughts I have in the shower in that way we have of considering and being inspired about what it means, what to do, what the opportunity is. Some of that is always happening for creative people – and research is ABSOLUTELY a creative process of sense making and understanding.

Q: Are there any topic(s) or technique(s) you hope more places of instruction cover for individuals entering into research? -Randolph D.

A: As you know, I love storytelling, and I think it’s such a powerful tool, but it’s just kind of considered to be something that is maybe part of your personality, your own personal toolbox. But obviously it can be taught, practiced, developed. Research is about gathering stories and creating new stories, and “story” is of course an ENORMOUSLY broad construct so one can take it however one likes.

I mentioned ethics in another question, I think as a field we haven’t reckoned with it sufficiently. I’d like people coming into the field to have a perspective on it. I know a researcher who wrote up their own research philosophy, not even as a document to publish but as a way of working out what they were trying to do. I was extremely impressed with that.

Research as a practice to me is a constant consideration of who we are in the world, as people, how we relate to other people, how we judge and don’t judge, and just how we move around and exist and perhaps make and help. So, being intentional about what you believe and what you want, damn. It’s a brilliant activity. I haven’t ever done that and hadn’t ever thought of doing it. So philosophy isn’t ethics literally but is adjacent?

I’d like something in training – and I don’t know that the places people are learning about research are the right environments to be considering this aspect – about what the researcher’s role is relative to the rest of the people they work with. I do not like the idea of research as “support” – you hear “oh I support three teams” – I know research is a helping profession like say librarians, and I don’t mean to squelch that strength, but I think we are partners and leaders, and if we don’t believe in the value we can bring in the role we can play, no one else will. I think it’s a hard field to break into and there are a lot of entry level people and so if they are told they have to be subservient, that can set a long running pattern for their career and for the practice.

Q: Someone interested in working with us recently asked how we recommend she improve her interview skills. I recommended your books, but are there any classes or seminars you might recommend as well? -Amy B.

A: I’m going to pitch my upcoming workshop. Also very very good is the cycle of “do, observe, reflect.” (which probably has a smarter name than I’m giving it). Listen to Terry Gross – listen to her technique. Reflect on her technique. Print out a transcript and mark it up. What choices did she make? What other choices could she make?

Listen to your own interviews. Print them out. Listen to a colleague’s interviews. Same same. Have someone ELSE listen to your interviews. Have someone else mark up Terry Gross (or anyone who does a lot of interviews).

I think training will get you further than you are, but practice, man practice is the way. Do a lot of interviews. A LOT. Reflect and analyze!

Q: What are some tips for building rapport in remote interviews? (with camera and without camera) -Erika

A: One avenue to explore is pointing to the medium, just acknowledging that you are doing what you are doing. And not pretending that you are as smooth as you are when you are in the same room. I did an interview yesterday where I had to share screen from Google Slides (not what I normally use for giving talks) and see speaker notes and it’s just a mess, and so I stopped and said what was going to happen, and then you heard the “unprofessional” sounds of me, saying, “Okay I’m going to hit share screen…yeah I think it’s shared now, okay, now I’m doing this, can you see this?” It just normalizes the interaction so you are both having a similar experience.

One thing I think needs to be explored is around shared sensory experiences. I saw Alice Waters talk recently and she described how she’d meet with people and she’d put a piece of fruit or something else down in between them and they’d just eat it beforehand and it created these interesting connections and well, rapport.

I don’t know how to operationalize this for remote research but I’m imagining having everyone pet their furry animal before starting and just sharing that moment that is about the senses, even though we are having our own experiences, we are having similar ones together. I think there’s probably some work to do to create that in a non-weird-sounding way.

Join us on Slack for an “Ask Me Anything” with Steve Portigal, author of Interviewing Users and Doorbells, Danger, and Dead Batteries. He’ll be answering your questions live on Thursday, August 13 from 1-1:45pm EDT in our #rm-chat channel. Join our Slack here. Haven’t read his books yet? You can access a discount code once you join.

Although we couldn’t make it down to Buenos Aires for Interaction South America, thanks to the magic of Skype I was able to present Designing the Problem at over the weekend.

Too often we assume that doing research with users means checking in with them to get feedback on the solution we’ve already outlined. But the biggest value from research is in uncovering the crucial details of the problem that people have; the problem that we should be solving.

As the design practices mature within companies, they need to play an active role in driving the creation of new and innovative solutions to the real unmet needs that people have. In part, driving towards this maturity means looking at one’s own culture and realizing the value of being open-minded and curious, not simply confident. This is a challenge to each of us personally and as leaders within our teams and communities.

Below you’ll find slides, audio and sketchnotes. I’ll repost when the video go up.

The talk is just over 40 minutes and there are two questions (which you can’t hear but which should be obvious enough from my response).
[audio:Steve Portigal – ISA14 – Designing the Problem.mp3]

To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac)

Here is my huge head during the Q&A segment (image via Juan Marcos Ortiz)

Sketchnote by Kat Davis (click for full size)

Sketchnote by Thiago Esser (click for full size)

There is a fantastic and disturbing episode of Love + Radio. Nick van der Kolk and Noah Morrison visit Jay Thunderbolt, who upon beginning the interview, aggressively reiterates his demand for payment (not possible for public radio, which Jay knew). Noah ends up going on a liquor run instead, as Jay offers him a pistol or Kevlar vest. Jay never stops insulting the interviewers, and stories of violence abound; indeed at one point he points a gun at Nick’s head. Meanwhile, they are interviewing Jay about the strip club he runs out of his house.

Yikes. This sounds like some of the War Stories, doesn’t it?

I don’t know what is going through Nick’s head as he’s doing this interview, but as I listen I find myself strongly repelled by Jay. And while the interview here is edited, so we don’t know all that happened, but Nick never reveals discomfort or lets Jay’s obvious provocations get to him. His patience and tolerance create room for Jay’s story to come out, and while Jay is not an appealing individual, you begin to understand and accept him as he is. Well, I did. Your experience may vary.

Nick finally responds to Jay’s taunting at the end, when he asks Jay “Do you think you understand the way I feel about you?” and Jay admits that he doesn’t. It’s a powerful moment in an intense interview.

The first episode of Working (a new podcast) features Stephen Colbert explaining in great detail the process of creating The Colbert Report. The entire episode (embedded above) is really good process stuff (creativity, collaboration, finding the story, media firehose, working under pressure) but I want to call out the section about how he prepares and uses the questions for interviewing his guest, as it’s is quite consistent with what I wrote in Interviewing Users.

And then I read the two sheets of questions that the writers have come up, what their ideas are. I usually pick 10 or 15 of those. But I don’t look at them. I don’t look at them until right before I go over [to the set], and then I read them over once again in front of my producers to get a sense of, oh, this is how my character feels about this person.

Come show time…I take them out and I go, oh, yes, these are the questions I chose. And then I try to forget them and I try to never look at the cards. I just have a sense in my head of how I feel. And the cards are in front of me, but I try not to look at them at all. I’m pretty good. Maybe I look once a week at the cards. I put my hand on them, so I know I have them if something terrible happens, but as long as I know what my first question is for the guest I kind of know what every other question is, because I really want to react to what their reaction to my first question is.

And I usually end up using four of the 15, and the rest of it is, what is the person just saying to me? Which makes that the most enjoyable part of the show for me. Because I started off as an improviser. I’m not a standup. I didn’t start off as a writer, I learned to write through improvisation, and so that’s the part of the show that can most surprise me. The written part of the show, I know I can get wrong. You can’t really get the interview “wrong.”

Steve spoke about user research at the HOW Interactive Design Conference [slides].

Patricia Colley is an experience designer and the Principal of Creative Catalysts in Portland, OR.

In 1984, I was 23, and working for a market & social research firm in San Antonio, Texas. They sent me down to McAllen to collect voter opinions on the upcoming national elections. McAllen is a sleepy little town near the bottom tip of the state, just a few miles from the Mexican border, mainly populated with low-to-moderate income Hispanic families.

I was on my second day of door-to-door polling, asking voters their opinions on policy matters, and their thoughts on the state and presidential candidates. The work was progressing well. As usual, I was getting a high rate of interview completions, with lots of useful data. After four years of working in market and social research, I was quite confident in my neutral, non-threatening “aw shucks, I’m just one of you” act, and its ability to deliver great results.

But my confidence was shaken when I met Maria, a shy housewife in her early 30’s.

It was about 4 pm on a warm, dry Thursday afternoon when I knocked on the door of a modest, well-kept ranch house in a suburban section of McAllen. Maria opened the door part way. She was half-hiding behind it, sizing me up like a rabbit peering through tall grass at a coyote in the distance…curious, but poised to flee.

Me: “Hello, my name is Patricia, and I’ve been sent here by (XYZ Research) to gather public opinions on the upcoming elections.”
Maria: “Oh, hi.”
Me (turning on the charm): “May I ask you some questions? Don’t worry, I’m not selling anything!”
Maria: “Uhh, sure, I guess?”&
Me: “Great, thanks! This won’t take long.”

Wide-eyed, Maria flashes a shy smile before her jaw slacks again. This one’s cagey, I thought to myself, but I’ll get her talking.

Me: “Now, thinking about (Candidate X), what comes to mind?”
Maria: “Uhh, I don’t know? Is he a good guy?”
Me (shrinking): “Well, I really don’t have any thoughts on (Candidate X). Besides, my bosses didn’t send me all this way to talk about my opinions. He wants to know your opinion.”
Maria: “I don’t know. He seems okay?”

Now, I don’t think Mary is incapable of forming opinions. I suspect she’s simply never been asked to share her thoughts about such important things, so far from home. And she may never be asked again. But on this day, I was determined to make her opinion count.

Me: “Well, you’ve heard of him, maybe seen him on TV?”
Maria: “Yes.”
Me: “So, what did you think of him? Is he someone you would vote for?”
Maria: “Um…(pause)”

Her eyes darted across my face, scanning every crease and twitch, searching for clues. Those big rabbit eyes begged mutely for help. I stared back, apologetically. I took a few slow breaths, trying to ground us both, so she might relax into talking more naturally. Each time she hesitates, I carefully repeat the question, altering the wording and inflection to make them sound as simple and benign as possible.

Me: “Really, we’re just interested in what you think. Whatever you think is fine. Do you think you’ll vote for him, or not?”
Maria: “Uh…yes?” (seeing no reaction from me) “No?”
Me: “Okay, that’s fine. Alright. Now, thinking about (Issue A), is that important to you? Do you think it’s good or bad?”
Maria: “Uhh…I think it’s good?”

The back and forth went on for several minutes. I’m trying to go completely neutral and void of any emotional expression, but my contortions only intensified the awkwardness. The interview was in free-fall. I was failing miserably to collect any genuine responses from Maria. A hot wave of panic washed over me. How can I get this back on track?

In that moment, I just had to let go.

I quit fighting it, and fell back on connecting with Maria as a person. As Maria answered my questions, I began riffing on her responses, affirming and adding detail to them. While trying not to reveal my personal opinions, I offered supportive words and gestures to elevate everything she said, so that she might open up and elaborate. Eventually, she did relax, and her answers flowed a bit more freely.

Me: “So, what about the presidential candidates?”
Maria: “I guess I’ll vote for (presidential candidate B).”
Me: “Great! Is it because he is for (issue B)?”
Maria: “Oh, that’s good. Yeah, (B) is good for us.”

Although Maria was warming up to me, I felt I was way off book. It seemed impossible not to sway her answers. Whatever I wrote down, I feared it might be swept away by the slightest shift in body position, or an eyebrow lift. Well – at least she was talking, I told myself.

Finally, we got to the end. Walking back to my car, I breathed a huge sigh of relief. The hardest interview I’d ever done was over. I went out for a well-earned drink and a tragicomic debrief with my co-workers.

Sometimes you just get a dud subject, and it is what it is. But something about that 15-minute exchange with Maria struck a deeper chord in me. As I drove out of town, troubling questions lingered. What is the value of a skewed interview? Was this the only time I’d failed to be impartial? Or, had this been happening all along, in more subtle ways? How can I ever know that the data I’m collecting is pure?

Maria taught me two important things that day.

1. People make stuff up as they go along. And, we can’t always see the flaws in self-reporting.
2. The observer effect is unavoidable. Interviewers shade their work in unpredictable ways.

I’m as diligent as ever about delivering valuable insights through my research. But ever since that incident in McAllen, I draw my conclusions with a fuzzy border, in humble deference to flawed inputs and shadow projections, on both sides of the clipboard.

Steve will be presenting Designing for Unmet Needs at the Warm Gun conference in San Francisco on December 4.

Steve spoke to the Denver UX Book Club (video).

Alex Blumberg has a podcast about his journey to start a podcast-related business. A recent episode of This American Life included an excerpt from this podcast (called StartUp), in which Blumberg is half-heartedly pitching his idea to investor Chris Sacca.

They talk for a while, and Alex is having difficulty in explaining his idea and what he’s asking for.

Alex Blumberg: So it’ll take a million and a half dollars, I think. And–
Chris Sacca: Take out the “I think.”
AB: Yeah. It’ll take a million and a half– I’m looking for a million and a half to $2 million in seed-stage funding.
CS: No, no, no, no, no.
AB: Yeah.
CS: You were looking for a very specific amount of money.
AB: I’m looking for– [LAUGHS NERVOUSLY]

Finally, Chris decides he’s just going to show Alex how to pitch his idea and he very masterfully riffs a confident and coherent bit of persuasion. It’s certainly worth listening to, but here’s the excerpt from the transcript.

Hey, look, can I get two minutes from you? So here’s the thing. You probably know me, producer of This American Life, been doing it for 15 years. You know it’s the most successful radio show, top of the podcasts in iTunes, et cetera.

So here’s the thing. I realize there’s a hunger for this kind of content out there and there’s none of this [BLEEP]. It’s just a bunch of jerk [BLEEP] podcasts. Nothing’s out there.

Advertisers are dying for it. Users are dying for it. And if you look at the macro environment, we’re seeing more and more podcast integrations into cars. People want this content. It’s a whole new button in the latest version of iOS.

So here’s the thing. Nobody else can make this [BLEEP]. I know how to make it better than anybody else in the world. And so I’ve already identified a few key areas where I know there’s hunger for the podcast. We’ve got the subject matter. We’re going to launch this [BLEEP]. I know there’s advertisers who want to get involved with it.

But here’s the unfair advantage I have. Because of what I’ve done in my past careers with This American Life and with Planet Money, people are actually willing to just straight-up pay for this stuff. And I’m not just talking about traditional subscriptions. I’m talking– we did this T-shirt experiment at Planet Money where we got $600,000 coming in, where people actually gave us money to buy a t-shirt with our logo on it as part of the content. It was integrated directly. And I know we can replicate that across these other platforms.

So here’s what we’re doing. We’re putting together a million and a half dollars. That’s going to buy us three, four guys who are going to launch these three podcasts in the next 12 months. We think very easily we could get to 300,000, 400,000 net subscribers across the whole thing.

With CPMs where they are in this market right now, I know on advertising alone, we could get to break even. But as we do more of this integration, we get people texting in to donate to this stuff, buying some of this product, doing some of these integrated episodes, I know that we’re going to have on our hands here something that will ultimately scale to be a network of 12, 15 podcasts. The audience is there. They want it. Nobody else can do it like we can. Are you in?

It’s so painful to hear Alex stumble and when Chris takes over, I felt a sense of relief and a certain excitement, to hear an idea presented in a way that was designed to engage and persuade. This is a valuable skill in many aspects of professional life, especially when we’re in the business of sharing ideas. The superlative example in this podcast is quite inspiring.

The relevant section starts at 19:21 in the embedded widget below.