Mental Models Blog

During our “Ask Me Anything” with Indi Young, author of Mental Models and Practical Empathy, we touched on subjects ranging from opportunity maps and research repositories to Jobs to Be Done and empathy as a design concept. Read on for a recap of the session, and please join our Slack here to stay informed about when our next #rm-chat author AMA will be!

Q: I’m wondering if you have any thoughts to share about using empathy as a concept to design for not just people but also the environment or any larger system. -Behnosh N.

A: I have used it and seen it used for designing systems for people, but not environmental ecosystems. Several government digital offices are using my approach to understand people more deeply, to see the differences in approach, to see different thinking styles … and to think in the problem space so as to discover people they’ve ignored.

Q: I’m curious about using the thinking styles approach in developing key archetypes within a body of population health research. Often, patients tend to get heavily sorted by demographic characteristics because they map to certain physical and social determinants of health conditions. But these don’t go far enough to capture attitudes, beliefs, behaviors etc. How have you used the thinking styles frame for rather large and diverse populations? – Jeremy B.

A: What I’ve done is framed several studies by “a person’s purpose.” Often, with health, their purpose is to “cope with,” so for example one study was “coping with my three ongoing conditions.” You must frame by a person’s purpose, so then you can get deep. (either in solution space or in problem space). You can go deep in listening sessions where you help the person trust you and get into their inner thinking, emotional reactions, and guiding principles as they were pursuing this purpose. Here’s my course on listening deeply.

Q: Research ultimately is about learning what we don’t know. Often we’re so focused on who our customers are that we forget that the real work in understanding how we’ve lost or who we’ve failed to win.How do you find, recruit, and drill down to the why of those who were near loses or recent loses? – Arpy. D

A: I would like to see us quit measuring by “engagement” altogether (“hey, someone looked at me through this glass pane!!”) and start measuring by how well we support each thinking style within each slice of their mental model toward accomplishing their purpose. I encourage people to do listening sessions with stakeholders, over and over, like monthly with each stakeholder at first, to develop rapport and trust. But you could totally make thinking styles if you do enough of them!

Q: I remember attending a talk you gave referencing the image below. A challenge I have is to group various user types based on thinking-style. Personae have been used by Agile coaches. I am having a hard time to convince folks to frame users based on thinking style instead of job titles. any thoughts? – Chika A.

A: You can totally use the word persona to mean thinking style, if that works better for your context. Or you can use “archetype.” There is a problem with any archetype that uses demographics to describe the group: invites subconscious bias. (Unless those demographics are related to a context of discrimination, physiology, and a couple of others.) When you explain to someone how demographics cause assumption, they don’t need to be convinced further. Here are two helpful articles: Challenging the Make-Believe in Personas and Demographic Assumptions.

Join us on Slack for an “Ask Me Anything” with Indi Young, author of Practical Empathy and Mental Models. She’ll be answering your questions live on Tuesday, December 1 from 2-2:45pm EST in our #rm-chat channel. Join our Slack here.

I received an enthusiastic, but bewildered cry for help from a UX designer in South Africa, Jeanne Marias. She wrote, “I am pioneering a service design project, part of which I’m wanting to do a Mental Model of ‘The New Member Journey.’ I’ve charged ahead and gotten the whole team excited about mental models, but after reading the section of your book about defining task-based audience segments, I’m feeling quite daunted and out of my depth! Especially when you speak about the Story, Craft and Companionship continuum.”

(Never mind that we don’t know what industry this is for–“new members” appear in lots of difference scenarios, like insurance or teachers unions or book clubs. So don’t worry about which industry she’s talking about.)

She wanted to know if there was an easier, friendlier way of creating the audience segments, similar to the comic strip explaining mental models. There is a cartoon about personas, but personas are things that you derive after you’ve done all the interviews. For the hypothetical audience segments before you recruit, you only need to keep in mind one thing: we traditionally recruit by demographic (like age or experience or frequency). This time we want to be sure we also cover all the possible behavior types that are important to us for this round of interviews. Behavior types are things like decision-making style, goals, motivations, attitude, etc. (Furthermore, remember that you don’t need to make up hypothetical segments you already have them from prior research. Use those as a basis for recruiting, or expand to additional groups if your business demands it at this point.)

The example in my book is pretty complex. Your situation might be simpler. Let’s explore an example. Pretend your industry is weight-loss, and new members are folks signing up for your program.

Remember, the only reason you do these hypothetical audience segments is to make sure to recruit from each group. They are not personas, and they don’t matter later. You delete them, in fact, after you’ve collected all the stories. Don’t get too caught up in all the details of these hypothetical groups.

The other day a university student named Maria Hernando wrote to ask me my opinion about the relationship between User Journey Maps, Customer Journey Maps, and User Experience Maps … and how a mental model diagram might relate to any one of them.

I told Maria that I think of the maps as the same, or similar enough. The maps try to represent an actual example of how a person (or persona) went through and did something they wanted to do. The maps are generally chronological, moving forward through the hours of the persona’s actions one stage at a time. I told her that I think the phrase “experience map” came about because we want to be agnostic of whether the persona was using digital tools or not, or a combination to tools. The map represents the journey a person takes from the idea of accomplishing something to having accomplished that thing in the end. We want to see how it all hangs together from the persona’s perspective.

There can be as many experience/journey maps for a particular persona as there are deviations in the way they do that thing. For example, if a persona was taking a commercial flight, there might be different maps for a business-related flight than a leisure-oriented flight. There might be different maps based on whether it’s a last-minute or urgent flight. There might be different maps for long versus short flights, flights where the persona has to get work done before landing, flights where the persona is scared of flying, etc.

The mental model represents a set of states of mind (mental spaces) that a person might pop into and out of during this journey toward accomplishing a goal. The states of mind might proceed in a nice linear fashion. Or they might represent a more cyclical approach, where the person revisits a previous state of mind again to re-evaluate something, to continue something, or to address something new that has come up. When I combine experience maps with the mental model, what I do is add little bubbles labeled with a mental space along the journey. Sometimes the bubbles repeat themselves in this fashion.

I wish I could share some actual diagrams, but here’s a quick sketch as an example. The experience is laid out left to right in brown, and the mental space bubbles appear in pink above, with red arrows showing the way this persona, Susan, popped into and out of Get Work Done several times during her trip. Apparently her boss had asked her to finish a report before she arrived at the meeting in Chicago the next day. (Feel free to tweak the concepts that are represented in the combined diagram of an experience map plus mental spaces. Feel free to make it prettier.)

Picking out an actual “guiding principle” (something that guides how I make
a decision) from a transcript is difficult. There is so much “brush” we need to clear away before we can see the “specimen trees” for what they are. Here is a perfect example.
It’s a page from an architecture firm’s web brochure. The page is titled “Our
philosophy.” http://www.philippetimmerman.com/ (You have to click “Philosophy”
in the navigation to get to this page.)

1. “Philippe Timmerman Architectural Designs [the company] is as much about an
   aesthetic attitude as it is about architecture and design. It is about defining
   and creating a …” This is an explanation/description of the company’s
   position in the market.
2. “… personal lifeworld that bears the watermark of an individual style.” This is a philosophy, “Watermark every person’s lifeworld with their individual
   style.” I’m not entirely sure I understood that right–I don’t know what a lifeworld is. Nevertheless, it is how they make decisions about watermarking their designs.
3. “Which is why in every design, every concept, every search for an object or
   work of art, we strive to recreate that original sense of harmony that resonates
   through everything we are accustomed to calling ‘beautiful’.” This is an
   explanation of their process, but you could pull a behavior out of it thus:
   “Recreate a sense of beauty, harmony in each design, concept, and search.” It describes each employee’s intent they create their designs.
4. “This relentless commitment to purity of design implies a similarly
   meticulous attention to detail.” This is a paragraph joiner–something that
   references the first paragraph and introduces the new subject.
5. “We believe that details are more than mere parts but reflections of the
   whole: the majesty and splendor of an entire palace is contained within a single
   door handle.” This is a philosophy, “Believe details (door handle) are
   reflections of the whole (palace).“
6. “A concept that permeates every corner and every layer of the design and
   finishing is not a superficial luxury but the natural outcome of a constancy to
   purpose and a consistency of perspective.” This is also an explanation of
   their process and a reassurance that the company is constant and consistent. You
   might be tempted to pull “be constant and consistent” out as a guiding
   principle, but that’s not how this is written. The intent it to reassure the
   potential client that the company is serious.
7. “At [the company], everything begins with the notion of craftsmanship.” This is also an explanation of method.
8. “We believe in craftsmanship simple because it stands the test of time.”
   This is a statement of fact: craftsmanship stands the test of time. We [the company] agree with this fact.
9. “Knowledge and knowhow, of materials and techniques, form an unconditional
   but limitless source of inspiration for our designs.” Here is another behavior, “Find inspiration for designs in my knowledge of materials and
   techniques.” It explains the method an employee of the company follows when looking for inspiration.
10. “… designs that encapsulate a personal vision of the here and now, but
    which also embody an orientation towards the future.” This is a statement of
    fact that their designs encapsulate this orientation.
11. “For over time, beautiful objects gain in beauty; houses develop character;
    furniture a certain patina; and works of art can but strengthen their force of
    presence.” These are several statements of fact. They may say they “believe
    in” these SOFs, but what do they do about them? We can’t include this set of phrases in a mental model because there is nothing here that guides a decision. (Note: Often the two words “believe in” are a red flag for something that is not really a guiding
    principle.)
12. “[The company]’s international design office comprises a team of specialized
    architects, interior decorators, designers and art historians resulting in a
    dynamic exchange of ideas and perspectives.” This describes their method,
    that they exchange ideas among the staff. See the next bit for the real guiding
    principle.
13. “… synergies which form the basis of our unique vision on architecture and
    design.” Well, this is not grammatically correct, but let’s go with the
    flow. They are stating another behavior here, “Exchange perspectives with our
    varied staff to gain unique vision.” This should definitely be included in the mental model.
14. “With an extensive portfolio of creations in countries such as France, Great
    Britain, Monaco, Italy and the United States, [the company] continues to stamp
    its mark on interior architecture in ways that are as diverse as our discerning
    clientele.” This is another reassurance to potential customers, stating that
    they have an extensive portfolio and discerning clients.

Tally: Two Guiding Principles

• “Watermark every person’s lifeworld with their individual style.”
• “Believe details (door handle) are reflections of the whole (palace).”

Tally: Three Behaviors

• “Recreate a sense of beauty, harmony in each design, concept, and search.”
• “Find inspiration for designs in my knowledge of materials and techniques.”
• “Exchange perspectives with our varied staff to gain unique vision.”

Put the pronoun “I” in front of each of those sentences and try them on for
size. This statements run through the minds of the employees or partners at this architecture company as they execute design work.

Also note: I use the term “guiding principle” here, but also use the words “belief” and “philosophy” in other places. I mean the same thing by them: something that guides decision-making.

At at recent workshop, I conducted a spontaneous interview as a demonstration of what I mean by “create a scope perimeter within which any conversation can happen.” I asked for a volunteer and for a topic. The volunteer was Daren. The topic was air travel. I scoped the topic down to “planning and booking air travel” just to have a good place to start, and also added “handling the day of travel.”

So, with both of us standing at the front of the room, I asked Daren about his thought process as he planned and booked his last flight. He said, “Well, it was a multi-leg flight, and so I knew it would be hard to set up online. So I called. I like to call, anyway. I fly Southwest mostly, and they have really nice reps.” I asked him what he meant by “nice reps.” The conversation flowed. He was great at describing how he thought. Then he said, “But actually, the customer service at Southwest has changed. It has gotten worse.” “How so?” I asked. “Well, recently I flew with my wife and our toddler. As we were walking down the aisle boarding, I was holding my son’s hand. Somehow he fell and cut his lip. Luckily we were right near a flight attendant, so I asked her for some gauze or a Band-aid or something. She told me there wasn’t any on board, turned away, and just started talking with another passenger. She totally saw my boy’s bloody lip! I was so angry!” I wondered out loud, “What did you do?” “I searched my pockets and found a tissue–a dirty tissue–and used it to clean up my son’s lip. And my wife was kind of upset at me for letting it happen, so I was also feeling guilty about the whole thing. But I did get up and find the flight attendant and write down her name. I was SO going to complain to management about her!” “And …?” I prompted. “Well, I cooled off during the flight. She actually came up to us later and was really friendly and helpful. So I decided it would be too much effort to write up a complaint–it wasn’t worth it. But I have switched airlines. I used JetBlue on my last business flight.” “Why them?” “Oh, I pass their billboard on the highway every morning. I heard they have seatback screens, and I’d like that. I don’t like craning my neck to see the screen all the time.” The conversation continued in this vein …

… then I switched topics to “handling the day of travel” just to demonstrate a different type of topic. Daren started out describing his latest business trip on JetBlue. He said, “I like to get to the airport early, like really early, to avoid stress. Maybe I’ll sit there and read or work or something.” I asked him about his reasons for avoiding stress. He told me, then gave me this example. “On this last trip I spent an hour looking for food. You don’t get food on the plane anymore, so you have to buy it ahead of time. Well, I have special dietary needs. Actually, my son has the allergies, but my wife and I eat the same as him just to make things easy. He’s allergic to wheat, dairy, nuts, and eggs. So I had to run around looking for something that I could eat. I told myself that morning that I wasn’t going to cheat. Sure, it would be easier to just grab something and go, because I’m not the one who’s allergic, and my son wasn’t with me. But, I wanted to not cheat. So I looked for an Asian place first. Those are usually good–rice is good. But the one I found had teriyaki, which has soy sauce in it. Soy sauce is made with wheat. So I finally ended up at a place that had hamburgers. I bought a hamburger and fries and threw out the buns. I ordered a half pounder because I thought I would need the extra calories if I was going to throw out the buns. I told them no cheese and no mayo. I actually bought two: one to eat then for breakfast and one to eat later on the plane.” He continued on with his description. “When I arrived in New York, it was late, but I was hungry again. The only place open was a Jamba Juice, so I thought I could get a smoothie. I spent 20 minutes looking at their menu and realized that all their drinks either had dairy in them or gluten in them.” “What did you decide to do?” I asked. “I was hungry, but I just convinced myself to walk away and go get a cab to the hotel. I stood there 20 minutes first.”

That’s a lot of emotion!

His determination to not cheat on the dietary restrictions of his son stuck with me 30 minutes later, when I went to lunch. I walked into my favorite quick lunch spot: Specialties Cafe & Bakery. It was a relatively new store, and they had these tethered iPads for placing orders. As I browsed through the sandwiches, I tried thinking like Daren did. What sandwich could I buy without wheat, dairy, nuts, or eggs?

This is what I mean by empathy. I felt bad about it.

Empathy sounds all wonderful, but it’s powerless unless you try out the life of the person you’re trying to empathize with. You won’t experience the remorse of cheating on dietary restrictions if you don’t try to apply those dietary restrictions honestly. I tell people it’s similar to what an actor must go through when studying a character. It’s the act of leaving yourself behind and stepping into the thought-processes of another person.

When you are designing, how much time do you spend in your own head, applying your own perspective, and how much time do you spend in someone else’s mindset? Next time you’re designing, try to spend more of the time outside of your own perspective. Make this into a practice. Say things about how you would encounter the design with an “I,” but this “I” is the “I” of another human being. “I am starving. I am tired from that long flight from San Francisco, and I’m slightly peeved that all the food places in the airport are closed this late at night. So I’m thrilled to see that Jamba Juice is open–I anticipate gulping down some fruity smoothie within a few minutes. But first I must adhere to my practice–what ingredients are in each drink available? I must read each description very carefully for wheat or wheat by-products. I must scan for dairy. I assume there are no nuts or eggs in these drinks, but I keep that in mind, too, as I study each drink, one by one. I have to set my backpack down beside my suitcase because it is so heavy and this is taking so long …”

Make empathy a bigger part of your design process.

BK: I was just re-reading some of the book and I was wondering how important it is to define the task-based segments. I’m not sure anyone here knows what users do well enough to make those guesses. I’m thinking of starting to talk to a few people who look promising and see what they say, but will this be usable later for a mental model?

Indi: Yes, anyone you can get to tell stories about their motivations will be usable in a mental model diagram. It will be a random or a homogeneous collection of stories, though. As far as defining some hypothetical audience segments up front, it’s an exercise to define and broaden your understanding of who you are supporting. Most people answer “everyone” when I ask, “Who will use this offering?” It’s not a good answer and keeps many organizations in chaos. It’s sometimes very difficult to undertake this exercise, but it is so helpful. I’m cajoling a client through the process this week in fact. Even if the groups you define now change completely once you have collected real stories from them, it’s important to try. If you don’t try to define some groups, then a) the scope of your research will be too broad, and b) you will miss talking to some people you might not have had in mind at first.

BK: Could I just ask how you honed your non-directive interviewing skills? Did you just start and then improve over time or did you study books, take courses or similar?

Indi: I started doing interviews in 1993 as a part of understanding the “lay of the land” for customer service reps at a call center in Baltimore. I considered myself a software engineer at the time, having graduated with a degree in Computer Science in 1987. So I think it was a skill I took from the deconstructive approach for writing code. They never taught us to interview people, but they did teach us to come at problems as neutrally as possible. So in the end, I guess my answer to your question is that I just dove into it. I try always to improve. I practice whenever I’m near people like at the checkout line or at community meetings. It’s far easier to be curious and neutral with “familiar strangers” like these situations than with people you know. Use every opportunity you can to practice.

BK: I’m gradually cajoling my colleagues into letting me loose on the customers. I have to introduce the idea of user-experience-based user research. I’m still at the stage of trying to convince them of the value (compared to analytics / market research), since you know the non-directive interviewing can raise a few eyebrows. I’ll be putting a case together. This will definitely be a ‘dive into it’ approach for sure.

Indi: Good! Dive into it! Employing a pop-up on a site with a few questions is a great way to collect leads. List these names and get back to each of them for a five minute chat (which needs to be by voice, but only 5 minutes) to find out who they are (in terms of your behavioral audience segments plus in terms of whatever demographics are interesting to your organization) and if they can talk story. This is really the reason why you need to call them. You can’t find out if a person can talk story by email.

Once you find some people you really want to talk to, set up a time to have a conversation for 30 or 60 minutes. This is the fun part! This is where you start by introducing the scope of what you’re doing and then simply ask, “So, what are your thought processes and reactions during this?” Then you let the stories flow for a while. No interview questions are needed. Simply be curious and ask for lots of explanations of thoughts, beliefs, and feelings. This is the information from which you will make your mental model.

Alternately, you’ve read about the lightning quick short-cut, right?

The letter begins, “You may have read in the newspaper–or heard on the radio or television–the official government figures on total employment and unemployment issued each month.” (The writer of the letter added to em-dashes to make sure we notice they are using new-fangled media channels. I assume the template for this letter was written in 1992 or something, since the Internet isn’t mentioned. There is no actual date on the letter.) The letter goes on to say, “We have selected your address and about 55,000 others throughout the United States for this survey … your participation in this voluntary survey is extremely important to ensure the completeness and accuracy of the final results. Although there are no penalties for failure to answer any question, each unanswered question lessens the accuracy of the final data. Your cooperation will be a distinct service to our country.” There is no explanation how to opt out, just an address to send comments. I didn’t want to opt out, though. I figured this opportunity is just like the time I was a Nielsen TV ratings household where I didn’t own a TV. This opportunity is similar because I don’t work for a company. I could be another statistical outlier that the analysts would have to contemplate … and probably throw out.

But what gets me is those adjectives (or noun-ified adjectives) in the letter. “Official,” “important,” “completeness,” “accuracy” (twice), “final” (twice), “distinct.” These are the adjectives that we have been fed for decades about surveys. Sure, I took my statistics courses in university–even worked as a teacher’s aide for one of the classes–so I respect the math and the statisticians. I don’t respect the people who write the survey. Their use of English is imprecise, which leads to crazy results. In addition, the Bureau insists on conducting the surveys in person, by voice. Your answers or side-chat with the human carrying the survey don’t influence how the answers are worded, and nothing matches up with reality. They ask for details you don’t know about other household members. Nothing is actually accurate, because they are using a survey to capture the information.

So, these two very normal 60-year-old ladies turn up at my front gate this morning. I was in the middle of my bike workout in the garage, so they weren’t interrupting my work, at least. They showed me their ID’s, handed me another copy of the letter, and introduced themselves as the Field Rep and her Supervisor. I got the feeling the Field Rep was on her first week of this survey-thing. I chatted with them cordially about where in the Bay Area they live while changing out of my bike shoes and invited them in the house, offered a cup of tea–everyone was smiles. They told me how not all the survey participants are so cheery and cooperative. They only accepted water. (I bet there’s a rule about what they can accept. I know there’s a rule to observe when I ask government employees to participate in an interview: I can’t give them any individual compensation. Group compensation is okay in some places, like a basket of muffins or something. A coffee cup, a check, a baseball cap–those are forbidden. No bribery implied.) Then the little gray laptop came out, and thus began three things: the circus of trying to get the laptop to behave the way the Field Rep intended, the defining and re-defining of the English used in the survey, and the hand-waving about what answer to put down, all resulting in the increasing unease of the Field Rep herself. I didn’t mean to, but I think I ruined her day.

1. The Circus of the Laptop

The survey began with all the traditional questions: is this your primary place of residence? What is your name? Who else lives here? What is his name? When I answered, “Philip” for that last question, at least the Field Rep had the presence of mind to say, “So I assume his gender is male.” That’s the only time she “slipped up” and allowed normal human discourse to dictate one of her answers. I asked about the rules of conducting the survey, and her Supervisor said they are required to read each survey question exactly as it is printed. And they have to read the answers out loud, too. Except the household income question. That struck me as humorous, because the Field Rep swung her laptop around to show me a list of radio buttons, each with an income range next to it for the household. I grinned and said, “We’re number sixteen on that list,” playing along with her. I guess it’s taboo to say income figures out loud.

But when we got to the question of race/ethnicity (I forget the actual wording on this one), that’s where we ran into trouble. Ever since participating in my first census as an adult in 1990, I have filled in the circle next to “Other” and hand-written “human” on the blank line next to that. I have a personal philosophy that paying attention to non-affective differences only perpetuates discord. I’ve not read anything yet that clearly correlates someone’s ability with their family tree. There’s a lot that goes into ability: nature, nurture, being in the right place at the right time, even love of doing something, as Malcolm Gladwell espouses. I remember taking an aptitude test in high school. Sitting in the chair next to me was my best friend, whose last name also happened to be Young. We called ourselves twins, even though we weren’t related. I remember feeling mortified that, when I looked over at the front cover of her test booklet, I had one bubble filled out differently than hers. Race. She’s descended from people who came from China six generations ago. I’m descended from people who came to California six generations ago from Canada and Europe. My best friend and I were twins to the core, so why did this stupid test booklet cover declare that we weren’t? I remember trembling I was so upset, right there in the classroom where they gave the high school aptitude test. So, my personal philosophy about this is pretty strong. And when the poor Field Rep asked, I instructed her to just choose the “Other” radio button. She read the possible answers out loud, and the only place the word “other” appeared was as “Other Pacific Islander.” She tried to skip it, but the survey software defaulted to “Latino.” The Supervisor stood up and came to bend over the laptop with the Field Rep. They took a few moments to figure out what they could do. The Field Rep looked up. “Is it okay if I just mark down “Caucasian” for you? Nope. It’s not okay. I make eye contact with the Supervisor and smile. She keeps a straight face, trying to hide that I’ve made her feel uncomfortable. The Supervisor pointed to something and they clicked, relief flooding their faces. I have no idea what they did, or how my answer was recorded. I smiled and chatted about how software sure isn’t designed well.

Imprecise English

The next part of the survey asked about my type of work. The first question was, “Do you run a business or a farm?” I laughed, because of all the possible things a person could do for a living, why pick just those two to begin with? I have a veggie garden but I don’t sell the produce, so I figured “farm” meant “selling produce” and I opted out of that one. But “business?” To me, that means I run an agency or a shop or a firm–something with employees and a location. I don’t have any employees, and I do my work in my living room, or on the ferry, or at the airport, or in client conference rooms. I didn’t think I was a “business” either, so I explained this to the Field Rep. She asked, “Do you work for someone else?” “Yes,” I said, “for my clients.” She shook her head, “No, you run your own business.” Okay. There is is. “Running a business” equals “not a W4 employee.”

“So, describe what work you do,” the Field Rep asks next. I pause, for a long time. You know why–how do we explain this field? How do we put it into a couple of words that have meaning in the traditional sense of “what do you do for work?” If I’m a mechanic, I say, “I fix cars.” If I’m an ex-software engineer trying to encourage big corporations, governments, universities, and start-ups to go talk to real people before investing in a product or service that might flop, I say, “Um. Uh.” The Supervisor, having heard my chatter about what I do before the survey started, tries to help, “You do software?” I try to un-fumble myself, saying, “I actually do research, and analysis, and …” The Field Rep starts typing in my response. She says out loud, “Researching, Analyzing … what? Analyzing what? They like ‘ing’ words.” Ech! My eyes roll up for half a second, as she has pressed another soapbox button on silly old me. It peeves me when people make verbs into gerunds by adding “ing” to them. It weakens the verb, makes it into a noun that can be boxed up and kept at a distance rather than tasted and experienced as an action. “I research and I analyze the data I collect.” I hear myself saying, “I hate gerunds.” Eek! Sorry ladies–am I turning into one of the uncooperative survey participants? Anyway, I finally concoct a sentence in my head that I’m willing to have recorded on the survey. “I understand people in real life situations so we can design products and services to support them.” The Field Rep types it in slowly. “Whoops,” she says toward the end. “We’re out of room.” I wonder how much of my explanation fit in the box.

“For the week of December 5th, a Sunday, through December 11th, a Saturday, how many people did you do work for, including part-time and evening?” I immediately wonder if having two clients last week counts as two jobs. I ask about this. The Field Rep tells me, “No, you count all your clients as one job.” Good to know.

Then the special just-for-this-week section of the survey begins. It’s a sub-survey by the Department of Agriculture wondering whether I have had enough food to eat since last December. I’m still not clear on what their scope of research was. They wanted to talk to the person who does the shopping and cooking for the household. I was stumped on that one, because Philip does most of the shopping, and I do most of the cooking. “The person most knowledgeable about food,” they amended. Well, we had just finished outlining how Philip got his Masters degree in Plant Science. “He’s technically more knowledgeable about food, but maybe I could answer for us both?” The ladies laughed politely, and I wondered why the survey writer assumed that a person shopping also does the cooking–that those two chores are always only done by one person. The survey writer must not have any idea of the mathematical properties of the word AND. Alas. (It means if shopping is true and cooking is false (in Philip’s case), then the answer is false. If shopping is false and cooking is true (in my case), then the answer is still false. It’s only a true answer if both shopping and cooking are true, which is not necessarily the case for anyone in our household.)

Another of the questions was about whether I or another household member (meaning Philip) had shopped at a grocery store or supermarket last week. “Does Trader Joe’s count?” I ask jokingly. “Philip does the shopping there. I do that shopping at the farmers market.” The ladies do not smile. “How much did you spend there last week?” I don’t know, maybe around $40, and I explain he has a bowl full of receipts that I can rummage through if they want me to. Yes, they want me to. To simplify things, I just grab the top receipt from Trader Joe’s. Later, I remember he went to Target last week for his monthly stock-up on his favorite cereal. I didn’t feel like really looking for all the Trader Joe’s receipts from last week, and I didn’t want to paw through his stuff and find other receipts from Carl’s Junior or Burger King that I really didn’t want to know about. That’s his business, not mine. So I brought the receipt back to the table. “Fifty dollars.” I hold out the receipt. The Field Rep looks doubtful. The Supervisor says, “Hey, you were pretty close!” I read the total at the bottom, “It actually says $49.59, but I think you want to round up for these surveys, right?” The Field Rep continues frowning. She asks, “Have you bought food elsewhere …” And I interrupt, repeating, “I shop at the farmers market. Every Thursday.” “… at a butcher, a bakery, a produce stand, or a convenience store? A convenience store is like a 7-11.” I look at her. She suggests, “Maybe a farmers market is like a produce stand?” The Supervisor says, “They have produce stands in Fresno,” (or was it Modesto?) from which comment I deduce that’s where the survey writers work. “Let’s put down ‘yes’ to this one.” I look at the Supervisor and muse out loud why they don’t have farmers market in the answer set, and was this survey written 15 years ago? The Supervisor laughs and says, “It’s because the programmers are antiquated.” I think she means the survey writers. In Fresno. Or Modesto. I begin to wonder if she has hung out with them, to begin to know that they’re “antiquated” in their thinking patterns. I agree with her thinking, then realize she said that just to agree with my musings, which are growing darker because this line of thinking only illustrates how much the world view of the survey writer influences the results coming out of the survey.

The final Department of Agriculture question makes me laugh, because the wording is so cunning. I actually feel impressed by the writer. “Have you had enough money to buy the foods that you have wanted to eat in the past year?” Yes, that’s what you read: “Wanted to eat.” It just works on so many levels! The writer adroitly steps around the guilt and (resulting prevarication) a participant might feel if the question asked about basic food groups or foods you needed for basic nutrition. If a participant lives on coffee and hamburgers, this question works just fine. There were five answers, varying from getting all the food you wanted to eat to getting very little food you wanted to eat. I could have said I was not getting enough chocolate cookies or brownies to eat, but I decided to go with the first answer. I get enough food, yes.

Fooling with the Answers

The third thing that really left me with a need to write this essay is that none of the answers I gave were precise. Quite a few of them were guesses or “close-enoughs.” If the statisticians are using the data from 55,000 U.S. households to calculate the official, important, complete, and accurate final results, and 55,000 real-live, human, unique participants are approximating numbers and shrugging their shoulders about which radio button to select, then how helpful is that information?

Here are some examples. One question about work asked how many hours I and the other household members worked last week. “Do you mean billable hours, in which I profited, or just all the hours I spent working on the computer? I also write and create workshop materials for various conferences I attend.” They are silent, pondering this. I walk across the room and grab my laptop from my desk, set it between the Field Rep and the Supervisor, and show them my spreadsheet. For the past two decades, I have recorded data in 15-minute increments about what projects and articles I’m working on, or if I’m “networking” or “reading” materials germane to my industry, or having “sales” calls, etc. I have twenty spreadsheets with this data, one per year since 1991. I am a bit of a data junkie. In last week’s column my spreadsheet shows I worked 10.5 billable hours, 0 sales hours, 12.75 hours doing stuff like maintaining my computer, touching base with people, reading articles, and commuting to a holiday party, 2.25 hours on this blog, and 2.5 hours making arrangements for speaking and teaching workshops. That’s a total of 28 hour last week. “Since this is less than 35 hours, is there a reason you did not work full time last week?” the Field Rep asks. I cannot resist musing out loud that there is a judgement in that phrase “did not work full time” and a strange cut off where something like 34.5 hours does not qualify as full time. I was surely at my computer doing work every day. I just don’t count the 15 minutes every hour that I spent writing to a friend or looking for a Christmas gift for my nephew. People who are employees also write to their friends and buy gifts for their nephews, surreptitiously, but it gets counted as “full time.” I admit I shouldn’t have reacted emotionally to the fact that my careful accounting cast aspersions unto me, but I did react. I think the ladies realized it, too.

If the statisticians could get their hands on my spreadsheets, they’d probably be thrilled. If they could just get information from payroll data, they’d be happy. Instead, they have to coax imprecise data from participants through the gauzy medium of a spoken survey.

Another example was the question, “How much could you save a month on food items?” I explained that I don’t look at food prices, I just buy what is locally grown and fresh and in season. I don’t know what the other version of this would cost at the grocery store. I never look. So I guessed $15. Shrug. Who would know how much they could save, anyway? What an odd question. What were they after?

Then the Field Rep read a question about household members buying food from a list of places that included vending machines. I never buy from vending machines, but I have no idea whether Philip does. The subject has never come up in the 11 years we’ve been together. So I had to guess about the answer to that question, too. Sorry, Department of Agriculture.

Oh, and I forgot. After the ladies left, I realized last week I had placed a $100 order for 65% chocolate chips from my favorite vendor, Sweet Earth in San Luis Obispo. I buy in bulk, 10 pounds at a time. Mint says I’ve bought these chocolate chips in bulk every other month. I forgot to mention it because the Department of Agriculture sub-survey never asked about online food purchases. I had totaled about $116 in food spending last week for the survey, which would average to around $550 a month, $1100 every two months. (Mint says this is about average for the U.S. participants that have signed up for their service.) The Department of Agriculture is missing almost 10% of my food spending by omitting online food purchases.

What Are the Numbers About?

In the end, all this effort that participants, field reps, supervisors, statisticians, and everyone at the U.S. Census Bureau put in adds up to … what? The letter says, “total employment and unemployment issued each month … estimates of the number of people working … help direct programs … and … provide … summary information about … participation in various government programs.” The letter says they include retirees as participants, and obviously they include the self-employed, since they are asking me to participate. I wonder how this particular collection of guesses and approximations got started. Who needed to know what, exactly? Is this question still valid today, or is the process/program self-perpetuating? I know one thing for certain: when I hear that monthly unemployment figures are up or down, I won’t pay any attention. It’s a statement based on 55,000 people waving their hands and rolling their eyes.

I have another seven months to look forward to participating in this survey. Giving my important and accurate guesses to the individuals at the Census Bureau (who are not bold enough to improve this data-collection project) is my distinct honor.

(I ought to state here that by the end of their visit, the supervisor thanked me and gave me some very nice compliments while the Field Rep turned gray-faced and made an abrupt dash for the front door. I was not sure how to react. I started apologizing, as if my musings had been too much for her, but perhaps she was only hungry. After all, they showed up at 11:45am and left an hour later. I did hear the Field Rep’s stomach grumble. I don’t think they had expected me to be at home, available to participate immediately. Hopefully she was just hungry, and wasn’t worried sick about the next time she has to talk to me.)

What Can Research Help Us Discover?

Like you, I’ve been experimenting with eReaders. Google’s announcement of its software for several platforms helped me consolidate my thoughts on the subject. My conclusion is that they’re not where I want them to be, functionally. Couple that with the fact the number of titles in my favorite genre available from the state-wide library can be counted on four hands, and you can guess that I’m not ready to invest money yet in this first generation of eReaders. Title availability aside, I believe this first few years worth of effort behind eReaders represents a transparent reach by companies for my pocketbook. There’s no heart behind their products yet, no deep attempt on their part to support my reading in a better way. They merely allow me to read in an obvious, flat, linear way. I am convinced that they need to understand the things people currently do in the real world in reaction to their reading, which will open up many options for functionality that will truly bring a step change to the experience of reading.

What do I mean? I don’t mean evaluating how people use eReaders today. All the companies have practices in place already to conduct evaluative research into the experience with their existing products. They need to turn to generative research to obtain the insights that I mean. They need an understanding of people’s intentions and thought-processes while reading, regardless of tools, medium, or services. From this understanding they can generate truly new support for readers. For example:

• Shujie puts her book down to ask her husband, there on the couch with her, whether he agrees with a particular viewpoint she just read.
• Adam folds a corner of a page to look at later when he’s in front of the computer and can Google the concept he read about there.
• Gerardo reads the three chapters of the textbook assigned, knowing there will be a quiz tomorrow. He highlights the sentences he wants to remember.
• Johan wonders while reading a famous passage how it might have influenced Winston Churchill during World War II, which he just watched a program about.
• Jocelyn reads something and smiles, thinking briefly of her friend Karen who would laugh at that sentence. She then reads on.

What are all the real-life thoughts, actions, and emotional reactions that go through people’s minds and sometimes get translated to immediate or delayed action? Can these activities and philosophies be compared and grouped into a model that might illuminate our path forward? If we can understand these behaviors, then we can discover the strongest threads that will pull readers toward a smoothly supportive digital reading experience. Generative research like this have the side benefit of producing stable results that shift only over decades and human generations. Therefore research needed now can serve designers for a decade to come.

How Can eReader Software Improve?

Currently, all eReader software features are pretty similar. People can highlight text and write notes, and of course change the fonts and background colors and sizes. They can flow text differently to different devices, and they can synchronize across those devices. They can skip around and bookmark pages. But none of this is a completely compelling reason to switch from an analog to digital reading medium. People making the jump to digital right now are true pioneers, putting up with very limited functionality in their digital readers. Not everyone sees benefits to switching from analog to digital. To be compelling, a digital reading experience must support the little things that flit through people’s heads as they read. Without having completed any research yet myself, here are three imagined examples of how a digital experience can be made more powerful:

• Katrin reads a beautiful passage in a book by her favorite author. She re-reads it several times, savoring it like fine wine. She wishes to “re-enjoy” that passage later, perhaps six months from now, without actually reaching out to the book itself, which remains “on her shelf.” She also wants to mention it to her friend Marisa, whom she thinks will enjoy it, too. Using the eReader software, she emails the passage to herself in six months, and to Marisa today. Katrin can enjoy the passage with her friend today.Then, six months later, she finds an email in her inbox from her eReader software with this passage, and she smiles, enjoying it once more.
• Paul finds a meaningful passage in a book on philosophy. He wants to incorporate it in his life–make it a part of his daily practice. Using the eReader software, he sets up a daily afternoon text to his smart phone with the passage, accompanied by a message to himself saying, “Paul, my friend, have you followed this today?” Each day he sees this text from his eReader software and evaluates whether he has done well with regard to the philosophy.
• AeSook pulls all the highlighted phrases from the chapters she has been studying into a separate study list. She finds most of the phrases in the list understandable, but one of them puzzles her. Wanting to remember the context of that phrase, she looks up the original paragraph and reads backward and forward a bit to recall the point the text is trying to make. With this enhanced understanding, she feels ready for the quiz tomorrow.

We are on the cusp of the digital reading age. There are hundreds of possibilities that will become “the norm” for humanity with regard to reading. The potential is massive. Many companies are jostling for position to guide the design of the new digital experience. Using a deep understanding of the things people think and do the way they read now, regardless of the analog or digital medium, will have a strong impact on the future direction of eReader software.

I sure wish I was on a team doing this research instead of dreaming about it!

One of my clients, who has spent her career getting to know customers, is in the midst of conducting a series of generative interviews. Under time pressure to finish (and analyze) these interviews in a less-than-ideal length of time, she is setting up all sorts of tools to help her get great results quickly. She says, “I want to improve my skills and generate more tasks-per-million–a denser, richer interview–if you will. I need to whittle down the five minutes of my stumbling and stuttering to just two minutes.”

• I pull my own questions from the transcript–just the questions–and critique them. This way I can see where I should have asked him to tell me a story or get more detail.
• I’ve decided I can’t use adjectives because when I use them, I am leading the participant.
• I have a bad habit: I’m programmed to take notes. I find myself writing instead of listening intently to the participant. I need to get better at just remembering key words they say, so I can get back to them in the conversation, instead of writing lots of notes.

I am impressed with the power of her approaches.

If you’re going to the IxDA 2011 conference in Boulder, Colorado in February but haven’t signed up for a workshop, I’m teaching about interviewing styles.