First Prototyping Survey Results

Back in June 2008, I ran a survey on prototyping. Close to 200 people participated, primarily from the US. The respondents were nearly 60/40 Male/Female and represented a mix of roles in the UX community.

The goal — challenge some insights I’ve found through interviews, see who’s prototyping, what tools/methods they’re using, find some succss stories, find some failures.

The result — as expected, a number of assumptions were validated, while others were challenged. I was surprised a bit by the number of people who’s activities were those of a Business Analyst (BAs), which was not an audience I expected to hear from. However, Axure and iRise, two prototyping tools, are relatively common for BAs.

I’m not shy about the fact that I’m not a big fan of surveys, however, they do have their place. My main dislike for surveys is their lack of ability to properly address the "Why?" question.

Good survey design is part art, part science. And in hindsight, I would have changed a few things. Something to learn for future surveys:

• There are a lot of tools used in combination to produce prototypes. Asking participants to weight scale their tools to total 100% might have been helpful.
• Title is rarely an accurate representation of what people do for a living — at least in the UX field. Close to 20% of people responded perform Business Analyst tasks, but just over 2% self identify as BAs.
• A large percentage of participants participate in cross-role activities (e.g. design and usability testing). Maybe next time I’ll have role and title as two separate questions, or possibly just dump title all together. I’m really more interested in activities people are performing than what their title is.

Summary Findings

• Paper tops the list as the most method, followed by hand-coded HTML is the second most popular method.
• Paper, Visio, PowerPoint, Dreamweaver, Axure, and OmniGraffle topped the list for most common tools.
• There were an equal number of people using Dreamweaver and Axure.
• Visio and PowerPoint are very common tools, especially in larger corporate environments.
• OmniGraffle and Keynote are very common in mid-small sized companies.
• Visual designers are more likely to use a tool like Dreamweaver to slap-and-map (link together screens with HTML), or PowerPoint/Keynote to present screens as a prototyping method.
• Respondants were almost equally split across 1-3 years, 3-5 years, and more than 5 years experience with prototyping.
• The most important factor for using a tool/method is time and effort to produce a working prototype, followed by a prototype usable for testing, and price.

I’ll be running some additional surveys in the future, which will be more focused on specific methods and tools.

Survey Results

Most Important Factors in a Prototyping Tool/Method

1. Time and effort to produce a working prototype.
2. Creating usable prototypes for testing.
3. Price
4. Learning curve.
5. Ability to create own GUI widgets and patterns.
6. Platform independence.
7. Collaborative/remote design capabilities.
8. Built-in solutions/patterns (e.g. AJAX transitions)
9. Built-in GUI widgets.
10. Creating usable source code.