Search Analytics for Your Site

Search Analytics for your Site is an excellent, comprehensive approach to understanding and demystifying the black box of search. From understanding the basics of search frequencies along the Zipf curve, to discerning and interpreting search patterns, deriving clusters, understanding tone and intent, analyzing time variance, performing failure analysis and even using search analytics as an input to (my favorites) taxonomy development and contextual navigation, this book is an excellent resource for taking the guesswork and bias out of search remediation and improvement projects. Lou provides anecdotes that are easy to read along with more advanced approaches for interpreting findings. Not just about theory, the book finishes with practical tips and actionable approaches for improving search performance. More than anything, Search Analytics for your Site eliminates the guesswork and allows you to justify search remediation projects as well as fund and justify a variety of interventions that will improve information findability. Highly recommended. Nice job.

Seth Earley, CEO, Earley & Associates, Inc.

Search is one of those mission-critical aspects of every web site that is sadly all to often forgotten until the damage has already been done. Lou, on the other hand, is one of those guys who understands search analytics and the opportunity associated with digging into the nuance of customer and search behaviors to mine for organizational gold. In Search Analytics for Your Site, Lou lays out pretty much everything you need to know to mine for that gold and convert it into positive customer experience on your site.

Eric T. Peterson, Founder and Author, Web Analytics Demystified

Clients have asked me countless times to pretty up their search results page design, as if this would distract users from realizing that they’re getting lousy results. That’s no longer necessary, thanks to Lou’s book.

Karen McGrane, Managing Partner, Bond Art + Science

At last a book that explains exactly how to get the best from search analytics so that users can actually find what they are looking for.

Martin White, Managing Director, Intranet Focus Ltd, and author of Making Search Work

Analytics are the single most important tool you have to improve your search experience, and Lou Rosenfeld’s world-class expertise in user-centered design is the place to start.

Pete Bell, co-founder, Endeca

Louis Rosenfeld’s Search Analytics for Your Site is a superlative work from the initial story to the final chapter on bridging web analytics and UX practice. I’m somewhat experienced with event logging methods, but Louis’ book opened my mind to new ways to use analytics. Each chapter is packed with useful information, clear examples, and refreshing caveats that could only come from a master of search analytics. The book is written in an engaging style that makes you feel like Louis is with you on every page. I plan to apply some of the knowledge and techniques immediately. Great book!

Chauncey Wilson, Senior Manager, User Research

If we all agree that user feedback will improve any site’s user experience, why aren’t we spending more time with the actual words our audience uses when asking us for stuff? I can’t imagine a more experienced guide than Lou Rosenfeld to help us put this amazing data to work.

Jeffrey Veen, Founder & CEO, Typekit

Lou is the perfect author to tackle what is essentially unexplored territory in the UX community. With Search Analytics for Your Site, he has uncovered a huge goldmine for UX professionals of all stripes: now we have the tools to finally, finally fix our website and intranet search experiences. This is one of those rare books that makes me pound the table with my fist and yell, ‘Yes! Exactly! Awesome!’ while I’m reading it.

Kristina Halvorson, CEO, Brain Traffic, and author, Content Strategy for the Web

Lou Rosenfeld provides remarkable clarity, insight, and humor on the complicated world of search site analytics, Search Analytics for Your Site will no doubt be an indispensable resource for anyone involved in user experience and web analytics.

Bill Albert, Ph.D, Director, Design and Usability Center, Bentley University

The potential value behind the queries issued by your customers is in practice unbounded. So do not waste this potential— use the knowledge behind these queries. For that you have to understand search analytics and hence you must read this book.

Ricardo Baeza-Yates, VP of Yahoo! Research

• Chapter 1: How Site Search Analytics Can Save Your Butt
• Chapter 2: Site Search Analytics in a Nutshell
• Chapter 3: Pattern Analysis
• Chapter 4: Failure Analysis
• Chapter 5: Session Analysis
• Chapter 6: Audience Analysis
• Chapter 7: Goal-Based Analysis
• Chapter 8: Practical Tips for Improving Search
• Chapter 9: Practical Tips for Improving Site Navigation and Metadata
• Chapter 10: Practical Tips for Improving Content
• Chapter 11: Bridging Web Analytics and User Experience

These common questions about site search analytics and their short answers are taken from Louis Rosenfeld’s book Search Analytics for your SIte [NEW LINK]. You can find longer answers to each in your copy of the book, either printed or digital version.

1. What is site search analytics (SSA)?
   If your Web site or intranet has a search engine, then you can log what users are searching for, tally queries to see what’s most important to your users, find out if they’re succeeding, and if they’re not, determine what might be getting in their way. Chapter 2 provides a short introduction to SSA (which is often also known as search log analysis).
2. Isn’t SSA the same as SEO?
   Not at all. Search engine optimization looks for ways to make Web-wide searches (for example, via Google and Bing) more likely to find your site. SSA looks for ways to improve how searching works on your site, using your site’s own search engine. That said, SSA and SEO share much in common, and can influence each other; for example, pp 90–91 shows how SSA may help you determine better keywords to bid on.
3. How does SSA differ from other kinds of analytics?
   SSA is based on data that comes from actual usage of your site, just like other forms of web analytics. But it’s far more semantic, as it is made of search queries—users’ expressions of what information they want from your site in their own words. That’s why SSA does a better job of depicting and helping you understand users’ intentions than any other form of web analytics. Chapter 3 provides some tools for analyzing and understanding users intentions.
4. Why do I need SSA?
   Because SSA can help drive—and back up—your design decisions. Because you already have query data and want to put it to good use. Because you want to make your search engine find content better (Chapter 8), your site easier to navigate (Chapter 9), and your content more effective (Chapter 10). And because your competitors aren’t using it, and you’d like to destroy your competitors.
5. Where does query data come from, and what tools do I need to analyze it?
   When someone uses your site’s search engine, they type a query that the engine will try to match with your site’s content. That query can be (and likely is being) saved. It’s either logged in a text file by your search engine or intercepted and kept in a database by your analytics application. Search engines occasionally and analytics tools increasingly provide reports that help you analyze the data, but ideally you’ll explore and learn more from the data in a spreadsheet. Unfortunately, there’s no one way to get your hands on the data, because how you can get at the data often depends on what search engine you’re using. Talk to your organization’s IT people for help and show them this book if they ask why you need access to the data.
6. I’m not a “data person,” so why should I read this book?
   Organizations are putting more and more pressure on designers to justify their decisions with evidence. Fortunately, SSA is real data that’s also semantically rich, so you won’t be just looking at numbers. And you won’t need to perform statistical tests to learn from it; in fact, it will be immediately obvious to you how it can help improve your design work.
7. This isn’t part of my job description (or anyone else’s) so who should do this work?
   User researchers and the designers who rely upon user research—such as information architects, content strategists, interaction designers, and knowledge managers—should at least consider SSA as a part of their standard kits of research tools, even if it’s not something they use on a regular basis. The same goes for web analytics practitioners: SSA is an important tool, just like clickstream analysis. The best part is that no one needs to do SSA as a job—it scales nicely, depending on the time you have available (see our discussion of the Zipf Distribution on pp.19–21).
8. How do I actually analyze query data?
   First, “play” with the data by looking for patterns and surprises that suggest what’s important to your searchers and what kinds of content will best meet their needs (Chapter 3). Then identify and learn from searchers’ failures (Chapter 4). See what happens in the course of single search sessions (Chapter 5), and tease out what’s important to specific audiences of searchers (Chapter 6). Finally, measure your site’s performance better by injecting search metrics into how your site is performing at meeting its goals (Chapter 7).
9. How does SSA fit with other user research methods?
   SSA is based mostly on quantitative behavioral data; therefore, it’s useful to combine it with your qualitative user research methods and tools. For example, use query data to help determine candidates’ tasks for your task analysis studies or to beef up your personas with real data. Chapter 11 talks about how SSA fits into the broader worlds of both user experience research and web analytics, and how it may be a great means for bringing them together.
10. If it’s so great, how come more people aren’t taking advantage of SSA?
    Good question. Most people don’t know that query data even exists, much less that their organizations likely already own some. Those who do often run into political problems when they try to get their hands on the data, because it is usually owned by IT or some other group. (This is getting easier thanks to ever-improving analytics tools.) Finally, there hasn’t been much practical information on how to analyze the data. Maybe this book will help.

• Chapter 2 (PDF)

A funny thing happened the first time Lou and I teamed up to teach our day-long public workshops (mine on usability, his on information architecture), probably eight years ago now. I went to his workshop—the day before mine—partly out of due diligence, but mostly because I’ve always enjoyed listening to Lou, and I knew I’d learn a lot. Halfway through the day, Lou spent 10 minutes talking about something I think he called “search log analysis” at the time. Basically, you get your hands on the log data for your site’s search engine so you can see what terms people are searching for most often. Then you take the most searched- for items (say, the top 25) for the current month, execute the searches yourself, and see what you can learn from them.

For instance:

• Were there any results? If not, maybe you need to add content, or at least figure out why people on your site are looking for something you don’t have.
• If there were results, were they the best content your site has on the topic? If not, you may want to tweak your search engine or fiddle with some keywords.
• And why were people using Search to find these things? Was it because it’s not obvious how to get to them through your navigation?

Basically, Lou was suggesting that you spend a very small amount of time each month to see if people are finding the things they’re looking for on your site. Then you tweak your navigation, your content, or your search engine as needed to make sure they can. The next month, you do it again, with the new top 25. The funny thing was that I’d been making almost exactly the same speech in my workshops for a long time, pretty much word for word.

When I mentioned it to Lou later, it turned out that even though the technique was really off-topic for both of our workshops, we both thought that we just had to tell people about it, because we thought it was the most cost-effective thing people could do to improve their site. Very little effort, very big payoff, and almost no skills required. And virtually no one knew about it.

Fast-forward, several years. Besides starting up a publishing company, having two great kids, and relocating from Ann Arbor to Brooklyn, Lou decided to write a book. About this topic. Which he now called Site Search Analytics.

Fast-forward several more years to today, when you’re holding the book in your hands. (Or reading it on a digital device that didn’t exist when this all started.) Lucky you. You get the benefit of years of pondering, researching, inventing, and fine-tuning by somebody as smart as Lou. (In this case, Lou.)

One caution: Don’t be intimidated by the soup-to-nuts scope of the book. Even though Lou spells out a lot that you can do, you don’t have to absorb (or even read; sorry, Lou) all of it. I encourage you to just try a little bit; dip a toe in the water, if you will. With nothing more than this book, a few hours of your time, and perhaps a copy of Excel, you’ll be amazed at how much better your site can be. And once you see how much you can learn in a few hours, don’t be surprised if you get hooked and want to do a lot more. That’s when you’ll be glad Lou wrote the whole book.

Have fun.

—Steve Krug
Author, Don’t Make Me Think! and Rocket Surgery Made Easy

I love internal site search data. Completely. My love emanates from a singular fact: Of all the data we have access to, site search is the only place where we have direct access to visitor intent. When people click on links to visit your Web site, you know the sites or search engines they come from. How much intent does that communicate to you? 10%. It is really hard to know from that data why people might be showing up.

How much intent is there in the keywords that people type into search engines like Google, Bing, or Baidu? Maybe a bit more than 10%, but honestly not that much. Our beloved visitors are notorious for being deliberately vague when they use search engines. Yet when people search Web sites, they become astonishingly precise about why they are there. The queries they type into site search engines contain oodles of intent, just waiting for us to convert into insights that drive greater customer satisfaction. Over the last couple of years, it’s been amazing to see how much valuable intent data is now available from almost all analytics tools, including Google Analytics. What words did people type? How many of them left your site because search results were so horrible? How many people had to refine their queries to get your search engine to cough up the right answer? Is there a material difference between conversion rates for people who use site search users and those who don’t?

With site search analytics, all these questions and more can now be more easily answered. So now that the data is available, how does your site go from merely okay to magnificently glorious? That’s where this lovely book by Lou comes into play. Gently holding your hands, whispering soothing words, Lou will guide you through this rich and untapped world. You’ll start simple: just reading Chapter 2 will bring you 10x the return on what you paid for the book. Subsequent chapters will take you deeper and empower you to answer invaluable questions. How do you understand the patterns in your data (Chapter 4)? How do you analyze the audience (Chapter 7)? How do you achieve the nirvana of bridging the world of quantitative and the qualitative (Chapter 11)?

All will be revealed using real-world examples, practical actionable tips, and a precision that will yield immediate benefit to your website visitors (and long-term benefits to your own salary!). Carpe Diem!

—Avinash Kaushik
Author, Web Analytics 2.0 and Web Analytics: An Hour A Day

I’ve heard it’s lonely at the top. It also must be lonely at the (fore)front. I say that because the smartest people I know—the ones five or ten years ahead of the rest of us—are invariably the ones who are most willing to share what they know. Perhaps they’re just dying for someone to see that they’ve stumbled upon a fantastic new way to solve a problem. Or maybe being brilliant and generous just go hand in hand.

Whatever it is, a few of those really smart people shared what they knew with me, and without them, this book wouldn’t have happened. One was my old friend from Michigan, Rich Wiggins. He introduced me to site search analytics about 10 years ago. Rich knew that SSA could close a critical feedback loop in the user experience of search, and somehow managed to convince his bosses at Michigan State University to let him institute an SSA program. Today, MSU is one of the most successful institutional users of SSA, and many of this book’s examples are a product of its—and Rich’s—excellent work.

Rich’s and my shared interest in SSA connected us with two other smart people, Avi Rappoport and Walter Underwood, who have both made short contributions to this book. If you’re looking for people to analyze your query data, you won’t find any better than Avi and Walter.

Like a lot of people in user experience, I’m very much a right-brain person, and I’m not entirely comfortable with data. Marko Hurst is one of those few people fortunate to function with a full brain. Marko can combine qualitative and quantitative analyses like no one I’ve met in the field, and he made sure this book took a far more balanced perspective on how to derive value from query data.

I was fortunate to have five tough technical reviewers who reflected a healthy variety of perspectives. Steve Krug, Avi Rappoport, Vivian Bliss, John Godinez, and Anil Batra were truly a dream team of practical advisors and healthy challengers; the book is much improved thanks to their efforts. And it benefited further from great contributions by these experts: Gary Angel, Martin Belam, Tom Chi, and Greg Nudelman.

A huge cadre contributed stories (and sometimes data) about their work for me to use in this book (most of which I used). They were often the same people who kept at me over too many years to get the book written, and I’m grateful to them: Mary Ambrosio, Meredith Anderson, Ricardo Baeza-Yates, Jeannine Bartlett, Fred Beecher, Christopher Billick, Mandy Brown, Lorelei Brown, Christine Connors, Justin Cutroni, Richard Dalton, Liz Danzico, John Ferrara, Alexandra Fox, Manya Kapikian, Nick Finck, Zach Gemignani, Dave Gray, Steve Hatch, Jason Hibbets, Caroline Jarrett, Avinash Kaushik, Phil Kemelor, Neil Kohl, Chris Kutler, Jeff Lash, Fred Leise, Helen Lippell, Karen Loasby, Lydia Mann, Matthew Marco, Ravi Mynampaty, Cynthia Osiecki, Robert Piddocke, Whitney Quesenbery, Brendan Quinn, Susan Rogers, Shaun Ryan, Denise Shanks, Tito Sierra, Johnny Snellgrove, Jared Spool, Mike Steckel, R. Todd Stephens, Walter Underwood, Guy Valerio, Javier Velasco, Abbie Walsh, Jennifer Whalen, Denise Wood, and Jeffrey Zeldman.

It’s strange when the author is also the publisher. Yet everyone involved in the book’s development and production made life easier for me, regardless of which hat I wore. It was an absolute joy to work with Stephanie Zhong, the book’s developmental editor. Given that she is using SSA at her day job at Teach For America, she was the ideal editor to work with. (It also helps that she’s very patient.) Karen Corbett, Rosenfeld Media’s Director of Operations, helped me in more ways on a daily basis than I can recount. (She’s also very patient.) And Rosenfeld Media’s crack production team maintained its high standards in creating and assembling the final product; many, many thanks to Marta Justak (our managing editor), Danielle Foster (our interior designer), The Heads of State (who create our covers), Chuck Hutchinson (proofreader), and Nancy Guenther (indexer).

Finally, my wife, Mary Jean Babic, always looks at my new book’s acknowledgments pages first—as well she should. She knows when she’s owed big-time, and this book—which took me forever to write—is such a case. Hopefully, Iris and Nate will inherit her genes for supportiveness and good counsel, rather than mine for procrastination and metaphor mixology.

I’m very glad that brilliant and generous often do go hand in hand. Thanks all.

Search Analytics for Your Site Blog View all Blog posts

• Zipf, meet Smashing,
  on May 17, 2012
• Updated site search analytics deck,
  on October 12, 2011
• An interview and a review,
  on August 2, 2011
• Interviewed in SearchEngineLand,
  on July 11, 2011
• Now on sale!,
  on July 6, 2011