This week in my class on usability we learned about usability tests with some specifics on the number of participants to involve. Some of what we learned was frankly, in my opinion, fascinating and other parts seemed to contradict each other.
I’ll start with a bit of context for clarity’s sake. If you have ever bothered to fill out one of those pop-up surveys that many corporate web sites seem to have then you’ve participated in a usability (or market research) study. When you see websites that are asking for feedback on their new site then they are conducting a usability study, and they want you to volunteer some time and participate. Realizing this I connection I can now say I’ve been an unwitting test participant more times than I can count.
What I learned from some of my readings (Jakob Nielsen: “Why you only need to test with five users.”) this week was that these companies don’t (usually) need to involve many people at all. That on average a mere six (yes 6) participants will find 90% of the problems with any product whether it be a website, app, or physical product. This is a concept that is amazing to me because I know in the past most research usually involves hundreds if not thousands of participants to get results, but with usability problems a mere six or so will do.
This idea does have its competition though.
“In a recent study, we decided to put the widely held belief that “eight users is enough” to the test. …
When we tested the site with 18 users, we identified 247 total obstacles-to-purchase. Contrary to our expectations, we saw new usability problems throughout the testing sessions. In fact, we saw more than five new obstacles for each user we tested.
Equally important, we found many serious problems for the first time with some of our later users. What was even more surprising to us was that repeat usability problems did not increase as testing progressed.” (Perfetti & Landesman: “Eight is not enough.”)
Needless to say this is certainly a contradiction of ideas that needs to be noted, at least in my opinion, by myself and other new recruits to the usability field.
Looking forward to learning and sharing more with you all, and more than likely learning from you also.