03 Aug Heuristic Analysis for UX – How to Run a Usability Evaluation
Design is an investment,
not an expense. At the risk of stating the obvious: It’s not enough to design a nice-looking product; it also has to be usable, and if you are to extract the largest ROI from a product, its usability—which generally refers to ease-of-use—takes on a vital importance.
Well-designed products have great usability. Great usability is a significant contributor to product quality and a seamless user experience.
There are a few ways a product’s usability can be tested: an inspection method called a heuristic analysis is one of them. This usually means running a heuristic evaluation on a product, whether it already exists or is brand new.
What Are Heuristics and What Is a Heuristic Analysis?
A heuristic analysis is used to identify a product’s common usability issues so that the problems can be resolved, consequently improving the user’s satisfaction and experience and raising the chances of a digital product’s success overall.
Focusing on usability, a heuristic analysis is an evaluation method in which one or more experts compare a digital product’s design to a list of predefined design principles (commonly referred to as heuristics) and identify where the product is not following those principles.
An expert reviewer performs a heuristic analysis of a website to identifying usability issues
A usability expert performs a heuristic analysis against a set of heuristics to identify usability issues.
A specific set of heuristics contains empirical rules of thumb, best practices, standards, rules, and conventions that have been tested or observed over long periods of time. Sticking to these heuristic standards produce UX designs that simply work better.
“Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles (the ‘heuristics’).” — Jakob Nielsen, The Nielsen Norman Group
A heuristic evaluation is not a usability test (for example, a one-on-one moderated test with users). Neither is it a cognitive walkthrough, which is also a usability inspection method. With cognitive walkthroughs, the emphasis is on user tasks. The process involves identifying the user’s goals and coming up with a task list to achieve those goals. Evaluators then flag problems users may have as they use the product.
A heuristic evaluation expert—the evaluator—is ideally a usability testing expert who has deep understanding of the chosen set of heuristics. They would typically come from the disciplines of human factors, interaction design (IXD), HCI (human-computer interaction) and/or UX design, with complementary backgrounds in disciplines such as psychology, computer science, information sciences, and commerce/business.
During the evaluation, individual evaluators assign a “severity rating” to each of the usability issues identified. As a rule, UX designers work their way down from the most critical issues on the backlog to the least critical. (In order to get the biggest UX bang for the buck from a heuristic evaluation, it is typical for the design team to give issues with the highest severity rating the most attention.)
It’s useful to note that even though a single experienced UX pro is usually adept at identifying the most critical usability issues, a group of evaluators is generally the best option. Between 5 and 8 individuals is the sweet spot: They should be able to flag over 80% of usability problems. However—as the graph below demonstrates—using more than 10 heuristic evaluators will not yield better results.
The proportion of usability problems identified when using a group of heuristic evaluators
As the number of evaluators increases, the proportion of usability problems identified increases. But after 10 evaluators, the potential gains are insignificant.
Why do it?
The core reason to perform a heuristic analysis is to improve the usability of a digital product. Another reason is efficiency (in this context, “efficiency” is the speed with which a product can be used as a direct consequence of better usability). “Usability” refers to quality components such as learnability, discoverability, memorability, flexibility, user satisfaction, and the handling of errors. A product’s UX is greatly improved when these components are delivered at a high quality.
When to do it?
There are no hard and fast rules. A heuristic analysis can be performed at any advanced stage of the design process (Obviously, it would not be productive to do it too early). With new products, a heuristic analysis is usually performed later in the design phase—after wireframing and prototyping and before visual design and UI development begins. Do it too late and making changes will become costly. Existing products found to have poor usability will often have a heuristic analysis run on them before a redesign begins.
What is the expected deliverable?
As with other usability tests or inspection methods, the typical deliverable is a consolidated report which not only identifies usability issues, but ranks them on a scale from severe to mildly problematic. For the most part, a heuristic evaluation report doesn’t include solutions—fortunately, many usability problems have fairly obvious fixes, and once identified the design team can start working on them.
A heuristic evaluation example: usability problems identified using an expert heuristic evaluator
Using an unbiased expert heuristic evaluator, usability problems are quickly identified and better UI design solutions often become obvious.
Advantages and Disadvantages of a Heuristic Evaluation
Uncovers many usability problems and significantly improves a product’s UX
Cheaper and faster than full-blown usability tests that require the recruitment of participants, coordination, equipment, running the test, recording, analyzing, etc.
Heuristics can help the evaluators focus on specific problems (i.e., lack of system feedback, poor discoverability, error prevention, etc.)
Heuristic evaluation does not carry the ethical and practical issues/problems associated with inspection methods involving real users
Evaluating designs using a set of heuristics can help identify usability problems with specific user flows and determine the impact on the overall user experience
Experienced usability experts are often hard to find and may be expensive
The value of issues uncovered by evaluators is limited by their skill level
At times, a heuristic analysis may set off false alarms: Issues that would not necessarily have a negative effect on the overall UX if left alone are sometimes flagged to be fixed
Unlike cognitive walkthroughs, heuristic evaluation is based on prejudged notions of what makes “good” usability
If the evaluators are not part of the design or dev team, they may be unaware of any technical limitations on the design
How to Run an Effective Heuristic Analysis
Preparation is key to running the analysis well. Following an established set of steps ensures that a heuristic analysis will run efficiently and yield maximum results. Here’s a heuristic analysis checklist:
Define the scope.
Know the business requirements and demographic of the end-users.
Decide on which reporting tools and heuristics to use.
Evaluate the experience and identify usability issues.
Analyze, aggregate, and present the results.