Insights
A heuristic evaluation should deliver scoped, evidenced findings with reasoned severity and buildable fixes. What it finds, misses, and when it is enough. A heuristic evaluation service should give you a scoped report in which several evaluators have inspected named parts of your product against named usability princip
A heuristic evaluation service should give you a scoped report in which several evaluators have inspected named parts of your product against named usability principles, and every finding has a location, evidence, a reasoned severity and a recommendation specific enough to build. Read those findings as predictions, not observations: the method is quick at catching broken conventions, but research shows it also flags problems real users never meet and misses problems that depend on knowing the work. In our judgement, it is enough on its own when changes are cheap and the interface is conventional; when a finding would trigger costly work, or your users do specialised work, confirm it with usability testing or a fuller audit.
This guide is about buying the service. If you are still deciding which method you need, start with UX audit vs heuristic evaluation vs usability testing. If you are choosing between providers, the 10 questions to ask before you sign cover that. The method itself was introduced by Jakob Nielsen and Rolf Molich in 1990: evaluators judge an interface against a short list of principles, without users taking part.
What should a heuristic evaluation service deliver?
The provider pages we reviewed promise a prioritised list of issues with severity ratings. That describes the format, not the quality. These are the parts of the deliverable worth checking, and what good looks like in each.
| Part of the deliverable | What good looks like | Why it matters |
|---|---|---|
| Scope | The flows, user roles, states, devices and build version inspected, and what was left out | A finding only speaks for what was inspected; a missing out-of-scope list hides the gaps |
| Evaluators | How many, whether each inspected independently before the lists were merged, and how familiar they were with your kind of product | Evaluators find different problems; a merged list can hide how little they overlapped |
| Criteria | The named heuristic set, plus any additions for your domain and the reason for them | You can see what the evaluators were looking for, and what they were not |
| Each finding | Where it happens and how to reach it, a screenshot, the principle it breaks, what the person would likely experience, a severity with its reasons, and how confident the evaluators are | Without location and reasons, your team cannot reproduce, challenge or rank it |
| Recommendations | A specific change, written so a designer or developer knows what to do, that respects the constraints you named | In one study of professional reports, most recommendations fell short of being both useful and usable (see below) |
| Patterns and strengths | Repeated problems grouped into a few patterns, and the parts that work and should be kept | Grouped problems are easier to rank and assign than a long list of one-off items |
| Next evidence | Which findings to confirm with users, analytics or code before money is spent | It keeps a prediction from turning into an expensive redesign by default |
Four findings from the research explain why some of these rows matter more than they look.
Ask to see who found what. In a 2002 study by Morten Hertzum, Niels Ebbe Jacobsen and Rolf Molich, 11 usability specialists inspected the same website and then merged their findings in groups. Any two of them shared about 9% of their reported problems on average, yet the groups felt they largely agreed. Merging raised their confidence rather than revealing the disagreement. A report that shows which evaluators reported each finding lets you see that for yourself.
The report format affects the findings. Gilbert Cockton, Alan Woolrych and colleagues (2004) found that asking evaluators to use an extended problem report format, one that made them record how they found and judged each problem, reduced false positives and improved how the heuristics were applied. Their reading is that the format made evaluators more careful. A provider whose findings must state their evidence is doing a version of the same thing.
Recommendations deserve the closest reading. Rolf Molich, Robin Jeffries and Joseph Dumas studied 84 comments, on six usability problems, from 17 professional teams that evaluated the same hotel website in 2003 (published 2007); nine teams ran usability tests and eight ran expert reviews. Only 14 (17%) contained recommendations that were both useful and usable. The weak ones were vague, not actionable, or might not improve the product's usability overall. Their advice for good recommendations: clear at the level of the idea, mindful of business and technical constraints, solving the whole problem rather than a special case, and illustrated with a specific example.
Developers value proposals over problem lists. When Kasper Hornbæk and Erik Frøkjær asked four members of the development team of a large Danish job portal to rate the output of 43 student evaluators, who had used an inspection technique or think-aloud testing, the team rated redesign proposals more useful than problem descriptions (2005). Few of the problems were new to the developers, but the problems helped them prioritise. No developer wanted only one or the other. So a report should do both: confirm and rank what your team half-knows, and propose what to do about it.
What it finds, and what it misses
What it finds well. Heuristic evaluation is good at problems that break a known principle and are visible on the screen: missing feedback after an action, inconsistent controls, no way back from a mistake, unclear system state, jargon. In a 1991 comparison of four evaluation methods at Hewlett-Packard, heuristic evaluation of one pre-release software product by four experienced interface specialists found the most serious problems with the least effort. The same study notes the method also produced a large number of specific, one-time and low-priority problems. Both results matter when you buy: experienced evaluators can be very efficient, and you should expect, and ask for, a filtered list.
It also predicts problems users never meet. In Cockton and Woolrych's 2001 assessment, 99 analysts working in groups evaluated a drawing editor, and their predictions were checked against user testing. 65% of the predictions were false. The heuristics helped little with discovering problems and were poor at ruling out improbable ones. The authors concluded that knowledge of the users, their tasks, the application and its domain were better resources for an evaluator than the heuristics themselves. That is one study of one application, and the paper's summary does not say how experienced the analysts were, but the direction fits the other studies cited here: a heuristic finding is a hypothesis.
Different evaluators find different problems. This is the evaluator effect, documented across eleven studies in Hertzum and Jacobsen's 2001 review, and it is why a credible service uses more than one evaluator working independently. The comparison article gives the numbers.
Severity is a judgement, not a measurement. An inspection cannot observe how often people hit a problem, so a severity rating from an inspection estimates frequency and impact rather than recording them. In an experiment reported by Hertzum (2006), evaluators' ratings of impact and persistence correlated only weakly with how many evaluators had reported each problem, which the author reads as either different evaluators standing for different user groups or the ratings being of questionable reliability. Ask for the reasons behind each rating, so your team can disagree with the reasoning rather than the number.
What it cannot tell you. Whether your users actually struggle, how often, and how much it costs them. Problems that come from the work rather than the screen: business rules, permissions, data that looks right but is wrong, handoffs between roles. Evaluators without access to that knowledge are unlikely to see those problems, however skilled they are.
AI-generated reviews. Some services now run part of the evaluation through a language model, and the evidence on how well that works is uneven. In a 2025 study by Guilherme Guerino and colleagues, GPT-4o reviewed screenshots of a web system against Nielsen's heuristics. It found 14 of the 66 issues three human experts had found (21.2%), surfaced some valid issues the experts missed, and 27 of its 111 reported issues (24.3%) were judged false positives. It did best on aesthetic issues and on matching users' language and worst on control, flexibility and efficiency; the authors suggest this may be because the model saw only screenshots and could not use the system. A 2025 preprint by Ruican Zhong, David McDonald and Gary Hsieh points the other way on coverage: multimodal models reviewing screenshots of two apps found 73% and 77% of the known issues, against 57% and 63% for five experienced evaluators combined. That figure is coverage only: the models also reported many non-issues, which were left out of it, and the models' own findings helped define the issue list. Each study looks at one or two products seen through screenshots, the Zhong study is a preprint, and the models are changing quickly. Our reading is that detection depends on the setup and is improving, so candidate issues are getting cheaper to produce. The buying rule stays simple: ask which parts were machine-generated and who verified each finding.
Accessibility is a separate evaluation. A heuristic pass may note accessibility problems, but it is not a conformance check against WCAG. The W3C's guidance says knowledgeable human evaluation is required to determine whether a site is accessible and describes a separate conformance method and the involvement of disabled users. If you need to know whether you meet an accessibility standard, buy that evaluation explicitly.
A weak finding and a strong one, side by side
Fictional example written for this article, not a client finding. The same problem in an invoice-approval screen, written two ways.
| Weak write-up | Strong write-up | |
|---|---|---|
| Where | "Approval screens" | Approvals, invoice detail view, approver role, invoice above the approver's limit |
| What was seen | "Inconsistent buttons" | Approve sits on the right in the list view and on the left in the detail view; Reject has the same filled style as Approve |
| Principle | "Consistency" | Consistency and standards; error prevention |
| Likely effect | Not stated | An approver working through a queue may press Reject when they meant Approve |
| Severity | "High" | Major: the action changes an invoice's status for other people, and how often it happens is unknown |
| Confidence | Not stated | Reported by two of three evaluators; not observed with users |
| Recommendation | "Improve button consistency" | Keep Approve in one position across both views; give Reject a secondary style; confirm a rejection with the invoice number and allow it to be undone where the workflow permits |
| Next evidence | Not stated | Watch approvers clear a realistic queue before changing the shared action bar used elsewhere |
The strong version takes more effort to write. It is also the only one your team can reproduce, argue with and build.
When is a heuristic evaluation enough on its own?
The rows below are our judgement, built from the evidence above: inspection is efficient and fallible, so the question is what a wrong prediction would cost you.
| Your situation | Heuristic evaluation alone? | What to add |
|---|---|---|
| A design or prototype before you test it with users | Yes, as the first pass | Usability testing of the revised prototype |
| A conventional flow (sign-up, settings, checkout) where fixes are cheap | Yes, if you check afterwards | The effect after release, in analytics or support themes |
| An AI-assisted first pass on a conventional flow, reviewed by one experienced person who checks every finding | Yes, as a cheap first pass | Expect to discard some machine findings; confirm anything costly with users or analytics |
| A finding would lead to a costly redesign or rebuild | No | Usability testing of that flow before committing the budget |
| Your users do specialised work the evaluators do not know (approvals, scheduling, compliance, operations) | Not without domain access | Walkthroughs with people who do the work, or testing with real users |
| You need to know how often a problem happens or how many people it affects | No | Analytics, support data or a larger-sample study |
| Friction keeps returning and your team disagrees about the cause | No | A fuller audit that follows the workflow into rules, roles, data and implementation |
| You need to know whether you meet an accessibility standard | No | A conformance evaluation, with disabled users where possible |
The last two rows describe the point where screen-level inspection stops being the right tool. When the question is why the same friction keeps returning, an audit has to look past the interface. That is the job of the Tcules UX Product Audit, which starts from the product decision, the work affected and the cost of being wrong rather than from scoring screens, and labels each conclusion as observed, reported or inferred.
What are you really paying for now?
Candidate issues are becoming cheap to produce. A model can draft a list of possible heuristic violations from screenshots, and in the Zhong study above that list covered more of the known issues than five experienced evaluators did on two apps. The parts that stay expensive are the ones a screenshot cannot supply.
- Severity grounded in your evidence. How often a problem happens and what it costs, read from your analytics, support history and sales notes, not estimated from the screen. Evaluators disagree about severity too (see above), so a rating anchored in your data is worth more than a confident label from either a person or a model.
- Cause. Whether a visible problem comes from the screen or from a rule, a permission, the data or a handoff behind it. That needs the logged-in product, real roles and realistic data.
- Decisions. Which findings to fix, test, defer or leave alone, with an owner, given your roadmap and constraints.
- Accountability. A named person who verified each finding and will answer for it.
So price the parts separately. In our judgement, a plain heuristic pass on screens, with or without AI help, should be lean, quick and priced like the commodity it is becoming. Pay more only for work that uses your evidence and ends in decisions. This is our reading of where the value is moving, not a measured market trend.
How to judge a provider's sample report
Ask every provider for a sample report before you sign; a fictional or anonymised one is fine if it is labelled as such. Then read it for these things.
Signs of a sound report
- The scope page names what was inspected and what was not.
- The number of evaluators is stated, and you can tell they worked independently before merging.
- The heuristic set is named. If the provider uses domain-specific heuristics, it says why; a 2016 review of 70 studies by Setia Hermawati and Glyn Lawson identified weak validation of domain-specific sets as the field's most urgent problem, so a custom list is not better by default.
- Each finding has a location you could find again, a screenshot, a predicted effect on a person and a severity with reasons.
- Recommendations are specific enough that a developer could estimate them, and they acknowledge constraints.
- Findings are grouped into patterns, and the report names what works and should stay.
- Some findings are explicitly marked as needing confirmation with users.
Warning signs
- A single "UX score" or percentage with no account of how it was reached.
- Every finding rated high, or no severity reasons at all.
- Findings with no location, or advice that would fit any product ("improve visual hierarchy", "simplify navigation").
- Recommendations that remove features or overturn business rules for a minor problem.
- No statement of what the report cannot tell you.
- No answer to which parts were produced by software and who checked them.
If you can, give two providers the same short brief and compare their samples on one of your own screens. The differences will tell you more than any proposal.
Who offers heuristic evaluation services?
The service is sold by independent practitioners, UX research firms, design studios, some university usability services, and, increasingly, software tools that generate a review automatically. The type of provider matters less than the deliverable: whoever you choose should be able to show you the parts described above. What separates them is how many evaluators they assign, how much they learn about your domain before inspecting, and whether they can continue into testing or design if the findings call for it. The 10 questions to ask before you sign help you compare them.
Tcules is a digital product design studio creating ambitious new software products and making established ones better, with a particular focus on SaaS. We sell audits, so read this guide with that in mind. Our own offer in this space is the UX Product Audit linked above, where heuristic review can be one input among several; it suits teams whose question has outgrown a heuristic pass. If a heuristic evaluation alone is enough for your situation, the table above should tell you so.