Methods for Evaluating Websites

Why this matters

"The site seems fine" is not an evaluation. There are two honest ways to check, and they answer different questions. Confusing them is how teams end up with a pile of numbers and no idea what to change.

The idea

Qualitative evaluation explores why. It uses usability testing, watching real users operate the site to see what trips them up, and interviews, hearing directly what they need and dislike. It reaches things numbers cannot express.

Quantitative evaluation measures how much, objectively, from numerical data. It uses web analytics and A/B testing.

The division is clean: qualitative finds the cause of "why is this hard to use", quantitative grasps the overall trend of "how much is this being used". Combining them is what gives an accurate picture.

The main analytics indicators:

Heuristic evaluation is a third angle: experts inspect the interface against rules of thumb. It is a form of qualitative evaluation, used alongside usability testing. The usual perspectives are consistency (is the design and behaviour unified across the site), error prevention (is it hard for a user to go wrong), and visibility of system status (does the user get told what is happening, through progress indicators and clear error messages).

Picture it

flowchart TD
  Q["Quantitative: PV, bounce rate, CVR"] --> W["Where is the problem?"]
  W --> L["Qualitative: usability testing, interviews, heuristic review"]
  L --> Y["Why is it a problem?"]
  Y --> C[A change worth making]

Worked example

Analytics shows one page with a bounce rate of 80 percent. What now?

The number has told you something real and stopped exactly there: four out of five visitors arrive and leave without going further. It has not told you whether the page is confusing, slow, or simply not what the link promised. No amount of extra quantitative data will say, because the question is about reasons.

So the next step is qualitative. A usability test on that page might show people scrolling past the thing they came for. An interview might reveal they expected prices and found a brochure. A heuristic evaluation might catch that there is no visible indication the page is still loading, so people give up before it appears.

Then the loop closes: make the change, and watch the bounce rate again. The numbers find where to look and confirm whether a fix worked; the user research explains what to do. Either on its own leaves you guessing.

Your turn

Try the practice questions for this lesson. Sort statements about the two kinds of evaluation into true and false, match each analytics indicator to its meaning, and pick the method that answers a "why" question.

Take it further

An online bookstore wants to know whether its site really works for customers. Should it rely on the owner's impression, or use both kinds of evaluation?

1. Collect data. Ask five classmates to visit any online shop. For each, record one number you could measure and one thing they said was confusing.
2. Analyse the signals. The bookstore's PV is high but its conversion rate is very low. What might that mean, and which method would tell you why? Then name one customer the data may hide, such as someone on a small phone screen, and one difficulty they might have.
3. Decide. Recommend an evaluation plan using both methods, and give two reasons.

High PV with low CVR means people are arriving and not acting. Only one of the two methods can tell you what stopped them.

Recap

• Qualitative evaluation answers why; quantitative answers how much.
• PV is exposure, bounce rate is whether the page met expectations, CVR is whether it achieved anything.
• Heuristic evaluation is experts checking the interface against rules of thumb.
• Numbers tell you where to look; users tell you what to change.