How to Read an SEO Agency Case Study Critically

Case studies are marketing artefacts. That does not make them useless — it makes them documents to interrogate rather than accept. A well-constructed case study is one of the highest-signal things an agency publishes, because writing an honest one requires the discipline that good client work also requires. A weak one is equally informative, just not in the way it was intended.

This is a reading method. Work through it with any case study in front of you and you will finish with a short list of specific questions that the sales call has to answer.

Demand the baseline

A 400% traffic increase from a near-zero baseline is arithmetic, not achievement. Percentages exist in case studies precisely because they survive the removal of context. Ask for three things every time: the absolute starting number, the absolute ending number, and the length of the window.

Then ask what the site's condition was at the start. There is an enormous difference between these four situations, and the same percentage can describe any of them:

• A brand-new domain with no history, where any growth looks exponential.
• A site recovering from a manual action or a core update hit, where the win is restoration rather than creation.
• A mid-migration site whose baseline was artificially depressed by a botched replatform.
• A healthy, established site where a 25% non-brand lift represents genuinely hard work.

The fourth is the most impressive and photographs the worst. Agencies that publish it anyway are telling you something about their standards.

Separate lift from seasonality and market movement

Many reported wins coincide with something the agency did not cause. Retail traffic rises into Q4. Tax services peak in spring. B2B software dips through August and December. A competitor's migration failure can hand you six months of gains that reverse when they fix it. A core update that rewarded existing quality can lift a site whose team changed nothing that quarter.

The defences are simple and rarely applied. Look for year-over-year comparisons rather than month-over-month. Look for a stated note about algorithm updates inside the window. Look for whether the chart begins at a suspiciously low point — a window that starts in the trough of a known seasonal cycle is a choice, not an accident.

Check the brand and non-brand split

Brand search grows when paid media, PR, product launches, or funding announcements do well. It is the easiest way for an underperforming SEO programme to produce a rising chart. If a case study reports total organic sessions without splitting brand from non-brand, you cannot tell which department earned the result.

This is not a nitpick. In our reading of published US case studies, unsplit totals are the single most common reason a headline number cannot be evaluated. The split is trivially available in Search Console, so its absence is a choice.

Look for the boring work

Durable results usually come from unglamorous causes: fixing indexation, consolidating duplicated and cannibalising pages, rewriting weak commercial pages, restructuring internal links, improving templates, and shipping consistently for eighteen months. Case studies that describe only a creative campaign usually describe a spike — real, sometimes valuable, but not a system you can buy repeatedly.

When you see a campaign-led case study, ask what happened in the two quarters afterwards. A campaign that produced links which then lifted a whole category of commercial pages is a very different thing from a campaign that produced coverage and a traffic bump with no commercial tail.

Four numbers that make a case study checkable

The fourth is where most agencies stop. It is also the one you are actually buying: efficiency, not raw outcome. A 60% non-brand lift on $4,000 a month is a different business proposition from the same lift on $28,000 a month, and both may be excellent depending on the revenue at stake.

Verify what you can independently

You do not need client access to sanity-check a public case study.

1. 1Look at the client's site now. Do the pages described in the case study still exist, and do they still look like the work claimed?
2. 2Check third-party visibility trends for the client domain across the stated window. Directional agreement is enough; exact numbers are not the point.
3. 3Read the archived version of a key page from before the engagement, where available, and compare it with the current one. The delta tells you what was actually shipped.
4. 4Search the target queries and see whether the client still ranks. Sustained positions are more informative than a peak screenshot.
5. 5Look for the client's own account of the work — a conference talk, a podcast, a LinkedIn post. In-house teams describe the division of labour more precisely than agencies do.

Questions to bring to the call

• What was the client's internal capacity, and what did they do themselves during this engagement?
• What was the single decision that produced the largest share of the result?
• What did you try that did not work?
• Was there an algorithm update or a competitor event inside this window?
• What is the current state of this account, and is the client still with you?

Ask for the reference that did not renew

Every agency has churn. The ones worth hiring can describe why a client left, whether the split was amicable, and what they changed internally afterwards. Ask for a reference from an engagement that ended, alongside the two glowing ones. The refusal is itself an answer, and the acceptance usually produces the most useful thirty minutes of the entire evaluation.

Read enough case studies this way and a pattern emerges quickly. The agencies whose published work survives scrutiny tend to be the ones whose monthly reporting will also survive scrutiny — which is the actual thing you are trying to predict.