Reviewing baseline updates in a pull request

This page is part of the baseline management section of the visual regression guide, which covers keeping the reference image honest.

The specific problem: a pull request changes a component, a design token and sixty baselines. The reviewer cannot tell which change produced which diff, so they approve the lot.

Problem statement

A baseline update is a change to the definition of correct. Every subsequent run is measured against it, so an unreviewed update is indistinguishable from a regression that was never caught — and it will never be caught, because the suite now asserts the broken rendering is right.

The symptom is a pull request whose file list is mostly binary images, with a code diff small enough that the reviewer reads the code and scrolls past the images.

Root cause

Two things make baseline diffs hard to review, and they compound.

The first is volume: a token change legitimately touches every component, and sixty images is past the point where anyone looks at each one.

The second is attribution: when a pull request contains more than one change that could produce a diff, no individual diff can be traced to a cause. A reviewer facing sixty images and three possible explanations has no way to reason about them, and approval becomes the only available action.

One cause per pull request Top lane: a single pull request contains a token change, a bug fix and sixty updated baselines, and no diff can be attributed to a specific cause. Bottom lane: the token change lands alone with its sixty diffs, all explained by one edit, and the bug fix lands separately with two — so both are reviewable. Mixed PR — the diffs have several possible causes token change + a bug fix + 60 baselines which caused which? Split PRs — every diff has one explanation PR 1: token 60 diffs, one cause PR 2: bug fix 2 diffs both reviewable

Minimal reproduction 

git log --stat -1
#  src/tokens.css                       |  4 +-
#  src/components/Card/Card.tsx         | 12 +++--
#  __screenshots__/desktop/*.png        | 60 files changed
# which of the two edits caused each image to change?

Step-by-step fix

1. Split by cause, not by size

Land the token change on its own, with no other edits. Every diff in that pull request then has exactly one possible explanation, which makes sampling a legitimate review strategy rather than a shortcut.

git checkout -b tokens/spacing-scale
# tokens only — no component edits
git commit -am "Update spacing scale from 4px to 8px base"

What this does: buys the one-cause property, which is what makes the remaining steps work.

2. Put the baselines in their own commit 

git commit -am "Update spacing scale"
npx playwright test --update-snapshots
git commit -am "Regenerate baselines for the spacing scale"

What this does: lets a reviewer read the code change and the image change separately, and lets git revert undo one without the other.

3. Require a reviewer who knows what the component should look like 

# CODEOWNERS
/__screenshots__/    @acme/design-systems

What this does: guarantees that someone who can judge the rendering sees every baseline change. Without it, a backend reviewer approving a frontend pull request approves the images too.

4. Sample deliberately rather than clicking through

For a change with one cause, review one component per family plus every component that uses the changed token unusually. Sixty images become eight decisions, and the eight are chosen rather than the first eight in the list.

# list what changed, grouped, so the sample is chosen rather than arbitrary
git diff --name-only HEAD~1 | sed 's#.*/##' | sort | head -60

5. Never update baselines in CI 

      - run: npx playwright test          # no --update-snapshots, ever

What this does: makes it impossible for a pipeline to approve its own diff. A baseline that appears without a human having seen it is the exact failure this whole section exists to prevent.

Verification

What a reviewable baseline change looks like A baseline review on pull request 421. Two screenshot files are modified and nothing outside that path changed. CODEOWNERS requires a design-systems review. The reviewer opened both images in the rich diff and approved, recording that the spacing change was intended. Baseline review — PR #421 __screenshots__/desktop/card-default.png modified __screenshots__/desktop/card-compact.png modified CODEOWNERS: @design-systems required 2 files changed, 0 additions to other paths reviewer opened both images in the rich diff approved: spacing change is intended

The check is the file list: the pull request should contain the baselines you intended and nothing else under that path. An unexpected extra image means the update was not scoped, and the safest response is to reset the path and redo it with a filter.

Edge cases and caveats

Five controls that keep a baseline honest Five rows. A CODEOWNERS entry on the screenshots directory prevents an unreviewed baseline at the cost of one required reviewer. Putting baselines in their own commit prevents image diffs being mixed with code at the cost of an extra commit. A scoped update command prevents rewriting the whole set and costs naming the stories. Never passing the update flag in CI prevents auto-approved drift and costs nothing. Keeping one cause per pull request prevents unattributable diffs at the cost of more, smaller pull requests. Control Prevents Cost CODEOWNERS on __screenshots__ an unreviewed baseline one required reviewer Baselines in their own commit diffs mixed with code one extra commit Scoped update command rewriting the whole set naming the stories No -u in CI auto-approved drift none One cause per PR unattributable diffs more, smaller PRs

  • Binary merge conflicts. Two branches updating the same baseline conflict, and Git cannot merge PNGs. Resolve by taking neither: check out the target branch’s version, rerun the suite on the merged code, and regenerate.

  • Rebasing baseline commits. A rebase replays the update commit against different code, producing images for a combination that was never tested. Prefer merging, or regenerate after the rebase.

  • Very large redesigns. Past a few hundred images, sampling stops being enough. Splitting by component family into several pull requests keeps each reviewable, at the cost of a longer sequence of merges.

FAQ

Is it acceptable to approve baselines without opening every image?

Yes, when the pull request has one cause and the sample is chosen rather than arbitrary. What is not acceptable is approving without opening any, or approving a mixed pull request where the images have several possible explanations. The property that makes sampling sound is that every diff in the run must have the same explanation — take that away and the sample tells you nothing about the rest.

Should baseline updates be a separate pull request from the code change?

A separate commit, in the same pull request, is usually the right balance. The reviewer sees the code and its visual consequence together, which is the context needed to judge whether the rendering change was intended, and git revert can still undo either alone. Separate pull requests break that link and leave a window where the code and its baselines disagree.

What stops someone bypassing all of this with an unscoped update?

Nothing technical, which is why the CODEOWNERS entry matters more than the process. A required reviewer on the snapshot directory means an unscoped update shows up as sixty unexpected files in front of someone who will notice. Combined with never passing the update flag in CI, that is enough in practice — the remaining risk is a reviewer approving without looking, which is a human problem rather than a tooling one.