Loki vs BackstopJS for Storybook visual testing

This page is part of the tool comparisons section of the visual regression guide, which covers choosing between self-hosted and cloud visual testing.

The specific question: Loki and BackstopJS are both mature, self-hosted, free visual regression tools. They are frequently shortlisted together, and they are built around different assumptions about what you are capturing.

Problem statement

Both compare screenshots against committed references and both run locally and in CI at no licence cost. The similarity ends at the input: Loki takes a Storybook and captures its stories; BackstopJS takes a list of scenarios, each a URL with optional interaction steps.

Choosing the one that does not match your input produces avoidable work — either maintaining a scenario per story by hand, or bending a Storybook-shaped tool around page-level flows it was not designed for.

Root cause

Loki reads Storybook’s story index, so its unit of work is a story and it needs almost no configuration: point it at a Storybook and it captures everything. That is exactly right for a component library and useless for an application page that is not in Storybook.

BackstopJS predates Storybook’s dominance and is built around scenarios. Each names a URL, a selector, viewports, and optional scripts that run before capture — which makes it capable of things Loki cannot express, at the cost of a configuration file that grows with the surface you cover.

Loki and BackstopJS compared Seven rows. Loki is built for Storybook and discovers stories automatically; BackstopJS targets any URL and requires you to list scenarios. Loki runs Chrome in Docker; BackstopJS drives Puppeteer or Playwright. Configuration is a Loki config file against BackstopJS's scenario list. Interaction support is limited in Loki and available through ready-script hooks in BackstopJS. Loki reports on the command line with an approve command; BackstopJS produces an HTML report. Loki is best at covering a whole Storybook quickly, BackstopJS at specific pages and flows. Loki BackstopJS Built for Storybook any URL Story discovery automatic you list scenarios Browser Chrome, Docker Puppeteer or Playwright Config loki.config.js backstop.json scenarios Interaction support limited onReadyScript hooks Report CLI + approve command HTML report Best at a whole Storybook, fast specific pages and flows

Minimal reproduction 

// loki.config.js — the whole configuration
module.exports = {
  configurations: {
    'chrome.laptop': { target: 'chrome.docker', width: 1366, height: 768, deviceScaleFactor: 1 },
  },
};
// backstop.json — one entry per thing you capture
{
  "scenarios": [
    { "label": "Card default", "url": "http://localhost:6006/iframe.html?id=card--default",
      "selectors": ["#storybook-root"], "misMatchThreshold": 0.2 }
  ],
  "viewports": [{ "label": "desktop", "width": 1366, "height": 768 }]
}

The asymmetry is the point: Loki’s file does not mention any component, and BackstopJS’s grows by one entry per capture.

Step-by-step comparison

1. Match the tool to the unit of work

Choosing by what you are capturing A decision node asking what is being captured. Covering every Storybook story suits Loki, whose discovery is zero-configuration. Capturing specific pages or flows suits BackstopJS, whose scenarios support steps. Capturing stories while also asserting behaviour suits the Storybook test-runner with Playwright, which is one tool with more control. A legacy application with no Storybook suits BackstopJS, since URLs are all it needs. What are you capturing? Loki zero-config discovery every Storybook story BackstopJS scenarios with steps specific pages or flows Test-runner + Playwright one tool, more control stories, with assertions BackstopJS URLs are all it needs a legacy app, no Storybook

A component library’s unit is a story, and Loki’s discovery means adding a component adds coverage with no configuration change. An application’s unit is a page or a flow, and BackstopJS’s scenarios express those directly.

2. Compare interaction support 

// BackstopJS: a script that runs before the capture
module.exports = async (page, scenario) => {
  await page.click('[data-testid="open-menu"]');
  await page.waitForSelector('[role="menu"]');
};

Loki has no equivalent hook of comparable depth. A story that needs interaction before capture has to reach that state through its own args or a play function — which is a reasonable pattern, and a limitation if the state cannot be expressed that way.

3. Compare the review workflow

Loki reports differences on the command line and provides loki approve to accept them wholesale. BackstopJS generates an HTML report with side-by-side and overlay views, which is meaningfully better for judging whether a change was intended.

4. Compare runtime

Runtime over three hundred captures Horizontal bars of runtime. Loki covers three hundred stories in three and a half minutes. BackstopJS covers three hundred scenarios in eight minutes, because each scenario navigates separately. The Storybook test-runner with Playwright takes three minutes ten seconds, highlighted. Sharding that same run four ways brings it to fifty-five seconds. Same components, same runner class Loki, 300 stories 3m 30s BackstopJS, 300 scenarios 8m 00s test-runner + Playwright 3m 10s the same, 4 shards 0m 55s

Loki is fast over a Storybook because it reuses one browser across stories. BackstopJS navigates per scenario, which costs more per capture but is unavoidable when each scenario is a different URL.

5. Consider whether either is the right answer now

Both predate the Storybook test-runner and Playwright’s screenshot assertions. A team already running the test-runner can add visual assertions to the tool it already has, with sharding, traces and reports included — which for a component library is usually a better answer than adding a third tool.

Verification

Whichever you pick, the check is the same: run it three times against an unchanged commit and confirm the diffs are near zero. Both tools capture through a real browser, so both inherit every determinism concern in dynamic content masking — and neither will be stable until those are addressed.

# Loki
npx loki test --requireReference --reactUri file:./storybook-static
# BackstopJS
npx backstop test --config=backstop.json

Edge cases and caveats

  • Loki’s approve command is unscoped by default. loki approve accepts every difference, which is the blind update this guide’s baseline management section warns against. Filter it, or review before running it.

  • BackstopJS scenario drift. Scenarios are maintained by hand, so a removed page leaves a scenario that fails forever and a new page has no coverage until someone adds one. Auditing the list periodically is part of the cost.

  • Docker for determinism. Loki’s Docker target exists precisely to pin the rendering environment. Running it against a local Chrome instead reintroduces the font and rasteriser differences the container was there to remove.

What consolidating actually saves

The argument for folding visual coverage into a tool you already run is not aesthetic. Each separate visual tool brings four things that have to be maintained independently, and the cost is easy to underestimate because it arrives gradually.

A second baseline set. Two tools mean two directories of reference images, updated by two commands, reviewed by two workflows. When a token changes, both sets need regenerating, and a team that regenerates one and forgets the other has a suite that disagrees with itself.

A second CI job. Another container image to pin, another cache to configure, another set of artefacts to publish and another required status check to keep green. The job is not hard to write and it is one more thing that breaks on an upgrade.

A second determinism configuration. Every technique in dynamic content masking — the fixed clock, the disabled animations, the pinned scale factor — has to be expressed twice, in two different vocabularies. In practice one of the two ends up less thoroughly configured, and that is the one that produces the flaky check.

A second review surface. Reviewers learn one tool’s diff view well and the other’s badly, so diffs from the less familiar tool get less scrutiny.

None of these is decisive on its own, and together they explain why a consolidated setup tends to stay healthy while a split one drifts. The corollary is that adding a second visual tool should be a deliberate decision with a named reason, not something that happens because a new team member preferred a different one.

FAQ

Is either still a reasonable choice for a new project?

BackstopJS, yes, when the thing being captured is pages rather than stories and there is no Storybook to build on. For a component library, a new project is usually better served by the Storybook test-runner with Playwright assertions, because it consolidates behaviour, accessibility and visual coverage into one run rather than adding a separate tool with its own baselines and its own CI job.

Can they capture the same Storybook?

Yes, and doing both is redundant. If you already have BackstopJS scenarios pointing at story iframe URLs, you have hand-maintained what Loki discovers automatically — which is a reason to consolidate rather than to run both.

How do their thresholds compare?

BackstopJS’s misMatchThreshold is a percentage of differing pixels, which is the same idea as a ratio budget. Loki exposes a per-configuration tolerance in similar terms. Neither offers the separate per-pixel colour distance that lets anti-aliasing noise be absorbed before the ratio is computed, so both tend to need slightly looser whole-image budgets than a Playwright setup for the same components.

Do either of these integrate with a cloud review service?

Not directly — both are self-hosted end to end, which is the point of them. What they can do is publish their reports the way any other job does, so a reviewer gets a link rather than a download. If a purpose-built review workflow is what you actually want, that is the argument for a hosted service rather than for either of these.