Research
Lighthouse variance: how much does one audit move?
The same page, the same engine, ten audits in a row — and a performance score that won't sit still.
By Roger Campos · Last updated: September 2026
TL;DR
Ten back-to-back mobile Lighthouse 13 audits of the same page gave performance scores that ranged from 0 points apart (Hacker News, 100 every time) to 61 points apart (the BBC News front page, 12 to 73). Two of five pages stayed within 2 points; three moved by 16 points or more. Accessibility, best practices and SEO moved by at most 7. One audit is a sample, not a verdict.
Free plan, no credit card. 1,000 credits a month.
Summary
The results
Performance score of 10 consecutive mobile audits per page, measured on 2026-09-24.
| Page | Min | Median | Max | Range | Runs within 5 points of the median |
|---|---|---|---|---|---|
| news.ycombinator.com | 100 | 100 | 100 | 0 | 10 of 10 |
| tailwindcss.com | 58 | 59 | 60 | 2 | 10 of 10 |
| stripe.com | 32 | 37.5 | 48 | 16 | 7 of 10 |
| en.wikipedia.org/wiki/Headless_browser | 59 | 77 | 90 | 31 | 3 of 10 |
| bbc.com/news | 12 | 47 | 73 | 61 | 5 of 10 |
The spread is a property of the page, not a constant. A light page with nothing competing for the main thread scores the same every time. A heavy page that loads ads, experiments or a banner that is only sometimes there can land anywhere in a 30- or 60-point band — on the same engine, minutes apart.
The other three categories are far steadier, because they are mostly checks on the markup rather than timings: across all 50 audits, accessibility moved by at most 3 points on any page, best practices by at most 4 and SEO by at most 7 — and the one page where they moved at all was also the one whose layout shift says its content changed between loads.
Under the score
Which metrics move
The performance score is a weighted blend of five lab metrics — Total Blocking Time 30%, Largest Contentful Paint 25%, Cumulative Layout Shift 25%, First Contentful Paint 10%, Speed Index 10% — so a swing in a heavily weighted metric moves the score most.
| Page | Metric that moved most | Min | Max |
|---|---|---|---|
| bbc.com/news | Cumulative Layout Shift | 0.091 | 0.814 |
| bbc.com/news | Total Blocking Time | 409 ms | 1,994 ms |
| stripe.com | Total Blocking Time | 1,081 ms | 5,302 ms |
| stripe.com | Largest Contentful Paint | 4.7 s | 11.1 s |
| en.wikipedia.org/wiki/Headless_browser | Largest Contentful Paint | 1.8 s | 5.2 s |
| en.wikipedia.org/wiki/Headless_browser | Cumulative Layout Shift | 0.151 | 0.656 |
| tailwindcss.com | Largest Contentful Paint | 15.7 s | 22.6 s |
Layout shift is the least intuitive one: it is not noise in the measurement but a different page. When something is injected above the content on one load and not on the next, CLS jumps from a pass to a clear fail, and because it carries a quarter of the weight the score follows. Tailwind's page shows the opposite case — a Largest Contentful Paint that swung by seven seconds while the score moved two points, because it was deep in the failing range either way.
In practice
What to do about it
- Never compare two single runs. A 10-point drop between two audits of the BBC front page is well inside what one page did to itself in ten minutes.
- Take the median of several runs. Medians of three consecutive runs here were 73, 70 and 88 for the Wikipedia page — still wide — so for a noisy page use five or more, and compare medians.
- Read the metrics, not the score. A CLS that flips between 0.09 and 0.8 is telling you about an element that only sometimes appears, which is a real bug worth finding.
- Track categories separately. Accessibility, best practices and SEO hardly moved, so a change in them is a change in the page.
- For monitoring, use field data or a trend. Lab audits are for diagnosing; a site's real-user Core Web Vitals, or a rolling median of many audits, is what should trigger an alert.
With URLpipe each audit of the same URL and options is stored, so a repeat inside max_age returns the same report, free. To collect several independent samples, send max_age: 0, which forces a fresh audit each time.
for i in 1 2 3 4 5; do
curl -s -X POST https://urlpipe.dev/lighthouse \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "max_age": 0, "sync": true}' \
| jq '.categories.performance.score'
doneIf you would rather have someone run the audits on a schedule, keep the history and tell you what regressed, that is what our sister product Full Stack Audit does.
How it was measured
Method
- Lighthouse 13.4.0 driving Chrome 153 — the Lighthouse service URLpipe runs in production, as a local container — with Lighthouse's default mobile emulation and simulated throttling, which is what
/lighthouseuses when you don't passdevice. - Five public pages chosen to span light to heavy. Ten audits of each, round-robin — every page once, then every page again — so a slow minute on the network affects all five pages rather than one. 50 audits between 15:31 and 15:42 UTC on 2026-09-24, one at a time, every one successful.
- Each audit in a fresh Chrome, as the service always does.
- Run on a developer machine, not on the production servers, with other work running on it. Contention on the machine is part of the variance; a quieter machine would likely narrow the stable pages' bands a little and do nothing about a page that changes between loads.
Read before citing
Limits of this study
- Five pages and ten runs each is enough to show that the spread depends on the page, and not enough to put a number on "typical" variance.
- Mobile emulation only; desktop audits were not measured here.
- Live pages: part of what moved is the pages themselves — banners, ads, experiments — which is also what your own audits will see.
Research
More research
Web-to-Markdown, measured
Four HTML-to-Markdown strategies on the same 33 real pages: how much of the visible text each keeps, and how much of its output the reader never saw.
What stops a headless browser in 2026
46 well-known, frequently protected sites, visited from a datacentre and a residential address: which refuse outright, which answer 200 with a block page, and what a residential exit changes.
FAQ
Frequently asked questions
Why does my Lighthouse score change every time I run it?
How many Lighthouse runs should I average?
Do accessibility and SEO scores vary between runs too?
Which Lighthouse metric varies the most?
Does URLpipe return the same Lighthouse report twice?
Try the converter on your own pages.
Free plan, no card. Confirm your email and your API key is live — you'll be making real requests in minutes.