Collecting Lighthouse data for many URLs requires planning around hardware, parallelism, and result stability. Before scaling up, read Dealing with score variability — the same variability concerns that affect individual runs are amplified at scale.
Options for running at scale
Option 1: PageSpeed Insights API
The PageSpeed Insights API runs Lighthouse on Google infrastructure and returns results via a simple HTTP request. The default quota is 25,000 requests per day.
Best for: publicly accessible URLs where you do not need to maintain your own test environment.
curl "https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=https://example.com&key=YOUR_API_KEY"
Option 2: Lighthouse CLI on cloud hardware
Running the CLI on your own machines gives you full control over configuration. Most teams wrap the CLI with a bash, Python, or Node script that iterates over a list of URLs.
Popular wrappers:
lighthouse-batch — run Lighthouse over a list of URLs and produce a summarymultihouse — parallel Lighthouse runs over multiple URLs
Best for: private URLs, custom configurations, or workflows that need programmatic access to raw results.
Option 3: Managed services
Commercial services handle infrastructure, scheduling, and result storage for you. See the CI Integration page for a list of third-party integrations including WebPageTest, Calibre, DebugBear, and Treo.
Gather mode and audit mode
Lighthouse supports splitting a run into two separate phases using the -G (gather) and -A (audit) flags. This is useful at scale because it decouples browser time (expensive, requires Chrome) from audit time (cheap, CPU-only).
# Phase 1: launch Chrome, collect page artifacts, save to disk
lighthouse https://example.com -G ./artifacts/example
# Phase 2: load saved artifacts from disk, run audits, generate report
lighthouse https://example.com -A ./artifacts/example
-GA to run both phases in a single invocation while still saving artifacts to disk for later re-auditing:
lighthouse https://example.com -GA ./artifacts/example
- Re-run audits against already-collected artifacts without re-launching Chrome
- Distribute gathering across many machines and consolidate auditing on one
- Iterate on audit logic without incurring browser startup cost
Launching headless Chrome
In CI and server environments, Chrome must run in headless mode. Pass the following flags via --chrome-flags:
lighthouse https://example.com \
--chrome-flags="--headless --disable-gpu --no-sandbox"
| Flag | Purpose |
|---|
--headless | Runs Chrome without a visible window |
--disable-gpu | Disables GPU hardware acceleration (required in many server environments) |
--no-sandbox | Disables the Chrome sandbox (required in Docker and some Linux environments) |
--no-sandbox reduces Chrome’s isolation. Only use it in environments you control, such as a dedicated CI machine or container. Review the sandbox tradeoffs before enabling this flag.Launching Chrome programmatically with chrome-launcher
For Node.js scripts that need direct control over Chrome, use the chrome-launcher package that ships with Lighthouse.
npm install chrome-launcher lighthouse
import chromeLauncher from 'chrome-launcher';
import lighthouse from 'lighthouse';
import fs from 'fs';
async function runLighthouse(url) {
const chrome = await chromeLauncher.launch({
chromeFlags: ['--headless', '--disable-gpu', '--no-sandbox'],
});
const options = {
logLevel: 'error',
output: 'json',
onlyCategories: ['performance'],
port: chrome.port,
};
const result = await lighthouse(url, options);
await chrome.kill();
return result.lhr;
}
// Process a list of URLs sequentially.
const urls = [
'https://example.com',
'https://example.com/about',
'https://example.com/contact',
];
for (const url of urls) {
const report = await runLighthouse(url);
const score = report.categories.performance.score * 100;
console.log(`${url}: ${score}`);
fs.writeFileSync(
`./reports/${encodeURIComponent(url)}.json`,
JSON.stringify(report, null, 2)
);
}
- Chrome is launched and killed for each URL. Reusing a Chrome instance across many runs allows state to accumulate in the profile and degrades result reliability above roughly 100 loads.
onlyCategories: ['performance'] skips audits you do not need, which reduces run time.- Results are written to individual JSON files for later processing.
Memory and stability tips
Running Lighthouse repeatedly puts sustained pressure on Chrome and the machine. A few practices reduce crashes and noise:
- Restart Chrome between runs. Reusing a single Chrome instance is convenient but accumulates memory and profile state. For reliable results, launch a fresh Chrome for every URL.
- Limit parallelism. Running more Lighthouse instances simultaneously is not always faster — beyond a certain point, resource contention degrades results and causes crashes.
- Use dedicated hardware. Burstable or shared-core instance types (AWS
t series, GCP shared-core N1/E2) produce inconsistent timing results. Use fixed-performance instances: AWS m5.large, GCP n2-standard-2, or Azure D2.
- Do not run Lighthouse alongside other CPU-intensive workloads. Anti-virus scans, builds, or other test suites running concurrently skew performance metrics.
Running many Lighthouse instances in parallel on a single machine will cause instability and unreliable performance scores. Scale horizontally across machines rather than vertically on one machine. A single n2-standard-2 instance running one Lighthouse run at a time produces more reliable results than an n2-standard-8 running eight in parallel.
Choosing instance specs
The minimum recommended specs for a single concurrent Lighthouse run:
- 2 dedicated CPU cores (4 recommended)
- 2 GB RAM (4–8 GB recommended)
- Avoid function-as-a-service infrastructure (AWS Lambda, Google Cloud Functions) — cold starts and CPU throttling produce highly variable results
At approximately 0.10/hourand 30secondspertest,eachLighthousereportcostsroughly0.0008 on a suitable instance. 