Guide
The 80% crawl-budget loss on JavaScript sites — and how to fix it
Most JavaScript sites lose roughly 80% of their crawl budget to Google's render queue. This guide quantifies where the loss happens and ships a repeatable fix pattern.
Introduction
This page focuses on the specific 80% loss pattern. For the broader mental model of how crawl budget works on JavaScript sites — what shapes it, what wastes it, and what the three engineering controls are — start with crawl budget fundamentals.
Googlebot's crawl-then-render pipeline has two stages. The HTML crawl is fast and cheap; the JavaScript render stage is slow and rate-limited by the Web Rendering Service queue. On JS-heavy sites, most of the budget is spent in stage two.
The fix pattern is deterministic: move the render work out of the crawl path by pre-rendering canonical URLs. This section walks through how to measure the loss, ship the fix, and verify the recovery.
How to: 80% crawl-budget loss
- 1
Measure the gap between crawl count and render count
In Search Console's Crawl stats, compare total crawl requests to requests marked "HTML-rendered." The delta is the render-queue queue depth. On JS sites the delta is often 70-85%.
- 2
Identify the top 20 page templates by impression
In Search Console, open Performance → Pages and group by URL pattern. The top 20 templates typically cover 80% of impressions. These are the templates to pre-render first.
- 3
Pre-render the top 20 templates
Route crawler traffic on those templates to a pre-rendered cache. The render queue depth on the rest of the site drops mechanically because the crawler has more spare capacity per cycle.
- 4
Add lastmod to the sitemap for recently-changed URLs
Googlebot weights recrawl priority toward URLs with recent lastmod. Pre-rendering without lastmod hints still works but recovers budget more slowly.
- 5
Re-measure after 14 days and after 45 days
The immediate recovery happens within 14 days. The sustained plateau arrives at 45 days. If either checkpoint shows no change, re-check bot-detection coverage and sitemap freshness.
Budget allocation — before vs after
Breakdown of where crawl time is spent on a representative JavaScript site, before and after pre-rendering the top 20 templates.
Category | Before | BestAfter |
|---|---|---|
| HTML crawl (cheap) | 18% | 62% |
| Render queue wait | 72% | 12% |
| Soft 404 + errors | 7% | 3% |
| Useful re-crawls | 3% | 23% |
Questions engineers ask about this guide
It is a median across JS-heavy sites we have audited. Individual sites range from 50% to 90%. Static-first sites (Next.js SSG, Astro) lose < 20% and do not benefit as much from pre-rendering.
Yes. Compare timestamps of the first Googlebot-smartphone crawl to the first Googlebot-smartphone WRS render on the same URL. The delta is the render-queue wait. On affected sites this is 4-48 hours.
Yes for the canonical set you pre-render. The crawler receives pre-rendered HTML on the first fetch, no render stage required. The rest of the site still goes through the queue as before.
Related guides and deep dives
Guide — crawl budget fundamentals
Concept primer if you are new.
Guide — expand crawl budget
Seven levers for systematic recovery.
Guide — large sites 100k+
Scale-specific tactics.
Compare — vs Prerender.io
Vendor comparison.
Technology — dynamic rendering vs SSR
When pre-rendering is the right fix vs SSR.
Guide — JavaScript rendering cost
Budget-side analysis.
Editorial trust
Written by ostr.io engineering team · Engineering Team. We build and run pre-rendering infrastructure for more than 200 engineering teams, which is where the numbers and code samples on this page come from.
Last updated . Editorial scope and review policy: About prerender.info.