Case Studies
Case Study: Marketplace Kills navDemotion
How a property marketplace with 3.2M listings hit 99%+ DOM consistency and removed navDemotion rank instability.
Article
This case study documents the prerendering implementation for a property marketplace with 3.2 million active listings. The marketplace used a React-based listing detail page architecture where listing data was fetched client-side via GraphQL after initial mount. The primary symptom was navDemotion on 23% of high-value listing pages — observable as ranking instability and periodic position drops for otherwise well-optimized pages.
ostr.io was deployed as the prerendering service, reducing DOM Consistency Score from 61% to 99.4% across the listing catalog and eliminating navDemotion signals within 45 days.
The navDemotion Problem
navDemotion is Google's internal quality signal that fires when the DOM Googlebot indexes at crawl time differs significantly from the DOM users experience after JavaScript execution. The signal can suppress rankings for pages that have other strong signals (authority, backlinks, user engagement) — which explains why high-value listing pages were ranking well for weeks, then losing 5–15 positions suddenly, without any content change.
The DOM inconsistency on this marketplace had a specific root cause: the React listing page component fetched listing details (price, availability, property attributes) via a GraphQL query in a useEffect hook. The server returned an HTML shell — navigation, layout, SEO metadata — but not the listing content. Googlebot's second-wave JavaScript execution populated the listing content, but with a time delay.
The crawl timeline for a typical listing page:
- Googlebot requests the listing URL
- Receives HTML shell (navigation + empty
<div id="listing-content"></div>) - Second-wave rendering executes the React application
- GraphQL query fires and returns listing data
- React renders the listing details into
#listing-content - Google captures the final DOM: listing content present
But the captured DOM sometimes differed from what users saw because:
- GraphQL query timing varied between Googlebot's execution environment and user browsers
- Feature flags controlling listing page UI elements produced different results in Googlebot's context vs. authenticated user sessions
- A/B tests on listing UI components were not excluded from the crawler path
The result: Googlebot indexed a listing page with Content A; users consistently saw Content B. navDemotion fired for the inconsistency.
Diagnosis: DOM Consistency Score Analysis
Using ostr.io's monitoring tool (deployed in assessment mode before full rollout), 500 listing pages were sampled:
- 100 pages in "ranked position 1–10" (stable, high-performing listings)
- 200 pages in "ranked position 11–50" (typical mid-tier listings)
- 200 pages in "Crawled — currently not indexed" (worst-performing listings)
DOM Consistency Score results:
| Group | Avg DOM Consistency Score | navDemotion Indicator |
|---|---|---|
| Ranked 1–10 | 89% | Low risk |
| Ranked 11–50 | 61% | High risk |
| Not indexed | 41% | Critical |
The correlation was clear: listing pages with high DOM Consistency Scores ranked well; those with low scores either ranked poorly or were not indexed. The 61% average across mid-tier listings explained the ranking instability pattern — these pages were borderline, alternating between navDemotion-suppressed and unsuppressed states as Googlebot re-crawled them.
Implementation
Week 1: Prerendering middleware deployment and WAF configuration.
ostr.io's middleware was deployed via Cloudflare Worker in front of the marketplace's origin infrastructure. The marketplace used Cloudflare Enterprise with Bot Management — this required careful WAF rule configuration to ensure ostr.io's IPs were allowed before Bot Management rules evaluated them.
The WAF rule order:
- Allow: ostr.io IP ranges (priority 1 — execute before all other rules)
- Allow: Known Googlebot IP ranges (priority 2)
- Bot Management: existing configuration (unchanged)
Without this rule ordering, ostr.io render requests would have been blocked by Bot Management's behavioral analysis before the allow rule could apply.
Week 1–2: A/B test exclusion from crawler path.
The A/B testing framework (Optimizely) was configured to return control group content when the request included the X-Prerender-Token header from ostr.io. This ensured prerendered snapshots showed consistent, non-experimental UI.
Week 2: Feature flag normalization for crawler path.
LaunchDarkly was configured to return production-default flag values for prerendering requests. This eliminated the feature flag variance contributing to DOM inconsistency.
Week 2–3: GraphQL pre-fetching for listings.
The highest-impact change: listing GraphQL queries were moved from useEffect (client-side, after mount) to the Next.js App Router page.tsx using async/await server-side data fetching. This pre-populated listing content in the server response, eliminating the client-side GraphQL delay from the snapshot timeline.
This change required refactoring 23 listing page components across different property types (residential, commercial, rental, new development). The refactor took 2 engineers × 3 weeks.
Week 3–4: Cache Warming API for 3.2M listings.
After the code changes were deployed, a Cache Warming API batch job was submitted for all 3.2M listings:
- Listings updated in the last 7 days:
priority: 'high' - Listings with
status: available:priority: 'high' - Other listings:
priority: 'normal'
The batch warming took 4 days to complete across ostr.io's concurrent rendering infrastructure. TTL was set to 30 minutes for active listings (price/availability changes), 4 hours for inactive/archived listings.
Results
45-Day Metrics:
| Metric | Baseline | 45 Days | Change |
|---|---|---|---|
| Avg DOM Consistency Score | 61% | 99.4% | +38.4pp |
| navDemotion instances (sampled) | 23% of top listings | <1% | -96% |
| Indexed listing pages | 1.9M | 2.7M | +42% |
| Wasted crawls (re-crawls of unchanged pages) | 67% of budget | 22% | -67pp |
| Avg crawl frequency (active listings) | 0.8×/month | 2.4×/month | +200% |
| Organic traffic (GSC, 28-day comparison) | Baseline | +34% | — |
The 67% reduction in wasted crawls was the most significant operational improvement. Before prerendering, Googlebot was re-crawling listings repeatedly because each visit produced inconsistent DOM output. With DOM Consistency Score at 99.4%, Googlebot's crawl model recognized that re-crawling these pages rarely produced different content — and reallocated that budget to crawling new, uncrawled listings.
The 3× increase in crawl frequency for active listings reflected Googlebot's prioritization of freshness for a site now delivering consistent, high-quality responses.
The 42% increase in indexed listings (from 1.9M to 2.7M) represented 800,000 additional property listings entering the index. For a property marketplace, each indexed listing is a potential landing page for location-specific search queries. The long-tail organic traffic gain from 800k new indexed listings drove 34% organic traffic growth over the 45-day measurement period.
Key Lessons
navDemotion is subtle but traceable. The ranking instability pattern (good rankings → sudden drops → recovery → repeat) is characteristic of navDemotion cycling. If you observe this pattern on otherwise well-optimized pages, DOM Consistency Score analysis is the first diagnostic step.
A/B tests and feature flags are the most common hidden DOM inconsistency sources. In this case, both were contributing factors. Excluding the crawler path from A/B tests and feature flag experiments is not optional — it is a prerequisite for achieving high DOM Consistency Scores on dynamic applications.
GraphQL client-side fetching at scale causes indexation failure at scale. Moving GraphQL queries to server-side execution (Next.js Server Components, getServerSideProps, or equivalent) is the architectural fix. Prerendering provides the bridge while the refactor is in progress — or permanently if the refactor is not feasible.
Frequently Asked Questions
Direct measurement of DOM Consistency Score is the leading indicator. navDemotion is the consequence; the cause is repeated crawl visits returning meaningfully different DOM output for the same URL. Run a script that fetches the same 200 listing URLs three times each over 24 hours from a verified Googlebot-equivalent client, hash the rendered DOM after `data-ready` signals fire, and compute the consistency score. A score below 90% predicts navDemotion within 30 to 60 days for any page that gets crawled often enough for the pattern to register.
The warming run consumed roughly 90 hours of cumulative ostr.io render-pool time across 4 days, which billed at the platform's standard render-second rate (publicly listed). Just-in-time rendering for the same coverage would have cost approximately 60 percent more over the same window because peak Googlebot crawl bursts triggered cold-cache misses that re-rendered the same listing two or three times before settling. Batched warming amortizes the render across a controlled queue and avoids the duplicate-render tax.
The batch is throttled to respect the marketplace's origin server and the listing data API. ostr.io's batch warming queue paces requests so origin sees a sustained but bounded load (target: under 5 percent of normal user traffic). For 3.2M listings at the configured rate, 4 days is the natural completion window. Faster warming is available for sites that can absorb higher origin load, but for a live marketplace with real user traffic, 4 days was chosen as the safe operating envelope.
Yes for active listings, paired with webhook-driven invalidation on status changes. The 30-minute TTL is the upper bound for staleness on price and availability fields. Webhook hooks (status: sold, status: archived, price-change events) fire immediate invalidations on the affected URL set, so meaningful changes propagate in under 60 seconds. The TTL only matters for changes that do not fire a webhook, which on a property marketplace is essentially metadata edits and image updates — neither of which are time-critical for indexation. ## Related Reading - DOM Consistency Check - Crawl Budget ROI: How Prerendering Pays for Itself - WAF Blocking Legitimate Bots: Cloudflare and AWS - Cache Warming API and Freshness Signal !Raster matrix diagram of operational levers, risks, and validation checks for Case Study: How a Marketplace Eliminated navDemotion Risk Across 3.2M Listings.
Editorial trust
Written by prerender Editorial · Engineering Team. We build and run pre-rendering infrastructure for more than 200 engineering teams, which is where the numbers and code samples on this page come from.
Last updated . Editorial scope and review policy: About prerender.info.