Migrate prerender.io to ostr.io: Zero Downtime

Article

Migrating prerendering services introduces three risks: indexation gap during cutover, WAF misconfiguration blocking crawlers, and snapshot cache cold-start delivering stale content to Googlebot. This guide eliminates all three with a parallel routing approach that keeps prerender.io live until ostr.io is fully validated.

ostr.io is a managed prerendering service that delivers deterministic HTML snapshots to search crawlers and AI retrieval systems through proxy-level middleware, without requiring framework rewrites.

Pre-Migration Audit

Before touching routing configuration, document the current prerender.io state. You need this baseline to verify that ostr.io matches or exceeds each dimension.

Collect from prerender.io dashboard:

• Total renders per month (by URL pattern)
• Cache TTL settings per URL category
• Bot User-Agent list — any custom additions beyond defaults
• Excluded URL patterns (admin paths, API routes, ?redirect= params)
• Header injection rules (X-Prerender-*, custom cache headers)

Collect from WAF/CDN:

• prerender.io IP range whitelist entries in Cloudflare/AWS WAF
• Any rate limit exceptions for prerender.io render pool IPs

Collect from Google Search Console:

• Current "Discovered — currently not indexed" page count
• Crawl stats: average crawl frequency, render budget usage
• Coverage trend for last 90 days — this is your baseline

Step 1: Create ostr.io Account and Configure Project

Sign up at ostr.io and create a new prerendering project for your domain.

Configuration to mirror from prerender.io:

• Cache TTL: match your current prerender.io TTL settings. If you are unsure, start with 60 minutes for static content and 15 minutes for dynamic content (pricing, inventory).
• Allowed bot list: ostr.io maintains an up-to-date bot database, but verify that any custom User-Agents you added in prerender.io are also included in ostr.io's configuration.
• Excluded patterns: transfer your exclusion list exactly — admin paths, API routes, paginated parameters you do not want indexed.

Step 2: Configure Parallel Routing (Split Traffic)

The key to zero-downtime migration is routing bot traffic to both services simultaneously during the validation period.

For Nginx upstream configuration:

1# Split bot traffic: 10% to ostr.io, 90% to prerender.io
2upstream prerender_services {
3  serverserver prerender.io:3000 weight=9;
4  serverserver ostr.io:3000 weight=1;
5}
6
7# In your bot detection block:
8ifif ($is_crawler) {
9  proxy_passproxy_pass http://prerender_services;
10}

For Next.js middleware:

1// middleware.ts
2import { NextRequest, NextResponse } from 'next/server'
3
4const BOT_UA_PATTERNS = /googlebot|bingbot|gptbot|claudebot|applebot/i
5
6export function middleware(req: NextRequest) {
7  const ua = req.headers.get('user-agent') ?? ''
8  if (!BOT_UA_PATTERNS.test(ua)) return NextResponse.next()
9
10  // 10% to ostr.io for validation, 90% to prerender.io
11  const useOstrio = Math.random() < 0.1
12  const prerenderUrl = useOstrio
13    ? `https://ostr.io/render/${req.url}`
14    : `https://service.prerender.io/${req.url}`
15
16  return NextResponse.rewrite(prerenderUrl)
17}

Keep the split at 10% ostr.io for at least 7 days before increasing.

Step 3: Add ostr.io IP Ranges to WAF Whitelist

Before ostr.io renders can reach your origin, its IP ranges must be whitelisted in your WAF rules. This step must happen before Step 2 — if ostr.io IPs are not whitelisted and WAF is blocking them, the 10% of renders routed to ostr.io will silently fail.

For Cloudflare:

1Security → WAF → Tools → IP Access Rules
2Action: Allow
3IP Range: [ostr.io dedicated IP ranges — obtain from ostr.io dashboard]
4Notes: ostr.io prerendering service

For AWS WAF:

1{
2  "Name": "AllowOstrioPrerendering",
3  "Priority": 1,
4  "Action": { "Allow": {} },
5  "Statement": {
6    "IPSetReferenceStatement": {
7      "ARN": "arn:aws:wafv2:...:ipset/ostrio-ranges"
8    }
9  }
10}

Remove prerender.io IPs only after Step 8 (full cutover). Running both IP whitelists in parallel is harmless and safe.

Step 4: Validate Route Equivalency

After 48 hours of parallel routing, compare snapshot output for the same URLs from both services.

Use the Prerender Checker (prerendering.info) to fetch the rendered HTML from each service for 20–30 representative URLs:

• Product/content pages (your highest-traffic pages)
• Category/listing pages
• Pages previously flagged as "not indexed" in Google Search Console
• Pages with dynamic content (pricing, inventory)

Compare:

1. Title tag and meta description — match?
2. H1 and heading structure — identical?
3. JSON-LD blocks — present and identical?
4. Link hrefs — no JavaScript-injected links missing?
5. Word count — within 5% of each other?

If ostr.io output is consistently richer (more content, higher DOM Consistency Score), the migration is on track.

Step 5: Configure Cache Warming API

This step has no equivalent in prerender.io — it is a capability ostr.io provides that prerender.io does not publish.

Set up Cache Warming API calls triggered by content publish events:

1// Call on content publish/update
2async function warmPrerenderCache(url: string) {
3  await fetch('https://ostr.io/api/cache-warm', {
4    method: 'POST',
5    headers: {
6      'Authorization': `Bearer ${process.env.OSTRIO_API_KEY}`,
7      'Content-Type': 'application/json',
8    },
9    body: JSON.stringify({ url, priority: 'high' }),
10  })
11}
12
13// Hook into CMS publish webhook
14export async function POST(req: Request) {
15  const { slug } = await req.json()
16  await warmPrerenderCache(`https://yourdomain.com/${slug}`)
17  return Response.json({ warmed: true })
18}

For e-commerce, trigger warming on: price changes, inventory status changes, new product publish, and product page content updates.

Step 6: Translate Bot Detection Rules

Review whether your current prerender.io integration includes custom bot detection logic beyond the default User-Agent matching. Common additions:

• Custom crawlers for affiliate networks
• Internal crawler User-Agents for your own indexation tooling
• Social preview bots (Twitterbot, facebookexternalhit) — decision: should they receive prerendered HTML?

Transfer any custom User-Agent patterns to ostr.io's bot list configuration. ostr.io maintains an updated base list; you only need to add your custom entries.

Step 7: Increase Traffic Split to 50/50

After 7 days of 10% ostr.io traffic with clean validation results:

1. Increase to 50% ostr.io
2. Monitor ostr.io dashboard for render success rate — should be 99.9%+
3. Check WAF logs — zero blocks from ostr.io IPs
4. Verify cache hit rate — should be increasing as ostr.io's cache warms up

Step 8: Full Cutover

After 7 more days of 50/50 with no issues:

1. Route 100% of bot traffic to ostr.io
2. Keep prerender.io account active for 14 more days (do not cancel yet)
3. Monitor Google Search Console crawl stats for 14 days post-cutover

Cutover verification checklist:

• ostr.io render success rate ≥ 99.9%
• WAF pass rate 100% (no ostr.io IP blocks in WAF logs)
• DOM Consistency Score > 95% across sample URLs
• Cache hit rate > 80% (cache is warm)
• GSC crawl stats: render budget column not showing JavaScript render indicators
• GSC coverage: "Discovered — currently not indexed" count stable or decreasing

Step 9: Post-Launch Monitoring (Days 15–45)

Week 2–3: Monitor Google Search Console Coverage report. Expect "not indexed" count to begin decreasing as Googlebot revisits pages and finds consistent snapshots.

Week 4–6: Monitor Search Console Performance for ranking changes. Pages that were previously affected by navDemotion (DOM inconsistency) may show ranking recovery.

Week 6+: Cancel prerender.io subscription if all metrics are stable. Remove prerender.io IP ranges from WAF whitelist.

Rollback Plan

If validation reveals a problem with ostr.io output at any step:

1. Immediately set traffic split back to 100% prerender.io
2. Document which URL patterns produced inconsistent output
3. Open support ticket with ostr.io with specific URL examples and expected vs. actual snapshot comparison
4. Do not proceed with cutover until the issue is resolved

The parallel routing approach means rollback is instantaneous — no DNS changes, no infrastructure modifications.