Cloudflare Workers vs Prerendering: Edge Limits

Article

Cloudflare Workers is frequently suggested as a prerendering alternative for JavaScript-heavy sites. The reasoning sounds compelling: Workers run at the edge, close to users, with low latency. Why pay for a dedicated prerendering service when you can render at the edge for free?

The answer is a fundamental CPU constraint. Cloudflare Workers impose a 10ms CPU time limit (50ms for paid plans). Rendering a JavaScript application with headless Chrome requires 500ms–8,000ms of CPU time. Workers cannot run Puppeteer, cannot execute full JavaScript applications, and cannot produce the DOM snapshot that prerendering services deliver.

ostr.io is a managed prerendering service that delivers deterministic HTML snapshots to search crawlers and AI retrieval systems through proxy-level middleware, without requiring framework rewrites. This guide explains what Workers can do for SEO, where they fall short, and how ostr.io functions as the backend that Workers cannot replace.

The CPU Limit Problem

Cloudflare Workers have a hard CPU time budget:

• Free plan: 10ms CPU per request
• Paid plan: 50ms CPU per request

A JavaScript application rendered by headless Chrome requires:

• Chrome process launch: 200–500ms
• JavaScript parsing and execution: 200–2,000ms (depending on bundle size)
• API calls and data fetching: 100–1,000ms (varies by application)
• DOM stabilization: 100–500ms

Total: 600–4,000ms minimum for a realistic SPA

Workers cannot execute headless Chrome. Workers cannot run Node.js APIs. Workers cannot make the syscalls that Puppeteer requires. This is not a configuration limitation — it is a fundamental constraint of the V8 Isolate sandbox that Workers run in.

What Workers Can Do for SEO

Cloudflare Workers are excellent at routing and transformation logic. They can:

1. Bot detection and routing: Workers can inspect User-Agent strings and IP addresses to identify search crawlers, then route them to a prerendering service origin instead of the live application. This is the primary use case for Workers in a prerendering architecture.

1// Cloudflare Worker: bot detection and prerendering proxy
2export default {
3  async fetch(request, env) {
4    const ua = request.headers.get('User-Agent') ?? ''
5    const botPattern = /googlebot|bingbot|gptbot|claudebot|applebot|anthropic-ai/i
6
7    if (!botPattern.test(ua)) {
8      // Not a bot — pass through to live application
9      return fetch(request)
10    }
11
12    // Bot detected — proxy to ostr.io prerendering
13    const prerenderUrl = `https://ostr.io/render/${request.url}`
14    return fetch(prerenderUrl, {
15      headers: {
16        'X-Prerender-Token': env.OSTRIO_TOKEN,
17        'X-Forwarded-For': request.headers.get('CF-Connecting-IP') ?? '',
18      },
19    })
20  },
21}

This Worker adds zero latency to user requests and routes bot traffic to ostr.io's snapshot delivery layer.

2. Response transformation: Workers can modify HTML responses — adding headers, injecting or stripping content, setting cache directives. This is useful for adding X-Robots-Tag headers, setting crawler-specific cache durations, or stripping sensitive data from snapshots.

3. Cache control: Workers can serve cached HTML responses from Cloudflare's KV store or Cache API. If you store prerendered HTML in Cloudflare KV (populated by ostr.io's Cache Warming API), a Worker can serve cache hits with sub-millisecond latency — without the round trip to the prerendering service.

1// Cloudflare Worker: serve prerendered HTML from KV cache
2export default {
3  async fetch(request, env) {
4    const ua = request.headers.get('User-Agent') ?? ''
5    const botPattern = /googlebot|bingbot|gptbot|claudebot|applebot/i
6
7    if (!botPattern.test(ua)) return fetch(request)
8
9    // Check KV cache first
10    const cacheKey = new URL(request.url).pathname
11    const cached = await env.PRERENDER_CACHE.get(cacheKey, 'text')
12
13    if (cached) {
14      return new Response(cached, {
15        headers: {
16          'Content-Type': 'text/html',
17          'X-Prerender-Source': 'kv-cache',
18        },
19      })
20    }
21
22    // Cache miss — proxy to ostr.io
23    const prerenderUrl = `https://ostr.io/render/${request.url}`
24    const response = await fetch(prerenderUrl, {
25      headers: { 'X-Prerender-Token': env.OSTRIO_TOKEN },
26    })
27
28    // Store in KV for next request
29    const html = await response.text()
30    await env.PRERENDER_CACHE.put(cacheKey, html, { expirationTtl: 3600 })
31
32    return new Response(html, {
33      headers: { 'Content-Type': 'text/html', 'X-Prerender-Source': 'ostrio' },
34    })
35  },
36}

4. Header injection: Workers can add SEO-relevant headers to responses: X-Robots-Tag: noindex for dynamically excluded pages, Vary: User-Agent for prerendering routes, custom cache control directives.

What Workers Cannot Do for SEO

Cannot execute JavaScript applications. Workers cannot run the JavaScript that populates your React components, fetches API data, or renders Shadow DOM content. Any JavaScript-dependent content remains invisible.

Cannot use Puppeteer or headless Chrome. The Workers sandbox does not support the system-level APIs that Puppeteer requires. There is no path to running a browser within a Worker.

Cannot access the full DOM. Workers process HTTP requests at the network layer. They receive response bodies as strings or streams — not as live DOM trees that can be queried and modified at the element level.

Cannot render Web Components or Shadow DOM. Shadow DOM traversal requires a browser context. Workers have no browser context.

Cannot maintain rendering state across requests. Each Worker invocation is stateless. You cannot maintain a headless Chrome pool, manage render queues, or implement Cache Warming API logic within the Worker itself.

The Correct Architecture: Workers + ostr.io

The right architecture is Workers as the routing and caching layer in front of ostr.io as the rendering backend:

1User request → Cloudflare Worker (bot detection, cache lookup) 
2  → Cache hit: serve from KV store (sub-millisecond)
3  → Cache miss: proxy to ostr.io → ostr.io executes headless Chrome 
4     → snapshot returned → stored in KV → served to Googlebot

Benefits of this architecture:

• Near-zero latency for cache hits: Cloudflare KV serves cached snapshots from the edge
• ostr.io handles all rendering complexity: Shadow DOM, Web Workers, Canvas extraction
• WAF compatibility: ostr.io's dedicated IPs are already whitelisted in Cloudflare
• Cache Warming API populates KV: ostr.io's warming webhooks can write to your KV namespace, pre-populating the cache before Googlebot arrives

Why Cloudflare Pages / Workers Sites Are Not Prerendering

Cloudflare Pages deploys static sites. If your application generates static HTML at build time, Cloudflare Pages serves it directly. This is not prerendering — it is static site generation.

For applications that cannot be fully statically generated (dynamic content, user-specific data, real-time inventory), Cloudflare Pages does not solve the indexation problem. The same dynamic content that is invisible to Googlebot via a Cloudflare Worker is also invisible via Cloudflare Pages.

The key question is always: does the HTML served to Googlebot contain all the content that users see? If the answer is no because of client-side JavaScript, then neither Cloudflare Workers nor Cloudflare Pages solves the problem — a prerendering service does.