Warmup Cache Request Errors: Fix Common Cache Failures Fast

A warmup cache request fails silently more often than it fails loudly. The crawler runs, the dashboard shows a status, and the cache still stays cold. This guide walks through the errors that actually show up in production, why they happen, and the exact fix for each one.

Why Warmup Cache Requests Fail More Often Than You Expect

A warmup cache request depends on several systems agreeing at once: the crawler, the web server, the caching plugin, and often a firewall or CDN in front of everything. Any one of these can silently block or slow the process without throwing an obvious error.

Most failures fall into a small set of patterns. The crawler never starts. Requests time out before the page finishes rendering. URLs get blocklisted after one bad response. A firewall blocks the crawler’s own requests. Cron stops firing on schedule. The cache shows a hit, but the content served is still wrong. Each pattern has a specific fix, and guessing at random wastes hours.

How to Diagnose a Failed Warmup Cache Request First

Step 1: Confirm the Crawler Actually Ran

Check the plugin’s log or status page before assuming a config problem. LiteSpeed Cache shows a live crawler map with color-coded status. WP Rocket logs preload activity in the wpr_rocket_cache database table. If the log shows zero activity, the problem is upstream of the crawl itself.

Step 2: Test One URL Manually

Run a single request outside the plugin to isolate the layer causing the failure:

curl -I -A "Mozilla/5.0" --max-time 15 https://yoursite.com/sample-page/

Check the returned status code and cache header. A 200 with a cache-miss header points to a caching config issue. A 403, 502, or 504 points to a firewall or server-level block. A timeout points to slow page generation or a proxy limit.

Step 3: Check Site Health for Loopback Requests

WordPress uses loopback requests, the server calling itself, for many background tasks including cron and some crawler implementations. Open Tools, Site Health, and check the “Loopback requests” test. A failure here breaks warmup regardless of which plugin you use.

Error 1: The Crawler Never Starts or Gets Stuck

Cause: Object Cache Conflicts

Some crawlers stop after a fixed number of visits or freeze at “Start watching” with no further status update. This often traces back to a broken or disabled object cache. If Redis or Memcached is misconfigured, the crawler can complete a partial run and silently stop instead of throwing a visible error.

Cause: Server-Level Crawler Disabled

LiteSpeed Cache requires the crawler engine to be enabled at the web server level, not just inside the plugin. Some hosts run LiteSpeed Web ADC as a load balancer in front of an Apache origin server, which looks like LiteSpeed but does not support the crawler at all. In that setup, no plugin setting fixes the problem, since the server itself lacks the required component.

Fix: Reset and Restart the Crawler Cleanly

Deactivate and reinstall the caching plugin through WP-CLI, clear any leftover crawler-related rows from wp_options, then reconfigure from a blank state:

wp plugin deactivate litespeed-cache --uninstall
wp plugin install litespeed-cache --activate

If the crawler still fails to start after a clean install, contact your host and ask directly whether the crawler engine is enabled at the server level. This single question resolves a large share of “stuck crawler” tickets.

Error 2: Warmup Requests Time Out

Cause: Slow Page Generation Exceeding Timeout Limits

A page that takes 16 seconds to generate without cache will fail a crawler set to a 10-second timeout, even though the page eventually loads fine in a browser. The crawler receives no response header in time and logs a timeout rather than a slow success.

Cause: CDN or Proxy Timeout Ceilings

If you run Cloudflare in front of your site, free-tier connections often cap at 100 seconds before returning a 524 error. Increasing your plugin’s timeout setting past that ceiling has no effect, since the shortest timeout in the chain always wins.

Fix: Increase Timeout and Isolate the Slow Layer

Raise the crawler timeout in your plugin settings to at least 30 seconds as a starting point. Then measure where time is actually going:

curl -w "Connect: %{time_connect}s\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n" -o /dev/null -s https://yoursite.com/slow-page/

If TTFB alone accounts for most of the total time, the bottleneck is server-side rendering, not the crawler configuration. Fix the underlying page speed issue rather than continuing to raise timeout values.

Error 3: URLs End Up Blocklisted

Cause: Pages Returning No-Cache Headers

Crawlers automatically blocklist any URL that responds with a no-cache control header, since warming a page that will never be cached wastes a request. If pages you expect to be cacheable keep landing on the blocklist, check whether another plugin is forcing DONOTCACHEPAGE on them unintentionally.

Cause: Non-200 Status Codes

Redirect loops, expired password protection, and broken permalinks all return non-200 responses that stall the crawler queue. A single misconfigured redirect can block an entire batch from completing if the crawler processes URLs sequentially.

Fix: Clear the Blocklist and Re-Test Individually

Empty the blocklist, then re-run a manual crawl on a small sample of the previously blocked URLs. Check each response code directly:

curl -I https://yoursite.com/previously-blocked-page/

A 301 or 302 means a redirect is intercepting the request. Fix the redirect chain first, then re-add the URL to your warmup queue.

Error 4: Warmup Requests Get Blocked with 403, 502, or 504

Cause: WAF or Security Plugin Flagging the Crawler

A firewall or bot-protection plugin can treat crawler traffic as suspicious, since the request pattern looks automated by design. Cloudflare’s Bot Fight Mode, Wordfence, and similar tools commonly block crawler requests using the same rules meant to stop malicious bots.

Fix: Allowlist the Crawler User Agent and IP

Identify your crawler’s user agent string from the plugin settings, then add an explicit allow rule in your firewall for that string combined with your server’s outbound IP. In Cloudflare, this means creating a Firewall Rule that skips WAF checks specifically for that user agent on your own domain, not a blanket bypass for all traffic.

Temporarily disable security plugins one at a time to confirm which one is responsible, rather than disabling all of them at once. This isolates the exact rule instead of leaving your site unprotected longer than necessary.

Error 5: Scheduled Warmup Stops Running

Cause: WP-Cron Disabled Without a Server Cron Replacement

If DISABLE_WP_CRON is set to true in wp-config.php, WordPress stops firing its own background tasks on page load, including scheduled warmup runs. Without a real server cron calling wp-cron.php on a schedule, warmup never fires again after the first manual run.

Cause: Plugin Update Breaking Scheduled Actions

Plugin updates occasionally break the internal scheduling hooks a crawler depends on. Store owners have reported scheduled warmup working fine after a manual run, then silently failing to repeat after an update, with no error message anywhere in the dashboard.

Fix: Verify Cron Execution Directly

Confirm whether WP-Cron is disabled, then check for a working server cron entry:

wp cron event list

If the expected warmup event is missing or shows a past due date with no execution, add a direct server cron entry as a reliable fallback:

*/15 * * * * wget -q -O /dev/null "https://yoursite.com/wp-cron.php?doing_wp_cron"

Error 6: Cache Shows a Hit but Content Looks Wrong

Cause: Missing Mobile or WebP Variants

If your site serves separate cache files for mobile and desktop, or for WebP versus standard images, a crawler using only one request header set leaves the other variant cold. Visitors on the uncrawled variant still hit a slow first load, even though your dashboard shows the cache as fully warmed.

Cause: Cart Fragments Cached by Mistake

On WooCommerce sites, a mini-cart widget appearing pre-filled for new visitors is a classic sign that a dynamic AJAX fragment got cached as if it were static content. This is a caching configuration error surfaced by warmup, not caused by it, since warmup simply reveals whatever the underlying cache rules allow.

Error 7: Warmup Overloads the Server

Cause: No Rate Limiting or Concurrency Control

Crawling a large site without a delay between requests can spike CPU to 100% and return 429 or 500 errors from your own server, which then get logged as crawler failures rather than the actual cause: too much concurrent load.

Fix: Apply Backoff and Batch Sizing

Set an explicit delay between requests, starting around 200 to 500 milliseconds, and reduce concurrency on shared hosting to a single request at a time. Some crawlers automatically slow down after repeated 429 or 500 responses; confirm this setting is enabled rather than assuming it by default.

Error 8: Loopback and DNS Resolution Failures

Cause: Server Cannot Reach Itself

Some hosting environments block a server from calling its own public domain, a setup problem known as a loopback failure. A warmup cache request that relies on the server visiting its own URLs fails immediately in this case, regardless of plugin settings.

Cause: DNS Misconfiguration After a Migration

Sites that recently changed hosts or DNS providers sometimes still resolve internally to an old IP address. The crawler sends requests that never reach the current server, producing connection errors that look like a plugin bug but are actually a DNS problem.

Fix: Run the Built-In Self-Check

Most modern caching and warmup plugins include a diagnostic tool that tests whether WordPress can reach the site using its own HTTP API. Run this check first, since it isolates loopback and DNS issues from crawler-specific configuration problems. If the self-check fails, confirm your site’s DNS A record points to the correct current IP, and ask your host whether outbound self-requests are blocked at the firewall level.

Why Diagnosing the Right Layer Matters

Every error above looks similar from the plugin dashboard: a stalled crawl, an incomplete log, or a page that never gets marked as warmed. The difference between a five-minute fix and a multi-hour debugging session comes down to identifying which layer actually caused the failure.

A useful habit is testing outside WordPress entirely before touching plugin settings. A raw curl request bypasses the plugin, the theme, and most caching logic, showing you exactly what the server returns without any interpretation layer in between. If curl reproduces the same failure the crawler shows, the problem sits at the server or network level. If curl succeeds where the crawler fails, the problem is specific to how that crawler builds its requests, such as a missing header, cookie, or user agent the server expects.

Keep a simple log of what you tested and when, especially on sites managed by more than one person. A warmup cache request that worked last month and stopped working this month almost always traces back to a specific change: a plugin update, a new security rule, a DNS migration, or a hosting plan change. Comparing the failure date against a changelog cuts diagnosis time significantly.

A Quick Troubleshooting Checklist

Symptom	Likely Cause	First Fix to Try
Crawler never starts	Server-level crawler disabled or object cache broken	Confirm crawler engine enabled with host
Requests time out	Slow page render or CDN timeout ceiling	Measure TTFB directly with curl
URLs blocklisted	No-cache headers or redirect loops	Clear blocklist, re-test individually
403/502/504 errors	WAF or security plugin blocking crawler	Allowlist crawler user agent and IP
Scheduled warmup stops	WP-Cron disabled, no server cron fallback	Add direct server cron entry
Cache hit but wrong content	Missing device/format variant or cached fragment	Warm all variants, check dynamic exclusions
Server overload during warmup	No rate limiting	Add delay and reduce concurrency
Crawler fails with connection errors	Loopback blocked or DNS misconfigured	Run the plugin’s self-check, verify DNS

FAQ

How do I know if a warmup cache request failure is a plugin bug or a server problem?

Test the same URL with a plain curl command outside WordPress. If curl reproduces the same error, the issue is at the server or network level rather than inside the caching plugin. If the request succeeds with curl but fails through the plugin, investigate the plugin configuration, crawler settings, or compatibility with other WordPress plugins.

Why does my crawler fail right after a hosting migration?

After a hosting migration, your domain’s DNS may still resolve to the old server until propagation completes. Verify that your domain’s A record points to the correct IP address before troubleshooting the crawler. Also check that SSL certificates, firewall rules, and server permissions were migrated correctly.

Why does my cache warmup crawler stop after a fixed number of pages?

This usually indicates an object cache issue. Verify that Redis or Memcached is running properly and not disconnecting during the crawl. If necessary, temporarily disable object caching and rerun the warmup process to determine whether it completes successfully without the object cache layer.

Why do I get a 403 error only from the crawler, not from a browser?

A firewall, Web Application Firewall (WAF), or security plugin may identify the crawler as automated traffic and block its requests. Allowlist the crawler’s user agent and your server’s outbound IP address instead of disabling security protections entirely. This keeps your site secure while allowing cache warming to run normally.

Why did my scheduled warmup stop running after a plugin update?

Plugin updates can occasionally break scheduled crawler events or internal cron hooks. Run wp cron event list to confirm the scheduled warmup event still exists. If it is missing or no longer executes, create a server-level cron job as a reliable fallback.

Can a warmup cache request crash my server?

Yes, if the crawler sends too many requests simultaneously without rate limiting. Large warmup jobs can overload CPU, memory, or database resources on smaller hosting plans. Reduce concurrency, add delays between requests, and monitor server load to keep the warmup process safe and stable.

Why does the cache show a hit but the page still looks broken?

A cache hit does not always mean the correct content was cached. The crawler may have warmed only one cache variant, such as the desktop version, while mobile or WebP variants remain cold. Another common cause is a dynamic element, such as a WooCommerce cart fragment or personalized widget, being cached as static content. Review your cache variation rules and exclude dynamic content where appropriate.