Google cannot buy from you if it cannot crawl you: crawl errors that hurt stores

A product page that sells nothing has a lot of possible causes. Bad photos, weak copy, a price that's out of line with competitors. But there's a quieter cause that has nothing to do with the page itself: Google never saw it. If a crawler can't reach a page, index it, and understand it, that page might as well not exist in search results. No amount of merchandising fixes an invisible product.

Why crawlability is a store problem, not just an seo problem

Ecommerce sites are structurally harder to crawl than blogs or brochure sites. They generate pages dynamically from a database: variants, filters, collections, paginated category lists. A single product might be reachable through six different URLs depending on how a shopper filtered to get there. Search engines have a finite crawl budget for any given site, and if that budget gets spent on duplicate filter URLs and broken redirects, your best-selling products can end up crawled less often, or not at all.

This matters because most shopping journeys still start with a search engine. If a crawler hits a wall, that shopper never gets the chance to see your product, no matter how good it is once they land on it. Ranking first no longer means being recommended, but you still need to be crawled and indexed before you can be recommended by anything, human or AI.

The crawl errors specific to online stores

A few patterns show up again and again in ecommerce crawl reports.

Soft 404s on out-of-stock and discontinued products. Instead of returning a real 404 or 410, many platforms leave the page live with an "unavailable" banner while returning a 200 status. Google may still index it, and it may still show up in search results pointing shoppers to a dead end. Worse, if a product is temporarily out of stock but expected back, killing the page outright loses the ranking history it built up. The fix is deliberate: use noindex for genuinely discontinued items, keep live pages for temporary stockouts, and make sure your team can tell the difference. Stockouts that go unnoticed cause their own revenue problems well beyond SEO, which is covered in the quiet revenue leak in fast-selling products.

Faceted navigation creating infinite URL variations. Filtering by size, color, and price often generates a unique URL for every combination, most of which have near-identical content. Crawlers will happily follow these into a maze that consumes crawl budget without adding indexable value. Canonical tags, robots.txt rules, or parameter handling in Search Console can contain this, but it needs to be configured, not assumed.

Redirect chains from theme changes and re-platforming. Every time a URL structure changes, whether from a new theme, a platform migration, or a simple slug edit, old links either need a direct redirect or they start returning errors. Chains of three or four redirects in a row slow crawling and sometimes get dropped entirely. This is one of the reasons theme updates quietly cost Shopify stores orders: the damage isn't limited to checkout, it extends to how the store is found in the first place.

Server errors during traffic spikes or plugin conflicts. If your store returns 500 errors intermittently, whether from an overloaded server during a sale or a WooCommerce plugin conflict, Google's crawler experiences that the same way a shopper does: a broken page. Repeated server errors on a domain can slow crawling across the entire site, not just the pages that failed. Plugin conflicts are the most common way WooCommerce stores break, and crawl errors are often the first external sign something's wrong internally.

What search console actually tells you

Google Search Console is free, and it's the closest thing to a direct line into how Googlebot experiences your site. The Page Indexing report (under Indexing) breaks down every URL Google knows about into categories: indexed, or excluded with a specific reason. The reasons worth watching closely for a store are "Not found (404)", "Server error (5xx)", "Crawled - currently not indexed", and "Duplicate without user-selected canonical".

A rising count in "Crawled - currently not indexed" is one of the more telling signals. It means Google visited the page, decided it wasn't worth including in the index, and moved on. For product pages, that often points to thin content, duplication with a similar product, or a page that looks like it was auto-generated with little unique value. It's rarely a technical crawl failure and more often a content quality signal wearing a technical disguise.

The Sitemaps report matters just as much. It shows how many URLs you submitted versus how many were actually indexed from that sitemap, and whether Google was able to read the file at all. A sitemap that hasn't been regenerated since a catalog change will list dead URLs and omit new ones, which wastes crawl attention on pages that no longer matter. If you haven't checked this recently, it's worth reading through how to tell if your store's sitemap is quietly broken, since sitemap decay is one of the more common, least visible causes of a slow index-coverage decline.

How crawl problems show up as a traffic decline you can't explain

Crawl issues rarely announce themselves. They show up weeks later as a slow, confusing drop in organic traffic that doesn't match any obvious cause. No Google algorithm update was announced, nothing changed in your merchandising, and yet sessions from search are down 15 to 20 percent quarter over quarter. If you're in that position right now, the 48-hour checklist for diagnosing a sudden drop in store sales and traffic dropped 40 percent, is it your store or is it Google both walk through how to separate a Google-side ranking shift from a self-inflicted crawl or indexing problem.

The practical difference is timing and scope. A ranking algorithm change tends to move an entire category of pages at once, often in step with a documented update from Google. A crawl or indexing problem tends to be gradual and page-specific: certain URLs quietly disappear from search results over weeks as Google recrawls them less often or drops them from the index. Checking Search Console's coverage report against your traffic timeline usually settles the question.

Building crawl health into a regular routine

The stores that stay ahead of this treat crawl health as a recurring check, not a one-time audit. A reasonable cadence: review the Page Indexing report monthly, check the sitemap after any bulk catalog change, and watch for spikes in server errors any time you change themes, plugins, or hosting. If you run a Shopify store, the checks in how to monitor a Shopify store cover this alongside uptime and speed. WooCommerce owners face an extra layer of complexity because plugins and server configuration both affect crawlability independently, which is why how to monitor a WooCommerce store treats each layer separately.

This is also where automated monitoring earns its keep. Cassian™ runs SEO health checks and sitemap checks continuously across Shopify, WooCommerce, and BigCommerce stores, and rolls the results into the Cassian Score™, so a sitemap that's gone stale or a spike in broken links shows up as a specific alert rather than a mystery you piece together three weeks later from a traffic graph.

The bottom line

Crawl errors don't crash your store and they don't show up in a support ticket. They just quietly narrow how much of your catalog Google is willing to show anyone, one excluded URL at a time. Checking Search Console regularly, keeping your sitemap current, and treating server errors as urgent even when they seem to resolve themselves are the basics that keep a store visible to the search engines that still send most of its traffic.