Crystal Ball checks covered

CRAWL_SKIPPED_URL, CRAWL_DUPLICATE_URL, CRAWL_MISSED_PAGE

What the flag means

Crystal Ball may skip, duplicate, or miss pages during a site scan when URLs are hard to discover or appear to describe the same page. This does not always mean the site is broken.

It is a problem when important pages are not linked, final URLs are unclear, or duplicate variants split the crawl path. The scanner is reading the map you gave it, including the smudges.

Common causes

Common causes include HTTP/HTTPS variants, www/non-www versions, trailing slash conflicts, redirects, canonical tags, sitemap quirks, unpublished pages, noindex rules, JavaScript-only links, faceted URLs, and duplicate product paths.

How to fix it

  • Custom HTML/static: Normalize internal links to the final canonical URL, avoid mixed trailing slashes, remove redirecting sitemap URLs, and make sure important pages are linked in HTML.
  • WordPress: Check permalinks, canonical tags, noindex settings, categories/tags, sitemap settings, and redirects. Page-builder buttons should use real links, not script-only actions.
  • Shopify: Review product/collection URL variants, unpublished products, redirects, app filters, and canonical product URLs. Collection and product paths can create duplicates if not managed carefully.
  • Wix: Check indexing settings, page visibility, menu links, dynamic pages, and redirects. Publish changes before rescanning.
  • Webflow: Check published status, collection item URLs, redirects, canonical tags, and whether links are real anchor links in the page HTML.

Need help?

If the fix gets murky, visit Support and send the details. Bug reports and Crystal Ball questions are free support. Implementation and development work may be paid support, but we will tell you clearly before anything becomes paid. No surprise invoices from the tower.