Crystal Ball checks covered

CRAWL_SKIPPED_URL, CRAWL_DUPLICATE_URL, CRAWL_MISSED_PAGE

Guide detail

Choose Your Technical Level

This setting applies to the meaning, causes, and fix sections. Your choice carries to other Fix Library guides on this device.

Practical explanations and page-specific steps. Recommended for most readers.

What the flag means

Crystal Ball may skip, duplicate, or miss pages during a site scan when URLs are hard to discover or appear to describe the same page. This does not always mean the site is broken.

Review the result when important pages are not linked, final URLs are unclear, or duplicate variants split discovery paths. The skipped URL may be intentional; the important question is whether the canonical page was found and evaluated.

Common causes

Common causes include HTTP/HTTPS variants, www/non-www versions, trailing slash conflicts, redirects, canonical tags, sitemap quirks, unpublished pages, noindex rules, JavaScript-only links, faceted URLs, and duplicate product paths.

What to check next

Check whether the issue appears on one page or every page using the same template. Review recent content, theme, plugin, deployment, and server changes, then compare the live output with a known-good page.

How to fix it

Confirm the warning first and make a backup before editing the live site. If the change involves templates, code, DNS, redirects, or server settings, switch to Intermediate or ask for help before proceeding.

  • Custom HTML/static: Normalize internal links to the final canonical URL, avoid mixed trailing slashes, remove redirecting sitemap URLs, and make sure important pages are linked in HTML.
  • WordPress: Check permalinks, canonical tags, noindex settings, categories/tags, sitemap settings, and redirects. Page-builder buttons should use real links, not script-only actions.
  • Shopify: Review product/collection URL variants, unpublished products, redirects, app filters, and canonical product URLs. Collection and product paths can create duplicates if not managed carefully.
  • Wix: Check indexing settings, page visibility, menu links, dynamic pages, and redirects. Publish changes before rescanning.
  • Webflow: Check published status, collection item URLs, redirects, canonical tags, and whether links are real anchor links in the page HTML.

Need help?

If the fix gets murky, visit Support and send the details. Bug reports and Crystal Ball questions are free support. Implementation and development work may be paid support, but we will tell you clearly before anything becomes paid. No surprise invoices from the tower.