v0.6.3: Orphan Page Detection URL Fix
Fixed false positives in orphan page detection caused by URL normalization mismatch.
What’s New in v0.6.3
URL Normalization Fix for Orphan Detection
Fixed false positives in orphan page detection. Internal links collected during the crawl were normalized (trailing slashes removed, fragments stripped), but page URLs were compared in their raw form. This mismatch caused some pages to appear as orphans even when they had incoming internal links.
Both sides of the comparison now use the same normalization, so orphan detection results are accurate.
Concrete example
Before this fix, a page reachable from the homepage by a link with a trailing slash was sometimes flagged as orphaned because the crawled URL did not have one:
Internal link found: https://example.com/about/ (normalized: example.com/about)Page URL recorded: https://example.com/about (raw)Result in 0.6.2: false positive - flagged as orphanResult in 0.6.3: correctly matched as linkedQuery strings, fragments, and uppercase host components are now handled the same way on both sides of the comparison.
How to verify
Re-run a crawl on any site that previously produced a long orphan list and open the Orphan Pages card in the desktop app, or run crawler seo your-site.crawl --filter orphan-pages. Pages that were false positives should disappear from the report.
Who Benefits
- SEO professionals get reliable orphan page reports without manual verification
- Content teams can trust that flagged orphan pages are genuinely unlinked
- Developers get consistent URL handling across crawl data
Related
More on orphan pages, internal linking, and adjacent releases that touched this area:
About crawler.sh
crawler.sh is a fast Rust-based web crawler and SEO auditing tool that runs entirely on your own machine. Use the CLI for automation, scripts, and CI pipelines, or the desktop app for a visual dashboard with live crawl progress, SEO issue charts, and one-click exports.
Every release ships across both the CLI and the desktop app.
Download the latest version
or run crawler update
from the terminal to upgrade an existing install.