v0.2.4: Improved Link Discovery Tracking
Fixed an issue where "discovered from" link tracking was not recording the source page correctly, improving redirect audits and crawl analysis.
What’s New in v0.2.4
Discovered-From Link Fix
Every crawled page tracks which page it was discovered from - this is essential for understanding your site’s internal link structure and debugging redirect chains. A bug in v0.2.3 caused this field to sometimes point to the wrong source page, making it difficult to trace how the crawler reached a particular URL.
This is now fixed. The discovered_from field in crawl output and the Redirects card in the desktop app accurately show where each URL was first found.
Why Link Discovery Tracking Matters
Internal linking is one of the most important aspects of technical SEO. When you know exactly which page links to which, you can identify orphaned content that no other page points to, find redirect chains that waste crawl budget, and understand how link equity flows through your site. The discovered_from field makes this analysis possible without manually clicking through hundreds of pages. Combined with the seo subcommand’s redirect audit, you can trace a complete path from discovery to final destination for every URL on your site.
How to Use It
After running a crawl, use the info subcommand to inspect your .crawl file and review internal link relationships. The desktop app’s Redirects card provides a visual overview of all redirect chains, showing the source page, each hop in the chain, and the final resolved URL. This makes it easy to spot chains with three or more hops that should be shortened to a single redirect.
Other Changes
- Updated llms.txt documentation
Related
Wrap-up
A CMS shouldn't slow you down. Crawler aims to expand into your workflow — whether you're coding content models, collaborating on product copy, or launching updates at 2am.
If that sounds like the kind of tooling you want to use — try Crawler .
