Product Roadmap
Windows support
Bring the CLI and desktop app to Windows with full installer support.
Core TeamCrawl diffing
Compare two .crawl files to detect added, removed, and changed pages between crawl runs.
Core TeamCustom crawl rules
Define include/exclude URL patterns, respect robots.txt, and configure custom request headers.
Core TeamAPI server mode
Run crawler.sh as a local HTTP server with REST endpoints for triggering crawls and retrieving results programmatically.
Core TeamJavaScript rendering
Built-in JS rendering engine with auto-detection. Samples pages and enables rendering when client-side frameworks are detected. Three modes: Auto, Always, and Never.
Core TeamSEO analysis CLI
The `crawler seo` subcommand with 24 check categories, CSV and TXT export, and canonical-aware duplicate detection.
Core TeamContent extraction
Content extraction with Markdown output, word count, byline, and excerpt for every crawled page.
Core TeamDesktop app v0.1
Desktop app with 8 dashboard cards, real-time crawl feed, SEO analysis panel, status code charts, and multi-format export.
Core TeamCLI v0.1
Initial release with crawl, info, and export subcommands. NDJSON, JSON, and Sitemap XML output formats. Configurable concurrency, depth, and delay.
Core Team