The meta description is the snippet of text that appears below your page title in search results. When a page has no meta description, search engines pull random text from the page body - often resulting in awkward, incomplete, or irrelevant snippets that fail to attract clicks.

This guide shows you how to find every page missing a meta description using the crawler.sh CLI.

Step 1: Install crawler.sh CLI

Install the CLI with a single command:

curl -fsSL https://install.crawler.sh | sh

This downloads the correct binary for your operating system and architecture, places it in ~/.crawler/bin/, and adds it to your PATH. Restart your terminal or run source ~/.bashrc (or ~/.zshrc) to pick up the new PATH entry.

Verify the installation:

crawler --version

Step 2: Crawl the target website

Run a full crawl of the website you want to audit:

crawler crawl https://example.com

The crawler records each page’s meta description along with its URL and other metadata. Results are saved as an NDJSON file (.crawl) in the current directory. For larger sites:

crawler crawl https://example.com --max-pages 5000

Step 3: Run SEO audit

Run the SEO analysis on your crawl data:

crawler seo example-com.crawl

The missing descriptions check flags every page where no <meta name="description"> tag was found.

Step 4: Identify missing descriptions

Look for the Missing Descriptions section in the SEO report. Pages commonly missing meta descriptions include:

Blog posts where the CMS does not auto-generate descriptions
Paginated archive pages (page 2, page 3, etc.)
Tag and category pages
Utility pages like search results, login, and account pages
Dynamically generated pages

Step 5: Fix and re-crawl

Add a unique meta description to every flagged page. Each description should:

Summarize the page content in 50–160 characters
Include the primary keyword naturally
Contain a clear value proposition or call to action
Be unique - never duplicate descriptions across pages

After fixing, re-crawl to verify:

crawler crawl https://example.com
crawler seo example-com.crawl

Why missing descriptions matter for SEO

While meta descriptions are not a direct ranking factor, they significantly impact click-through rates. A well-written description acts as ad copy for your page in search results. Without one, Google auto-generates a snippet that may not highlight your most compelling content. Pages with custom meta descriptions consistently outperform those without in terms of organic clicks.