Guides
March 6, 2026

How to Find Duplicate H1 with CLI

Learn how to detect pages sharing the same H1 tag using crawler.sh CLI. Find duplicate headings that confuse search engines and differentiate your page topics.

Mehmet Kose
3 mins read

When multiple pages share the same H1 tag, search engines receive conflicting signals about which page best matches a given topic. Like duplicate titles, duplicate H1 tags create internal competition and make it harder for any single page to rank. They often indicate template issues or content that needs better differentiation.

This guide shows you how to find every instance of duplicate H1 tags on your website using the crawler.sh CLI.

Step 1: Install crawler.sh CLI

Install the CLI with a single command:

curl -fsSL https://install.crawler.sh | sh

This downloads the correct binary for your operating system and architecture, places it in ~/.crawler/bin/, and adds it to your PATH. Restart your terminal or run source ~/.bashrc (or ~/.zshrc) to pick up the new PATH entry.

Verify the installation:

crawler --version

Step 2: Crawl the target website

Run a full crawl of the website you want to audit:

crawler crawl https://example.com

The crawler extracts H1 tag content from every page. Results are saved as an NDJSON file (.crawl) in the current directory. For larger sites:

crawler crawl https://example.com --max-pages 5000

Step 3: Run SEO audit

Run the SEO analysis on your crawl data:

crawler seo example-com.crawl

The duplicate H1 check compares H1 tags across all crawled pages and flags groups of pages that share the same heading text.

Step 4: Identify duplicate H1 tags

Look for the Duplicate H1 section in the SEO report. Each group shows the shared H1 text and all pages using it. Common causes:

  • Template components that hardcode the same H1 across page types
  • Category or archive pages with a generic “Latest Posts” or “All Products” H1
  • Product variants sharing the parent product’s H1
  • CMS default headings that were never customized
  • The site logo or brand name used as H1 on every page via the header template

Step 5: Fix and re-crawl

Make each page’s H1 unique and descriptive:

  • Customize each page’s H1 to reflect its specific content
  • Add differentiators to similar pages (category names, dates, variant details)
  • Fix template logic that produces the same H1 across multiple pages
  • Ensure the H1 matches the page’s unique focus - it should align with the title tag but not necessarily be identical

After fixing, re-crawl to verify:

crawler crawl https://example.com
crawler seo example-com.crawl

Why duplicate H1 tags matter for SEO

Unique H1 tags help search engines differentiate your pages and understand what each one is uniquely about. When multiple pages share an H1, search engines may cluster them together or pick an arbitrary one to rank - often not the one you want. Unique headings strengthen the topical clarity of every page on your site and reduce internal keyword cannibalization.

Crawler.sh - Free Local AEO & SEO Spider and a Markdown content extractor | Product Hunt