How to Find Duplicate H1 with CLI
Learn how to detect pages sharing the same H1 tag using crawler.sh CLI. Find duplicate headings that confuse search engines and differentiate your page topics.
When multiple pages share the same H1 tag, search engines receive conflicting signals about which page best matches a given topic. Like duplicate titles, duplicate H1 tags create internal competition and make it harder for any single page to rank. They often indicate template issues or content that needs better differentiation.
This guide shows you how to find every instance of duplicate H1 tags on your website using the crawler.sh CLI.
Step 1: Install crawler.sh CLI
Install the CLI with a single command:
curl -fsSL https://install.crawler.sh | shThis downloads the correct binary for your operating system and architecture, places it in ~/.crawler/bin/, and adds it to your PATH. Restart your terminal or run source ~/.bashrc (or ~/.zshrc) to pick up the new PATH entry.
Verify the installation:
crawler --versionStep 2: Crawl the target website
Run a full crawl of the website you want to audit:
crawler crawl https://example.comThe crawler extracts H1 tag content from every page. Results are saved as an NDJSON file (.crawl) in the current directory. For larger sites:
crawler crawl https://example.com --max-pages 5000Step 3: Run SEO audit
Run the SEO analysis on your crawl data:
crawler seo example-com.crawlThe duplicate H1 check compares H1 tags across all crawled pages and flags groups of pages that share the same heading text.
Step 4: Identify duplicate H1 tags
Look for the Duplicate H1 section in the SEO report. Each group shows the shared H1 text and all pages using it. Common causes:
- Template components that hardcode the same H1 across page types
- Category or archive pages with a generic “Latest Posts” or “All Products” H1
- Product variants sharing the parent product’s H1
- CMS default headings that were never customized
- The site logo or brand name used as H1 on every page via the header template
Step 5: Fix and re-crawl
Make each page’s H1 unique and descriptive:
- Customize each page’s H1 to reflect its specific content
- Add differentiators to similar pages (category names, dates, variant details)
- Fix template logic that produces the same H1 across multiple pages
- Ensure the H1 matches the page’s unique focus - it should align with the title tag but not necessarily be identical
After fixing, re-crawl to verify:
crawler crawl https://example.comcrawler seo example-com.crawlWhy duplicate H1 tags matter for SEO
Unique H1 tags help search engines differentiate your pages and understand what each one is uniquely about. When multiple pages share an H1, search engines may cluster them together or pick an arbitrary one to rank - often not the one you want. Unique headings strengthen the topical clarity of every page on your site and reduce internal keyword cannibalization.