When multiple pages share the same title tag, search engines cannot distinguish between them. This creates keyword cannibalization where your own pages compete against each other for the same search queries. The result is diluted rankings - instead of one strong page, you have several weak ones.

This guide shows you how to find every instance of duplicate titles on your website using the crawler.sh CLI.

Step 1: Install crawler.sh CLI

Install the CLI with a single command:

curl -fsSL https://install.crawler.sh | sh

This downloads the correct binary for your operating system and architecture, places it in ~/.crawler/bin/, and adds it to your PATH. Restart your terminal or run source ~/.bashrc (or ~/.zshrc) to pick up the new PATH entry.

Verify the installation:

crawler --version

Step 2: Crawl the target website

Run a full crawl of the website you want to audit:

crawler crawl https://example.com

The crawler records every page’s title tag. Results are saved as an NDJSON file (.crawl) in the current directory. For larger sites:

crawler crawl https://example.com --max-pages 5000

Step 3: Run SEO audit

Run the SEO analysis on your crawl data:

crawler seo example-com.crawl

The duplicate titles check compares title tags across all crawled pages and flags groups of pages that share the same title.

Step 4: Identify duplicate titles

Look for the Duplicate Titles section in the SEO report. Each group shows the shared title and all pages using it. Common causes:

CMS templates that use a static title instead of the page-specific title
Category and tag pages using the same “Archive” or “Blog” title
Product variants (sizes, colors) sharing the parent product title
Paginated pages all using the same base title without page numbers
Default titles from theme or template boilerplate

Step 5: Fix and re-crawl

Make each page’s title unique and descriptive:

Add page-specific prefixes or suffixes to differentiate similar pages (e.g., “Running Shoes - Men’s” vs “Running Shoes - Women’s”)
Include page numbers on paginated content (e.g., “Blog - Page 2”)
Fix CMS template logic to pull page-specific titles instead of hardcoded defaults
Consolidate truly duplicate pages using canonical tags or 301 redirects

After fixing, re-crawl to verify:

crawler crawl https://example.com
crawler seo example-com.crawl

Why duplicate titles matter for SEO

Unique titles help search engines understand what each page is about and which page to show for a given query. Duplicate titles force search engines to guess, and they often choose the wrong page. This wastes the ranking potential of all the duplicated pages. Fixing duplicate titles is one of the most reliable ways to improve organic visibility across your entire site.