Skip to content
Crawler.sh Logo Crawler.sh Logo Crawler

Crawler MCP Server Setup for Claude, Cursor & Zed

What is the Crawler MCP server?

Section titled “What is the Crawler MCP server?”

crawler-mcp is a small local binary that exposes crawler.sh as a Model Context Protocol server. Agents like Claude Code, Claude Desktop, Cursor, OpenCode, Zed, and any other MCP-aware client can call into it over stdio to crawl a site, fetch a single page, or discover links - and get back clean Markdown they can reason over.

The server is a thin shim over the same crawling engine the CLI and desktop app use. JavaScript rendering, robots.txt politeness, and per-host backoff are all on by default.

Get started

Section titled “Get started”

  1. Install the binary

    Section titled “Install the binary”
    curl -fsSL https://install.crawler.sh/install-mcp.sh | sh

    See the Installation guide for manual downloads, custom install directories, and tier caps.

  2. Wire it into your client

    Section titled “Wire it into your client”

    Pick the agent you use:

    Claude Code One-command setup with `claude mcp add`.
    Claude Desktop Edit `claude_desktop_config.json` on macOS or Windows.
    Cursor Global or per-project `mcp.json`.
    OpenCode Add to `opencode.json` under the `mcp` key.
    Zed Declare under `context_servers` in `settings.json`.

  3. Call the tools

    Section titled “Call the tools”

    See the Tool reference for the three exposed tools - crawl_site, fetch_page, discover_links - with parameters, defaults, and return shapes.

Section titled “Related”