Comparison
crawler.sh vs Jina Reader
Single-URL cloud reader vs local crawler with full-site sweep and Markdown archive export.
Jina Reader (r.jina.ai) is a cloud endpoint that turns a single URL into Markdown with a short HTTP call. It is excellent for one-shot lookups inside an agent loop. crawler.sh is built for the next step: sweeping the whole site, deduping, and producing a Markdown corpus on disk. Both produce clean Markdown; they answer different questions.
Side by side
Honest comparison on the axes that decide which tool fits your workflow.
Pricing and feature notes reflect publicly listed information at the time of writing.
When to pick which
Both tools solve real problems. Pick based on where the work actually runs and what you are billing against.
Pick Jina Reader when
You need to convert individual URLs to Markdown on demand from inside an agent or script, the volume is low to moderate, and the simplicity of a single HTTP endpoint outweighs the per-call cost. Jina Reader is genuinely fast at that shape of work.
Pick crawler.sh when
You need the whole site, not a URL. You want a Markdown directory on disk you can hand to a tokenizer or RAG indexer. You want crawls to run locally, with no per-token billing, no API key, and politeness built in.
