HTTP status codes
HTTP status codes are three-digit numbers returned by a server to indicate the result of a browser or crawler request.
HTTP status codes are three-digit numbers that a web server returns in response to a request. They indicate whether the request was successful, redirected, resulted in an error, or requires further action. Status codes are grouped into five categories by their first digit.
Status code categories
- 1xx (Informational) - Request received, processing continues
- 2xx (Success) - Request was successfully received and processed. 200 OK is the most common.
- 3xx (Redirection) - Further action is needed to complete the request. Includes 301 (permanent redirect) and 302 (temporary redirect).
- 4xx (Client Error) - The request contains an error. 404 (Not Found) is the most common. 403 (Forbidden) and 410 (Gone) are also frequent.
- 5xx (Server Error) - The server failed to fulfill a valid request. 500 (Internal Server Error) and 503 (Service Unavailable) indicate server-side problems.
Why status codes matter for SEO
Search engines use status codes to understand how to treat each URL. A 200 means the page is valid and should be indexed. A 301 tells the engine to transfer ranking signals to the new URL. A 404 signals the page is gone and should be removed from the index. Incorrect status codes send wrong signals and can harm rankings.
A common mistake is returning a 200 status code for pages that should be 404s (soft 404s). This wastes crawl budget and confuses search engines.
How crawler.sh helps
Run crawler crawl to record the HTTP status code for every page on your site. Use crawler info to see a full breakdown of status codes across the crawl. The crawler seo command flags specific issues like redirect chains, broken links (4xx), and server errors (5xx).
