Free tool · runs in your browser
robots.txt tester
Paste a robots.txt and the URLs you care about. See instantly whether each one is allowed or blocked — and exactly which rule decides it.
How Google reads robots.txt
- One group applies: the most specific matching User-agent (e.g. Googlebot-Image beats Googlebot), otherwise
*. Groups don't combine. - Longest rule wins: among the Allow/Disallow paths that match, the longest one decides. On a tie, Allow wins.
- Wildcards:
*matches any characters,$marks the end of the URL. Paths are case-sensitive. - Blocking isn't de-indexing: a blocked URL can still be indexed from links — see "Indexed, though blocked by robots.txt". Use noindex to keep a page out of Google.
More free Search Console tools
Search Console action list
Upload an export, get what to fix first.
SERP snippet preview
See your title and description as Google shows them.
Expected CTR calculator
How many clicks a position should bring.
Search Console regex generator
Filters for brand, questions and long-tail queries.
hreflang generator
Correct hreflang tags for HTML and sitemaps.
Indexing errors explained
What every Page indexing status means and how to fix it.
Search Console guides
How to submit sitemaps, export data, filter with regex and more.