robots.txt Tester
Enter the full URL (including https://). The tool doesn't fetch your robots.txt automatically - paste it below.
Choose the bot to simulate.
Results and validity update instantly as you type.
Matching rules: longest User-agent group wins; longest path match wins; Allow beatsDisallow on ties; supports * and $.
robots.txt Checker
A robots.txt checker lets you see how search engines and AI crawlers interpret your site's crawling rules. By simulating different user-agents, you can confirm whether specific pages are being blocked or allowed as intended. This is especially useful if you manage multiple sections of a site, have private areas, or want to control how AI tools interact with your content.
With this checker, you paste your robots.txt content and instantly see whether a bot would be able to crawl a given URL; without waiting for it to be discovered in the wild.
robots.txt Validator
A robots.txt validator ensures that your file follows the correct syntax and structure so search engines can parse it without errors. Even small formatting issues - e.g. missing colons or misplaced wildcards - can lead to rules being ignored.
Using a validator helps catch these mistakes before they cause crawling problems. It also confirms that your rules match your intended behaviour for different bots, from traditional search crawlers to newer AI agents.