Runs locally — nothing is uploaded
Result
Nothing yet — enter data or press Example.
How to use
- 1Choose whether bots may crawl by default.
- 2List the paths that should stay out of search.
- 3Pick a policy for AI crawlers and add your sitemap.
No. It asks well-behaved crawlers not to fetch them; blocked URLs can still appear in search if linked. Use noindex or authentication for privacy.
No, Google ignores it. Bing and some others respect it.
Related tools
SERP snippet previewPreview title and description in searchrobots.txt testerIs this path allowed for this bot?hreflang generatorLanguage versions for search enginesSchema.org generatorStructured data in JSON-LDRedirect generatorRedirect rules for your serverMeta tags generatorEssential head tags in one block
Found mistakes on your site?
We can fix them in one sprint. Reply within one business day.