Robots.txt Generator
Control exactly which pages search engines and AI bots can crawl on your site.
Quick start templates
Configure rules
Group 1
Always include your sitemap URL for better discoverability.
robots.txt preview
User-agent: *
How to use
- Download or copy the generated file
- Upload it to your website’s root directory (e.g.,
example.com/robots.txt) - Test with Google’s Robots Testing Tool
- Always include your sitemap URL for better discoverability