robots.txt Generator

Use the robots.txt Generator to create a simple crawler rules file with allow or disallow settings and an optional sitemap URL for search engines, locally.

A basic robots.txt template for simple allow-all or disallow-all configurations, with an optional sitemap directive.

Other Tools

About

The robots.txt Generator creates a basic robots.txt file based on your preferences. You can choose to allow or disallow all crawlers and specify a sitemap URL. The resulting file can be pasted directly into your site’s root to guide search engine bots in a straightforward way.