Developer tool

Robots.txt Generator

Set crawler rules and generate a ready-to-upload robots.txt.

Allow all crawlers by default

Sitemap URL (optional)

Disallowed paths (one per line)

Crawl-delay in seconds (optional)

robots.txt

Related tools