CHECK CRAWL ACCESS · FREE

Robots.txt tester

Test whether your robots.txt rules allow a crawler to access a specific URL.

Robots.txt contents

URL to testCrawler name

Make the next improvement.

Use the analyzer for a live check, or save a research action in your workspace.

Your inputs stay in this browser. No account or paid API required.