Crawlability

Robots.txt Tester

Fetch robots.txt and flag common indexing conflicts.