Check a private account path
User-agent: * Disallow: /account/ Path: /account/billing
Check the path against simple Disallow prefixes. This is not a crawler simulation.
Check one path against simple Disallow rules. This is not a crawler simulator.
Check one path against simple Disallow rules. This is not a crawler simulator. Use it to review how a page may appear to search engines and searchers. Generation and inspection run in this browser.
Create and test crawler rules. Workspace preferences are saved locally. Generator values are kept only for this browser session.
This is the indexing half of the SEO journey; Search Preview covers metadata and snippets. Basic Robots Path Check handles Disallow prefixes and is not a crawler simulator. JSON-LD inspection checks structure, not search eligibility. Download text, XML, JSON-LD findings, or HTML link elements.
User-agent: * Disallow: /account/ Path: /account/billing
Check the path against simple Disallow prefixes. This is not a crawler simulation.
Generation and inspection run in this browser. Workspace preferences are saved locally. Generator values are kept only for this browser session.
robots.txt, XML sitemap, Canonical URLs, hreflang links, JSON-LD. This is the indexing half of the SEO journey; Search Preview covers metadata and snippets. Basic Robots Path Check handles Disallow prefixes and is not a crawler simulator. JSON-LD inspection checks structure, not search eligibility.
Download text, XML, JSON-LD findings, or HTML link elements.
Test crawler directives with the target search engine before deployment.
Build crawler rules and sitemap directives.
Web tools
Create an XML sitemap from a URL list.
Web tools
Check canonical URL syntax and flag URL fragments.
Web tools
Build crawl rules as decisions, not mysterious syntax.
Crawl directives
2 rules