Free Marketing Tool
robots.txt Generator for Law Firms
Build a valid robots.txt in seconds with one-click toggles for every major AI crawler — so ChatGPT, Perplexity, and Claude can find and cite your firm. Free, instant, private.
Optional but recommended — helps search and AI engines find every page.
GPTBot
PerplexityBot
Google-Extended
anthropic-ai
ClaudeBot
CCBot
Bytespider
Green = allowed. Click a crawler to block it. Blocking AI crawlers removes your firm from those AI answers.
Googlebot
Bingbot
All others (*)
Keep these allowed unless you have a specific reason to block them.
Frequently Asked Questions
Should law firms allow AI crawlers like GPTBot and ClaudeBot?
Yes, in almost every case. Allowing AI crawlers such as GPTBot, PerplexityBot, ClaudeBot, anthropic-ai, and Google-Extended lets ChatGPT, Perplexity, Claude, and Google AI Overviews read and cite your firm's pages. Blocking them removes your firm from those AI answers, where a growing share of potential clients now research lawyers.
What is a robots.txt file and where does it go?
robots.txt is a plain text file that tells search and AI crawlers which parts of your site they may access. It must live at the root of your domain — for example https://yourfirm.com/robots.txt. Generate the file here, save it as robots.txt, and upload it to your website's root directory.
Does blocking a crawler hide my pages from Google?
Blocking a standard crawler like Googlebot can stop your pages from being indexed and effectively make your firm invisible in search. Only block a crawler if you have a specific reason. For most law firms across North America, keeping Googlebot, Bingbot, and the AI crawlers allowed is the right call.
What should I put in the Disallow lines?
Disallow paths you do not want indexed — admin areas, thank-you pages, shopping-cart or checkout URLs, and internal search results. Do not disallow your practice-area pages, blog, or contact page, since those are exactly what you want clients and AI engines to find.
Is this robots.txt generator free and private?
Yes. It runs entirely in your browser, builds the file instantly, and sends nothing to any server. Nothing you type is stored or transmitted — copy the generated robots.txt and upload it to your site.