Rankawy
Free generator · runs in your browser

Build a robots.txt file without deleting existing rules

Paste your current file to merge rules while preserving the rest, or confirm that your site has no file to start a new one.

  • Preserves current rules
  • Checks Googlebot before copy
  • 18 sourced tokens
Does the site already have robots.txt?
AI crawler rules

Enable a capability to block all of its tokens, then adjust individual tokens as exceptions. Blocking search or on-demand retrieval may reduce your site's visibility in some answers.

Block every token in this capability
Block every token in this capability
Block every token in this capability
Block every token in this capability
Block every token in this capability
Block every token in this capability
Optional Content-Signal

An emerging preference signal recipients may ignore; it does not technically enforce blocking.

Output

The output passed semantic access checks.

Size: 0 bytes

Check the result after publishing

Current file compared with output

− Before+ After
   

What does the generator protect?

  • Group boundaries

    Splits a blocked token from a shared group, keeps its non-path directives, and gives it no path.

  • Google access

    Checks that the merge never opens a path previously denied to Googlebot or Googlebot-Image.

  • Existing lines and size

    Keeps pasted duplicates, deduplicates only additions and refuses files over 500 KiB.

  • Usage signal

    Writes the three Content-Signal keys with yes or no values only.