Robots.txt Generator

It must be placed at the root of your domain, e.g. https://example.com/robots.txt — search engines won't look for it anywhere else.

Disallow tells crawlers not to access a path; Allow can carve out an exception within a disallowed path (for example, disallowing /private/ but allowing /private/public-page).

Not reliably — it only asks crawlers not to fetch the page. A disallowed page can still be indexed (e.g. via links from elsewhere) without its content being crawled. Use a noindex meta tag if you need a page kept out of search results entirely.