Build crawling instructions for your EmDash website.
Robots.txt controls crawling. Use authentication for private content and noindex to prevent indexing.