User-Agent: * Disallow: /terms.html Disallow: /internal/ Disallow: /admin