openstack-manuals/www/robots.txt
Roger Luethi 3558d2961d robots.txt: keep web crawlers out of /trunk
We don't want users to accidentally refer to documentation for the
upcoming release. Search engines should not lead them to work in
progress.

With this patch, robots.txt instructs web search engines to ignore
docs.openstack.org/trunk/.

Change-Id: I9484d7d527d025c5ff3631bccb9321df583920b3
2014-06-24 09:01:06 +02:00

8 lines
185 B
Plaintext

# openstack-manuals/www/robots.txt
User-agent: Googlebot
Disallow: /*.pdf$
# Keep OpenStack users from accidentally using docs for the upcoming release
User-agent: *
Disallow: /trunk/