From 3558d2961d5716cdbcfde1d80d19989e38068f85 Mon Sep 17 00:00:00 2001 From: Roger Luethi Date: Sun, 22 Jun 2014 07:50:53 +0200 Subject: [PATCH] robots.txt: keep web crawlers out of /trunk We don't want users to accidentally refer to documentation for the upcoming release. Search engines should not lead them to work in progress. With this patch, robots.txt instructs web search engines to ignore docs.openstack.org/trunk/. Change-Id: I9484d7d527d025c5ff3631bccb9321df583920b3 --- www/robots.txt | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/www/robots.txt b/www/robots.txt index e65975c459..5528593348 100644 --- a/www/robots.txt +++ b/www/robots.txt @@ -1,2 +1,7 @@ -User-agent: Googlebot -Disallow: /*.pdf$ +# openstack-manuals/www/robots.txt +User-agent: Googlebot +Disallow: /*.pdf$ + +# Keep OpenStack users from accidentally using docs for the upcoming release +User-agent: * +Disallow: /trunk/