Add our site robots.txt from Gerrit1

Although most of the data is public, crawling the site isn't
very useful because we are JavaScript/AJAX based.

Signed-off-by: Shawn O. Pearce <sop@google.com>

appjar/src/main/java/com/google/gerrit/public/robots.txt

@@ -0,0 +1,8 @@
+# Directions for web crawlers.
+# See http://www.robotstxt.org/wc/norobots.html.
+
+User-agent: HTTrack
+User-agent: puf
+User-agent: MSIECrawler
+User-agent: Nutch
+Disallow: /