Add bandersnatch element

Add an element to create a bandersnatch pypi mirror. We can then
use this element to create ci mirrors local to each CI rack.

Change-Id: Ie29b38823c63cf320ecce42300c78b5e7ac80a59

elements/bandersnatch/README.md

@@ -0,0 +1,8 @@
+Element to install bandersnatch
+
+Installs a bandersnatch pypi mirror in the image, when booted a cronjob will
+populate the mirror and keep it uptodate.
+
+Note: Booting an image including this element will result in a full mirror
+of https://pypi.python.org (over 100G of of data being downloaded), don't
+do this unless that is your intention.

elements/bandersnatch/element-deps

@@ -0,0 +1,5 @@
+apache2
+os-svc-install
+pip-and-virtualenv
+source-repositories
+use-ephemeral

elements/bandersnatch/install.d/76-bandersnatch

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+set -eu
+
+mkdir -p /opt/stack /srv/pypi/web
+
+virtualenv /opt/stack/bandersnatch
+/opt/stack/bandersnatch/bin/pip install bandersnatch
+ln -s /opt/stack/bandersnatch/bin/bandersnatch /usr/local/bin/bandersnatch
+register-state-path --leave-symlink /srv/pypi/web
+
+# Run bandersnatch once to create a config file
+/usr/local/bin/bandersnatch mirror || true
+
+# Run bandersnatch every hour
+echo "0 * * * * root flock -n /var/run/bandersnatch -c \"/usr/local/bin/bandersnatch mirror &>> /var/log/bandersnatch\"" > /etc/cron.d/bandersnatch.cron
+
+cat > /etc/httpd/conf.d/bandersnatch.conf<<'EOF'
+Alias /pypi /srv/pypi/web
+<Directory "/srv/pypi/web">
+    Options Indexes
+    AllowOverride None
+    Require all granted
+</Directory>
+EOF

elements/bandersnatch/os-refresh-config/post-configure.d/50-bandersnatch

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -eux
+set -o pipefail
+
+# Start building mirror in the background, this can take hours and downloads over 100G of data
+flock -n /var/run/bandersnatch -c \
+    "/usr/local/bin/bandersnatch mirror &>> /var/log/bandersnatch" &