introduce bifrost-deploy container

- This change adds support for building and deploying
  a bifrost container for baremetal provisioning.
- This change documents how to manually deploy and bootstrap
  the bifrost container.

Implements: blueprint bifrost-support
Change-Id: I7d895839b11cbf916be33225875465c3358b5aa4

doc/bifrost.rst

@@ -0,0 +1,210 @@
+=============
+Bifrost Guide
+=============
+
+
+Prep host
+=========
+
+clone kolla
+-----------
+git clone https://github.com/openstack/kolla
+
+cd kolla
+
+set up kolla dependcies `doc`:quickstart.rst
+
+fix hosts file
+--------------
+Docker bind mounts ``/etc/hosts`` into the container from a volume.
+This prevents atomic renames which will prevent ansible from fixing
+the ``/etc/host`` file automatically.
+
+to enable bifrost to be bootstrapped correctly
+add the deployment hosts hostname to 127.0.0.1 line
+e.g.
+
+ubuntu@bifrost:/repo/kolla$ cat /etc/hosts
+127.0.0.1 bifrost localhost
+
+# The following lines are desirable for IPv6 capable hosts
+::1 ip6-localhost ip6-loopback
+fe00::0 ip6-localnet
+ff00::0 ip6-mcastprefix
+ff02::1 ip6-allnodes
+ff02::2 ip6-allrouters
+ff02::3 ip6-allhosts
+192.168.100.15 bifrost
+
+
+enable source build type
+========================
+
+via config file
+---------------
+
+tox -e genconfig
+
+modify kolla-build.conf as follows.
+
+set install_type to source
+
+command line
+------------
+alternitivly if you do not wish to use the kolla-build.conf
+you can enable a source build by appending ``-t source`` to
+your kolla-build or tools/build.py command.
+
+build container
+===============
+
+Development
+-----------
+tools/build.py bifrost-deploy
+
+Production
+----------
+kolla-build bifrost-deploy
+
+launch bifrost
+==============
+
+docker run -it --net=host -v /dev:/dev -d --privileged --name bifrost 192.168.1.51:5000/kollaglue/ubuntu-source-bifrost-deploy:3.0.0
+
+bootstrap bifrost ansible
+=========================
+
+TODO
+
+bootstrap bifrost manual
+========================
+
+docker exec -it bifrost bash
+
+generate ssh key
+----------------
+ssh-keygen
+
+# source env variables
+cd /bifrost
+. env-vars
+. /opt/stack/ansible/hacking/env-setup
+cd playbooks/
+
+
+create /tmp/bootstrap_args
+--------------------------
+
+    skip_package_install: true
+    mysql_service_name: mysql
+    ansible_python_interpreter: /var/lib/kolla/venv/bin/python
+    network_interface: < add you network interface here >
+    # uncomment below if needed
+    # dhcp_pool_start: 192.168.2.200
+    # dhcp_pool_end: 192.168.2.250
+    # dhcp_lease_time: 12h
+    # dhcp_static_mask: 255.255.255.0
+
+bootstap and start services
+---------------------------
+ansible-playbook -vvvv -i /bifrost/playbooks/inventory/localhost /bifrost/playbooks/install.yaml -e @/tmp/bootstrap_args
+
+
+Use bifrost
+===========
+
+check with "ironic node-list" should return with no nodes.
+
+create servers.yml
+------------------
+
+the servers.yml will discribing your physical nodes and list ipmi credentials.
+see bifrost dynamic inventory examples for mor details.
+
+e.g. /tmp/servers.yml
+
+---
+cloud1:
+    uuid: "31303735-3934-4247-3830-333132535336"
+    driver_info:
+      power:
+        ipmi_username: "admin"
+        ipmi_address: "192.168.1.30"
+        ipmi_password: "root"
+    nics:
+      -
+        mac: "1c:c1:de:1c:aa:53"
+      -
+        mac: "1c:c1:de:1c:aa:52"
+    driver: "agent_ipmitool"
+    ipv4_address: "192.168.1.10"
+    properties:
+      cpu_arch: "x86_64"
+      ram: "24576"
+      disk_size: "120"
+      cpus: "16"
+    name: "cloud1"
+
+adjust as appropriate for your deployment
+
+
+
+Enroll Physical Nodes
+---------------------
+
+export BIFROST_INVENTORY_SOURCE=/tmp/servers.yml
+ansible-playbook -vvvv -i inventory/bifrost_inventory.py enroll-dynamic.yaml -e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" -e network_interface=<provisioning interface>
+
+Deploy Nodes
+------------
+export BIFROST_INVENTORY_SOURCE=/tmp/servers.yml
+ansible-playbook -vvvv -i inventory/bifrost_inventory.py deploy-dynamic.yaml -e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" -e network_interface=<prvisioning interface>
+
+at this point ironic should clean down your nodes and install the default
+deabin image.
+
+Advanced configuration
+======================
+
+Custom images
+-------------
+TODO
+
+Bring your own image
+--------------------
+TODO
+
+Bring your own ssh key
+----------------------
+TODO
+
+Known issues
+============
+
+SSH deamon not running
+----------------------
+By default sshd is installed in the image but may not be enabled.
+If you encounter this issue you will have to acess the server phyically in recovery mode to enable the ssh service.
+if your hardware supports it, this can be done remotely with ipmitool and serial over lan.
+e.g.
+ipmitool -I lanplus -H 192.168.1.30 -U admin -P root sol activate
+
+
+References
+==========
+
+Bifrost
+-------
+docs
+____
+http://docs.openstack.org/developer/bifrost/
+
+troubleshooting
+_______________
+http://docs.openstack.org/developer/bifrost/troubleshooting.html
+
+code
+____
+https://github.com/openstack/bifrost
+
+

doc/index.rst

@@ -57,6 +57,7 @@ Kolla Services
    manila-guide
    swift-guide
    kibana-guide
+   bifrost
 
 Developer Docs
 ==============

docker/bifrost/bifrost-base/Dockerfile.j2

@@ -0,0 +1,42 @@
+FROM {{ namespace }}/{{ image_prefix }}openstack-base:{{ tag }}
+MAINTAINER {{ maintainer }}
+
+{% if install_type == 'binary' %}
+
+RUN echo '{{ install_type }} not yet available for {{ base_distro }}' && /bin/false
+
+{% elif install_type == 'source' %}
+ADD bifrost-base-archive /bifrost-base-source
+COPY build_arg.yml /tmp/build_arg.yml
+RUN ln -s bifrost-base-source/* bifrost \
+    && useradd --user-group bifrost \
+    && /var/lib/kolla/venv/bin/pip --no-cache-dir install --upgrade -c requirements/upper-constraints.txt /bifrost
+
+WORKDIR /bifrost
+
+{% if base_distro in ['ubuntu', 'debian'] %}
+RUN sed -e "s/.*mariadb.*//Ig" -i /etc/apt/sources.list && apt-get purge -y mariadb* mysql*
+{% endif %}
+
+
+RUN bash -c './scripts/env-setup.sh && source ./env-vars && source /opt/stack/ansible/hacking/env-setup && \
+    ansible-playbook -vvvv -i /bifrost/playbooks/inventory/localhost /bifrost/playbooks/install.yaml \
+    -e @/tmp/build_arg.yml && \
+{% if base_distro in ['ubuntu', 'debian'] %}
+    apt-get clean'
+{% else %}
+    yum clean all'
+{% endif %}
+
+
+{% endif %}
+
+
+COPY bifrost_sudoers /etc/sudoers.d/bifrost_sudoers
+
+RUN usermod -a -G kolla bifrost \
+    && chmod 750 /etc/sudoers.d \
+    && chmod 440 /etc/sudoers.d/bifrost_sudoers \
+    && chown -R bifrost:bifrost /bifrost
+
+{% block bifrost_base_footer %}{% endblock %}

docker/bifrost/bifrost-base/bifrost_sudoers

@@ -0,0 +1 @@
+bifrost ALL=(ALL) NOPASSWD:ALL

docker/bifrost/bifrost-base/build_arg.yml

@@ -0,0 +1,4 @@
+skip_bootstrap: true
+skip_start: true
+install_dib: true
+create_image_via_dib: false

docker/bifrost/bifrost-deploy/Dockerfile.j2

@@ -0,0 +1,58 @@
+FROM {{ namespace }}/{{ image_prefix }}bifrost-base:{{ tag }}
+MAINTAINER {{ maintainer }}
+ENV container docker
+
+{% import "macros.j2" as macros with context %}
+
+{% if base_distro in ['centos', 'fedora', 'oraclelinux', 'rhel', 'debian'] or (base_distro == 'ubuntu' and base_distro_tag == '16.04') %}
+
+RUN bash -c ' $(cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ \"$i\" == \"systemd-tmpfiles-setup.service\" ] || rm -f $i; done); \
+rm -f /lib/systemd/system/multi-user.target.wants/*;\
+rm -f /etc/systemd/system/*.wants/*;\
+rm -f /lib/systemd/system/local-fs.target.wants/*; \
+rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
+rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
+rm -f /lib/systemd/system/basic.target.wants/*;\
+rm -f /lib/systemd/system/anaconda.target.wants/*;'
+
+{% elif base_distro in ['ubuntu'] %}
+
+# derived from https://raw.githubusercontent.com/tianon/dockerfiles/4d24a12b54b75b3e0904d8a285900d88d3326361/sbin-init/ubuntu/upstart/14.04/Dockerfile
+ADD init-fake.conf /etc/init/fake-container-events.conf
+
+# undo some leet hax of the base image
+RUN rm /usr/sbin/policy-rc.d; \
+    rm /sbin/initctl; dpkg-divert --rename --remove /sbin/initctl
+
+# remove some pointless services
+RUN /usr/sbin/update-rc.d -f ondemand remove; \
+    for f in \
+        /etc/init/u*.conf \
+        /etc/init/mounted-dev.conf \
+        /etc/init/mounted-proc.conf \
+        /etc/init/mounted-run.conf \
+        /etc/init/mounted-tmp.conf \
+        /etc/init/mounted-var.conf \
+        /etc/init/hostname.conf \
+        /etc/init/networking.conf \
+        /etc/init/tty*.conf \
+        /etc/init/plymouth*.conf \
+        /etc/init/hwclock*.conf \
+        /etc/init/module*.conf\
+    ; do \
+        dpkg-divert --local --rename --add "$f"; \
+    done; \
+    echo '# /lib/init/fstab: cleared out for bare-bones Docker' > /lib/init/fstab
+{% endif %}
+
+{% set bifrost_deploy_packages = [
+    'e2fsprogs'
+] %}
+
+{{ macros.install_packages(bifrost_deploy_packages | customizable("packages")) }}
+
+CMD [ "/sbin/init" ]
+
+{% block bifrost_deploy_footer %}{% endblock %}
+{% block footer %}{% endblock %}
+{{ include_footer }}

docker/bifrost/bifrost-deploy/init-fake.conf

@@ -0,0 +1,14 @@
+# copied from https://github.com/tianon/dockerfiles/blob/4d24a12b54b75b3e0904d8a285900d88d3326361/sbin-init/ubuntu/upstart/14.04/init-fake.conf
+# fake some events needed for correct startup other services
+
+description     "In-Container Upstart Fake Events"
+
+start on startup
+
+script
+    rm -rf /var/run/*.pid
+    rm -rf /var/run/network/*
+    /sbin/initctl emit stopped JOB=udevtrigger --no-wait
+    /sbin/initctl emit started JOB=udev --no-wait
+    /sbin/initctl emit runlevel RUNLEVEL=3 --no-wait
+end script

kolla/common/config.py

@@ -173,6 +173,10 @@ SOURCES = {
         'type': 'url',
         'location': ('http://tarballs.openstack.org/aodh/'
                      'aodh-master.tar.gz')},
+    'bifrost-base': {
+        'type': 'url',
+        'location': ('http://tarballs.openstack.org/bifrost/'
+                     'bifrost-master.tar.gz')},
     'ceilometer-base': {
         'type': 'url',
         'location': ('http://tarballs.openstack.org/ceilometer/'

releasenotes/notes/bifrost-f080de99005ad38e.yaml

@@ -0,0 +1,7 @@
+---
+prelude: >
+    During the Newton cycle support for baremetal
+    provisioning with bifrost was added to kolla.
+    Bifrost is a collection of ansible roles and
+    playbooks to deploy a standalone instance of
+    ironic.

tests/test_build.py

@@ -70,7 +70,8 @@ class BuildTestCentosBinary(BuildTest, base.BaseTestCase):
                        "senlin-base",
                        "vmtp",
                        "watcher-base",
-                       "congress-base"]
+                       "congress-base",
+                       "bifrost-base"]
 
     def setUp(self):
         super(BuildTestCentosBinary, self).setUp()
@@ -95,7 +96,8 @@ class BuildTestUbuntuBinary(BuildTest, base.BaseTestCase):
                        "vmtp",
                        "zaqar",
                        "watcher-base",
-                       "congress-base"]
+                       "congress-base",
+                       "bifrost-base"]
 
     def setUp(self):
         super(BuildTestUbuntuBinary, self).setUp()
@@ -118,7 +120,8 @@ class BuildTestOracleLinuxBinary(BuildTest, base.BaseTestCase):
                        "senlin-base",
                        "vmtp",
                        "watcher-base",
-                       "congress-base"]
+                       "congress-base",
+                       "bifrost-base"]
 
     def setUp(self):
         super(BuildTestOracleLinuxBinary, self).setUp()