add the containers prep role to the quickstart-extras

add container prep to tripleo-quickstart-extras

Related-Change: I676b429cab920516a151b124fca2e26dd5c5e87b
Related-Change: Ie1ca08de17ff0fddd9c9cbd124ae65735ea4b6bc

Change-Id: Id91cfae8aff8652222a4e9adab0635be6c0f8f64
This commit is contained in:
Wes Hayutin 2016-11-22 17:09:15 -05:00
parent 593f15e741
commit c9896602c7
8 changed files with 357 additions and 0 deletions

View File

@ -0,0 +1,136 @@
overcloud-prep-containers
=========
This role that prepares an environment for a containerized compute is under active development along
with active development of the containerized computer feature itself.
This role walks through the developer setup for a tripleo deployment with a containerized compute.
The developer documentation can be found here: https://etherpad.openstack.org/p/tripleo-containers-work
The instructions below use the master branch from delorean that has been vetted by CI, and then updates
any tripleo rpm to the latest version available from delorean. It should be the same content as what
currently runs in tripleo-ci
This also git checks out https://git.openstack.org/openstack/tripleo-heat-templates
with refspec: refs/changes/59/330659/43 , this can be updated in config/general_config/containers_minimal.yml
Requirements
------------
https://github.com/openstack/tripleo-quickstart/blob/master/README.rst
overcloud-prep-containers variables
--------------
* working_dir: /home/stack
* containerized_overcloud: false
* overcloud_prep_containers_script: overcloud-prep-containers.sh.j2
* containers_default_parameters: container-default-parameters.yaml.j2
* overcloud_prep_containers_log: overcloud_prep_containers.log
* container_image: CentOS-Atomic-Host-7-GenericCloud.qcow2
* container_url: "http://cloud.centos.org/centos/7/atomic/images/{{ container_image}}.gz"
* undercloud_network_cidr: 192.168.24.0/24
* ctl_plane_ip: "{{undercloud_network_gateway|default(undercloud_network_cidr|nthhost(1))}}"
overcloud-prep-config variables
-------------------------------
* overcloud_templates_path: /home/stack/tripleo-heat-templates
* overcloud_templates_repo: https://git.openstack.org/openstack/tripleo-heat-templates
* overcloud_templates_branch: master
tripleo-quickstart variables
----------------------------
* see config/general_config/containers_minimal.yml
Dependencies
------------
These dependencies are accounted for in the unmerged tripleo-quickstart review https://review.openstack.org/#/c/393348/
* Depends-On: https://review.openstack.org/#/c/393348/
* Depends-On: https://review.gerrithub.io/#/c/300328/
How to Execute:
---------------
Review https://github.com/openstack/tripleo-quickstart/blob/master/README.rst::
mkdir quickstart_containers
export WORKSPACE=$PWD/quickstart_containers
cd $WORKSPACE
git clone https://github.com/openstack/tripleo-quickstart.git
git clone https://github.com/openstack/tripleo-quickstart-extras.git
# Update quickstart to use the right review
pushd tripleo-quickstart
git remote add gerrit https://review.openstack.org/openstack/tripleo-quickstart
git fetch --all
git review -d I676b429cab920516a151b124fca2e26dd5c5e87b
popd
# Update quickstart-extras to use the right review
pushd tripleo-quickstart-extras
git remote add gerrit https://review.openstack.org/openstack/tripleo-quickstart-extras
git fetch --all
git-review -d Id91cfae8aff8652222a4e9adab0635be6c0f8f64
git-review -x Ie1ca08de17ff0fddd9c9cbd124ae65735ea4b6bc
popd
mkdir /var/tmp/containers
export WD=/var/tmp/containers
export VIRTHOST=<virthost>
pushd tripleo-quickstart
sed -i "s|git+https://git.openstack.org/openstack|file://$WORKSPACE|g" quickstart-extras-requirements.txt
./quickstart.sh --no-clone --working-dir $WD --teardown all --requirements quickstart-extras-requirements.txt --playbook quickstart-extras.yml --config $PWD/config/general_config/containers_minimal.yml --tags all --release master-tripleo-ci $VIRTHOST
How to Execute with Additional gerrit reviews
---------------------------------------------
This will install a local delorean instance and build the reviews into the undercloud/overcloud
Example change https://review.openstack.org/#/c/396460/
STEPS::
export GERRIT_HOST=review.openstack.org
export GERRIT_BRANCH=master
export GERRIT_CHANGE_ID=396460
export GERRIT_PATCHSET_REVISION=3ea99ef27f60157699c13acb64f88d2cd03d237b
# Note.. FOR RHEL VIRTHOST's
* ensure mock is installed on the virthost *, for rhel it comes from epel.. then remove the epel repo
# Build the yum repo in /home/stack of the $VIRTHOST
./quickstart.sh \
--no-clone \
--working-dir $WD \
--teardown all \
--requirements quickstart-extras-requirements.txt \
--playbook dlrn-gate.yml \
--config $PWD/config/general_config/containers_minimal.yml \
--extra-vars compressed_gating_repo="/home/stack/gating_repo.tar.gz" \
--tags all \
--release master-tripleo-ci \
$VIRTHOST
# Consume the local delorean repo in addition to the normal deployment
./quickstart.sh \
--no-clone \
--working-dir $WD \
--teardown none \
--retain-inventory \
--requirements quickstart-extras-requirements.txt \
--playbook quickstart-extras.yml \
--config $PWD/config/general_config/containers_minimal.yml \
--extra-vars compressed_gating_repo="/home/stack/gating_repo.tar.gz" \
--skip-tags provision \
--tags all \
--release master-tripleo-ci \
$VIRTHOST

View File

@ -0,0 +1,11 @@
---
# defaults file for ansible-role-tripleo-overcloud-prep-containers
working_dir: /home/stack
containerized_overcloud: false
overcloud_prep_containers_script: overcloud-prep-containers.sh.j2
containers_default_parameters: container-default-parameters.yaml.j2
overcloud_prep_containers_log: overcloud_prep_containers.log
container_image: CentOS-Atomic-Host-7-GenericCloud.qcow2
container_url: "http://cloud.centos.org/centos/7/atomic/images/{{ container_image }}.gz"
undercloud_network_cidr: 192.168.24.0/24
ctl_plane_ip: "{{undercloud_network_gateway|default(undercloud_network_cidr|nthhost(1))}}"

View File

@ -0,0 +1,20 @@
# Create the scripts that will be used to setup the overcloud images
- name: Create overcloud prep-containers script
template:
src: "{{ overcloud_prep_containers_script }}"
dest: "{{ working_dir }}/overcloud-prep-containers.sh"
mode: 0755
- name: Create the default parameters for the container deployment
template:
src: "{{ containers_default_parameters }}"
dest: "{{ working_dir }}/containers-default-parameters.yaml"
mode: 0755
- name: Create the python script to upload images to the local registry
template:
src: upload_images_to_local_registry.py.j2
dest: "{{ working_dir }}/upload_images_to_local_registry.py"
mode: 0755

View File

@ -0,0 +1,9 @@
---
# tasks file for ansible-role-tripleo-overcloud-prep-containers
- include: create-scripts.yml
tags:
- undercloud-scripts
- include: overcloud-prep-containers.yml
tags:
- containers

View File

@ -0,0 +1,6 @@
---
- name: Prepare for the containerized deployment
shell: |
"{{ working_dir }}/overcloud-prep-containers.sh" > \
{{ overcloud_prep_containers_log }} 2>&1
when: containerized_overcloud|bool

View File

@ -0,0 +1,14 @@
parameter_defaults:
# NovaImage: atomic-image # FYI the team is using overcloud-full currently https://review.openstack.org/#/c/395880/
# Defaults to 'tripleoupstream'. Specify a local docker registry
# Example: 192.0.2.1:8787/tripleoupstream
DockerNamespace: 192.168.24.1:8787/tripleoupstream
# Enable local Docker registry
DockerNamespaceIsRegistry: true
DockerAgentImage: heat-docker-agents:newton
# Docker containers
DockerNovaComputeImage: centos-binary-nova-compute:newton
DockerLibvirtImage: centos-binary-nova-libvirt:newton
DockerOpenvswitchImage: centos-binary-neutron-openvswitch-agent:newton
# DockerOvsVswitchdImage: centos-binary-openvswitch-vswitchd:newton
# DockerOpenvswitchDBImage: centos-binary-openvswitch-db-server:newton

View File

@ -0,0 +1,141 @@
#!/bin/bash
set -eux
### --start_docs
## Prepare the undercloud for deploying the containerized compute node
## ===================================================================
## note:: In progress documentation is available at https://etherpad.openstack.org/p/tripleo-containers-work
##::
## Prepare Your Environment
## ------------------------
## * Source in the undercloud credentials.
## ::
source {{ working_dir }}/stackrc
cd {{ working_dir }}
## * Set the DNS Server for neutron
## ::
subnet_uuid=`neutron subnet-list | grep 192.168.24 | awk '{ print $2 }'`
echo $subnet_uuid
neutron subnet-update $subnet_uuid --dns-nameserver 192.168.23.1
neutron subnet-show $subnet_uuid
{% if containerized_overcloud %}
## * Download the Cloud Atomic Image and upload to glance
## ::
wget {{ container_url }}
gunzip -f "{{ container_image }}.gz"
glance image-create --name atomic-image --file {{ container_image }} --disk-format qcow2 --container-format bare
{% endif %}
{% if release != "master" %}
## note:: On TripleO master, docker and docker-registry are installed and enabled. On older releases the following may be required:
## ::
sudo yum install -y docker
sudo systemctl enable docker
sudo systemctl start docker
sudo systemctl enable docker-registry
sudo systemctl start docker-registry
sudo easy_install docker-py
{% endif %}
## * Update heat's max_nested_stack_depth
## ::
sudo sed -i 's/#max_nested_stack_depth = 5/max_nested_stack_depth = 6/' /etc/heat/heat.conf
sudo systemctl restart openstack-heat-engine
## * Set the REGISTRY ADDRESS to 127.0.0.1 for the upload
## ::
# **WORKAROUND** unable to push when registry address is 192.168.24.1 / control_plane_ip
# testing fix https://review.openstack.org/#/c/396460/
# **WORKAROUND** port 8787 is not open on the undercloud
sudo iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 8787 -j ACCEPT
## * Checkout TripleO-Common
## ::
if [ ! -d tripleo-common ]; then
git clone https://github.com/openstack/tripleo-common.git
fi
## * Create Overcloud Containers Uploads template
## ::
bash -c 'cat <<EOF > {{ working_dir }}/tripleo-common/contrib/overcloud_containers.yaml
# put this in {{ working_dir }}/tripleo-common/contrib/
uploads:
- imagename: tripleoupstream/centos-binary-nova-compute:newton
uploader: docker
pull_source: docker.io
push_destination: {{ ctl_plane_ip }}:8787
- imagename: tripleoupstream/centos-binary-nova-libvirt:newton
uploader: docker
pull_source: docker.io
push_destination: {{ ctl_plane_ip }}:8787
- imagename: tripleoupstream/centos-binary-neutron-openvswitch-agent:newton
uploader: docker
pull_source: docker.io
push_destination: {{ ctl_plane_ip }}:8787
- imagename: tripleoupstream/centos-binary-openvswitch-vswitchd:newton
uploader: docker
pull_source: docker.io
push_destination: {{ ctl_plane_ip }}:8787
- imagename: tripleoupstream/centos-binary-openvswitch-db-server:newton
uploader: docker
pull_source: docker.io
push_destination: {{ ctl_plane_ip }}:8787
EOF'
## * Populate the docker registery
## ::
sudo python {{ working_dir }}/upload_images_to_local_registry.py
## * Configure the environments/docker.yaml, this is done automatically.
## ::
echo "============================="
echo "Containers default parameters:"
cat {{ working_dir }}/containers-default-parameters.yaml
echo "============================="
## * Rebuild heat-agents image and push to local registry
## ::
sudo docker build -t "{{ ctl_plane_ip }}:8787/tripleoupstream/heat-docker-agents:newton" {{ working_dir }}/tripleo-common/heat_docker_agent/
sudo docker push {{ ctl_plane_ip }}:8787/tripleoupstream/heat-docker-agents:newton
## * Set the REGISTRY ADDRESS back to the original ip
## ::
# **WORKAROUND** unable to push when registry address is 192.168.24.1 / control_plane_ip
# testing fix https://review.openstack.org/#/c/396460/
## * Deploy Puppet Artifacts to the Overcloud
## ::
wget https://raw.githubusercontent.com/mandre/dotfiles/master/bin/pull_puppet_modules.sh
chmod +x pull_puppet_modules.sh
bash pull_puppet_modules.sh
bash tripleo-common/scripts/upload-puppet-modules --directory ~/puppet-modules/
## * Get the journal logs for docker
sudo journalctl -u docker > docker_journalctl.log
### --stop_docs

View File

@ -0,0 +1,20 @@
# Upload images to local registry
import time
from socket import error as SocketError
from tripleo_common.image.image_uploader import ImageUploadManager
mgr = ImageUploadManager(['{{ working_dir }}/tripleo-common/contrib/overcloud_containers.yaml'])
for i in range(0,3):
try:
mgr.upload()
print("The upload was successfull")
# this seems to break in various ways
except Exception as e:
print(str(e.args) + " retry attempt: " + str(i+1))
time.sleep(20)
continue
break