openstack
/
openstack-ansible
Code Issues Proposed changes

Convert AIO bootstrap from bash to Ansible 

This patch converts the AIO bootstrap process to use Ansible
instead of bash scripting. The patch also minimises the options
available to focus the role concerned to just handle an AIO
bootstrap, but gives it just enough flexibility to allow the
use of an external MongoDB database for Ceilometer/Aodh and
for a deployer to specify a secondary disk for the AIO to
consume.

A major change is that the AIO bootstrap process no longer
assumes that it can destroy a secondary boot device. It
requires a device name to be provided. This prevents horrible
surprises.

Backport Note:
While this backport respects the differences between the Master
branch and the Kilo branch, it also brings all the gating and
AIO related scripts into line with each other to ensure that all
methods and settings (where applicable) are the same.

The following differences from the Master patch are noted:
 - neutron_rpc_workers must be 0 for Kilo
 - nova_virt_type must be specifically configured for Kilo
 - the in-tree pip_install role is used (configured in
   tests/ansible.cfg) instead of the new independent role
 - All references to Aodh have been removed
 - The haproxy env.d related config task has been removed as
   this does not apply to Kilo
 - Change I98bcb478e99e11956618b2327399e18471aab534 is included
   (Add reliable OS detection for ansible bootstrap)
 - Change I31f63a2998080c6fedab3af7c2b1bb1a987d8c08 is included
   (Use pure variable name in $(()) statement)
 - Change If3a914650749d72c2eb13b9f1307ef7b4319bd2f is included
   (Remove quotes from subshell call in bash script)
 - Change Iec03c768a9fadf82831af4b90db1907abd399966 is included
   (use fastest mirror available in OpenStack-CI)
 - Change I666f53e01066bf8bff4d28fa012eadae7c958116 is included
   (Bootstrap Host with updated CPU Map for OpenStack-CI)
 - Change Ic20ab65dc68a6f82eb5bf0751a968aa662b29eb5 is included
   (Make instance info output to stdout on failure)
 - Change I28af8a587bde629fd0c81a6c4ba47dd64c2e44db is included
   (Use Ansible AIO playbook for run-aio-build.sh)
 - Change I730c838a67ca03413fbdfa43ef7606d8cf2f9cd0 is included
   (Updating AIO docs for Ansible playbook)

Conflicts:
>------ansible-role-requirements.yml
>------scripts/bootstrap-aio.sh
>------scripts/gate-check-commit.sh
>------scripts/scripts-library.sh

Closes-Bug: #1522957
Implements: blueprint convert-aio-bootstrap-to-ansible
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Change-Id: I6028952e7260388873f57db47cc3e08126ecc530
(cherry picked from commit 892c7fe46c)
This commit is contained in:
Major Hayden 2015-12-01 15:41:32 +00:00 committed by Jesse Pretorius
parent ff00d6ffd3
commit 6a6b66f83b
31 changed files with 1555 additions and 631 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
ansible-role-requirements.yml Normal file
View File

@ -0,0 +1,2 @@
- src: mattwillsher.sshd
name: sshd

2
doc/source/developer-docs/extending.rst
View File

@ -113,6 +113,8 @@ Ordering and Precedence
``user_*.yml`` variables are just YAML variable files. They will be sourced
in alphanumeric order by ``openstack-ansible``.
.. _adding-galaxy-roles:
Adding Galaxy roles
-------------------

101
doc/source/developer-docs/quickstart-aio.rst
View File

@ -13,11 +13,17 @@ for:
Although AIO builds aren't recommended for large production deployments,
they're great for smaller proof-of-concept deployments.
It's strongly recommended to have hardware that meets the following
requirements before starting an AIO build:
Absolute minimum server resources (currently used for gate checks):
* 8 vCPU's
* 50GB disk space
* 8GB RAM
Recommended server resources:
* CPU/motherboard that supports `hardware-assisted virtualization`_
* At least 80GB disk space (more than 175GB if you have a lxc lvm volume group)
* 8 CPU Cores
* 80GB disk space
* 16GB RAM
It's `possible` to perform AIO builds within a virtual machine but your
@ -29,7 +35,7 @@ Running an AIO build in one step
--------------------------------
For a one-step build, there is a `convenient script`_ within the
openstack-ansible repository that will run a AIO build with defaults:
Openstack-Ansible repository that will run a AIO build with defaults:
.. _convenient script: https://raw.githubusercontent.com/openstack/openstack-ansible/kilo/scripts/run-aio-build.sh
@ -47,8 +53,8 @@ Running a customized AIO build
There are four main steps for running a customized AIO build:
* Configuration *(this step is optional)*
* Initial host bootstrap
* Install and bootstrap Ansible
* Initial host bootstrap
* Run playbooks
Start by cloning the openstack-ansible repository and changing into the
@ -73,18 +79,41 @@ development) build it is usually best to checkout the latest tagged version.
$ # Checkout the latest tag from the previous command.
$ git checkout 11.2.5
By default the scripts deploy all OpenStack services. At this point you may
optionally adjust which services are deployed within your AIO build. Look at
the ``DEPLOY_`` environment variables at the top of
``scripts/run-playbooks.sh`` for more details. For example, if you'd like to
skip the deployment of ceilometer, you would execute the following:
By default the scripts deploy all OpenStack services with sensible defaults
for the purpose of a gate check, development or testing system.
Review the ``tests/roles/bootstrap-host/defaults/main.yml`` file to see
various configuration options. Deployers have the option to change how the
host is bootstrapped. This is useful when you wish the AIO to make use of
a secondary data disk, or when using this role to bootstrap a multi-node
development environment.
The bootstrap script is pre-set to pass the environment variable
``BOOTSTRAP_OPTS`` as an additional option to the bootstrap process. For
example, if you wish to set the bootstrap to re-partition a specific
secondary storage device (/dev/sdb), which will erase all of the data on the
device, then execute:
.. code-block:: bash
$ export BOOTSTRAP_OPTS="bootstrap_host_data_disk_device=sdb"
Additional options may be implemented by simply concatenating them with
a space between each set of options, for example:
.. code-block:: bash
$ export BOOTSTRAP_OPTS="bootstrap_host_data_disk_device=sdb"
$ export BOOTSTRAP_OPTS="${BOOTSTRAP_OPTS} bootstrap_host_ubuntu_repo=http://mymirror.example.com/ubuntu"
The next step is to bootstrap Ansible and the Ansible roles for the
development environment. Deployers can customize roles by adding variables to
override the defaults in each role (see :ref:`adding-galaxy-roles`). Run the
following to bootstrap Ansible:
.. code-block:: bash
$ export DEPLOY_CEILOMETER="no"
Note that the scripts still build containers for any service that you disable,
but do not deploy the service.
$ scripts/bootstrap-ansible.sh
In order for all the services to run, the host must be prepared with the
appropriate disks, packages, network configuration and a base configuration
@ -99,15 +128,6 @@ configuration then this can be done now by editing
``/etc/openstack_deploy/user_variables.yml``. Please see the `Install Guide`_
for more details.
Note that the host bootstrap is not idempotent and should only be executed
once.
Once you're ready to deploy, bootstrap Ansible by executing:
.. code-block:: bash
$ scripts/bootstrap-ansible.sh
Finally, run the playbooks by executing:
.. code-block:: bash
@ -131,6 +151,10 @@ Keystone service, execute:
$ cd /opt/openstack-ansible/playbooks
$ openstack-ansible os-keystone-install.yml
**Note:** The AIO bootstrap playbook will still build containers for services
that are not requested for deployment, but the service will not be deployed
in that container.
.. _Install Guide: ../install-guide/
Rebuilding the AIO
@ -171,30 +195,6 @@ will destroy whole environments and should be used WITH CAUTION.
After the teardown is complete, ``run-playbooks.sh`` may be executed again to
rebuild the AIO.
AIO Host Bootstrap Customisation
--------------------------------
The AIO makes a number of assumptions about the system it is being deployed
on. Many of these assumptions may be changed through the use of environment
variables which can be changed prior to the AIO host bootstrap.
As these options change on a regular basis it is best to examine each of the
scripts used to find the environment variables which can be overridden:
* ``scripts/bootstrap-aio.sh`` (this sets the AIO host up)
* ``scripts/bootstrap-ansible.sh`` (this sets Ansible up)
* ``scripts/scripts-library.sh`` (this is used by all the other scripts)
As an example, if you wish the Keystone Admin password to be a value which
you set (rather than the default of a random string), then execute:
.. code-block:: bash
export ADMIN_PASSWORD="secrete"
More details about the scripts used can be found in the `scripts page`_.
.. _scripts page: scripts.html
Quick AIO build on Rackspace Cloud
----------------------------------
@ -217,15 +217,12 @@ soon as the instance starts. Save this file as ``user_data.yml``:
- export REPO=https://github.com/openstack/openstack-ansible
- export BRANCH=kilo
- git clone -b ${BRANCH} ${REPO} /opt/openstack-ansible
- export DEPLOY_CEILOMETER="no"
- cd /opt/openstack-ansible && scripts/bootstrap-aio.sh
- cd /opt/openstack-ansible && scripts/bootstrap-ansible.sh
- cd /opt/openstack-ansible && scripts/bootstrap-aio.sh
- cd /opt/openstack-ansible && scripts/run-playbooks.sh
output: { all: '| tee -a /var/log/cloud-init-output.log' }
Feel free to customize the YAML file to meet your requirements. As an example
above, the deployment of ceilometer will be skipped due to the
``DEPLOY_CEILOMETER`` export line.
Feel free to customize the YAML file to meet your requirements.
We can pass this YAML file to nova and build a Cloud Server at Rackspace:

2
etc/openstack_deploy/openstack_user_config.yml.aio
View File

@ -12,7 +12,7 @@ used_ips:
global_overrides:
internal_lb_vip_address: 172.29.236.100
external_lb_vip_address: 192.168.1.1
external_lb_vip_address: {{ bootstrap_host_public_address | default(ansible_default_ipv4.address) }}
tunnel_bridge: "br-vxlan"
management_bridge: "br-mgmt"
provider_networks:

409
scripts/bootstrap-aio.sh
View File

@ -1,5 +1,5 @@
#!/usr/bin/env bash
#
# Copyright 2014, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
@ -17,405 +17,16 @@
## Shell Opts ----------------------------------------------------------------
set -e -u -x
## Vars ----------------------------------------------------------------------
DEFAULT_PASSWORD=$(tr -dc 'a-zA-Z0-9' < /dev/urandom | head -c 32)
export BOOTSTRAP_AIO="yes"
export HTTP_PROXY=${HTTP_PROXY:-""}
export HTTPS_PROXY=${HTTPS_PROXY:-""}
export ADMIN_PASSWORD=${ADMIN_PASSWORD:-$DEFAULT_PASSWORD}
export SERVICE_REGION=${SERVICE_REGION:-"RegionOne"}
export DEPLOY_OPENSTACK=${DEPLOY_OPENSTACK:-"yes"}
export DEPLOY_SWIFT=${DEPLOY_SWIFT:-"yes"}
export DEPLOY_CEILOMETER=${DEPLOY_CEILOMETER:-"yes"}
export PUBLIC_INTERFACE=${PUBLIC_INTERFACE:-$(ip route show | awk '/default/ { print $NF }')}
export PUBLIC_ADDRESS=${PUBLIC_ADDRESS:-$(ip -o -4 addr show dev ${PUBLIC_INTERFACE} | awk -F '[ /]+' '/global/ {print $4}')}
export NOVA_VIRT_TYPE=${NOVA_VIRT_TYPE:-"qemu"}
export TEMPEST_FLAT_CIDR=${TEMPEST_FLAT_CIDR:-"172.29.248.0/22"}
export FLUSH_IPTABLES=${FLUSH_IPTABLES:-"yes"}
export RABBITMQ_PACKAGE_URL=${RABBITMQ_PACKAGE_URL:-""}
export MONGO_HOST=${MONGO_HOST:-"172.29.236.100"}
export DEBIAN_FRONTEND=${DEBIAN_FRONTEND:-"noninteractive"}
# Default disabled fatal deprecation warnings
export CINDER_FATAL_DEPRECATIONS=${CINDER_FATAL_DEPRECATIONS:-"no"}
export GLANCE_FATAL_DEPRECATIONS=${GLANCE_FATAL_DEPRECATIONS:-"no"}
export HEAT_FATAL_DEPRECATIONS=${HEAT_FATAL_DEPRECATIONS:-"no"}
export KEYSTONE_FATAL_DEPRECATIONS=${KEYSTONE_FATAL_DEPRECATIONS:-"no"}
export NEUTRON_FATAL_DEPRECATIONS=${NEUTRON_FATAL_DEPRECATIONS:-"no"}
export NOVA_FATAL_DEPRECATIONS=${NOVA_FATAL_DEPRECATIONS:-"no"}
export TEMPEST_FATAL_DEPRECATIONS=${TEMPEST_FATAL_DEPRECATIONS:-"no"}
# Ubuntu Repository Determination (based on existing host OS configuration)
UBUNTU_RELEASE=$(lsb_release -sc)
UBUNTU_REPO=${UBUNTU_REPO:-$(awk "/^deb .*ubuntu\/? ${UBUNTU_RELEASE} main/ {print \$2; exit}" /etc/apt/sources.list)}
UBUNTU_SEC_REPO=${UBUNTU_SEC_REPO:-$(awk "/^deb .*ubuntu\/? ${UBUNTU_RELEASE}-security main/ {print \$2; exit}" /etc/apt/sources.list)}
## Library Check -------------------------------------------------------------
info_block "Checking for required libraries." 2> /dev/null ||
source $(dirname ${0})/scripts-library.sh ||
source scripts/scripts-library.sh
## Variables -----------------------------------------------------------------
# Extra options to pass to the AIO bootstrap process
export BOOTSTRAP_OPTS=${BOOTSTRAP_OPTS:-''}
## Main ----------------------------------------------------------------------
# Log some data about the instance and the rest of the system
log_instance_info
# Run AIO bootstrap playbook
pushd tests
ansible-playbook -i "localhost ansible-connection=local," \
-e "${BOOTSTRAP_OPTS}" \
bootstrap-aio.yml
popd
# Ensure that the current kernel can support vxlan
if ! modprobe vxlan; then
echo "VXLAN support is required for this to work. And the Kernel module was not found."
echo "This build will not work without it."
exit_fail
fi
info_block "Running AIO Setup"
# Set base DNS to google, ensuring consistent DNS in different environments
if [ ! "$(grep -e '^nameserver 8.8.8.8' -e '^nameserver 8.8.4.4' /etc/resolv.conf)" ];then
echo -e '\n# Adding google name servers\nnameserver 8.8.8.8\nnameserver 8.8.4.4' | tee -a /etc/resolv.conf
fi
# Ensure that the https apt transport is available before doing anything else
apt-get update && apt-get install -y apt-transport-https < /dev/null
# Set the host repositories to only use the same ones, always, for the sake of consistency.
cat > /etc/apt/sources.list <<EOF
# Base repositories
deb ${UBUNTU_REPO} ${UBUNTU_RELEASE} main restricted universe multiverse
# Updates repositories
deb ${UBUNTU_REPO} ${UBUNTU_RELEASE}-updates main restricted universe multiverse
# Backports repositories
deb ${UBUNTU_REPO} ${UBUNTU_RELEASE}-backports main restricted universe multiverse
# Security repositories
deb ${UBUNTU_SEC_REPO} ${UBUNTU_RELEASE}-security main restricted universe multiverse
EOF
# Update the package cache
apt-get update
# Remove known conflicting packages in the base image
apt-get purge -y libmysqlclient18 mysql-common < /dev/null
# Install required packages
apt-get install -y bridge-utils \
build-essential \
curl \
ethtool \
git-core \
ipython \
linux-image-extra-$(uname -r) \
lvm2 \
python2.7 \
python-dev \
tmux \
vim \
vlan \
xfsprogs < /dev/null
# Flush all the iptables rules set by openstack-infra
if [ "${FLUSH_IPTABLES}" == "yes" ]; then
# Flush all the iptables rules set by openstack-infra
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
fi
# Ensure newline at end of file (missing on Rackspace public cloud Trusty image)
if ! cat -E /etc/ssh/sshd_config | tail -1 | grep -q "\$$"; then
echo >> /etc/ssh/sshd_config
fi
# Ensure that sshd permits root login, or ansible won't be able to connect
if grep "^PermitRootLogin" /etc/ssh/sshd_config > /dev/null; then
sed -i 's/^PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config
else
echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config
fi
# Create /opt if it doesn't already exist
if [ ! -d "/opt" ];then
mkdir /opt
fi
# Remove the pip directory if its found
if [ -d "${HOME}/.pip" ];then
rm -rf "${HOME}/.pip"
fi
# Install pip
get_pip
# Ensure we use the HTTPS/HTTP proxy with pip if it is specified
PIP_OPTS=""
if [ -n "$HTTPS_PROXY" ]; then
PIP_OPTS="--proxy $HTTPS_PROXY"
elif [ -n "$HTTP_PROXY" ]; then
PIP_OPTS="--proxy $HTTP_PROXY"
fi
# Install requirements if there are any
if [ -f "requirements.txt" ];then
pip2 install $PIP_OPTS -r requirements.txt || pip install $PIP_OPTS -r requirements.txt
fi
# Configure all disk space
configure_diskspace
# Create /etc/rc.local if it doesn't already exist
if [ ! -f "/etc/rc.local" ];then
touch /etc/rc.local
chmod +x /etc/rc.local
fi
# Make the system key used for bootstrapping self
if [ ! -d /root/.ssh ];then
mkdir -p /root/.ssh
chmod 700 /root/.ssh
fi
ssh_key_create
# Make sure everything is mounted.
mount -a || true
# Build the loopback drive for swap to use
if [ ! "$(swapon -s | grep -v Filename)" ]; then
memory_kb=$(awk '/MemTotal/ {print $2}' /proc/meminfo)
if [ "${memory_kb}" -lt "8388608" ]; then
swap_size="4294967296"
else
swap_size="8589934592"
fi
loopback_create "/opt/swap.img" ${swap_size} thick swap
# Ensure swap will be used on the host
if [ ! $(sysctl vm.swappiness | awk '{print $3}') == "10" ];then
sysctl -w vm.swappiness=10 | tee -a /etc/sysctl.conf
fi
swapon -a
fi
if [ "${DEPLOY_OPENSTACK}" == "yes" ]; then
# Build the loopback drive for cinder to use
CINDER="cinder.img"
if ! vgs cinder-volumes; then
loopback_create "/opt/${CINDER}" 1073741824000 thin rc
CINDER_DEVICE=$(losetup -a | awk -F: "/${CINDER}/ {print \$1}")
pvcreate ${CINDER_DEVICE}
pvscan
# Check for the volume group
if ! vgs cinder-volumes; then
vgcreate cinder-volumes ${CINDER_DEVICE}
fi
# Ensure that the cinder loopback is enabled after reboot
if ! grep ${CINDER} /etc/rc.local && ! vgs cinder-volumes; then
sed -i "\$i losetup \$(losetup -f) /opt/${CINDER}" /etc/rc.local
fi
fi
fi
# Enable swift deployment
if [ "${DEPLOY_SWIFT}" == "yes" ]; then
# build the loopback drives for swift to use
for SWIFT in swift1 swift2 swift3; do
if ! grep "${SWIFT}" /proc/mounts > /dev/null; then
loopback_create "/opt/${SWIFT}.img" 1073741824000 thin none
if ! grep -w "^/opt/${SWIFT}.img" /etc/fstab > /dev/null; then
echo "/opt/${SWIFT}.img /srv/${SWIFT}.img xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
fi
# Format the lo devices
mkfs.xfs -f "/opt/${SWIFT}.img"
mkdir -p "/srv/${SWIFT}.img"
mount "/opt/${SWIFT}.img" "/srv/${SWIFT}.img"
fi
done
fi
# Copy aio network config into place.
if [ ! -d "/etc/network/interfaces.d" ];then
mkdir -p /etc/network/interfaces.d/
fi
# Copy the basic aio network interfaces over
cp -R etc/network/interfaces.d/aio_interfaces.cfg /etc/network/interfaces.d/
# Ensure the network source is in place
if [ ! "$(grep -Rni '^source\ /etc/network/interfaces.d/\*.cfg' /etc/network/interfaces)" ]; then
echo "source /etc/network/interfaces.d/*.cfg" | tee -a /etc/network/interfaces
fi
# Bring up the new interfaces
for i in $(awk '/^iface/ {print $2}' /etc/network/interfaces.d/aio_interfaces.cfg); do
if grep "^$i\:" /proc/net/dev > /dev/null;then
/sbin/ifdown $i || true
fi
/sbin/ifup $i || true
done
# Remove an existing etc directory if already found
if [ -d "/etc/openstack_deploy" ];then
rm -rf "/etc/openstack_deploy"
fi
# Move the *.aio files into place for use within the AIO build.
cp -R etc/openstack_deploy /etc/
for i in $(find /etc/openstack_deploy/ -type f -name '*.aio');do
rename 's/\.aio$//g' $i
done
# Ensure the conf.d directory exists
if [ ! -d "/etc/openstack_deploy/conf.d" ];then
mkdir -p "/etc/openstack_deploy/conf.d"
fi
# Ensure containers are using the same resolvers as the host
RESOLVERS=$(grep nameserver /etc/resolv.conf | awk 'NF { print "\""$0"\""}' | tr '\n' ',' | sed 's/,$//' )
if [ ! "$(grep -Rni '^lxc_cache_resolvers' /etc/openstack_deploy/user_variables.yml)" ]; then
echo "lxc_cache_resolvers: [$RESOLVERS]" | tee -a /etc/openstack_deploy/user_variables.yml
fi
# Add tempest settings for particular use-cases
if [ ${DEPLOY_OPENSTACK} == "no" ]; then
for svc in cinder glance heat horizon neutron nova; do
echo "tempest_service_available_${svc}: False" | tee -a /etc/openstack_deploy/user_variables.yml
done
fi
if [ ${DEPLOY_SWIFT} == "no" ]; then
echo "tempest_service_available_swift: False" | tee -a /etc/openstack_deploy/user_variables.yml
fi
# Generate the passwords
scripts/pw-token-gen.py --file /etc/openstack_deploy/user_secrets.yml
# change the generated passwords for the OpenStack (admin)
sed -i "s/keystone_auth_admin_password:.*/keystone_auth_admin_password: ${ADMIN_PASSWORD}/" /etc/openstack_deploy/user_secrets.yml
sed -i "s/external_lb_vip_address:.*/external_lb_vip_address: ${PUBLIC_ADDRESS}/" /etc/openstack_deploy/openstack_user_config.yml
# Change affinities (number of containers per host) if the appropriate
# environment variables are set.
for container_type in keystone galera rabbit_mq horizon repo; do
var_name="NUM_${container_type}_CONTAINER"
set +u
num=${!var_name}
set -u
[[ -z $num ]] && continue
sed -i "s/${container_type}_container:.*/${container_type}_container: ${num}/" /etc/openstack_deploy/openstack_user_config.yml
done
if [ ${DEPLOY_CEILOMETER} == "yes" ]; then
# Install mongodb on the aio1 host
apt-get install mongodb-server mongodb-clients python-pymongo -y < /dev/null
# Change bind_ip to management ip
sed -i "s/^bind_ip.*/bind_ip = $MONGO_HOST/" /etc/mongodb.conf
# Asserting smallfiles key
sed -i "s/^smallfiles.*/smallfiles = true/" /etc/mongodb.conf
service mongodb restart
# Wait for mongodb to restart
for i in {1..12}; do
mongo --host $MONGO_HOST --eval ' ' && break
sleep 5
done
#Adding the ceilometer database
mongo --host $MONGO_HOST --eval '
db = db.getSiblingDB("ceilometer");
db.addUser({user: "ceilometer",
pwd: "ceilometer",
roles: [ "readWrite", "dbAdmin" ]})'
# change the generated passwords for mongodb access
sed -i "s/ceilometer_container_db_password:.*/ceilometer_container_db_password: ceilometer/" /etc/openstack_deploy/user_secrets.yml
# Change the Ceilometer user variables necessary for deployment
sed -i "s/ceilometer_db_ip:.*/ceilometer_db_ip: ${MONGO_HOST}/" /etc/openstack_deploy/user_variables.yml
# Enable Ceilometer for Swift
if [ ${DEPLOY_SWIFT} == "yes" ]; then
sed -i "s/swift_ceilometer_enabled:.*/swift_ceilometer_enabled: True/" /etc/openstack_deploy/user_variables.yml
fi
# Enable Ceilometer for other OpenStack Services
if [ ${DEPLOY_OPENSTACK} == "yes" ]; then
for svc in cinder glance heat nova; do
sed -i "s/${svc}_ceilometer_enabled:.*/${svc}_ceilometer_enabled: True/" /etc/openstack_deploy/user_variables.yml
done
fi
echo 'tempest_service_available_ceilometer: true' | tee -a /etc/openstack_deploy/user_variables.yml
fi
# Service region set
echo "service_region: ${SERVICE_REGION}" | tee -a /etc/openstack_deploy/user_variables.yml
# Virt type set
echo "nova_virt_type: ${NOVA_VIRT_TYPE}" | tee -a /etc/openstack_deploy/user_variables.yml
# Set network for tempest
echo "tempest_public_subnet_cidr: ${TEMPEST_FLAT_CIDR}" | tee -a /etc/openstack_deploy/user_variables.yml
# Minimize galera cache
echo 'galera_innodb_buffer_pool_size: 512M' | tee -a /etc/openstack_deploy/user_variables.yml
echo 'galera_innodb_log_buffer_size: 32M' | tee -a /etc/openstack_deploy/user_variables.yml
echo 'galera_wsrep_provider_options:
- { option: "gcache.size", value: "32M" }' | tee -a /etc/openstack_deploy/user_variables.yml
# Set the running kernel as the required kernel
echo "required_kernel: $(uname --kernel-release)" | tee -a /etc/openstack_deploy/user_variables.yml
# Set the Ubuntu apt repository used for containers to the same as the host
echo "lxc_container_template_main_apt_repo: ${UBUNTU_REPO}" | tee -a /etc/openstack_deploy/user_variables.yml
echo "lxc_container_template_security_apt_repo: ${UBUNTU_SEC_REPO}" | tee -a /etc/openstack_deploy/user_variables.yml
# Set the running neutron workers to 0/1
echo "neutron_api_workers: 0" | tee -a /etc/openstack_deploy/user_variables.yml
echo "neutron_rpc_workers: 0" | tee -a /etc/openstack_deploy/user_variables.yml
echo "neutron_metadata_workers: 1" | tee -a /etc/openstack_deploy/user_variables.yml
# Add in swift vars if needed
if [ "${DEPLOY_SWIFT}" == "yes" ]; then
# ensure that glance is configured to use swift
sed -i "s/glance_default_store:.*/glance_default_store: swift/" /etc/openstack_deploy/user_variables.yml
echo "cinder_service_backup_program_enabled: True" | tee -a /etc/openstack_deploy/user_variables.yml
echo "tempest_volume_backup_enabled: True" | tee -a /etc/openstack_deploy/user_variables.yml
fi
if [ ! -z "${RABBITMQ_PACKAGE_URL}" ]; then
echo "rabbitmq_package_url: ${RABBITMQ_PACKAGE_URL}" | tee -a /etc/openstack_deploy/user_variables.yml
fi
# Update fatal_deprecations settings
if [ "${CINDER_FATAL_DEPRECATIONS}" == "yes" ]; then
echo "cinder_fatal_deprecations: True" | tee -a /etc/openstack_deploy/user_variables.yml
fi
if [ "${GLANCE_FATAL_DEPRECATIONS}" == "yes" ]; then
echo "glance_fatal_deprecations: True" | tee -a /etc/openstack_deploy/user_variables.yml
fi
if [ "${HEAT_FATAL_DEPRECATIONS}" == "yes" ]; then
echo "heat_fatal_deprecations: True" | tee -a /etc/openstack_deploy/user_variables.yml
fi
if [ "${KEYSTONE_FATAL_DEPRECATIONS}" == "yes" ]; then
echo "keystone_fatal_deprecations: True" | tee -a /etc/openstack_deploy/user_variables.yml
fi
if [ "${NEUTRON_FATAL_DEPRECATIONS}" == "yes" ]; then
echo "neutron_fatal_deprecations: True" | tee -a /etc/openstack_deploy/user_variables.yml
fi
if [ "${NOVA_FATAL_DEPRECATIONS}" == "yes" ]; then
echo "nova_fatal_deprecations: True" | tee -a /etc/openstack_deploy/user_variables.yml
fi
if [ "${TEMPEST_FATAL_DEPRECATIONS}" == "yes" ]; then
echo "tempest_fatal_deprecations: True" | tee -a /etc/openstack_deploy/user_variables.yml
fi
# Log some data about the instance and the rest of the system
log_instance_info
info_block "The system has been prepared for an all-in-one build."

170
scripts/gate-check-commit.sh
View File

@ -17,105 +17,131 @@
set -e -u -x
## Variables -----------------------------------------------------------------
export BOOTSTRAP_ANSIBLE=${BOOTSTRAP_ANSIBLE:-"yes"}
export BOOTSTRAP_AIO=${BOOTSTRAP_AIO:-"yes"}
export RUN_PLAYBOOKS=${RUN_PLAYBOOKS:-"yes"}
export RUN_TEMPEST=${RUN_TEMPEST:-"yes"}
# Ansible options
export ANSIBLE_PARAMETERS=${ANSIBLE_PARAMETERS:-"-v"}
# Deployment options
export DEPLOY_HOST=${DEPLOY_HOST:-"yes"}
export DEPLOY_LB=${DEPLOY_LB:-"yes"}
export DEPLOY_INFRASTRUCTURE=${DEPLOY_INFRASTRUCTURE:-"yes"}
export DEPLOY_LOGGING=${DEPLOY_LOGGING:-"yes"}
export DEPLOY_OPENSTACK=${DEPLOY_OPENSTACK:-"yes"}
export DEPLOY_SWIFT=${DEPLOY_SWIFT:-"yes"}
export DEPLOY_TEMPEST=${DEPLOY_TEMPEST:-"yes"}
# Limit the gate check to only performing one attempt, unless already set
export MAX_RETRIES=${MAX_RETRIES:-"2"}
# tempest and testr options, default is to run tempest in serial
export TESTR_OPTS=${TESTR_OPTS:-''}
# Disable the python output buffering so that jenkins gets the output properly
export PYTHONUNBUFFERED=1
# Extra options to pass to the AIO bootstrap process
export BOOTSTRAP_OPTS=${BOOTSTRAP_OPTS:-''}
## Functions -----------------------------------------------------------------
info_block "Checking for required libraries." 2> /dev/null || source $(dirname ${0})/scripts-library.sh
## Main ----------------------------------------------------------------------
# Disable Ansible color output
sed -i 's/nocolor.*/nocolor = 1/' $(dirname ${0})/../playbooks/ansible.cfg
# Log some data about the instance and the rest of the system
log_instance_info
# Make the /openstack/log directory for openstack-infra gate check log publishing
mkdir -p /openstack/log
# Determine the largest secondary disk device available for repartitioning
DATA_DISK_DEVICE=$(lsblk -brndo NAME,TYPE,RO,SIZE | \
awk '/d[b-z]+ disk 0/{ if ($4>m){m=$4; d=$1}}; END{print d}')
# Implement the log directory link for openstack-infra log publishing
ln -sf /openstack/log $(dirname ${0})/../logs
# Only set the secondary disk device option if there is one
if [ -n "${DATA_DISK_DEVICE}" ]; then
export BOOTSTRAP_OPTS="${BOOTSTRAP_OPTS} bootstrap_host_data_disk_device=${DATA_DISK_DEVICE}"
fi
# Create ansible logging directory and add in a log file entry into ansible.cfg
mkdir -p /openstack/log/ansible-logging
sed -i '/\[defaults\]/a log_path = /openstack/log/ansible-logging/ansible.log' $(dirname ${0})/../playbooks/ansible.cfg
# Bootstrap Ansible
source $(dirname ${0})/bootstrap-ansible.sh
# Log some data about the instance and the rest of the system
log_instance_info
# Flush all the iptables rules set by openstack-infra
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
# Adjust settings based on the Cloud Provider info in OpenStack-CI
if [ -f /etc/nodepool/provider -a -s /etc/nodepool/provider ]; then
source /etc/nodepool/provider
if [[ ${NODEPOOL_PROVIDER} == "rax"* ]]; then
export UBUNTU_REPO="http://mirror.rackspace.com/ubuntu"
export UBUNTU_SEC_REPO="${UBUNTU_REPO}"
elif [[ ${NODEPOOL_PROVIDER} == "hpcloud"* ]]; then
export UBUNTU_REPO="http://${NODEPOOL_AZ}.clouds.archive.ubuntu.com/ubuntu"
export UBUNTU_SEC_REPO="${UBUNTU_REPO}"
# Get the fastest possible Linux mirror depending on the datacenter where the
# tests are running.
case ${NODEPOOL_PROVIDER} in
"rax-dfw"*)
export UBUNTU_REPO="http://dfw.mirror.rackspace.com/ubuntu"
;;
"rax-ord"*)
export UBUNTU_REPO="http://ord.mirror.rackspace.com/ubuntu"
;;
"rax-iad"*)
export UBUNTU_REPO="http://iad.mirror.rackspace.com/ubuntu"
;;
"hpcloud"*)
export UBUNTU_REPO="http://${NODEPOOL_AZ}.clouds.archive.ubuntu.com/ubuntu"
;;
"ovh-gra1"*)
export UBUNTU_REPO="http://ubuntu.mirrors.ovh.net/ubuntu"
;;
"ovh-bhs1"*)
export UBUNTU_REPO="http://ubuntu.bhs.mirrors.ovh.net/ubuntu"
;;
"bluebox-sjc1"*)
export UBUNTU_REPO="http://ord.mirror.rackspace.com/ubuntu"
;;
"internap-nyj01"*)
export UBUNTU_REPO="http://iad.mirror.rackspace.com/ubuntu"
;;
esac
if [ -n "${UBUNTU_REPO:-}" ]; then
export BOOTSTRAP_OPTS="${BOOTSTRAP_OPTS} bootstrap_host_ubuntu_repo=${UBUNTU_REPO}"
export BOOTSTRAP_OPTS="${BOOTSTRAP_OPTS} bootstrap_host_ubuntu_security_repo=${UBUNTU_REPO}"
fi
# Update the libvirt cpu map with a gate64 cpu model. This enables nova
# live migration for 64bit guest OSes on heterogenous cloud "hardware".
export BOOTSTRAP_OPTS="${BOOTSTRAP_OPTS} bootstrap_host_libvirt_config=yes"
fi
# Enable detailed task profiling
sed -i '/\[defaults\]/a callback_plugins = plugins/callbacks' $(dirname ${0})/../playbooks/ansible.cfg
# Bootstrap an AIO
pushd $(dirname ${0})/../tests
sed -i '/\[defaults\]/a nocolor = 1/' ansible.cfg
ansible-playbook -i "localhost ansible-connection=local," \
-e "${BOOTSTRAP_OPTS}" \
${ANSIBLE_PARAMETERS} \
bootstrap-aio.yml
popd
# Bootstrap an AIO setup if required
if [ "${BOOTSTRAP_AIO}" == "yes" ]; then
source $(dirname ${0})/bootstrap-aio.sh
fi
# Implement the log directory link for openstack-infra log publishing
mkdir -p /openstack/log
ln -sf /openstack/log $(dirname ${0})/../logs
# Bootstrap ansible if required
if [ "${BOOTSTRAP_ANSIBLE}" == "yes" ]; then
source $(dirname ${0})/bootstrap-ansible.sh
fi
pushd $(dirname ${0})/../playbooks
# Disable Ansible color output
sed -i 's/nocolor.*/nocolor = 1/' ansible.cfg
# Enable debug logging for all services to make failure debugging easier
echo "debug: True" | tee -a /etc/openstack_deploy/user_variables.yml
# Create ansible logging directory and add in a log file entry into ansible.cfg
mkdir -p /openstack/log/ansible-logging
sed -i '/\[defaults\]/a log_path = /openstack/log/ansible-logging/ansible.log' ansible.cfg
# NOTE: hpcloud-b4's eth0 uses 10.0.3.0/24, which overlaps with the
# lxc_net_address default
# TODO: We'll need to implement a mechanism to determine valid lxc_net_address
# value which will not overlap with an IP already assigned to the host.
echo "lxc_net_address: 10.255.255.1" | tee -a /etc/openstack_deploy/user_variables.yml
echo "lxc_net_netmask: 255.255.255.0" | tee -a /etc/openstack_deploy/user_variables.yml
echo "lxc_net_dhcp_range: 10.255.255.2,10.255.255.253" | tee -a /etc/openstack_deploy/user_variables.yml
# Enable detailed task profiling
sed -i '/\[defaults\]/a callback_plugins = plugins/callbacks' ansible.cfg
popd
# Limit the number of processes used by Keystone
# The defaults cause tempest failures in OpenStack CI due to resource constraints
echo "keystone_wsgi_processes: 4" | tee -a /etc/openstack_deploy/user_variables.yml
# Log some data about the instance and the rest of the system
log_instance_info
# Disable the python output buffering so that jenkins gets the output properly
export PYTHONUNBUFFERED=1
# Execute the Playbooks
bash $(dirname ${0})/run-playbooks.sh
# Run the ansible playbooks if required
if [ "${RUN_PLAYBOOKS}" == "yes" ]; then
# Set-up our tiny awk script.
strip_debug="
!/(^[ 0-9|:.-]+<[0-9.]|localhost+>)|Extracting/ {
gsub(/{.*/, \"\");
gsub(/\\n.*/, \"\");
gsub(/\=\>.*/, \"\");
print
}
"
set -o pipefail
bash $(dirname ${0})/run-playbooks.sh | awk "${strip_debug}"
set +o pipefail
fi
# Log some data about the instance and the rest of the system
log_instance_info
# Run the tempest tests if required
if [ "${RUN_TEMPEST}" == "yes" ]; then
source $(dirname ${0})/run-tempest.sh
fi
# Run the tempest tests
source $(dirname ${0})/run-tempest.sh
# Log some data about the instance and the rest of the system
log_instance_info
exit_success

25
scripts/os-detection.py Executable file
View File

@ -0,0 +1,25 @@
#!/usr/bin/env python
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Determines the operating system of a host and returns bash variables."""
import platform
template = """
HOST_DISTRO="{0}"
HOST_VERSION="{1}"
HOST_CODENAME="{2}"
""".format(*platform.linux_distribution())
print(template)

33
scripts/run-aio-build.sh
View File

@ -12,6 +12,13 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# ----------------------------------------------------------------------------
#
# This script configures an all-in-one (AIO) deployment. For more details, see
# the quick start documentation for openstack-ansible:
#
# http://docs.openstack.org/developer/openstack-ansible/developer-docs/quickstart-aio.html#running-an-aio-build-in-one-step
## Shell Opts ----------------------------------------------------------------
@ -25,31 +32,32 @@ export WORKING_FOLDER=${WORKING_FOLDER:-"/opt/openstack-ansible"}
## Main ----------------------------------------------------------------------
# set verbosity
# Set verbosity
set -x
# install git so that we can fetch the repo
# note: the redirect of stdin to /dev/null is necessary for when this script is
# Install git so that we can fetch various git repositories.
# Note: the redirect of stdin to /dev/null is necessary for when this script is
# run as part of a curl-pipe-shell. otherwise apt-get will consume the rest of
# this file as if it was its own stdin (despite using -y to skip interaction).
apt-get update && apt-get install -y git < /dev/null
# fetch the repo
# Fetch the openstack-ansible repository.
git clone -b ${REPO_BRANCH} ${REPO_URL} ${WORKING_FOLDER}
# change into the expected root directory
# Change into the expected root directory.
cd ${WORKING_FOLDER}
# first, bootstrap the AIO host
source scripts/bootstrap-aio.sh
# next, bootstrap Ansible
# Start by bootstrapping Ansible from source.
source scripts/bootstrap-ansible.sh
# finally, run all the playbooks
# Next, bootstrap the AIO host.
source scripts/bootstrap-aio.sh
# Finally, run all of the playbooks.
bash scripts/run-playbooks.sh
# put a motd in place to help the user know what stuff is accessible once the build is complete
# Add a MOTD to explain to the deployer what is accessible once the build
# is complete.
cat > /etc/update-motd.d/20-openstack<< EOF
#!/usr/bin/env bash
echo ""
@ -60,7 +68,8 @@ echo ""
EOF
chmod +x /etc/update-motd.d/20-openstack
# put an motd in place to help the user know how to restart galera after reboot
# Add a MOTD to explain to the deployer how to restart galera properly after a
# reboot.
cat > /etc/update-motd.d/21-galera<< EOF
#!/usr/bin/env bash
echo ""

4
scripts/run-playbooks.sh
View File

@ -25,9 +25,9 @@ DEPLOY_LOGGING=${DEPLOY_LOGGING:-"yes"}
DEPLOY_OPENSTACK=${DEPLOY_OPENSTACK:-"yes"}
DEPLOY_SWIFT=${DEPLOY_SWIFT:-"yes"}
DEPLOY_CEILOMETER=${DEPLOY_CEILOMETER:-"yes"}
DEPLOY_TEMPEST=${DEPLOY_TEMPEST:-"no"}
DEPLOY_TEMPEST=${DEPLOY_TEMPEST:-"yes"}
COMMAND_LOGS=${COMMAND_LOGS:-"/openstack/log/ansible_cmd_logs/"}
ADD_NEUTRON_AGENT_CHECKSUM_RULE=${BOOTSTRAP_AIO:-"no"}
ADD_NEUTRON_AGENT_CHECKSUM_RULE=${ADD_NEUTRON_AGENT_CHECKSUM_RULE:-"yes"}
## Functions -----------------------------------------------------------------

106
scripts/scripts-library.sh
View File

@ -18,7 +18,6 @@
## Vars ----------------------------------------------------------------------
LINE='----------------------------------------------------------------------'
MAX_RETRIES=${MAX_RETRIES:-5}
MIN_LXC_VG_SIZE_GB=${MIN_LXC_VG_SIZE_GB:-250}
REPORT_DATA=${REPORT_DATA:-""}
ANSIBLE_PARAMETERS=${ANSIBLE_PARAMETERS:-""}
STARTTIME="${STARTTIME:-$(date +%s)}"
@ -43,12 +42,12 @@ fi
function successerator {
set +e
# Get the time that the method was started.
OP_START_TIME="$(date +%s)"
OP_START_TIME=$(date +%s)
RETRY=0
# Set the initial return value to failure.
false
while [ $? -ne 0 -a ${RETRY} -lt ${MAX_RETRIES} ];do
RETRY=$((${RETRY}+1))
((RETRY++))
if [ ${RETRY} -gt 1 ];then
$@ -vvvv
else
@ -61,7 +60,7 @@ function successerator {
exit_fail
fi
# Print the time that the method completed.
OP_TOTAL_SECONDS="$(( $(date +%s) - $OP_START_TIME ))"
OP_TOTAL_SECONDS="$(( $(date +%s) - OP_START_TIME ))"
REPORT_OUTPUT="${OP_TOTAL_SECONDS} seconds"
REPORT_DATA+="- Operation: [ $@ ]\t${REPORT_OUTPUT}\tNumber of Attempts [ ${RETRY} ]\n"
echo -e "Run Time = ${REPORT_OUTPUT}"
@ -74,54 +73,6 @@ function install_bits {
successerator openstack-ansible ${ANSIBLE_PARAMETERS} --forks ${FORKS} $@
}
function configure_diskspace {
# If there are any block devices available other than the one
# used for the root disk, repurpose it for our needs.
MIN_LXC_VG_SIZE_B=$((${MIN_LXC_VG_SIZE_GB} * 1024 * 1024 * 1024))
# only do this if the lxc vg doesn't already exist
if ! vgs lxc > /dev/null 2>&1; then
blk_devices=$(lsblk -nrdo NAME,TYPE,RO | awk '/d[b-z]+ disk [^1]/ {print $1}')
for blk_dev in ${blk_devices}; do
# dismount any mount points on the device
mount_points=$(awk "/^\/dev\/${blk_dev}[0-9]* / {print \$2}" /proc/mounts)
for mount_point in ${mount_points}; do
umount ${mount_point}
sed -i ":${mount_point}:d" /etc/fstab
done
# add a vg for lxc
blk_dev_size_b=$(lsblk -nrdbo NAME,TYPE,SIZE | awk "/^${blk_dev} disk/ {print \$3}")
if [ "${blk_dev_size_b}" -gt "${MIN_LXC_VG_SIZE_B}" ]; then
if ! vgs lxc > /dev/null 2>&1; then
parted --script /dev/${blk_dev} mklabel gpt
parted --align optimal --script /dev/${blk_dev} mkpart lxc 0% 80%
part_num=$(parted /dev/${blk_dev} print --machine | awk -F':' '/lxc/ {print $1}')
pvcreate -ff -y /dev/${blk_dev}${part_num}
vgcreate lxc /dev/${blk_dev}${part_num}
fi
# add a vg for cinder volumes, but only if it doesn't already exist
if ! vgs cinder-volumes > /dev/null 2>&1; then
parted --align optimal --script /dev/${blk_dev} mkpart cinder 80% 100%
part_num=$(parted /dev/${blk_dev} print --machine | awk -F':' '/cinder/ {print $1}')
pvcreate -ff -y /dev/${blk_dev}${part_num}
vgcreate cinder-volumes /dev/${blk_dev}${part_num}
fi
else
if ! grep '/var/lib/lxc' /proc/mounts 2>&1; then
parted --script /dev/${blk_dev} mklabel gpt
parted --script /dev/${blk_dev} mkpart lxc ext4 0% 100%
part_num=$(parted /dev/${blk_dev} print --machine | awk -F':' '/lxc/ {print $1}')
# Format, Create, and Mount it all up.
mkfs.ext4 /dev/${blk_dev}${part_num}
mkdir -p /var/lib/lxc
mount /dev/${blk_dev}${part_num} /var/lib/lxc
fi
fi
done
fi
}
function ssh_key_create {
# Ensure that the ssh key exists and is an authorized_key
key_path="${HOME}/.ssh"
@ -145,48 +96,10 @@ function ssh_key_create {
fi
}
function loopback_create {
LOOP_FILENAME=${1}
LOOP_FILESIZE=${2}
LOOP_FILE_TYPE=${3} # thin, thick
LOOP_MOUNT_METHOD=${4} # swap, rc, none
if [ ! -f "${LOOP_FILENAME}" ]; then
if [ "${LOOP_FILE_TYPE}" = "thin" ]; then
truncate -s ${LOOP_FILESIZE} ${LOOP_FILENAME}
elif [ "${LOOP_FILE_TYPE}" = "thick" ]; then
fallocate -l ${LOOP_FILESIZE} ${LOOP_FILENAME} &> /dev/null || \
dd if=/dev/zero of=${LOOP_FILENAME} bs=1M count=$(( ${LOOP_FILESIZE} / 1024 / 1024 ))
else
exit_fail "No valid option ${LOOP_FILE_TYPE} found."
fi
fi
if [ "${LOOP_MOUNT_METHOD}" = "rc" ]; then
if ! losetup -a | grep -q "(${LOOP_FILENAME})$"; then
LOOP_DEVICE=$(losetup -f)
losetup ${LOOP_DEVICE} ${LOOP_FILENAME}
fi
if ! grep -q ${LOOP_FILENAME} /etc/rc.local; then
sed -i "\$i losetup \$(losetup -f) ${LOOP_FILENAME}" /etc/rc.local
fi
fi
if [ "${LOOP_MOUNT_METHOD}" = "swap" ]; then
if ! swapon -s | grep -q ${LOOP_FILENAME}; then
mkswap ${LOOP_FILENAME}
swapon -a
fi
if ! grep -q "^${LOOP_FILENAME} " /etc/fstab; then
echo "${LOOP_FILENAME} none swap loop 0 0" >> /etc/fstab
fi
fi
}
function exit_state {
set +x
TOTALSECONDS="$(( $(date +%s) - $STARTTIME ))"
info_block "Run Time = ${TOTALSECONDS} seconds || $(($TOTALSECONDS / 60)) minutes"
TOTALSECONDS="$(( $(date +%s) - STARTTIME ))"
info_block "Run Time = ${TOTALSECONDS} seconds || $((TOTALSECONDS / 60)) minutes"
if [ "${1}" == 0 ];then
info_block "Status: Success"
else
@ -203,6 +116,7 @@ function exit_success {
function exit_fail {
set +x
log_instance_info
cat ${INFO_FILENAME}
info_block "Error Info - $@"
exit_state 1
}
@ -224,7 +138,8 @@ function log_instance_info {
if [ ! -d "/openstack/log/instance-info" ];then
mkdir -p "/openstack/log/instance-info"
fi
get_instance_info &> /openstack/log/instance-info/host_info_$(date +%s).log
export INFO_FILENAME="/openstack/log/instance-info/host_info_$(date +%s).log"
get_instance_info &> ${INFO_FILENAME}
set -x
}
@ -333,6 +248,11 @@ function get_pip {
trap "exit_fail ${LINENO} $? 'Received STOP Signal'" SIGHUP SIGINT SIGTERM
trap "exit_fail ${LINENO} $?" ERR
## Determine OS --------------------------------------------------------------
# Determine the operating system of the base host
# Adds the $HOST_DISTRO, $HOST_VERSION, and $HOST_CODENAME bash variables.
eval "$(python $(dirname ${BASH_SOURCE})/os-detection.py)"
echo "Detected ${HOST_DISTRO} ${HOST_VERSION} (codename: ${HOST_CODENAME})"
## Pre-flight check ----------------------------------------------------------
# Make sure only root can run our script

6
tests/ansible.cfg Normal file
View File

@ -0,0 +1,6 @@
[defaults]
action_plugins = ../playbooks/plugins/actions
callback_plugins = ../playbooks/plugins/callbacks
library = ../playbooks/library
roles_path = ../playbooks/roles:/etc/ansible/roles
host_key_checking = False

21
tests/bootstrap-aio.yml Normal file
View File

@ -0,0 +1,21 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- hosts: localhost
user: root
roles:
- sshd
- pip_install
- bootstrap-host

104
tests/roles/bootstrap-host/defaults/main.yml Normal file
View File

@ -0,0 +1,104 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Boolean option to implement OpenStack-Ansible configuration for an AIO
bootstrap_host_aio_config: yes
# Boolean option to implement OpenStack-CI LibVirt Configuration Changes
bootstrap_host_libvirt_config: no
## Swap memory
# If there is no swap memory present, the bootstrap will create a loopback disk
# for the purpose of having swap memory available. Swap is required for some of
# the services deployed and is useful for AIO's built with less than 16GB memory.
# By default the swap size is set to 8GB unless the host memory is less than 8GB,
# in which case it is set to 4GB.
bootstrap_host_swap_size: "{% if ansible_memory_mb['real']['total'] < 8*1024 %}4{% else %}8{% endif %}"
## Loopback volumes
# Sparse loopback disks are used for Cinder, Swift and Nova (instance storage).
# The size of the loopback volumes can be customized here (in gigabytes).
#
# Boolean option to deploy the loopback disk for Cinder
bootstrap_host_loopback_cinder: yes
# Size of the Cinder loopback disk in gigabytes (GB).
bootstrap_host_loopback_cinder_size: 1024
#
# Boolean option to deploy the loopback disk for Swift
bootstrap_host_loopback_swift: yes
# Size of the Swift loopback disk in gigabytes (GB).
bootstrap_host_loopback_swift_size: 1024
#
# Boolean option to deploy the loopback disk for Nova
bootstrap_host_loopback_nova: yes
# Size of the Nova loopback disk in gigabytes (GB).
bootstrap_host_loopback_nova_size: 1024
## Bridge configuration
# The AIO bootstrap configures bridges for use with the AIO deployment.
# By default, these bridges are configured to be independent of any physical
# interfaces, and they have their 'bridge_ports' set to 'none'. However,
# deployers can add a physical interface to 'bridge_ports' to connect the
# bridge to a real physical interface.
#
# A setting of 'none' keeps the bridges as independent from physical
# interfaces (the default).
#
# Setting the value to 'eth1' would mean that the bridge is directly connected
# to the eth1 device.
#
# See https://wiki.debian.org/BridgeNetworkConnections for more details.
bootstrap_host_bridge_mgmt_ports: none
bootstrap_host_bridge_vxlan_ports: none
bootstrap_host_bridge_storage_ports: none
## Extra storage
# An AIO may optionally be built using a second storage device. If a
# secondary disk device to use is not specified, then the AIO will be
# built on any existing disk partitions.
#
# WARNING: The data on a secondary storage device specified here will
# be destroyed and repartitioned.
#
# Specify the secondary disk device to use.
#bootstrap_host_data_disk_device: vdb
#
# Boolean value to force the repartitioning of the secondary device.
bootstrap_host_data_disk_device_force: no
#
# If the storage capacity on this device is greater than or equal to this
# size (in GB), the bootstrap process will use it.
bootstrap_host_data_disk_min_size: 80
### MongoDB Settings
# MongoDB is installed on the host in the AIO for Ceilometer to use.
#
# Boolean value to deploy and configure the MongoDB service on the host.
bootstrap_host_mongodb_service: yes
#
# Specify the IP address of a MongoDB Host.
bootstrap_host_mongodb_address: 172.29.236.100
### Optional Settings ###
# Set the apt repository URL's configured for the host and containers.
# By default the configuration will be derived from the host.
#bootstrap_host_ubuntu_repo: http://archive.ubuntu.com/ubuntu/
#bootstrap_host_ubuntu_security_repo: http://archive.ubuntu.com/ubuntu/
# Specify the public IP address for the host.
# By default the address will be set to the ipv4 address of the
# host's network interface that has the default route on it.
#bootstrap_host_public_address: 0.0.0.0

56
tests/roles/bootstrap-host/tasks/check-requirements.yml Normal file
View File

@ -0,0 +1,56 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Check for a supported Operating System
assert:
that: ansible_distribution | lower == 'ubuntu'
tags:
- check-operating-system
- name: Identify the space available in /
shell: |
df -BG / | awk '/^[^Filesystem]/ {print $4}' | sed 's/G//'
when:
- bootstrap_host_data_disk_device is not defined
changed_when: false
register: root_space_available
tags:
- check-disk-size
- name: Fail if there is not enough space available in /
assert:
that: |
root_space_available.stdout | int >= (bootstrap_host_data_disk_min_size * 0.75) | int
when:
- bootstrap_host_data_disk_device is not defined
tags:
- check-disk-size
- name: Fail if there is not enough disk space available (disk specified)
assert:
that: |
(ansible_devices[bootstrap_host_data_disk_device]['size'] | replace(' GB','')) | int
>= bootstrap_host_data_disk_min_size | int
when:
- bootstrap_host_data_disk_device is defined
tags:
- check-disk-size
- name: Ensure that the kernel has VXLAN support
modprobe:
name: vxlan
state: present
tags:
- check-vxlan

107
tests/roles/bootstrap-host/tasks/install-apt.yml Normal file
View File

@ -0,0 +1,107 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Ensure that apt supports HTTPS package sources
apt:
name: apt-transport-https
state: present
tags:
- apt-install-prerequisites
- name: Determine the existing Ubuntu repo configuration
shell: 'awk "/^deb .*ubuntu\/? {{ ansible_distribution_release }} main/ {print \$2; exit}" /etc/apt/sources.list'
register: ubuntu_repo
when:
- bootstrap_host_ubuntu_repo is not defined
changed_when: false
tags:
- find-apt-repo
- name: Determine the existing Ubuntu Security repo configuration
shell: 'awk "/^deb .*ubuntu\/? {{ ansible_distribution_release }}-security main/ {print \$2; exit}" /etc/apt/sources.list'
register: ubuntu_security_repo
when:
- bootstrap_host_ubuntu_security_repo is not defined
changed_when: false
tags:
- find-apt-security-repo
- name: Set apt repo facts based on discovered information
set_fact:
bootstrap_host_ubuntu_repo: "{{ ubuntu_repo.stdout }}"
bootstrap_host_ubuntu_security_repo: "{{ ubuntu_security_repo.stdout }}"
when:
- bootstrap_host_ubuntu_repo is not defined
- bootstrap_host_ubuntu_security_repo is not defined
- ubuntu_repo is defined
- ubuntu_security_repo is defined
- name: Configure apt's sources.list (Ubuntu only)
template:
src: apt-sources.list.j2
dest: /etc/apt/sources.list
backup: yes
when:
- ansible_distribution == 'Ubuntu'
- bootstrap_host_ubuntu_repo is defined
- bootstrap_host_ubuntu_security_repo is defined
register: apt_sources_configure
- name: Update apt-cache
apt:
update_cache: yes
when:
- apt_sources_configure is defined
- apt_sources_configure | changed
tags:
- apt-cache-update
- name: Remove known problem packages
apt:
name: "{{ item }}"
state: absent
with_items:
- "{{ packages_remove }}"
tags:
- remove-packages
- name: Install required packages
apt:
name: "{{ item }}"
state: present
with_items:
- "{{ packages_install }}"
tags:
- install-packages
- name: Install MongoDB packages
apt:
name: "{{ item }}"
state: present
with_items:
- "{{ packages_mongodb }}"
when: bootstrap_host_mongodb_service | bool
tags:
- install-mongodb
- name: Install LibVirt packages
apt:
name: "{{ item }}"
state: present
with_items:
- "{{ packages_libvirt }}"
when: bootstrap_host_libvirt_config | bool
tags:
- install-libvirt

116
tests/roles/bootstrap-host/tasks/main.yml Normal file
View File

@ -0,0 +1,116 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Before we do anything, check the minimum requirements
- include: check-requirements.yml
tags:
- check-requirements
# We will look for the most specific variable files first and eventually
# end up with the least-specific files.
- name: Gather variables for each operating system
include_vars: "{{ item }}"
with_first_found:
- "{{ ansible_distribution | lower }}-{{ ansible_distribution_version | lower }}.yml"
- "{{ ansible_distribution | lower }}.yml"
- "{{ ansible_os_family | lower }}.yml"
tags:
- always
- name: Create the required directories
file:
path: "{{ item }}"
state: directory
with_items:
- "/openstack"
tags:
- create-directories
# Configure apt in a known way to reduce the chance of unexpected failures
- include: install-apt.yml
when:
- ansible_pkg_mgr == 'apt'
tags:
- install-apt
# Prepare the data disk, if one is provided
- include: prepare_data_disk.yml
when:
- bootstrap_host_data_disk_device is defined
tags:
- prepare-data-disk
# Prepare the swap space loopback disk
# This is only necessary if there isn't swap already
- include: prepare_loopback_swap.yml
when:
- ansible_swaptotal_mb < 1
tags:
- prepare-loopback-swap
# Prepare the Cinder LVM VG loopback disk
# This is only necessary if bootstrap_host_loopback_cinder is set to yes
- include: prepare_loopback_cinder.yml
when:
- bootstrap_host_loopback_cinder | bool
tags:
- prepare-loopback-cinder
# Prepare the Nova instance storage loopback disk
- include: prepare_loopback_nova.yml
when:
- bootstrap_host_loopback_nova | bool
tags:
- prepare-loopback-nova
# Prepare the Swift data storage loopback disks
- include: prepare_loopback_swift.yml
when:
- bootstrap_host_loopback_swift | bool
tags:
- prepare-loopback-swift
# Prepare the network interfaces
- include: prepare_networking.yml
tags:
- prepare-networking
# Ensure that there are both private and public ssh keys for root
- include: prepare_ssh_keys.yml
tags:
- prepare-ssh-keys
# Put the OpenStack-Ansible configuration for an All-In-One on the host
- include: prepare_aio_config.yml
when: bootstrap_host_aio_config | bool
tags:
- prepare-aio-config
# Prepare the MongoDB Service for Ceilometer
- include: prepare_mongodb_service.yml
when: bootstrap_host_mongodb_service | bool
tags:
- prepare-mongodb-service
# Prepare the MongoDB Users for Ceilometer
- include: prepare_mongodb_users.yml
tags:
- prepare-mongodb-users
# Prepare the LibVirt Service for Nova
- include: prepare_libvirt_service.yml
when: bootstrap_host_libvirt_config | bool
tags:
- prepare-libvirt-service

143
tests/roles/bootstrap-host/tasks/prepare_aio_config.yml Normal file
View File

@ -0,0 +1,143 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Create the required deployment directories
file:
path: "{{ item }}"
state: directory
with_items:
- "/etc/openstack_deploy/"
- "/etc/openstack_deploy/conf.d"
- "/etc/openstack_deploy/env.d"
tags:
- create-directories
- name: Deploy environment (env.d) configuration
config_template:
src: "../etc/openstack_deploy/env.d/{{ item.name }}"
dest: "/etc/openstack_deploy/env.d/{{ item.name }}"
config_overrides: "{{ item.override }}"
config_type: "yaml"
with_items:
- name: ceilometer.yml
override: "{{ ceilometer_env_overrides | default({}) }}"
- name: cinder.yml
override: "{{ cinder_env_overrides | default({}) }}"
- name: galera.yml
override: "{{ galera_env_overrides | default({}) }}"
- name: glance.yml
override: "{{ glance_env_overrides | default({}) }}"
- name: heat.yml
override: "{{ heat_env_overrides | default({}) }}"
- name: horizon.yml
override: "{{ horizon_env_overrides | default({}) }}"
- name: infra.yml
override: "{{ infra_env_overrides | default({}) }}"
- name: keystone.yml
override: "{{ keystone_env_overrides | default({}) }}"
- name: memcache.yml
override: "{{ memcache_env_overrides | default({}) }}"
- name: neutron.yml
override: "{{ neutron_env_overrides | default({}) }}"
- name: nova.yml
override: "{{ nova_env_overrides | default({}) }}"
- name: os-infra.yml
override: "{{ os_infra_env_overrides | default({}) }}"
- name: pkg_repo.yml
override: "{{ pkg_repo_env_overrides | default({}) }}"
- name: rabbitmq.yml
override: "{{ rabbitmq_env_overrides | default({}) }}"
- name: rsyslog.yml
override: "{{ rsyslog_env_overrides | default({}) }}"
- name: shared-infra.yml
override: "{{ shared_infra_env_overrides | default({}) }}"
- name: swift-remote.yml
override: "{{ swift_remote_env_overrides | default({}) }}"
- name: swift.yml
override: "{{ swift_env_overrides | default({}) }}"
- name: utility.yml
override: "{{ utility_env_overrides | default({}) }}"
tags:
- deploy-envd
- name: Deploy user conf.d configuration
config_template:
src: "../etc/openstack_deploy/conf.d/{{ item.name }}"
dest: "/etc/openstack_deploy/conf.d/{{ item.name | regex_replace('.aio$', '') }}"
config_overrides: "{{ item.override }}"
config_type: "yaml"
with_items:
- name: ceilometer.yml.aio
override: "{{ ceilometer_conf_overrides | default({}) }}"
- name: swift.yml.aio
override: "{{ swift_conf_overrides | default({}) }}"
tags:
- deploy-confd
- name: Deploy openstack_user_config
config_template:
src: "../etc/openstack_deploy/openstack_user_config.yml.aio"
dest: "/etc/openstack_deploy/openstack_user_config.yml"
config_overrides: "{{ openstack_user_config_overrides | default({}) }}"
config_type: "yaml"
tags:
- deploy-openstack-user-config
- name: Deploy openstack_environment
config_template:
src: "../etc/openstack_deploy/openstack_environment.yml"
dest: "/etc/openstack_deploy/openstack_environment.yml"
config_overrides: "{{ openstack_environment_overrides | default({}) }}"
config_type: "yaml"
tags:
- deploy-openstack-environment
- name: Deploy user_secrets file
config_template:
src: "../etc/openstack_deploy/user_secrets.yml"
dest: "/etc/openstack_deploy/user_secrets.yml"
config_overrides: "{{ user_secrets_overrides | default({}) }}"
config_type: "yaml"
tags:
- deploy-user-secrets
- name: Generate any missing values in user_secrets
shell: ../scripts/pw-token-gen.py --file /etc/openstack_deploy/user_secrets.yml
tags:
- generate_secrets
- name: Get the DNS servers in use on the host
shell: grep nameserver /etc/resolv.conf
register: nameservers
when:
- lxc_cache_resolvers is not defined
tags:
- get-nameservers
- name: Set lxc_cache_resolvers fact
set_fact:
lxc_cache_resolvers: "[\"{{ nameservers.stdout_lines | join('\",\"') }}\"]"
when:
- nameservers is defined
tags:
- set-fact-lxc_cache_resolvers
- name: Set the user_variables
config_template:
src: user_variables.aio.yml.j2
dest: /etc/openstack_deploy/user_variables.yml
config_overrides: "{{ user_variables_overrides | default({}) }}"
config_type: yaml

73
tests/roles/bootstrap-host/tasks/prepare_data_disk.yml Normal file
View File

@ -0,0 +1,73 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Only execute the disk partitioning process if a partition labeled
# 'openstack-data{1,2}' is not present and that partition is not
# formatted as ext4. This is an attempt to achieve idempotency just
# in case these tasks are executed multiple times.
- name: Determine whether partitions labeled openstack-data{1,2} are present
shell: |
parted --script -l -m | egrep -q ':ext4:openstack-data[12]:;$'
register: data_disk_partitions
changed_when: false
ignore_errors: yes
tags:
- check-data-disk-partitions
- name: Dismount and remove fstab entries for anything on the data disk device
mount:
name: "{{ item.mount }}"
src: "{{ item.device }}"
fstype: ext4
state: absent
when:
- data_disk_partitions.rc == 1 or bootstrap_host_data_disk_device_force | bool
- item.device | search(bootstrap_host_data_disk_device)
with_items:
- "{{ ansible_mounts }}"
- name: Partition the whole data disk for our usage
shell: "{{ item }}"
when: data_disk_partitions.rc == 1 or bootstrap_host_data_disk_device_force | bool
with_items:
- "parted --script /dev/{{ bootstrap_host_data_disk_device }} mklabel gpt"
- "parted --align optimal --script /dev/{{ bootstrap_host_data_disk_device }} mkpart openstack-data1 ext4 0% 40%"
- "parted --align optimal --script /dev/{{ bootstrap_host_data_disk_device }} mkpart openstack-data2 ext4 40% 100%"
tags:
- create-data-disk-partitions
- name: Format the partitions
filesystem:
fstype: ext4
dev: "{{ item }}"
when: data_disk_partitions.rc == 1 or bootstrap_host_data_disk_device_force | bool
with_items:
- "/dev/{{ bootstrap_host_data_disk_device }}1"
- "/dev/{{ bootstrap_host_data_disk_device }}2"
tags:
- format-data-partitions
- name: Create the mount points, fstab entries and mount the file systems
mount:
name: "{{ item.mount_point }}"
src: "{{ item.device }}"
fstype: ext4
state: mounted
with_items:
- { mount_point: /openstack, device: "/dev/{{ bootstrap_host_data_disk_device }}1"}
- { mount_point: /var/lib/lxc, device: "/dev/{{ bootstrap_host_data_disk_device }}2"}
tags:
- mount-data-partitions

53
tests/roles/bootstrap-host/tasks/prepare_libvirt_service.yml Normal file
View File

@ -0,0 +1,53 @@
---
# Copyright 2016, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Download LibVirt CPU map configuration script
get_url:
url: "http://git.openstack.org/cgit/openstack-dev/devstack/plain/tools/cpu_map_update.py"
dest: /openstack/cpu_map_update.py
validate_certs: yes
mode: 755
register: libvirt_cpu_map_download
tags:
- libvirt-cpu-map-download
# Update the libvirt cpu map with a gate64 cpu model. This enables nova
# live migration for 64bit guest OSes on heterogenous cloud "hardware".
- name: Execute LibVirt CPU map configuration script
shell: /openstack/cpu_map_update.py /usr/share/libvirt/cpu_map.xml
when: libvirt_cpu_map_download | changed
tags:
- libvirt-cpu-map-updated
# libvirt detects various settings on startup, as we potentially changed
# the system configuration (modules, filesystems), we need to restart
# libvirt to detect those changes. Use a stop start as otherwise the new
# cpu_map is not loaded properly on some systems (Ubuntu).
- name: Stop libvirt-bin
service:
name: "{{ servicename_libvirt }}"
state: stopped
when: libvirt_cpu_map_download | changed
tags:
- libvirt-service-stop
- name: Start libvirt-bin
service:
name: "{{ servicename_libvirt }}"
state: started
when: libvirt_cpu_map_download | changed
tags:
- libvirt-service-start

72
tests/roles/bootstrap-host/tasks/prepare_loopback_cinder.yml Normal file
View File

@ -0,0 +1,72 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Create sparse Cinder file
shell: "truncate -s {{ bootstrap_host_loopback_cinder_size }}G /openstack/cinder.img"
args:
creates: /openstack/cinder.img
register: cinder_create
tags:
- cinder-file-create
- name: Get a loopback device for cinder file
shell: losetup -f
when: cinder_create | changed
register: cinder_losetup
tags:
- cinder-device-get
- name: Create the loopback device
shell: "losetup {{ cinder_losetup.stdout }} /openstack/cinder.img"
when: cinder_create | changed
tags:
- cinder-device-create
- name: Ensure that rc.local exists
file:
path: /etc/rc.local
state: touch
mode: "u+x"
tags:
- cinder-rc-file
# As the cinder loopback is an LVM VG, it needs to be mounted differently
# to the other loopback files. It requires the use of rc.local to attach
# the loopback device on boot so that the VG becomes available immediately
# after the boot process completes.
- name: Create loopback devices at boot time
lineinfile:
dest: /etc/rc.local
line: "losetup $(losetup -f) /openstack/cinder.img"
tags:
- cinder-rc-config
- name: Make LVM physical volume on the cinder device
shell: "{{ item }}"
when: cinder_create | changed
with_items:
- "pvcreate {{ cinder_losetup.stdout }}"
- "pvscan"
tags:
- cinder-lvm-pv
- name: Add cinder-volumes volume group
lvg:
vg: cinder-volumes
pvs: "{{ cinder_losetup.stdout }}"
when: cinder_create | changed
tags:
- cinder-lvm-vg

39
tests/roles/bootstrap-host/tasks/prepare_loopback_nova.yml Normal file
View File

@ -0,0 +1,39 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Create sparse Nova file
shell: "truncate -s {{ bootstrap_host_loopback_nova_size }}G /openstack/nova.img"
args:
creates: /openstack/nova.img
register: nova_create
tags:
- nova-file-create
- name: Format the Nova file
filesystem:
fstype: ext4
dev: /openstack/nova.img
when: nova_create | changed
tags:
- nova-format-file
- name: Create the mount points, fstab entries and mount the file systems
mount:
name: /var/lib/nova/instances
src: /openstack/nova.img
fstype: ext4
state: mounted
tags:
- nova-file-mount

61
tests/roles/bootstrap-host/tasks/prepare_loopback_swap.yml Normal file
View File

@ -0,0 +1,61 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Create swap file
shell: "fallocate -l {{ bootstrap_host_swap_size }}G /openstack/swap.img"
args:
creates: /openstack/swap.img
register: swap_create
tags:
- swap-file-create
- name: Set swap file permissions to 0600
file:
path: /openstack/swap.img
mode: 0600
tags:
- swap-permissions
- name: Format the swap file
shell: mkswap /openstack/swap.img
when: swap_create | changed
tags:
- swap-format
- name: Ensure that the swap file entry is in /etc/fstab
mount:
name: none
src: /openstack/swap.img
fstype: swap
opts: sw
passno: 0
dump: 0
state: present
tags:
- swap-fstab
- name: Bring swap file online
shell: swapon /openstack/swap.img
tags:
- swap-online
- name: Set system swappiness
sysctl:
name: vm.swappiness
value: 10
state: present
tags:
- swap-sysctl

54
tests/roles/bootstrap-host/tasks/prepare_loopback_swift.yml Normal file
View File

@ -0,0 +1,54 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Create sparse Swift files
shell: "truncate -s {{ bootstrap_host_loopback_cinder_size }}G /openstack/{{ item }}.img"
args:
creates: "/openstack/{{ item }}.img"
with_items:
- 'swift1'
- 'swift2'
- 'swift3'
register: swift_create
tags:
- swift-file-create
- name: Format the Swift files
filesystem:
fstype: xfs
dev: "/openstack/{{ item }}.img"
when: swift_create | changed
with_items:
- 'swift1'
- 'swift2'
- 'swift3'
tags:
- swift-format-file
- name: Create the Swift mount points, fstab entries and mount the file systems
mount:
name: "/srv/{{ item }}.img"
src: "/openstack/{{ item }}.img"
fstype: xfs
opts: 'loop,noatime,nodiratime,nobarrier,logbufs=8'
passno: 0
dump: 0
state: mounted
with_items:
- 'swift1'
- 'swift2'
- 'swift3'
tags:
- swift-file-mount

61
tests/roles/bootstrap-host/tasks/prepare_mongodb_service.yml Normal file
View File

@ -0,0 +1,61 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Configure the MongoDB bind address
lineinfile:
dest: /etc/mongodb.conf
regexp: "^(#)?bind_ip"
line: "bind_ip = {{ bootstrap_host_mongodb_address }}"
register: mongodb_bind
tags:
- mongodb-conf-bind_ip
- name: Enable the MongoDB smallfiles option
lineinfile:
dest: /etc/mongodb.conf
regexp: "^(#)?smallfiles"
line: "smallfiles = true"
register: mongodb_smallfiles
tags:
- mongodb-conf-smallfiles
- name: Restart mongodb
service:
name: mongodb
state: restarted
when:
- mongodb_bind | changed or mongodb_smallfiles | changed
register: mongodb_restart
tags:
- mongodb-restart
- name: Wait for mongodb to come back online after the restart
wait_for:
host: "{{ bootstrap_host_mongodb_address }}"
port: 27017
delay: 5
timeout: 30
when:
- mongodb_restart is defined
- mongodb_restart | changed
tags:
- mongodb-wait
- name: Test mongodb connectivity
command: "mongo --host {{ bootstrap_host_mongodb_address }} --eval ' '"
changed_when: False
tags:
- mongodb-test

30
tests/roles/bootstrap-host/tasks/prepare_mongodb_users.yml Normal file
View File

@ -0,0 +1,30 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Read user_secrets vars
include_vars: /etc/openstack_deploy/user_secrets.yml
tags:
- mongodb-secrets
- name: Add ceilometer database user
mongodb_user:
login_host: "{{ bootstrap_host_mongodb_address }}"
database: ceilometer
name: ceilometer
password: "{{ ceilometer_container_db_password }}"
roles: 'readWrite,dbAdmin'
state: present
tags:
- mongodb-create-user-ceilometer

58
tests/roles/bootstrap-host/tasks/prepare_networking.yml Normal file
View File

@ -0,0 +1,58 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Ensure that /etc/network/interfaces.d/ exists
file:
path: /etc/network/interfaces.d/
state: directory
tags:
- networking-dir-create
- name: Copy network configuration
template:
src: osa_interfaces.cfg.j2
dest: /etc/network/interfaces.d/osa_interfaces.cfg
register: osa_interfaces
tags:
- networking-interfaces-file
- name: Ensure our interfaces.d configuration files are loaded automatically
lineinfile:
dest: /etc/network/interfaces
line: "source /etc/network/interfaces.d/*.cfg"
tags:
- networking-interfaces-load
- name: Shut down the network interfaces
command: "ifdown {{ item }}"
when: osa_interfaces | changed
with_items:
- br-mgmt
- br-storage
- br-vlan
- br-vxlan
tags:
- networking-interfaces-stop
- name: Start the network interfaces
command: "ifup {{ item }}"
when: osa_interfaces | changed
with_items:
- br-mgmt
- br-storage
- br-vlan
- br-vxlan
tags:
- networking-interfaces-start

67
tests/roles/bootstrap-host/tasks/prepare_ssh_keys.yml Normal file
View File

@ -0,0 +1,67 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Ensure root has a .ssh directory
file:
path: /root/.ssh
state: directory
owner: root
group: root
mode: 0700
tags:
- ssh-key-dir
- name: Check for existing ssh private key file
stat:
path: /root/.ssh/id_rsa
register: ssh_key_private
tags:
- ssh-key-check
- name: Check for existing ssh public key file
stat:
path: /root/.ssh/id_rsa.pub
register: ssh_key_public
tags:
- ssh-key-check
- name: Remove an existing private/public ssh keys if one is missing
file:
path: "/root/.ssh/{{ item }}"
state: absent
when: not ssh_key_public.stat.exists or not ssh_key_private.stat.exists
with_items:
- 'id_rsa'
- 'id_rsa.pub'
tags:
- ssh-key-clean
- name: Create ssh key pair for root
user:
name: root
generate_ssh_key: yes
ssh_key_bits: 2048
ssh_key_file: /root/.ssh/id_rsa
tags:
- ssh-key-generate
- name: Ensure root's new public ssh key is in authorized_keys
authorized_key:
user: root
key: "{{ lookup('file', '/root/.ssh/id_rsa.pub') }}"
manage_dir: no
tags:
- ssh-key-authorized

10
tests/roles/bootstrap-host/templates/apt-sources.list.j2 Normal file
View File

@ -0,0 +1,10 @@
# {{ ansible_managed }}
# Base repositories
deb {{ bootstrap_host_ubuntu_repo }} {{ ansible_distribution_release }} main restricted universe multiverse
# Updates repositories
deb {{ bootstrap_host_ubuntu_repo }} {{ ansible_distribution_release }}-updates main restricted universe multiverse
# Backports repositories
deb {{ bootstrap_host_ubuntu_repo }} {{ ansible_distribution_release }}-backports main restricted universe multiverse
# Security repositories
deb {{ bootstrap_host_ubuntu_security_repo }} {{ ansible_distribution_release }}-security main restricted universe multiverse

68
tests/roles/bootstrap-host/templates/osa_interfaces.cfg.j2 Normal file
View File

@ -0,0 +1,68 @@
## The default networking requires several bridges. These bridges were named to be informative
## however they can be named what ever you like and is adaptable to any network infrastructure
## environment. This file serves as an example of how to setup basic networking and was ONLY
## built for the purpose of being an example and used expressly in the building of an ALL IN
## ONE development environment.
auto br-mgmt
iface br-mgmt inet static
bridge_stp off
bridge_waitport 0
bridge_fd 0
# Notice the bridge port is the vlan tagged interface
bridge_ports {{ bootstrap_host_bridge_mgmt_ports }}
address 172.29.236.100
netmask 255.255.252.0
offload-sg off
auto br-vxlan
iface br-vxlan inet static
bridge_stp off
bridge_waitport 0
bridge_fd 0
bridge_ports {{ bootstrap_host_bridge_vxlan_ports }}
address 172.29.240.100
netmask 255.255.252.0
offload-sg off
# To ensure ssh checksum is correct
up /sbin/iptables -A POSTROUTING -t mangle -p tcp --dport 22 -j CHECKSUM --checksum-fill
down /sbin/iptables -D POSTROUTING -t mangle -p tcp --dport 22 -j CHECKSUM --checksum-fill
# To provide internet connectivity to instances
up /sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
down /sbin/iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
auto br-storage
iface br-storage inet static
bridge_stp off
bridge_waitport 0
bridge_fd 0
bridge_ports {{ bootstrap_host_bridge_storage_ports }}
address 172.29.244.100
netmask 255.255.252.0
offload-sg off
auto br-vlan
iface br-vlan inet static
bridge_stp off
bridge_waitport 0
bridge_fd 0
address 172.29.248.100
netmask 255.255.252.0
offload-sg off
# Create veth pair, don't bomb if already exists
pre-up ip link add br-vlan-veth type veth peer name eth12 || true
# Set both ends UP
pre-up ip link set br-vlan-veth up
pre-up ip link set eth12 up
# Delete veth pair on DOWN
post-down ip link del br-vlan-veth || true
bridge_ports br-vlan-veth
# Add an additional address to br-vlan
iface br-vlan inet static
# Flat network default gateway
# -- This needs to exist somewhere for network reachability
# -- from the router namespace for floating IP paths.
# -- Putting this here is primarily for tempest to work.
address 172.29.248.1
netmask 255.255.252.0

89
tests/roles/bootstrap-host/templates/user_variables.aio.yml.j2 Normal file
View File

@ -0,0 +1,89 @@
---
# Copyright 2014, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
## General options
debug: True
## Ceilometer Options
ceilometer_db_type: mongodb
ceilometer_db_ip: {{ bootstrap_host_mongodb_address }}
ceilometer_db_port: 27017
cinder_ceilometer_enabled: True
glance_ceilometer_enabled: True
heat_ceilometer_enabled: True
neutron_ceilometer_enabled: True
nova_ceilometer_enabled: True
swift_ceilometer_enabled: True
## Nova Options
nova_virt_type: qemu
## Glance Options
glance_default_store: swift
## SSL Settings
ssl_protocol: "ALL -SSLv2 -SSLv3"
# Cipher suite string from https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl_cipher_suite: "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS"
## Cinder settings
cinder_service_backup_program_enabled: True
## Tempest settings
tempest_service_available_ceilometer: True
tempest_public_subnet_cidr: 172.29.248.0/22
tempest_volume_backup_enabled: True
## Galera settings
galera_innodb_buffer_pool_size: 512M
galera_innodb_log_buffer_size: 32M
galera_wsrep_provider_options:
- { option: "gcache.size", value: "32M" }
## Set workers for all services to optimise memory usage
ceilometer_api_workers: 2
ceilometer_collector_workers: 2
ceilometer_notification_workers: 2
cinder_osapi_volume_workers: 2
glance_api_workers: 2
glance_registry_workers: 2
heat_api_workers: 2
heat_engine_workers: 2
horizon_wsgi_processes: 2
horizon_wsgi_threads: 2
keystone_wsgi_processes: 2
neutron_api_workers: 2
neutron_metadata_workers: 1
neutron_rpc_workers: 0
nova_conductor_workers: 2
nova_metadata_workers: 2
nova_osapi_compute_workers: 2
swift_account_server_workers: 2
swift_container_server_workers: 2
swift_object_server_workers: 2
swift_proxy_server_workers: 2
# NOTE: hpcloud-b4's eth0 uses 10.0.3.0/24, which overlaps with the
# lxc_net_address default
# TODO: We'll need to implement a mechanism to determine valid lxc_net_address
# value which will not overlap with an IP already assigned to the host.
lxc_net_address: 10.255.255.1
lxc_net_netmask: 255.255.255.0
lxc_net_dhcp_range: 10.255.255.2,10.255.255.253
## LXC Container Settings
lxc_cache_resolvers: {{ lxc_cache_resolvers }}
lxc_container_template_main_apt_repo: {{ bootstrap_host_ubuntu_repo }}
lxc_container_template_security_apt_repo: {{ bootstrap_host_ubuntu_security_repo }}

44
tests/roles/bootstrap-host/vars/ubuntu.yml Normal file
View File

@ -0,0 +1,44 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
packages_install:
- bridge-utils
- build-essential
- curl
- ethtool
- git-core
- ipython
- linux-image-extra-{{ ansible_kernel }}
- lvm2
- python2.7
- python-dev
- tmux
- vim
- vlan
- xfsprogs
packages_remove:
- libmysqlclient18
- mysql-common
packages_mongodb:
- mongodb-clients
- mongodb-server
- python-pymongo
packages_libvirt:
- libvirt-bin
servicename_libvirt: libvirt-bin