Fix multinode mode for devstack

Extend the devstack job so that it can support both single and multinode
cases.  Multinode mode require extra settings in devstack configuration,
some of which as subnode specific, some controller specific.

Also keep a simple devstack-multinode job defined for now so we can run
a multinode job in devstack gate, until the full tempest multinode job
is ready to match the old
gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv.

Fixing multinode also requires sharing the CA configuration between
controller and peers, overlay network configuration for communication
between virtual machines and running discover_hosts for nova after the
subnode has been setup.

The extra orchestration required for multinode is encoded in a
dedicated role to allow for jobs in other repos to re-use it.

Change-Id: I2dcbd9bdb401860820e655d97aa3c4775af2827f
This commit is contained in:
Andrea Frittoli 2018-02-17 22:21:26 +00:00
parent d9c1275c5d
commit 5e714d3308
9 changed files with 179 additions and 21 deletions

View File

@ -16,19 +16,24 @@
- name: compute1
label: ubuntu-xenial
groups:
# Node where tests are executed and test results collected
- name: tempest
nodes:
- controller
# Nodes running the compute service
- name: compute
nodes:
- controller
- compute1
# Nodes that are not the controller
- name: subnode
nodes:
- compute1
# Switch node for multinode networking setup
- name: switch
nodes:
- controller
# Peer nodes for multinode networking setup
- name: peers
nodes:
- compute1
@ -45,7 +50,7 @@
all single Devstack jobs, single or multinode.
Variables are defined in job.vars, which is what is then used by single
node jobs and by multi node jobs for the controller, as well as in
job.group-vars.peers, which is what is used by multi node jobs for peer
job.group-vars.peers, which is what is used by multi node jobs for subnode
nodes (everything but the controller).
required-projects:
- openstack-dev/devstack
@ -76,7 +81,6 @@
# from the location below for all the CI jobs.
ETCD_DOWNLOAD_URL: http://tarballs.openstack.org/etcd/
devstack_services:
# Ignore base set of services setup by test-matrix.
# Ignore any default set by devstack. Emit a "disable_all_services".
base: false
zuul_copy_output:
@ -119,7 +123,7 @@
localrc: True
stackenv: True
group-vars:
peers:
subnode:
devstack_localrc:
DATABASE_PASSWORD: secretdatabase
RABBIT_PASSWORD: secretrabbit
@ -156,7 +160,9 @@
name: devstack
parent: devstack-base
description: |
Single or multi node devstack job for integration gate.
Base devstack job for integration gate.
This base job can be used for single node and multinode devstack jobs.
nodeset: openstack-single-node
required-projects:
- openstack/cinder
@ -178,6 +184,15 @@
NOVA_VNC_ENABLED: true
VNCSERVER_LISTEN: 0.0.0.0
VNCSERVER_PROXYCLIENT_ADDRESS: "{{ hostvars[inventory_hostname]['nodepool']['private_ipv4'] }}"
# Multinode specific settings
SERVICE_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
HOST_IP: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
PUBLIC_BRIDGE_MTU: "{{ external_bridge_mtu }}"
devstack_localconf:
post-config:
$NEUTRON_CONF:
DEFAULT:
global_physnet_mtu: "{{ external_bridge_mtu }}"
devstack_services:
# Core services enabled for this branch.
# This list replaces the test-matrix.
@ -254,16 +269,25 @@
# Test matrix emits ceilometer but ceilomenter is not installed in the
# integrated gate, so specifying the services has not effect.
# ceilometer-*: false
devstack_localrc:
# Multinode specific settings
HOST_IP: "{{ hostvars[inventory_hostname]['nodepool']['private_ipv4'] }}"
SERVICE_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
PUBLIC_BRIDGE_MTU: "{{ external_bridge_mtu }}"
# Subnode specific settings
DATABASE_TYPE: mysql
GLANCE_HOSTPORT: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}:9292"
Q_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
RABBIT_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
DATABASE_HOST: "{{ hostvars['controller']['nodepool']['private_ipv4'] }}"
- job:
name: devstack-multinode
parent: devstack-base
description: Base devstack multinode job
parent: devstack
nodeset: openstack-two-node
# NOTE(andreaf) The multinode job is useful to see the setup of different
# services on different nodes, however the subnode configuration is not
# ready yet. Until then this job should stay non-voting.
voting: false
description: |
Simple multinode test to verify multinode functionality on devstack side.
This is not meant to be used as a parent job.
- job:
name: devstack-tox-base

View File

@ -1,3 +1,3 @@
- hosts: all
roles:
- run-devstack
- orchestrate-devstack

View File

@ -1,15 +1,25 @@
- hosts: controller
roles:
- role: test-matrix
test_matrix_role: primary
- hosts: subnode
roles:
- role: test-matrix
test_matrix_role: subnode
- hosts: all
pre_tasks:
- name: Gather minimum local MTU
set_fact:
local_mtu: >
{% set mtus = [] -%}
{% for interface in ansible_interfaces -%}
{% set interface_variable = 'ansible_' + interface -%}
{% if interface_variable in hostvars[inventory_hostname] -%}
{% set _ = mtus.append(hostvars[inventory_hostname][interface_variable]['mtu']|int) -%}
{% endif -%}
{% endfor -%}
{{- mtus|min -}}
- name: Calculate external_bridge_mtu
# 50 bytes is overhead for vxlan (which is greater than GRE
# allowing us to use either overlay option with this MTU.
# TODO(andreaf) This should work, but it may have to be reconcilied with
# the MTU setting used by the multinode setup roles in multinode pre.yaml
set_fact:
external_bridge_mtu: "{{ local_mtu | int - 50 }}"
roles:
- test-matrix
- configure-swap
- setup-stack-user
- setup-tempest-user

View File

@ -0,0 +1,24 @@
Orchestrate a devstack
Runs devstack in a multinode scenario, with one controller node
and a group of subnodes.
The reason for this role is so that jobs in other repository may
run devstack in their plays with no need for re-implementing the
orchestration logic.
The "run-devstack" role is available to run devstack with no
orchestration.
This role sets up the controller and CA first, it then pushes CA
data to sub-nodes and run devstack there. The only requirement for
this role is for the controller inventory_hostname to be "controller"
and for all sub-nodes to be defined in a group called "subnode".
**Role Variables**
.. zuul:rolevar:: devstack_base_dir
:default: /opt/stack
The devstack base directory.

View File

@ -0,0 +1 @@
devstack_base_dir: /opt/stack

View File

@ -0,0 +1,38 @@
- name: Run devstack on the controller
include_role:
name: run-devstack
when: inventory_hostname == 'controller'
- name: Setup devstack on sub-nodes
block:
- name: Sync CA data to subnodes (when any)
# Only do this if the tls-proxy service is defined and enabled
include_role:
name: sync-devstack-data
when: devstack_services['tls-proxy']|default(false)
- name: Run devstack on the sub-nodes
include_role:
name: run-devstack
when: inventory_hostname in groups['subnode']
- name: Discover hosts
# Discovers compute nodes (subnodes) and maps them to cells. Only run
# on the controller node.
# NOTE(mriedem): We want to remove this if/when nova supports
# auto-registration of computes with cells, but that's not happening in
# Ocata.
# NOTE(andreaf) This is taken (NOTE included) from the discover_hosts
# function in devstack gate. Since this is now in devstack, which is
# branched, we know that the discover_hosts tool exists.
become: true
become_user: stack
shell: ./tools/discover_hosts.sh
args:
chdir: "{{ devstack_base_dir }}/devstack"
when: inventory_hostname == 'controller'
when:
- '"controller" in hostvars'
- '"subnode" in groups'

View File

@ -0,0 +1,12 @@
Sync devstack data for multinode configurations
Sync any data files which include certificates to be used if TLS is enabled.
This role must be executed on the controller and it pushes data to all
subnodes.
**Role Variables**
.. zuul:rolevar:: devstack_base_dir
:default: /opt/stack
The devstack base directory.

View File

@ -0,0 +1 @@
devstack_base_dir: /opt/stack

View File

@ -0,0 +1,48 @@
- name: Ensure the data folder exists
become: true
file:
path: "{{ devstack_base_dir }}/data"
state: directory
owner: stack
group: stack
mode: 0755
when: 'inventory_hostname in groups["subnode"]|default([])'
- name: Ensure the CA folder exists
become: true
file:
path: "{{ devstack_base_dir }}/data/CA"
state: directory
owner: stack
group: stack
mode: 0755
when: 'inventory_hostname in groups["subnode"]|default([])'
- name: Pull the CA certificate and folder
become: true
synchronize:
src: "{{ item }}"
dest: "{{ zuul.executor.work_root }}/{{ item | basename }}"
mode: pull
with_items:
- "{{ devstack_base_dir }}/data/ca-bundle.pem"
- "{{ devstack_base_dir }}/data/CA"
when: inventory_hostname == 'controller'
- name: Push the CA certificate
become: true
become_user: stack
synchronize:
src: "{{ zuul.executor.work_root }}/ca-bundle.pem"
dest: "{{ devstack_base_dir }}/data/ca-bundle.pem"
mode: push
when: 'inventory_hostname in groups["subnode"]|default([])'
- name: Push the CA folder
become: true
become_user: stack
synchronize:
src: "{{ zuul.executor.work_root }}/CA/"
dest: "{{ devstack_base_dir }}/data/"
mode: push
when: 'inventory_hostname in groups["subnode"]|default([])'