Decompose overcloud inventory file into top level, components, and services

This allows a deployer to customise their inventory at various levels, by
providing a custom inventory template for one or more of the sections of the
inventory.

* Top level groups define the roles of hosts, e.g. controller or compute.
* Components define groups of services, e.g. nova or ironic.
* Services define single containers, e.g. nova-compute or ironic-api.
This commit is contained in:
Mark Goddard 2017-08-29 11:35:21 +00:00
parent 7b02fe89d8
commit 5cf205502d
12 changed files with 694 additions and 252 deletions

View File

@ -153,6 +153,50 @@ kolla_build_blocks: {}
# remove. The value should be a list.
kolla_build_customizations: {}
###############################################################################
# Kolla-ansible inventory configuration.
# Full custom seed inventory contents.
kolla_seed_inventory_custom:
# Custom overcloud inventory containing a mapping from top level groups to
# hosts.
kolla_overcloud_inventory_custom_top_level:
# Custom overcloud inventory containing a mapping from components to top level
# groups.
kolla_overcloud_inventory_custom_components:
# Custom overcloud inventory containing a mapping from services to components.
kolla_overcloud_inventory_custom_services:
# Full custom overcloud inventory contents. By default this will be the
# concatenation of the top level, component, and service inventories.
kolla_overcloud_inventory_custom:
# Dict mapping from kolla-ansible groups to kayobe groups and variables. Each
# item is a dict with the following items:
# * groups: A list of kayobe ansible groups to map to this kolla-ansible group.
# * vars: A dict mapping variable names to values for hosts in this
# kolla-ansible group.
kolla_overcloud_inventory_top_level_group_map:
control:
groups:
- controllers
network:
groups:
- network
# List of names of top level kolla-ansible groups. Any of these groups which
# have no hosts mapped to them will be provided with an empty group definition.
kolla_overcloud_inventory_kolla_top_level_groups:
- "control"
- "network"
- "compute"
- "monitoring"
- "storage"
- "deployment"
###############################################################################
# Kolla-ansible configuration.

View File

@ -24,6 +24,41 @@ kolla_config_path:
# Directory where Kolla custom configuration files will be installed.
kolla_node_custom_config_path:
###############################################################################
# Kolla-ansible inventory configuration.
# Full custom seed inventory contents.
kolla_seed_inventory_custom:
# Custom overcloud inventory containing a mapping from top level groups to
# hosts.
kolla_overcloud_inventory_custom_top_level:
# Custom overcloud inventory containing a mapping from components to top level
# groups.
kolla_overcloud_inventory_custom_components:
# Custom overcloud inventory containing a mapping from services to components.
kolla_overcloud_inventory_custom_services:
# Full custom overcloud inventory contents. By default this will be the
# concatenation of the top level, component, and service inventories.
kolla_overcloud_inventory_custom:
# Dict mapping from kolla-ansible groups to kayobe groups and variables. Each
# item is a dict with the following items:
# * groups: A list of kayobe ansible groups to map to this kolla-ansible group.
# * vars: A dict mapping variable names to values for hosts in this
# kolla-ansible group.
kolla_overcloud_inventory_top_level_group_map: {}
# List of names of top level kolla-ansible groups. Any of these groups which
# have no hosts mapped to them will be provided with an empty group definition.
kolla_overcloud_inventory_kolla_top_level_groups: []
###############################################################################
# Kolla-ansible global configuration options.
# Valid options are [ centos, fedora, oraclelinux, ubuntu ]
kolla_base_distro:

View File

@ -12,15 +12,26 @@
- "{{ kolla_config_path }}/inventory"
- "{{ kolla_node_custom_config_path }}"
- name: Ensure the Kolla configuration files exist
- name: Ensure the Kolla global configuration file exists
template:
src: "{{ item.src }}"
dest: "{{ kolla_config_path }}/{{ item.dest }}"
src: "globals.yml.j2"
dest: "{{ kolla_config_path }}/globals.yml"
mode: 0640
# NOTE: We're not looping over the two inventory files to avoid having the file
# content displayed in the ansible-playbook output.
- name: Ensure the Kolla seed inventory file exists
copy:
content: "{{ kolla_seed_inventory }}"
dest: "{{ kolla_config_path }}/inventory/seed"
mode: 0640
- name: Ensure the Kolla overcloud inventory file exists
copy:
content: "{{ kolla_overcloud_inventory }}"
dest: "{{ kolla_config_path }}/inventory/overcloud"
mode: 0640
with_items:
- { src: seed.j2, dest: inventory/seed }
- { src: overcloud.j2, dest: inventory/overcloud }
- { src: globals.yml.j2, dest: globals.yml }
- name: Ensure the Kolla passwords file exists
kolla_passwords:

View File

@ -0,0 +1,201 @@
# This inventory section provides a mapping of the components to top level
# groups.
#
# Top level groups define the roles of hosts, e.g. controller or compute.
# Components define groups of services, e.g. nova or ironic.
# Services define single containers, e.g. nova-compute or ironic-api.
[baremetal:children]
control
network
compute
storage
monitoring
# You can explicitly specify which hosts run each project by updating the
# groups in the sections below. Common services are grouped together.
[chrony-server:children]
control
[chrony:children]
control
network
compute
storage
monitoring
[collectd:children]
compute
[grafana:children]
monitoring
[etcd:children]
control
[influxdb:children]
monitoring
[karbor:children]
control
[kibana:children]
control
[telegraf:children]
compute
control
monitoring
network
storage
[elasticsearch:children]
control
[haproxy:children]
network
[hyperv]
#hyperv_host
[hyperv:vars]
#ansible_user=user
#ansible_password=password
#ansible_port=5986
#ansible_connection=winrm
#ansible_winrm_server_cert_validation=ignore
[mariadb:children]
control
[rabbitmq:children]
control
[outward-rabbitmq:children]
control
[mongodb:children]
control
[keystone:children]
control
[glance:children]
control
[nova:children]
control
[neutron:children]
network
[openvswitch:children]
network
compute
manila-share
[cinder:children]
control
[cloudkitty:children]
control
[freezer:children]
control
[memcached:children]
control
[horizon:children]
control
[swift:children]
control
[barbican:children]
control
[heat:children]
control
[murano:children]
control
[solum:children]
control
[ironic:children]
control
[ceph:children]
control
[magnum:children]
control
[sahara:children]
control
[mistral:children]
control
[manila:children]
control
[ceilometer:children]
control
[aodh:children]
control
[congress:children]
control
[panko:children]
control
[gnocchi:children]
control
[tacker:children]
control
[trove:children]
control
# Tempest
[tempest:children]
control
[senlin:children]
control
[vmtp:children]
control
[watcher:children]
control
[rally:children]
control
[searchlight:children]
control
[octavia:children]
control
[designate:children]
control
[placement:children]
control
[bifrost:children]
deployment
[zun:children]
control
[skydive:children]
monitoring

View File

@ -1,243 +1,8 @@
# {{ ansible_managed }}
# Overcloud inventory for Kolla. Adapted from multinode inventory in Kolla
# repository.
[controllers]
# These hostnames must be resolvable from your deployment host
{% for controller in groups['controllers'] %}
{% set controller_hv=hostvars[controller] %}
{{ controller }}{% if "ansible_host" in controller_hv %} ansible_host={{ controller_hv["ansible_host"] }}{% endif %}
{% endfor %}
[controllers:vars]
ansible_user=kolla
ansible_become=true
# These initial groups are the only groups required to be modified. The
# additional groups are for more control of the environment.
[control:children]
controllers
# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network:children]
controllers
[compute:children]
[monitoring]
# These hostnames must be resolvable from your deployment host
{% for monitoring_host in groups['monitoring'] %}
{% set monitoring_hv=hostvars[monitoring_host] %}
{{ monitoring_host }}{% if "ansible_host" in monitoring_hv %} ansible_host={{ monitoring_hv["ansible_host"] }}{% endif %}
{% endfor %}
[monitoring:vars]
ansible_user=kolla
ansible_become=true
[storage:children]
controllers
[deployment]
[baremetal:children]
control
network
compute
storage
monitoring
# You can explicitly specify which hosts run each project by updating the
# groups in the sections below. Common services are grouped together.
[chrony-server:children]
control
[chrony:children]
control
network
compute
storage
monitoring
[collectd:children]
compute
[grafana:children]
monitoring
[etcd:children]
control
[influxdb:children]
monitoring
[karbor:children]
control
[kibana:children]
control
[telegraf:children]
compute
control
monitoring
network
storage
[elasticsearch:children]
control
[haproxy:children]
network
[hyperv]
#hyperv_host
[hyperv:vars]
#ansible_user=user
#ansible_password=password
#ansible_port=5986
#ansible_connection=winrm
#ansible_winrm_server_cert_validation=ignore
[mariadb:children]
control
[rabbitmq:children]
control
[outward-rabbitmq:children]
control
[mongodb:children]
control
[keystone:children]
control
[glance:children]
control
[nova:children]
control
[neutron:children]
network
[openvswitch:children]
network
compute
manila-share
[cinder:children]
control
[cloudkitty:children]
control
[freezer:children]
control
[memcached:children]
control
[horizon:children]
control
[swift:children]
control
[barbican:children]
control
[heat:children]
control
[murano:children]
control
[solum:children]
control
[ironic:children]
control
[ceph:children]
control
[magnum:children]
control
[sahara:children]
control
[mistral:children]
control
[manila:children]
control
[ceilometer:children]
control
[aodh:children]
control
[congress:children]
control
[panko:children]
control
[gnocchi:children]
control
[tacker:children]
control
[trove:children]
control
# Tempest
[tempest:children]
control
[senlin:children]
control
[vmtp:children]
control
[watcher:children]
control
[rally:children]
control
[searchlight:children]
control
[octavia:children]
control
[designate:children]
control
[placement:children]
control
[bifrost:children]
deployment
[zun:children]
control
[skydive:children]
monitoring
# This inventory section provides a mapping of services to components.
#
# Top level groups define the roles of hosts, e.g. controller or compute.
# Components define groups of services, e.g. nova or ironic.
# Services define single containers, e.g. nova-compute or ironic-api.
# Additional control implemented here. These groups allow you to control which
# services run on which hosts at a per-service level.
@ -413,8 +178,16 @@ ironic
[ironic-conductor:children]
ironic
[ironic-inspector:children]
ironic
#[ironic-inspector:children]
#ironic
[ironic-inspector]
# FIXME: Ideally we wouldn't reference controllers in here directly, but only
# one inspector service should exist, and groups can't be indexed in an
# inventory (e.g. ironic[0]).
{% if groups['controllers'] | length > 0 %}
{{ groups['controllers'][0] }}
{% endif %}
[ironic-pxe:children]
ironic

View File

@ -0,0 +1,63 @@
# This inventory section provides a mapping of the top level groups to hosts.
#
# Top level groups define the roles of hosts, e.g. controller or compute.
# Components define groups of services, e.g. nova or ironic.
# Services define single containers, e.g. nova-compute or ironic-api.
{% set top_level_groups = kolla_overcloud_inventory_top_level_group_map.values() |
selectattr('groups', 'defined') |
map(attribute='groups') |
sum(start=[]) |
unique |
list %}
{% for group in top_level_groups %}
# Top level {{ group }} group.
[{{ group }}]
# These hostnames must be resolvable from your deployment host
{% for host in groups[group] %}
{% set host_hv=hostvars[host] %}
{{ host }}{% if "ansible_host" in host_hv %} ansible_host={{ host_hv["ansible_host"] }}{% endif %}
{% endfor %}
{% endfor %}
[overcloud:children]
{% for group in top_level_groups %}
{{ group }}
{% endfor %}
[overcloud:vars]
ansible_user=kolla
ansible_become=true
{% for kolla_group, kolla_group_config in kolla_overcloud_inventory_top_level_group_map.items() %}
{% if 'groups' in kolla_group_config %}
{% set renamed_groups = kolla_group_config.groups | difference([kolla_group]) | list %}
{% if renamed_groups | length > 0 %}
# Mapping from kolla-ansible group {{ kolla_group }} to top level kayobe
# groups.
[{{ kolla_group }}:children]
{% for group in kolla_group_config.groups %}
{{ group }}
{% endfor %}
{% endif %}
{% endif %}
{% if 'vars' in kolla_group_config %}
# Mapping from kolla-ansible group {{ kolla_group }} to top level kayobe
# variables.
[{{ kolla_group }}:vars]
{% for var_name, var_value in kayobe_group_config.vars.items() %}
{{ var_name }}={{ var_value }}
{% endfor %}
{% endif %}
{% endfor %}
{% for group in kolla_overcloud_inventory_kolla_top_level_groups %}
{% if group not in kolla_overcloud_inventory_top_level_group_map %}
# Empty group definition for {{ group }}.
[{{ group }}]
{% endif %}
{% endfor %}

View File

@ -1,5 +1,3 @@
# {{ ansible_managed }}
# Simple inventory for bootstrapping Kolla seed node.
[seed]
{% for seed in groups['seed'] %}

View File

@ -8,6 +8,58 @@ kolla_ansible_module: "{% if kolla_ansible_is_standalone | bool %}kolla-ansible{
# Path to Kolla Ansible installation directory.
kolla_ansible_install_dir: "{{ kolla_venv }}/share/{{ kolla_ansible_module }}"
###############################################################################
# Inventory configuration.
# Full default seed inventory contents.
kolla_seed_inventory_default: |
# This file is managed by Ansible. Do not edit.
{{ lookup('template', "seed.j2") }}
# Full seed inventory contents.
kolla_seed_inventory: "{{ kolla_seed_inventory_custom or kolla_seed_inventory_default }}"
# Default overcloud inventory containing a mapping from top level groups to
# hosts.
kolla_overcloud_inventory_default_top_level: "{{ lookup('template', 'overcloud-top-level.j2') }}"
# Overcloud inventory containing a mapping from top level groups to hosts.
kolla_overcloud_inventory_top_level: "{{ kolla_overcloud_inventory_custom_top_level or kolla_overcloud_inventory_default_top_level }}"
# Default overcloud inventory containing a mapping from components to top level
# groups.
kolla_overcloud_inventory_default_components: "{{ lookup('template', 'overcloud-components.j2') }}"
# Overcloud inventory containing a mapping from components to top level groups.
kolla_overcloud_inventory_components: "{{ kolla_overcloud_inventory_custom_components or kolla_overcloud_inventory_default_components }}"
# Default overcloud inventory containing a mapping from services to components.
kolla_overcloud_inventory_default_services: "{{ lookup('template', 'overcloud-services.j2') }}"
# Overcloud inventory containing a mapping from services to components.
kolla_overcloud_inventory_services: "{{ kolla_overcloud_inventory_custom_services or kolla_overcloud_inventory_default_services }}"
# Full default overcloud inventory contents. By default this will be the
# concatenation of the top level, component, and service inventories.
kolla_overcloud_inventory_default: |
# This file is managed by Ansible. Do not edit.
# Overcloud inventory file for kolla-ansible.
{{ kolla_overcloud_inventory_top_level }}
{{ kolla_overcloud_inventory_components }}
{{ kolla_overcloud_inventory_services }}
# Full overcloud inventory contents. By default this will be the concatenation
# of the top level, component, and service inventories.
kolla_overcloud_inventory: "{{ kolla_overcloud_inventory_custom or kolla_overcloud_inventory_default }}"
###############################################################################
# Feature configuration.
# List of features supported by Kolla as enable_* flags.
kolla_feature_flags:
- aodh
@ -48,3 +100,4 @@ kolla_feature_flags:
- telegraf
- tempest
- watcher

View File

@ -90,7 +90,12 @@ Seed
should exist in the ``seed`` group.
Cloud hosts and bare metal compute hosts are not required to exist in the
inventory.
inventory if discovery of the control plane hardware is planned, although
entries for groups may still be required.
Use of advanced control planes with multiple server roles and customised
service placement across those servers is covered in
:ref:`control-plane-service-placement`.
Site Localisation and Customisation
-----------------------------------

View File

@ -0,0 +1,219 @@
.. _control-plane-service-placement:
===============================
Control Plane Service Placement
===============================
.. note::
This is an advanced topic and should only be attempted when familiar with
kayobe and OpenStack.
The default configuration in kayobe places all control plane services on a
single set of servers described as 'controllers'. In some cases it may be
necessary to introduce more than one server role into the control plane, and
control which services are placed onto the different server roles.
Configuration
=============
Overcloud Inventory Discovery
-----------------------------
If using a seed host to enable discovery of the control plane services, it is
necessary to configure how the discovered hosts map into kayobe groups. This
is done using the ``overcloud_group_hosts_map`` variable, which maps names of
kayobe groups to a list of the hosts to be added to that group.
This variable will be used during the command ``kayobe overcloud inventory
discover``. An inventory file will be generated in
``${KAYOBE_CONFIG_PATH}/inventory/overcloud`` with discovered hosts added to
appropriate kayobe groups based on ``overcloud_group_hosts_map``.
Kolla-ansible Inventory Mapping
-------------------------------
Once hosts have been discovered and enrolled into the kayobe inventory, they
must be added to the kolla-ansible inventory. This is done by mapping from top
level kayobe groups to top level kolla-ansible groups using the
``kolla_overcloud_inventory_top_level_group_map`` variable. This variable maps
from kolla-ansible groups to lists of kayobe groups, and variables to define
for those groups in the kolla-ansible inventory.
Variables For Custom Server Roles
---------------------------------
Certain variables must be defined for hosts in the ``overcloud`` group. For
hosts in the ``controllers`` group, many variables are mapped to other
variables with a ``controller_`` prefix in files under
``ansible/group_vars/controllers/``. This is done in order that they may be set
in a global extra variables file, typically ``controllers.yml``, with defaults
set in ``ansible/group_vars/all/controllers``. A similar scheme is used for
hosts in the ``monitoring`` group.
.. table:: Overcloud host variables
====================== =====================================================
Variable Purpose
====================== =====================================================
``ansible_user`` Username with which to access the host via SSH.
``bootstrap_user`` Username with which to access the host before
``ansible_user`` is configured.
``lvm_groups`` List of LVM volume groups to configure. See
`mrlesmithjr.manage-lvm role
<https://galaxy.ansible.com/mrlesmithjr/manage-lvm/>`_
for format.
``network_interfaces`` List of names of networks to which the host is
connected.
====================== =====================================================
If configuring BIOS and RAID via ``kayobe overcloud bios raid configure``, the
following variables should also be defined:
.. table:: Overcloud BIOS & RAID host variables
====================== =====================================================
Variable Purpose
====================== =====================================================
``bios_config`` Dict mapping BIOS configuration options to their
required values. See `stackhpc.drac role
<https://galaxy.ansible.com/stackhpc/drac/>`_ for
format.
``raid_config`` List of RAID virtual disks to configure. See
`stackhpc.drac role
<https://galaxy.ansible.com/stackhpc/drac/>`_ for
format.
====================== =====================================================
These variables can be defined in inventory host or group variables files,
under ``${KAYOBE_CONFIG_PATH}/inventory/host_vars/<host>`` or
``${KAYOBE_CONFIG_PATH}/inventory/group_vars/<group>`` respectively.
Custom Kolla-ansible Inventories
--------------------------------
As an advanced option, it is possible to fully customise the content of the
kolla-ansible inventory, at various levels. To facilitate this, kayobe breaks
the kolla-ansible inventory into three separate sections.
**Top level** groups define the roles of hosts, e.g. ``controller`` or ``compute``,
and it is to these groups that hosts are mapped directly.
**Components** define groups of services, e.g. ``nova`` or ``ironic``, which
are mapped to top level groups.
**Services** define single containers, e.g. ``nova-compute`` or ``ironic-api``,
which are mapped to components.
The default top level inventory is generated from
``kolla_overcloud_inventory_top_level_group_map``.
Kayobe's component- and service-level inventory for
kolla-ansible is static, and taken from the kolla-ansible example ``multinode``
inventory. The complete inventory is generated by concatenating these
inventories.
Each level may be separately overridden by setting the following variables:
.. table:: Custom kolla-ansible inventory variables
=============================================== =================================
Variable Purpose
=============================================== =================================
``kolla_overcloud_inventory_custom_top_level`` Overcloud inventory containing a
mapping from top level groups
to hosts.
``kolla_overcloud_inventory_custom_components`` Overcloud inventory
containing a mapping from
components to top level
groups.
``kolla_overcloud_inventory_custom_services`` Overcloud inventory
containing a mapping from
services to components.
``kolla_overcloud_inventory_custom`` Full overcloud inventory
contents.
=============================================== =================================
Examples
========
Example 1: Adding Network Hosts
-------------------------------
This example walks through the configuration that could be applied to enable
the use of separate hosts for neutron network services and load balancing.
The control plane consists of three controllers, ``controller-[0-2]``, and two
network hosts, ``network-[0-1]``. All file paths are relative to
``${KAYOBE_CONFIG_PATH}``.
First, we must map the hosts to kayobe groups.
.. code-block:: yaml
:caption: ``overcloud.yml``
overcloud_group_hosts_map:
controllers:
- controller-0
- controller-1
- controller-2
network:
- network-0
- network-1
Next, we must map these groups to kolla-ansible groups.
.. code-block:: yaml
:caption: ``kolla.yml``
kolla_overcloud_inventory_top_level_group_map:
control:
groups:
- controllers
network:
groups:
- network
Finally, we create a group variables file for hosts in the network group,
providing the necessary variables for a control plane host.
.. code-block:: yaml
:caption: ``inventory/group_vars/network``
ansible_user: "{{ kayobe_ansible_user }}"
bootstrap_user: "{{ controller_bootstrap_user }}"
lvm_groups: "{{ controller_lvm_groups }}"
network_interfaces: "{{ controller_network_host_network_interfaces }}"
Here we are using the controller-specific values for some of these variables,
but they could equally be different.
Example 2: Overriding the Kolla-ansible Inventory
-------------------------------------------------
This example shows how to override one or more sections of the kolla-ansible
inventory. All file paths are relative to ``${KAYOBE_CONFIG_PATH}``.
First, create a file containing the customised inventory section. We'll use the
**components** section in this example.
.. code-block:: console
:caption: ``kolla/inventory/overcloud-components.j2``
[nova]
control
[ironic]
{% if kolla_enable_ironic | bool %}
control
{% endif %}
...
Next, we must configure kayobe to use this inventory template.
.. code-block:: yaml
:caption: ``kolla.yml``
kolla_overcloud_inventory_custom_components: "{{ lookup('template', kayobe_config_path ~ '/kolla/inventory/overcloud-components.j2') }}"
Here we use the ``template`` lookup plugin to render the Jinja2-formatted
inventory template.

View File

@ -28,6 +28,14 @@ Documentation
upgrading
administration
Advanced Documentation
----------------------
.. toctree::
:maxdepth: 2
control-plane-service-placement
Developer Documentation
-----------------------

View File

@ -63,6 +63,38 @@
# remove. The value should be a list.
#kolla_build_customizations:
###############################################################################
# Kolla-ansible inventory configuration.
# Full custom seed inventory contents.
#kolla_seed_inventory_custom:
# Custom overcloud inventory containing a mapping from top level groups to
# hosts.
#kolla_overcloud_inventory_custom_top_level:
# Custom overcloud inventory containing a mapping from components to top level
# groups.
#kolla_overcloud_inventory_custom_components:
# Custom overcloud inventory containing a mapping from services to components.
#kolla_overcloud_inventory_custom_services:
# Full custom overcloud inventory contents. By default this will be the
# concatenation of the top level, component, and service inventories.
#kolla_overcloud_inventory_custom:
# Dict mapping from kolla-ansible groups to kayobe groups and variables. Each
# item is a dict with the following items:
# * groups: A list of kayobe ansible groups to map to this kolla-ansible group.
# * vars: A dict mapping variable names to values for hosts in this
# kolla-ansible group.
#kolla_overcloud_inventory_top_level_group_map:
# List of names of top level kolla-ansible groups. Any of these groups which
# have no hosts mapped to them will be provided with an empty group definition.
#kolla_overcloud_inventory_kolla_top_level_groups:
###############################################################################
# Kolla-ansible configuration.