Add documentation for Bifrost and IPA configuration

Depends-On: https://review.opendev.org/690559/

Change-Id: I4e766424607317e5817d26461f7a1e180e9b2146
Story: 2004337
Task: 37378
Task: 37381
This commit is contained in:
Mark Goddard 2019-11-06 16:38:38 +00:00
parent a02c65832c
commit 8c01771e29
5 changed files with 598 additions and 1 deletions

View File

@ -112,7 +112,10 @@ repository_name = 'openstack/kayobe'
bug_project = 'kayobe'
bug_tag = ''
openstack_projects = [
'bifrost',
'diskimage-builder',
'ironic',
'ironic-python-agent',
'kolla',
'kolla-ansible',
]

View File

@ -0,0 +1,294 @@
.. _configuration-bifrost:
=======
Bifrost
=======
This section covers configuration of the Bifrost service that runs on the seed
host. Bifrost configuration is typically applied in
``${KAYOBE_CONFIG_PATH}/bifrost.yml``. Consult the :bifrost-doc:`Bifrost
documentation <>` for further details of Bifrost usage and configuration.
Bifrost installation
====================
.. note::
This section may be skipped if using an upstream Bifrost container image.
The following options are used if building the Bifrost container image locally.
``kolla_bifrost_source_url``
URL of Bifrost source code repository. Default is
https://opendev.org/openstack/bifrost.
``kolla_bifrost_source_version``
Version (branch, tag, etc.) of Bifrost source code repository. Default is
``{{ openstack_branch }}``, which is the same as the Kayobe upstream branch
name.
For example, to install Bifrost from a custom git repository:
.. code-block:: yaml
:caption: ``bifrost.yml``
kolla_bifrost_source_url: https://git.example.com/bifrost
kolla_bifrost_source_version: downstream
.. _configuration-bifrost-overcloud-root-image:
Overcloud root disk image configuration
=======================================
Bifrost uses Diskimage builder (DIB) to build a root disk image that is
deployed to overcloud hosts when they are provisioned. The following options
configure how this image is built. Consult the
:diskimage-builder-doc:`Diskimage-builder documentation <>` for further
information on building disk images.
The default configuration builds a CentOS 7 whole disk (partitioned) image with
SELinux disabled and a serial console enabled. `Cloud-init
<https://cloudinit.readthedocs.io/en/latest/>`__ is used to process the
configuration drive built by Bifrost, rather than the Bifrost default of
:diskimage-builder-doc:`simple-init <elements/simple-init/README>`.
``kolla_bifrost_dib_os_element``
DIB base OS element. Default is ``centos7``.
``kolla_bifrost_dib_os_release``
DIB image OS release. Default is ``GenericCloud``.
``kolla_bifrost_dib_elements_default``
*Added in the Train release. Use kolla_bifrost_dib_elements in earlier
releases.*
List of default DIB elements. Default is ``["disable-selinux",
"enable-serial-console", "vm"]``. The ``vm`` element is poorly named, and
causes DIB to build a whole disk image rather than a single partition.
``kolla_bifrost_dib_elements_extra``
*Added in the Train release. Use kolla_bifrost_dib_elements in earlier
releases.*
List of additional DIB elements. Default is none.
``kolla_bifrost_dib_elements``
List of DIB elements. Default is a combination of
``kolla_bifrost_dib_elements_default`` and
``kolla_bifrost_dib_elements_extra``.
``kolla_bifrost_dib_init_element``
DIB init element. Default is ``cloud-init-datasources``.
``kolla_bifrost_dib_env_vars_default``
*Added in the Train release. Use kolla_bifrost_dib_env_vars in earlier
releases.*
DIB default environment variables. Default is
``{"DIB_CLOUD_INIT_DATASOURCES": "ConfigDrive"}``.
``kolla_bifrost_dib_env_vars_extra``
*Added in the Train release. Use kolla_bifrost_dib_env_vars in earlier
releases.*
DIB additional environment variables. Default is none.
``kolla_bifrost_dib_env_vars``
DIB environment variables. Default is combination of
``kolla_bifrost_dib_env_vars_default`` and
``kolla_bifrost_dib_env_vars_extra``.
``kolla_bifrost_dib_packages``
List of DIB packages to install. Default is to install no extra packages.
The disk image is built during the ``kayobe seed service deploy`` command. It
is worth noting that currently, the image will not be rebuilt if it already
exists. To force rebuilding the image, it is necessary to remove the file. On
the seed:
.. code-block:: console
docker exec bifrost_deploy rm /httpboot/deployment_image.qcow2
Example: Adding an element
--------------------------
In the following, we extend the list of DIB elements to add the ``growpart``
element:
.. code-block:: yaml
:caption: ``bifrost.yml``
kolla_bifrost_dib_elements_extra:
- "growpart"
Example: Building an XFS root filesystem image
----------------------------------------------
By default, DIB will format the image as ``ext4``. In some cases it might be
useful to use XFS, for example when using the ``overlay`` Docker storage driver
which can reach the maximum number of hardlinks allowed by ``ext4``.
In DIB, we achieve this by setting the ``FS_TYPE`` environment variable to
``xfs``.
.. code-block:: yaml
:caption: ``bifrost.yml``
kolla_bifrost_dib_env_vars_extra:
FS_TYPE: "xfs"
In order to build the image, we also require the ``xfsprogs`` package
to be installed in the Bifrost container. As of the Ussuri release of Bifrost
this will be done automatically, but until then, the following workaround
can be made on the seed host:
.. code-block:: console
docker exec bifrost_deploy yum -y install xfsprogs
Example: Configuring a development user account
-----------------------------------------------
.. warning::
A development user account should not be used in production.
When debugging a failed deployment, it can sometimes be necessary to allow
access to the image via a preconfigured user account with a known password.
This can be achieved via the :diskimage-builder-doc:`devuser
<elements/devuser/README>` element.
This example shows how to add the ``devuser`` element, and configure a username
and password for an account that has passwordless sudo:
.. code-block:: yaml
:caption: ``bifrost.yml``
kolla_bifrost_dib_elements_extra:
- "devuser"
kolla_bifrost_dib_env_vars_extra:
DIB_DEV_USER_USERNAME: "devuser"
DIB_DEV_USER_PASSWORD: "correct horse battery staple"
DIB_DEV_USER_PWDLESS_SUDO: "yes"
Alternatively, the :diskimage-builder-doc:`dynamic-login element
<elements/dynamic-login/README>` can be used to authorize SSH keys by appending
them to the kernel arguments.
Ironic configuration
====================
The following options configure the Ironic service in the ``bifrost-deploy``
container.
``kolla_bifrost_enabled_hardware_types``
List of :ironic-doc:`hardware types <admin/drivers>` to enable for
Bifrost's Ironic. Default is ``["ipmi"]``.
Ironic Inspector configuration
==============================
The following options configure the Ironic Inspector service in the
``bifrost-deploy`` container.
``kolla_bifrost_inspector_processing_hooks``
List of of inspector processing plugins. Default is ``{{
inspector_processing_hooks }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_inspector_port_addition``
Which MAC addresses to add as ports during introspection. One of ``all``,
``active`` or ``pxe``. Default is ``{{ inspector_add_ports }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_inspector_extra_kernel_options``
List of extra kernel parameters for the inspector default PXE
configuration. Default is ``{{ inspector_extra_kernel_options }}``, defined
in ``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_inspector_rules``
List of introspection rules for Bifrost's Ironic Inspector service. Default
is ``{{ inspector_rules }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_inspector_ipmi_username``
Ironic inspector IPMI username to set via an introspection rule. Default is
``{{ ipmi_username }}``, defined in ``${KAYOBE_CONFIG_PATH}/bmc.yml``.
``kolla_bifrost_inspector_ipmi_password``
Ironic inspector IPMI password to set via an introspection rule. Default is
``{{ ipmi_password }}``, defined in ``${KAYOBE_CONFIG_PATH}/bmc.yml``.
``kolla_bifrost_inspector_lldp_switch_port_interface``
Ironic inspector network interface name on which to check for an LLDP switch
port description to use as the node's name. Default is ``{{
inspector_lldp_switch_port_interface_default }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_inspector_deploy_kernel``
Ironic inspector deployment kernel location. Default is ``http://{{
provision_oc_net_name | net_ip }}:8080/ipa.vmlinuz``.
``kolla_bifrost_inspector_deploy_ramdisk``
Ironic inspector deployment ramdisk location. Default is ``http://{{
provision_oc_net_name | net_ip }}:8080/ipa.initramfs``.
Ironic Python Agent (IPA) configuration
=======================================
.. note::
If building IPA images locally (``ipa_build_images`` is ``true``) this
section can be skipped.
The following options configure the source of Ironic Python Agent images used
by Bifrost for inspection and deployment. Consult the
:ironic-python-agent-doc:`Ironic Python Agent documentation <>` for full
details.
``kolla_bifrost_ipa_kernel_upstream_url``
URL of Ironic Python Agent (IPA) kernel image. Default is ``{{
inspector_ipa_kernel_upstream_url }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_ipa_kernel_checksum_url``
URL of checksum of Ironic Python Agent (IPA) kernel image. Default is ``{{
inspector_ipa_kernel_checksum_url }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_ipa_kernel_checksum_algorithm``
Algorithm of checksum of Ironic Python Agent (IPA) kernel image. Default is
``{{ inspector_ipa_kernel_checksum_algorithm }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_ipa_ramdisk_upstream_url``
URL of Ironic Python Agent (IPA) ramdisk image. Default is ``{{
inspector_ipa_ramdisk_upstream_url }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_ipa_ramdisk_checksum_url``
URL of checksum of Ironic Python Agent (IPA) ramdisk image. Default is ``{{
inspector_ipa_ramdisk_checksum_url }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
``kolla_bifrost_ipa_ramdisk_checksum_algorithm``
Algorithm of checksum of Ironic Python Agent (IPA) ramdisk image. Default
is ``{{ inspector_ipa_ramdisk_checksum_algorithm }}``, defined in
``${KAYOBE_CONFIG_PATH}/inspector.yml``.
Inventory configuration
=======================
.. note::
This feature is currently not well tested. It is advisable to use
autodiscovery of overcloud servers instead.
The following options are used to configure a static inventory of servers for
Bifrost.
``kolla_bifrost_servers``
Server inventory for Bifrost in the :bifrost-doc:`JSON file format
<user/howto#json-file-format>`.
Custom Configuration
====================
Further configuration of arbitrary Ansible variables for Bifrost can be
provided via the following files:
* ``${KAYOBE_CONFIG_PATH}/kolla/config/bifrost/bifrost.yml``
* ``${KAYOBE_CONFIG_PATH}/kolla/config/bifrost/dib.yml``
These are both passed as extra variables files to ``ansible-playbook``, but the
naming scheme provides a separation of DIB image related variables from other
variables. It may be necessary to inspect the `Bifrost source code
<https://opendev.org/openstack/bifrost>`__ for the full set of variables that
may be configured.
For example, to configure debug logging for Ironic Inspector:
.. code-block:: yaml
:caption: ``kolla/config/bifrost/bifrost.yml``
inspector_debug: true

View File

@ -11,3 +11,5 @@ Configuration Guide
hosts
kolla
kolla-ansible
bifrost
ironic-python-agent

View File

@ -0,0 +1,297 @@
=========================
Ironic Python Agent (IPA)
=========================
This section covers configuration of Ironic Python Agent (IPA) which is used by
Ironic and Ironic Inspector to deploy and inspect bare metal nodes. This is
used by the Bifrost services that run on the seed host, and also by Ironic and
Ironic Inspector services running in the overcloud for bare metal compute, if
enabled (``kolla_enable_ironic`` is ``true``). IPA configuration is typically
applied in ``${KAYOBE_CONFIG_PATH}/ipa.yml``. Consult the
:ironic-python-agent-doc:`IPA documentation <>` for full details of IPA usage
and configuration.
.. _configuration-ipa-build:
Ironic Python Agent (IPA) image build configuration
===================================================
.. note::
This section may be skipped if not building IPA images locally
(``ipa_build_images`` is ``false``).
The following options cover building of IPA images via Diskimage-builder (DIB).
Consult the :diskimage-builder-doc:`Diskimage-builder documentation <>` for
full details.
The default configuration builds a CentOS 7 ramdisk image which includes the
upstream IPA source code, and has a serial console enabled.
The images are built for Bifrost via ``kayobe seed deployment image build``,
and for Ironic in the overcloud (if enabled) via ``kayobe overcloud deployment
image build``.
``ipa_build_images``
Whether to build IPA images from source. Default is ``False``.
``ipa_build_source_url``
URL of IPA source repository. Default is
https://opendev.org/openstack/ironic-python-agent
``ipa_build_source_version``
Version of IPA source repository. Default is ``{{ openstack_branch }}``.
``ipa_build_dib_elements_default``
List of default Diskimage Builder (DIB) elements to use when building IPA
images. Default is ``["centos7", "enable-serial-console",
"ironic-agent"]``.
``ipa_build_dib_elements_extra``
List of additional Diskimage Builder (DIB) elements to use when building IPA
images. Default is empty.
``ipa_build_dib_elements``
List of Diskimage Builder (DIB) elements to use when building IPA images.
Default is combination of ``ipa_build_dib_elements_default`` and
``ipa_build_dib_elements_extra``.
``ipa_build_dib_env_default``
Dictionary of default environment variables to provide to Diskimage Builder
(DIB) during IPA image build. Default is
``{"DIB_REPOLOCATION_ironic_agent": "{{ ipa_build_source_url }}",
"DIB_REPOREF_ironic_agent": "{{ ipa_build_source_version }}"}``.
``ipa_build_dib_env_extra``
Dictionary of additional environment variables to provide to Diskimage
Builder (DIB) during IPA image build. Default is empty.
``ipa_build_dib_env``
Dictionary of environment variables to provide to Diskimage Builder (DIB)
during IPA image build. Default is a combination of
``ipa_build_dib_env_default`` and ``ipa_build_dib_env_extra``.
``ipa_build_dib_git_elements``
List of git repositories containing Diskimage Builder (DIB) elements. See
`stackhpc.os-images <https://galaxy.ansible.com/stackhpc/os-images>`__ role
for usage. Default is none.
Example: Building IPA images locally
------------------------------------
To build IPA images locally:
.. code-block:: yaml
:caption: ``ipa.yml``
ipa_build_images: true
Example: Installing IPA from a custom git repository
----------------------------------------------------
To install IPA from a custom git repository:
.. code-block:: yaml
:caption: ``ipa.yml``
ipa_source_url: https://git.example.com/ironic-python-agent
ipa_source_version: downstream
Example: Adding an element
--------------------------
In the following example, we extend the list of DIB elements to add the
:diskimage-builder-doc:`mellanox element <elements/mellanox/README>`, which can
be useful for inspecting hardware with Mellanox InfiniBand NICs.
.. code-block:: yaml
:caption: ``ipa.yml``
ipa_build_dib_elements_extra:
- "mellanox"
Example: Configuring a development user account
-----------------------------------------------
.. warning::
A development user account should not be used in production.
When debugging a failed deployment, it can sometimes be necessary to allow
access to the image via a preconfigured user account with a known password.
This can be achieved via the :diskimage-builder-doc:`devuser
<elements/devuser/README>` element.
This example shows how to add the ``devuser`` element, and configure a username
and password for an account that has passwordless sudo:
.. code-block:: yaml
:caption: ``ipa.yml``
ipa_build_dib_elements_extra:
- "devuser"
ipa_build_dib_env_extra:
DIB_DEV_USER_USERNAME: "devuser"
DIB_DEV_USER_PASSWORD: "correct horse battery staple"
DIB_DEV_USER_PWDLESS_SUDO: "yes"
Alternatively, the :diskimage-builder-doc:`dynamic-login element
<elements/dynamic-login/README>` can be used to authorize SSH keys by appending
them to the kernel arguments.
Further information on troubleshooting IPA can be found
:ironic-python-agent-doc:`here <admin/troubleshooting>`.
Example: Configuring custom DIB elements
----------------------------------------
Sometimes it is useful to use custom DIB elements that are not shipped with DIB
itself. This can be done by sharing them in a git repository.
.. code-block:: yaml
:caption: ``ipa.yml``
ipa_build_dib_elements_extra:
- "my-element"
ipa_build_dib_git_elements:
- repo: "https://git.example.com/custom-dib-elements"
local: "{{ source_checkout_path }}/custom-dib-elements"
version: "master"
elements_path: "elements"
In this example the ``master`` branch of
https://git.example.com/custom-dib-elements would have a top level ``elements``
directory, containing a ``my-element`` directory for the element.
Ironic Python Agent (IPA) images configuration
==============================================
.. note::
If building IPA images locally (``ipa_build_images`` is ``true``) this
section can be skipped.
The following options configure the source of Ironic Python Agent images for
inspection and deployment. Consult the :ironic-python-agent-doc:`Ironic Python
Agent documentation <>` for full details.
``ipa_images_upstream_url_suffix``
Suffix of upstream Ironic deployment image files. Default is based on
``{{ openstack_branch }}``.
``ipa_images_kernel_name``
Name of Ironic deployment kernel image to register in Glance. Default is
``ipa.vmlinuz``.
``ipa_kernel_upstream_url``
URL of Ironic deployment kernel image to download. Default is
``https://tarballs.openstack.org/ironic-python-agent/coreos/files/coreos_production_pxe{{
ipa_images_upstream_url_suffix }}.vmlinuz``.
``ipa_kernel_checksum_url``
URL of checksum of Ironic deployment kernel image. Default is ``{{
ipa_kernel_upstream_url }}.{{ ipa_kernel_checksum_algorithm }}``.
``ipa_kernel_checksum_algorithm``
Algorithm of checksum of Ironic deployment kernel image. Default is
``sha256``.
``ipa_images_ramdisk_name``
Name of Ironic deployment ramdisk image to register in Glance. Default is
``ipa.initramfs``.
``ipa_ramdisk_upstream_url``
URL of Ironic deployment ramdisk image to download. Default is
``https://tarballs.openstack.org/ironic-python-agent/coreos/files/coreos_production_pxe_image-oem{{
ipa_images_upstream_url_suffix }}.cpio.gz``.
``ipa_ramdisk_checksum_url``
URL of checksum of Ironic deployment ramdisk image. Default is ``{{
ipa_ramdisk_upstream_url }}.{{ ipa_ramdisk_checksum_algorithm }}``.
``ipa_ramdisk_checksum_algorithm``
Algorithm of checksum of Ironic deployment ramdisk image. Default is
``sha256``.
Ironic Python Agent (IPA) deployment configuration
==================================================
The following options configure how IPA operates during deployment and
inspection.
``ipa_collect_lldp``
Whether to enable collection of LLDP TLVs. Default is ``True``.
``ipa_collectors_default``
.. note::
``extra-hardware`` is not currently included as it requires a ramdisk
with the ``hardware`` python module installed.
List of default inspection collectors to run. Default is ``["default",
"logs", "pci-devices"]``.
``ipa_collectors_extra``
List of additional inspection collectors to run. Default is none.
``ipa_collectors``
List of inspection collectors to run. Default is a combination of
``ipa_collectors_default`` and ``ipa_collectors_extra``.
``ipa_benchmarks_default``
List of default inspection benchmarks to run. Default is ``["cpu", "disk",
"ram"]``.
``ipa_benchmarks_extra``
List of extra inspection benchmarks to run. Default is none.
``ipa_benchmarks``
.. note::
The ``extra-hardware`` collector must be enabled in order to execute
benchmarks during inspection.
List of inspection benchmarks to run. Default is a combination of
``ipa_benchmarks_default`` and ``ipa_benchmarks_extra``.
``ipa_kernel_options_default``
List of default kernel parameters for Ironic python agent. Default includes
``ipa-collect-lldp``, ``ipa-inspection-collectors`` and
``ipa-inspection-benchmarks``, with arguments taken from
``ipa_collect_lldp``, ``ipa_collectors`` and ``ipa_benchmarks``.
``ipa_kernel_options_extra``
List of additional kernel parameters for Ironic python agent. Default is
none.
``ipa_kernel_options``
List of kernel parameters for Ironic python agent. Default is a combination
of ``ipa_kernel_options_default`` and ``ipa_kernel_options_extra``.
Example: Adding the ``extra-hardware`` collector
------------------------------------------------
The ``extra-hardware`` collector may be used to collect additional information
about hardware during inspection. It is also a requirement for running
benchmarks. This collector depends on the Python `hardware package
<https://pypi.org/project/hardware/>`__, which is not installed in IPA images
by default.
The following example enables the ``extra-hardware`` collector:
.. code-block:: yaml
:caption: ``ipa.yml``
ipa_collectors_extra:
- "extra-hardware"
The `StackHPC image elements
<https://github.com/stackhpc/stackhpc-image-elements>`__ git repository
provides an ``ipa-extra-hardware`` element which may be used to install this
package. It may be used as follows if building an IPA image locally:
.. code-block:: yaml
:caption: ``ipa.yml``
ipa_build_dib_elements_extra:
- "ipa-extra-hardware"
ipa_build_dib_git_elements:
- repo: "https://github.com/stackhpc/stackhpc-image-elements"
local: "{{ source_checkout_path }}/stackhpc-image-elements"
version: "master"
elements_path: "elements"
ipa_build_dib_env_extra:
# This is to workaround the fact that pip > 10 will produce an error if
# you try and uninstall a distuils installed package. Previous versions
# would remove the metadata only - leaving the code intact, see:
# https://bugs.launchpad.net/diskimage-builder/+bug/1768135
DIB_INSTALLTYPE_pip_and_virtualenv: package
Example: Passing additional kernel arguments to IPA
---------------------------------------------------
The following example shows how to pass additional kernel arguments to IPA:
.. code-block:: yaml
:caption: ``ipa.yml``
ipa_kernel_options_extra:
- "foo=bar"

View File

@ -45,7 +45,8 @@ installation of Kolla:
https://opendev.org/openstack/kolla.
``kolla_source_version``
Version (branch, tag, etc.) of Kolla source code repository if type is
``source``. Default is the same as the Kayobe upstream branch name.
``source``. Default is ``{{ openstack_branch }}``, which is the same as the
Kayobe upstream branch name.
``kolla_venv``
Path to virtualenv in which to install Kolla on the container image build
host. Default is ``{{ virtualenv_path ~ '/kolla' }}``.