diff --git a/doc/source/conf.py b/doc/source/conf.py index fba4509a5..0636ce794 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -112,7 +112,10 @@ repository_name = 'openstack/kayobe' bug_project = 'kayobe' bug_tag = '' openstack_projects = [ + 'bifrost', + 'diskimage-builder', 'ironic', + 'ironic-python-agent', 'kolla', 'kolla-ansible', ] diff --git a/doc/source/configuration/bifrost.rst b/doc/source/configuration/bifrost.rst new file mode 100644 index 000000000..c24b2e50e --- /dev/null +++ b/doc/source/configuration/bifrost.rst @@ -0,0 +1,294 @@ +.. _configuration-bifrost: + +======= +Bifrost +======= + +This section covers configuration of the Bifrost service that runs on the seed +host. Bifrost configuration is typically applied in +``${KAYOBE_CONFIG_PATH}/bifrost.yml``. Consult the :bifrost-doc:`Bifrost +documentation <>` for further details of Bifrost usage and configuration. + +Bifrost installation +==================== + +.. note:: + + This section may be skipped if using an upstream Bifrost container image. + +The following options are used if building the Bifrost container image locally. + +``kolla_bifrost_source_url`` + URL of Bifrost source code repository. Default is + https://opendev.org/openstack/bifrost. +``kolla_bifrost_source_version`` + Version (branch, tag, etc.) of Bifrost source code repository. Default is + ``{{ openstack_branch }}``, which is the same as the Kayobe upstream branch + name. + +For example, to install Bifrost from a custom git repository: + +.. code-block:: yaml + :caption: ``bifrost.yml`` + + kolla_bifrost_source_url: https://git.example.com/bifrost + kolla_bifrost_source_version: downstream + +.. _configuration-bifrost-overcloud-root-image: + +Overcloud root disk image configuration +======================================= + +Bifrost uses Diskimage builder (DIB) to build a root disk image that is +deployed to overcloud hosts when they are provisioned. The following options +configure how this image is built. Consult the +:diskimage-builder-doc:`Diskimage-builder documentation <>` for further +information on building disk images. + +The default configuration builds a CentOS 7 whole disk (partitioned) image with +SELinux disabled and a serial console enabled. `Cloud-init +`__ is used to process the +configuration drive built by Bifrost, rather than the Bifrost default of +:diskimage-builder-doc:`simple-init `. + +``kolla_bifrost_dib_os_element`` + DIB base OS element. Default is ``centos7``. +``kolla_bifrost_dib_os_release`` + DIB image OS release. Default is ``GenericCloud``. +``kolla_bifrost_dib_elements_default`` + *Added in the Train release. Use kolla_bifrost_dib_elements in earlier + releases.* + + List of default DIB elements. Default is ``["disable-selinux", + "enable-serial-console", "vm"]``. The ``vm`` element is poorly named, and + causes DIB to build a whole disk image rather than a single partition. +``kolla_bifrost_dib_elements_extra`` + *Added in the Train release. Use kolla_bifrost_dib_elements in earlier + releases.* + + List of additional DIB elements. Default is none. +``kolla_bifrost_dib_elements`` + List of DIB elements. Default is a combination of + ``kolla_bifrost_dib_elements_default`` and + ``kolla_bifrost_dib_elements_extra``. +``kolla_bifrost_dib_init_element`` + DIB init element. Default is ``cloud-init-datasources``. +``kolla_bifrost_dib_env_vars_default`` + *Added in the Train release. Use kolla_bifrost_dib_env_vars in earlier + releases.* + + DIB default environment variables. Default is + ``{"DIB_CLOUD_INIT_DATASOURCES": "ConfigDrive"}``. +``kolla_bifrost_dib_env_vars_extra`` + *Added in the Train release. Use kolla_bifrost_dib_env_vars in earlier + releases.* + + DIB additional environment variables. Default is none. +``kolla_bifrost_dib_env_vars`` + DIB environment variables. Default is combination of + ``kolla_bifrost_dib_env_vars_default`` and + ``kolla_bifrost_dib_env_vars_extra``. +``kolla_bifrost_dib_packages`` + List of DIB packages to install. Default is to install no extra packages. + +The disk image is built during the ``kayobe seed service deploy`` command. It +is worth noting that currently, the image will not be rebuilt if it already +exists. To force rebuilding the image, it is necessary to remove the file. On +the seed: + +.. code-block:: console + + docker exec bifrost_deploy rm /httpboot/deployment_image.qcow2 + +Example: Adding an element +-------------------------- + +In the following, we extend the list of DIB elements to add the ``growpart`` +element: + +.. code-block:: yaml + :caption: ``bifrost.yml`` + + kolla_bifrost_dib_elements_extra: + - "growpart" + +Example: Building an XFS root filesystem image +---------------------------------------------- + +By default, DIB will format the image as ``ext4``. In some cases it might be +useful to use XFS, for example when using the ``overlay`` Docker storage driver +which can reach the maximum number of hardlinks allowed by ``ext4``. + +In DIB, we achieve this by setting the ``FS_TYPE`` environment variable to +``xfs``. + +.. code-block:: yaml + :caption: ``bifrost.yml`` + + kolla_bifrost_dib_env_vars_extra: + FS_TYPE: "xfs" + +In order to build the image, we also require the ``xfsprogs`` package +to be installed in the Bifrost container. As of the Ussuri release of Bifrost +this will be done automatically, but until then, the following workaround +can be made on the seed host: + +.. code-block:: console + + docker exec bifrost_deploy yum -y install xfsprogs + +Example: Configuring a development user account +----------------------------------------------- + +.. warning:: + + A development user account should not be used in production. + +When debugging a failed deployment, it can sometimes be necessary to allow +access to the image via a preconfigured user account with a known password. +This can be achieved via the :diskimage-builder-doc:`devuser +` element. + +This example shows how to add the ``devuser`` element, and configure a username +and password for an account that has passwordless sudo: + +.. code-block:: yaml + :caption: ``bifrost.yml`` + + kolla_bifrost_dib_elements_extra: + - "devuser" + + kolla_bifrost_dib_env_vars_extra: + DIB_DEV_USER_USERNAME: "devuser" + DIB_DEV_USER_PASSWORD: "correct horse battery staple" + DIB_DEV_USER_PWDLESS_SUDO: "yes" + +Alternatively, the :diskimage-builder-doc:`dynamic-login element +` can be used to authorize SSH keys by appending +them to the kernel arguments. + +Ironic configuration +==================== + +The following options configure the Ironic service in the ``bifrost-deploy`` +container. + +``kolla_bifrost_enabled_hardware_types`` + List of :ironic-doc:`hardware types ` to enable for + Bifrost's Ironic. Default is ``["ipmi"]``. + +Ironic Inspector configuration +============================== + +The following options configure the Ironic Inspector service in the +``bifrost-deploy`` container. + +``kolla_bifrost_inspector_processing_hooks`` + List of of inspector processing plugins. Default is ``{{ + inspector_processing_hooks }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_inspector_port_addition`` + Which MAC addresses to add as ports during introspection. One of ``all``, + ``active`` or ``pxe``. Default is ``{{ inspector_add_ports }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_inspector_extra_kernel_options`` + List of extra kernel parameters for the inspector default PXE + configuration. Default is ``{{ inspector_extra_kernel_options }}``, defined + in ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_inspector_rules`` + List of introspection rules for Bifrost's Ironic Inspector service. Default + is ``{{ inspector_rules }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_inspector_ipmi_username`` + Ironic inspector IPMI username to set via an introspection rule. Default is + ``{{ ipmi_username }}``, defined in ``${KAYOBE_CONFIG_PATH}/bmc.yml``. +``kolla_bifrost_inspector_ipmi_password`` + Ironic inspector IPMI password to set via an introspection rule. Default is + ``{{ ipmi_password }}``, defined in ``${KAYOBE_CONFIG_PATH}/bmc.yml``. +``kolla_bifrost_inspector_lldp_switch_port_interface`` + Ironic inspector network interface name on which to check for an LLDP switch + port description to use as the node's name. Default is ``{{ + inspector_lldp_switch_port_interface_default }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_inspector_deploy_kernel`` + Ironic inspector deployment kernel location. Default is ``http://{{ + provision_oc_net_name | net_ip }}:8080/ipa.vmlinuz``. +``kolla_bifrost_inspector_deploy_ramdisk`` + Ironic inspector deployment ramdisk location. Default is ``http://{{ + provision_oc_net_name | net_ip }}:8080/ipa.initramfs``. + +Ironic Python Agent (IPA) configuration +======================================= + +.. note:: + + If building IPA images locally (``ipa_build_images`` is ``true``) this + section can be skipped. + +The following options configure the source of Ironic Python Agent images used +by Bifrost for inspection and deployment. Consult the +:ironic-python-agent-doc:`Ironic Python Agent documentation <>` for full +details. + +``kolla_bifrost_ipa_kernel_upstream_url`` + URL of Ironic Python Agent (IPA) kernel image. Default is ``{{ + inspector_ipa_kernel_upstream_url }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_ipa_kernel_checksum_url`` + URL of checksum of Ironic Python Agent (IPA) kernel image. Default is ``{{ + inspector_ipa_kernel_checksum_url }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_ipa_kernel_checksum_algorithm`` + Algorithm of checksum of Ironic Python Agent (IPA) kernel image. Default is + ``{{ inspector_ipa_kernel_checksum_algorithm }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_ipa_ramdisk_upstream_url`` + URL of Ironic Python Agent (IPA) ramdisk image. Default is ``{{ + inspector_ipa_ramdisk_upstream_url }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_ipa_ramdisk_checksum_url`` + URL of checksum of Ironic Python Agent (IPA) ramdisk image. Default is ``{{ + inspector_ipa_ramdisk_checksum_url }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. +``kolla_bifrost_ipa_ramdisk_checksum_algorithm`` + Algorithm of checksum of Ironic Python Agent (IPA) ramdisk image. Default + is ``{{ inspector_ipa_ramdisk_checksum_algorithm }}``, defined in + ``${KAYOBE_CONFIG_PATH}/inspector.yml``. + +Inventory configuration +======================= + +.. note:: + + This feature is currently not well tested. It is advisable to use + autodiscovery of overcloud servers instead. + +The following options are used to configure a static inventory of servers for +Bifrost. + +``kolla_bifrost_servers`` + + Server inventory for Bifrost in the :bifrost-doc:`JSON file format + `. + +Custom Configuration +==================== + +Further configuration of arbitrary Ansible variables for Bifrost can be +provided via the following files: + +* ``${KAYOBE_CONFIG_PATH}/kolla/config/bifrost/bifrost.yml`` +* ``${KAYOBE_CONFIG_PATH}/kolla/config/bifrost/dib.yml`` + +These are both passed as extra variables files to ``ansible-playbook``, but the +naming scheme provides a separation of DIB image related variables from other +variables. It may be necessary to inspect the `Bifrost source code +`__ for the full set of variables that +may be configured. + +For example, to configure debug logging for Ironic Inspector: + +.. code-block:: yaml + :caption: ``kolla/config/bifrost/bifrost.yml`` + + inspector_debug: true diff --git a/doc/source/configuration/index.rst b/doc/source/configuration/index.rst index 7493a4ab4..042c5baa8 100644 --- a/doc/source/configuration/index.rst +++ b/doc/source/configuration/index.rst @@ -11,3 +11,5 @@ Configuration Guide hosts kolla kolla-ansible + bifrost + ironic-python-agent diff --git a/doc/source/configuration/ironic-python-agent.rst b/doc/source/configuration/ironic-python-agent.rst new file mode 100644 index 000000000..74c8c4cef --- /dev/null +++ b/doc/source/configuration/ironic-python-agent.rst @@ -0,0 +1,297 @@ +========================= +Ironic Python Agent (IPA) +========================= + +This section covers configuration of Ironic Python Agent (IPA) which is used by +Ironic and Ironic Inspector to deploy and inspect bare metal nodes. This is +used by the Bifrost services that run on the seed host, and also by Ironic and +Ironic Inspector services running in the overcloud for bare metal compute, if +enabled (``kolla_enable_ironic`` is ``true``). IPA configuration is typically +applied in ``${KAYOBE_CONFIG_PATH}/ipa.yml``. Consult the +:ironic-python-agent-doc:`IPA documentation <>` for full details of IPA usage +and configuration. + +.. _configuration-ipa-build: + +Ironic Python Agent (IPA) image build configuration +=================================================== + +.. note:: + + This section may be skipped if not building IPA images locally + (``ipa_build_images`` is ``false``). + +The following options cover building of IPA images via Diskimage-builder (DIB). +Consult the :diskimage-builder-doc:`Diskimage-builder documentation <>` for +full details. + +The default configuration builds a CentOS 7 ramdisk image which includes the +upstream IPA source code, and has a serial console enabled. + +The images are built for Bifrost via ``kayobe seed deployment image build``, +and for Ironic in the overcloud (if enabled) via ``kayobe overcloud deployment +image build``. + +``ipa_build_images`` + Whether to build IPA images from source. Default is ``False``. +``ipa_build_source_url`` + URL of IPA source repository. Default is + https://opendev.org/openstack/ironic-python-agent +``ipa_build_source_version`` + Version of IPA source repository. Default is ``{{ openstack_branch }}``. +``ipa_build_dib_elements_default`` + List of default Diskimage Builder (DIB) elements to use when building IPA + images. Default is ``["centos7", "enable-serial-console", + "ironic-agent"]``. +``ipa_build_dib_elements_extra`` + List of additional Diskimage Builder (DIB) elements to use when building IPA + images. Default is empty. +``ipa_build_dib_elements`` + List of Diskimage Builder (DIB) elements to use when building IPA images. + Default is combination of ``ipa_build_dib_elements_default`` and + ``ipa_build_dib_elements_extra``. +``ipa_build_dib_env_default`` + Dictionary of default environment variables to provide to Diskimage Builder + (DIB) during IPA image build. Default is + ``{"DIB_REPOLOCATION_ironic_agent": "{{ ipa_build_source_url }}", + "DIB_REPOREF_ironic_agent": "{{ ipa_build_source_version }}"}``. +``ipa_build_dib_env_extra`` + Dictionary of additional environment variables to provide to Diskimage + Builder (DIB) during IPA image build. Default is empty. +``ipa_build_dib_env`` + Dictionary of environment variables to provide to Diskimage Builder (DIB) + during IPA image build. Default is a combination of + ``ipa_build_dib_env_default`` and ``ipa_build_dib_env_extra``. +``ipa_build_dib_git_elements`` + List of git repositories containing Diskimage Builder (DIB) elements. See + `stackhpc.os-images `__ role + for usage. Default is none. + +Example: Building IPA images locally +------------------------------------ + +To build IPA images locally: + +.. code-block:: yaml + :caption: ``ipa.yml`` + + ipa_build_images: true + +Example: Installing IPA from a custom git repository +---------------------------------------------------- + +To install IPA from a custom git repository: + +.. code-block:: yaml + :caption: ``ipa.yml`` + + ipa_source_url: https://git.example.com/ironic-python-agent + ipa_source_version: downstream + +Example: Adding an element +-------------------------- + +In the following example, we extend the list of DIB elements to add the +:diskimage-builder-doc:`mellanox element `, which can +be useful for inspecting hardware with Mellanox InfiniBand NICs. + +.. code-block:: yaml + :caption: ``ipa.yml`` + + ipa_build_dib_elements_extra: + - "mellanox" + +Example: Configuring a development user account +----------------------------------------------- + +.. warning:: + + A development user account should not be used in production. + +When debugging a failed deployment, it can sometimes be necessary to allow +access to the image via a preconfigured user account with a known password. +This can be achieved via the :diskimage-builder-doc:`devuser +` element. + +This example shows how to add the ``devuser`` element, and configure a username +and password for an account that has passwordless sudo: + +.. code-block:: yaml + :caption: ``ipa.yml`` + + ipa_build_dib_elements_extra: + - "devuser" + + ipa_build_dib_env_extra: + DIB_DEV_USER_USERNAME: "devuser" + DIB_DEV_USER_PASSWORD: "correct horse battery staple" + DIB_DEV_USER_PWDLESS_SUDO: "yes" + +Alternatively, the :diskimage-builder-doc:`dynamic-login element +` can be used to authorize SSH keys by appending +them to the kernel arguments. + +Further information on troubleshooting IPA can be found +:ironic-python-agent-doc:`here `. + +Example: Configuring custom DIB elements +---------------------------------------- + +Sometimes it is useful to use custom DIB elements that are not shipped with DIB +itself. This can be done by sharing them in a git repository. + +.. code-block:: yaml + :caption: ``ipa.yml`` + + ipa_build_dib_elements_extra: + - "my-element" + + ipa_build_dib_git_elements: + - repo: "https://git.example.com/custom-dib-elements" + local: "{{ source_checkout_path }}/custom-dib-elements" + version: "master" + elements_path: "elements" + +In this example the ``master`` branch of +https://git.example.com/custom-dib-elements would have a top level ``elements`` +directory, containing a ``my-element`` directory for the element. + +Ironic Python Agent (IPA) images configuration +============================================== + +.. note:: + + If building IPA images locally (``ipa_build_images`` is ``true``) this + section can be skipped. + +The following options configure the source of Ironic Python Agent images for +inspection and deployment. Consult the :ironic-python-agent-doc:`Ironic Python +Agent documentation <>` for full details. + +``ipa_images_upstream_url_suffix`` + Suffix of upstream Ironic deployment image files. Default is based on + ``{{ openstack_branch }}``. +``ipa_images_kernel_name`` + Name of Ironic deployment kernel image to register in Glance. Default is + ``ipa.vmlinuz``. +``ipa_kernel_upstream_url`` + URL of Ironic deployment kernel image to download. Default is + ``https://tarballs.openstack.org/ironic-python-agent/coreos/files/coreos_production_pxe{{ + ipa_images_upstream_url_suffix }}.vmlinuz``. +``ipa_kernel_checksum_url`` + URL of checksum of Ironic deployment kernel image. Default is ``{{ + ipa_kernel_upstream_url }}.{{ ipa_kernel_checksum_algorithm }}``. +``ipa_kernel_checksum_algorithm`` + Algorithm of checksum of Ironic deployment kernel image. Default is + ``sha256``. +``ipa_images_ramdisk_name`` + Name of Ironic deployment ramdisk image to register in Glance. Default is + ``ipa.initramfs``. +``ipa_ramdisk_upstream_url`` + URL of Ironic deployment ramdisk image to download. Default is + ``https://tarballs.openstack.org/ironic-python-agent/coreos/files/coreos_production_pxe_image-oem{{ + ipa_images_upstream_url_suffix }}.cpio.gz``. +``ipa_ramdisk_checksum_url`` + URL of checksum of Ironic deployment ramdisk image. Default is ``{{ + ipa_ramdisk_upstream_url }}.{{ ipa_ramdisk_checksum_algorithm }}``. +``ipa_ramdisk_checksum_algorithm`` + Algorithm of checksum of Ironic deployment ramdisk image. Default is + ``sha256``. + +Ironic Python Agent (IPA) deployment configuration +================================================== + +The following options configure how IPA operates during deployment and +inspection. + +``ipa_collect_lldp`` + Whether to enable collection of LLDP TLVs. Default is ``True``. +``ipa_collectors_default`` + .. note:: + + ``extra-hardware`` is not currently included as it requires a ramdisk + with the ``hardware`` python module installed. + + List of default inspection collectors to run. Default is ``["default", + "logs", "pci-devices"]``. +``ipa_collectors_extra`` + List of additional inspection collectors to run. Default is none. +``ipa_collectors`` + List of inspection collectors to run. Default is a combination of + ``ipa_collectors_default`` and ``ipa_collectors_extra``. +``ipa_benchmarks_default`` + List of default inspection benchmarks to run. Default is ``["cpu", "disk", + "ram"]``. +``ipa_benchmarks_extra`` + List of extra inspection benchmarks to run. Default is none. +``ipa_benchmarks`` + .. note:: + + The ``extra-hardware`` collector must be enabled in order to execute + benchmarks during inspection. + + List of inspection benchmarks to run. Default is a combination of + ``ipa_benchmarks_default`` and ``ipa_benchmarks_extra``. +``ipa_kernel_options_default`` + List of default kernel parameters for Ironic python agent. Default includes + ``ipa-collect-lldp``, ``ipa-inspection-collectors`` and + ``ipa-inspection-benchmarks``, with arguments taken from + ``ipa_collect_lldp``, ``ipa_collectors`` and ``ipa_benchmarks``. +``ipa_kernel_options_extra`` + List of additional kernel parameters for Ironic python agent. Default is + none. +``ipa_kernel_options`` + List of kernel parameters for Ironic python agent. Default is a combination + of ``ipa_kernel_options_default`` and ``ipa_kernel_options_extra``. + +Example: Adding the ``extra-hardware`` collector +------------------------------------------------ + +The ``extra-hardware`` collector may be used to collect additional information +about hardware during inspection. It is also a requirement for running +benchmarks. This collector depends on the Python `hardware package +`__, which is not installed in IPA images +by default. + +The following example enables the ``extra-hardware`` collector: + +.. code-block:: yaml + :caption: ``ipa.yml`` + + ipa_collectors_extra: + - "extra-hardware" + +The `StackHPC image elements +`__ git repository +provides an ``ipa-extra-hardware`` element which may be used to install this +package. It may be used as follows if building an IPA image locally: + +.. code-block:: yaml + :caption: ``ipa.yml`` + + ipa_build_dib_elements_extra: + - "ipa-extra-hardware" + + ipa_build_dib_git_elements: + - repo: "https://github.com/stackhpc/stackhpc-image-elements" + local: "{{ source_checkout_path }}/stackhpc-image-elements" + version: "master" + elements_path: "elements" + + ipa_build_dib_env_extra: + # This is to workaround the fact that pip > 10 will produce an error if + # you try and uninstall a distuils installed package. Previous versions + # would remove the metadata only - leaving the code intact, see: + # https://bugs.launchpad.net/diskimage-builder/+bug/1768135 + DIB_INSTALLTYPE_pip_and_virtualenv: package + +Example: Passing additional kernel arguments to IPA +--------------------------------------------------- + +The following example shows how to pass additional kernel arguments to IPA: + +.. code-block:: yaml + :caption: ``ipa.yml`` + + ipa_kernel_options_extra: + - "foo=bar" diff --git a/doc/source/configuration/kolla.rst b/doc/source/configuration/kolla.rst index 3273f4a7c..6a5497a1d 100644 --- a/doc/source/configuration/kolla.rst +++ b/doc/source/configuration/kolla.rst @@ -45,7 +45,8 @@ installation of Kolla: https://opendev.org/openstack/kolla. ``kolla_source_version`` Version (branch, tag, etc.) of Kolla source code repository if type is - ``source``. Default is the same as the Kayobe upstream branch name. + ``source``. Default is ``{{ openstack_branch }}``, which is the same as the + Kayobe upstream branch name. ``kolla_venv`` Path to virtualenv in which to install Kolla on the container image build host. Default is ``{{ virtualenv_path ~ '/kolla' }}``.