tripleo-heat-templates/deployment
Zuul 6e1433dd31 Merge "Add CephClientConfigOverrides resource" into stable/train 2020-10-22 11:39:02 +00:00
..
aide Convert heat template to use aide role 2019-07-03 12:00:17 +00:00
aodh Remove ffwd-upgrade leftovers from THT. 2020-08-13 01:31:20 +00:00
apache Skip both tenant and management networks when generating certs 2020-04-09 17:19:14 +00:00
auditd Move auditd, ca-cert, certmonger to deployment 2019-05-30 20:37:25 +00:00
backup-and-restore Use list join for rendering rear config file in heat 2020-01-20 19:00:10 +01:00
barbican Fix tripleo-hieradata role name in Barbican deployment 2020-10-15 12:08:19 +00:00
cavium Remove unnecessary slash volume maps 2020-02-10 12:01:02 -05:00
ceilometer Remove ffwd-upgrade leftovers from THT. 2020-08-13 01:31:20 +00:00
ceph-ansible Add CephClientConfigOverrides resource 2020-10-21 09:15:03 +00:00
certs Always set hieradata for certmonger_ca 2020-03-15 18:17:44 +00:00
cinder Merge "[FFU] Remove cinder's v1 keystone service" into stable/train 2020-10-21 07:05:59 +00:00
clients Revert "Remove panko" 2020-01-30 20:34:27 +00:00
container-image-prepare Properly place undercloud hosts record upgrade task 2020-04-09 09:39:53 +00:00
database Retry container pull 3 times 2020-10-09 22:44:35 +00:00
deprecated Adds new configuration for panko-expirer 2020-09-23 18:49:54 +02:00
etcd Make sure IPA has the right ACI 2020-09-02 15:20:11 -04:00
experimental Add mode option when creating persistent directories. 2020-05-15 20:41:12 +00:00
glance DCN: use FQDN in glance endpoint with internal TLS 2020-09-02 20:04:00 -07:00
gnocchi Merge "Add support for Gnocchi NFS Backend" into stable/train 2020-08-18 06:07:10 +00:00
haproxy Retry container pull 3 times 2020-10-09 22:44:35 +00:00
heat Remove ffwd-upgrade leftovers from THT. 2020-08-13 01:31:20 +00:00
horizon Sync httpd conf.modules.d configs 2020-06-24 06:53:39 -06:00
image-serve We have to allow httpd to listen on those ports in some cases. 2019-03-21 08:57:59 +01:00
ipa Fix delegation with FreeIPA cleanup 2020-08-12 14:31:47 -05:00
ipsec Remove extraconfig/services directory 2019-07-18 14:44:14 -04:00
ironic Remove ffwd-upgrade leftovers from THT. 2020-08-13 01:31:20 +00:00
iscsid Remove all ignore_errors to avoid confusion when debugging 2020-03-05 21:51:19 -05:00
keepalived Add mode option when creating persistent directories. 2020-05-15 20:41:12 +00:00
kernel Align kernel args for system upgrade using leapp 2020-08-06 03:59:04 +00:00
keystone Remove ffwd-upgrade leftovers from THT. 2020-08-13 01:31:20 +00:00
logging Add more metadata to logs 2020-09-23 09:40:10 +00:00
login-defs Use login-defs role from tripleo-ansible in sc004 2019-08-10 13:25:16 +03:00
logrotate Fix privilege escalation 2020-07-13 01:32:49 +00:00
manila [manila] Add "ManilaEnabledShareProtocols" param 2020-10-16 05:01:18 +00:00
masquerade-networks Move masq-nets, swift-external, and validations to deployment 2019-05-30 20:37:30 +00:00
memcached Memcached collectd plugin uses host URI instead of IP address. 2020-09-09 18:39:50 +00:00
messaging Add mode option when creating persistent directories. 2020-05-15 20:41:12 +00:00
metrics Merge "Revert "Adapt container health check for built-in podman health checks"" into stable/train 2020-10-19 18:55:10 +00:00
mistral [TRAIN ONLY] Ensure interim db migration containers work properly 2020-06-17 16:24:47 +02:00
multipathd Remove all ignore_errors to avoid confusion when debugging 2020-03-05 21:51:19 -05:00
neutron Merge "Remove ffwd-upgrade leftovers from THT." into stable/train 2020-08-13 16:20:12 +00:00
nova Merge "Improve documentations for NovaLibvirtFileBackedMemory" into stable/train 2020-10-13 20:12:08 +00:00
octavia Change permissions on /run/octavia to octavia 2020-10-14 10:39:31 -02:30
openvswitch depends_on: add .service to avoid errors in logs 2020-01-20 01:40:21 +00:00
ovn Retry container pull 3 times 2020-10-09 22:44:35 +00:00
pacemaker Centralized logging minor fixes 2020-09-23 09:40:45 +00:00
placement Sync httpd conf.modules.d configs 2020-06-24 06:53:39 -06:00
podman Merge "Force container fetch" into stable/train 2020-10-02 08:57:27 +00:00
qdr Add mode option when creating persistent directories. 2020-05-15 20:41:12 +00:00
rabbitmq Retry container pull 3 times 2020-10-09 22:44:35 +00:00
rhsm Remove extraconfig/services directory 2019-07-18 14:44:14 -04:00
sahara Remove ffwd-upgrade leftovers from THT. 2020-08-13 01:31:20 +00:00
securetty Configure securetty using tripleo-ansible 2019-09-04 01:22:40 +00:00
snmp Remove all ignore_errors to avoid confusion when debugging 2020-03-05 21:51:19 -05:00
sshd Avoid dangling firewall rule for ssh access 2019-01-22 14:49:46 +01:00
swift Configure rsyncd without pid file for Swift 2020-09-28 23:47:15 +02:00
time Restart only installed services 2019-07-23 10:21:30 +00:00
timesync Replace chronyc "waitsync" with "makestep" 2020-03-13 11:39:42 -04:00
tls Add composible service for tls enrollment 2020-06-11 01:32:53 -05:00
tripleo-firewall Revert "[train/backport] Prevent nftables to interfere with tripleo firewall" 2020-07-10 11:27:13 +05:30
tripleo-packages [train-only] Ensure obsolete module nf_conntrack_proto_sctp is not loaded 2020-09-21 21:22:33 +02:00
tuned Convert heat template to use tuned role 2019-06-26 23:01:43 +00:00
undercloud Add openvswitch special treatment to update too. 2020-07-30 12:53:21 +02:00
validations Use tripleo-validations-package role instead of puppet 2019-08-26 08:56:35 +00:00
veritas-hyperscale [train-squash] Backport "all Keystone resources with Ansible" 2020-02-19 02:56:15 +00:00
vpp Remove all ignore_errors to avoid confusion when debugging 2020-03-05 21:51:19 -05:00
zaqar Sync httpd conf.modules.d configs 2020-06-24 06:53:39 -06:00
README.rst Remove ffwd-upgrade leftovers from THT. 2020-08-13 01:31:20 +00:00
containers-common.yaml Bind mount /var/lib/container-config-scripts inside the restart bundles 2020-09-11 08:29:40 +00:00

README.rst

TripleO Deployments

This directory contains files that represent individual service deployments, orchestration tools, and the configuration tools used to deploy them.

Directory Structure

Each logical grouping of services will have a directory. Example: 'timesync'. Within this directory related timesync services would exist to for example configure timesync services on baremetal or via containers.

Filenaming conventions

As a convention each deployments service filename will reflect both the deployment engine (baremetal, or containers) along with the config tool used to deploy that service.

The convention is <service-name>-<engine>-<config management tool>.

Examples:

deployment/aodh/aodh-api-container-puppet.yaml (containerized Aodh service configured with Puppet)

deployment/aodh/aodh-api-container-ansible.yaml (containerized Aodh service configured with Ansible)

deployment/timesync/chrony-baremetal-ansible.yaml (baremetal Chrony service configured with Ansible)

deployment/timesync/chrony-baremetal-puppet.yaml (baremetal Chrony service configured with Puppet)

Building Kolla Images

TripleO currently relies on Kolla(Dockerfile) containers. Kolla supports container customization and we are making use of this feature within TripleO to inject puppet (our configuration tool of choice) into the Kolla base images. A variety of other customizations are being made via the tripleo-common/container-images/tripleo_kolla_template_overrides.j2 file.

To build Kolla images for TripleO adjust your kolla config1 to build your centos base image with puppet using the example below:

$ cat template-overrides.j2 {% extends parent_template %} {% set base_centos_binary_packages_append = ['puppet'] %} {% set nova_scheduler_packages_append = ['openstack-tripleo-common'] %}

kolla-build --base centos --template-override template-overrides.j2

Containerized Deployment Template Structure

Each deployment template may define a set of output values control the underlying service deployment in a variety of ways. These output sections are specific to the TripleO deployment architecture. The following sections are available for containerized services.

  • config_settings: This section contains service specific hiera data can be used to generate config files for each service. This data is ultimately processed via the container-puppet.py tool (in new versions it's handled by the container_puppet_config module in tripleo-ansible) which generates config files for each service according to the settings here.

  • kolla_config: Contains YAML that represents how to map config files into the kolla container. This config file is typically mapped into the container itself at the /var/lib/kolla/config_files/config.json location and drives how kolla's external config mechanisms work.

  • docker_config: Data that is passed to paunch tool to configure a container, or step of containers at each step. See the available steps documented below which are implemented by TripleO's cluster deployment architecture. If you want the tasks executed only once for the bootstrap node per a role in the cluster, use the /usr/bin/bootstrap_host_exec wrapper.

  • puppet_config: This section is a nested set of key value pairs that drive the creation of config files using puppet. Required parameters include:

    • puppet_tags: Puppet resource tag names that are used to generate config files with puppet. Only the named config resources are used to generate a config file. Any service that specifies tags will have the default tags of 'file,concat,file_line,augeas,cron' appended to the setting. Example: keystone_config
    • config_volume: The name of the volume (directory) where config files will be generated for this service. Use this as the location to bind mount into the running Kolla container for configuration.
    • config_image: The name of the container image that will be used for generating configuration files. This is often the same container that the runtime service uses. Some services share a common set of config files which are generated in a common base container.
    • step_config: This setting controls the manifest that is used to create container config files via puppet. The puppet tags below are used along with this manifest to generate a config directory for this container.
  • container_puppet_tasks: This section provides data to drive the puppet containers tooling directly. The task is executed for the defined steps before the corresponding docker_config's step. Puppet always sees the step number overrided as the step #6. It might be useful for initialization of things. Note that the tasks are executed only once for the bootstrap node per a role in the cluster. Make sure the puppet manifest ensures the wanted "at most once" semantics. That may be achieved via the <service_name>_short_bootstrap_node_name hiera parameters automatically evaluated for each service.

  • global_config_settings: the hiera keys will be distributed to all roles

  • service_config_settings: Takes an extra key to wire in values that are defined for a service that need to be consumed by some other service. For example: service_config_settings: haproxy: foo: bar This will set the hiera key 'foo' on all roles where haproxy is included.

Deployment steps

Similar to baremetal containers are brought up in a stepwise manner. The current architecture supports bringing up baremetal services alongside of containers. For each step the baremetal puppet manifests are executed first and then any containers are brought up afterwards.

Steps correlate to the following:

Pre) Containers config files generated per hiera settings. 1) Load Balancer configuration baremetal a) step 1 baremetal b) step 1 containers 2) Core Services (Database/Rabbit/NTP/etc.) a) step 2 baremetal b) step 2 containers 3) Early Openstack Service setup (Ringbuilder, etc.) a) step 3 baremetal b) step 3 containers 4) General OpenStack Services a) step 4 baremetal b) step 4 containers c) Keystone containers post initialization (tenant,service,endpoint creation) 5) Service activation (Pacemaker), online data migration a) step 5 baremetal b) step 5 containers

Update steps:

All services have an associated update_tasks output that is an ansible snippet that will be run during update in an rolling update that is expected to run in a rolling update fashion (one node at a time)

For Controller (where pacemaker is running) we have the following states:
  1. Step=1: stop the cluster on the updated node;
  2. Step=2: Pull the latest image and retag the it pcmklatest
  3. Step=3: yum upgrade happens on the host.
  4. Step=4: Restart the cluster on the node
  5. Step=5: Verification: Currently we test that the pacemaker services are running.

Then the usual deploy steps are run which pull in the latest image for all containerized services and the updated configuration if any.

Note: as pacemaker is not containerized, the points 1 and 4 happen in deployment/pacemaker/pacemaker-baremetal-puppet.yaml.

Input Parameters

Each service may define its own input parameters and defaults. Operators will use the parameter_defaults section of any Heat environment to set per service parameters.

Apart from sevice specific inputs, there are few default parameters for all the services. Following are the list of default parameters:

  • ServiceData: Mapping of service specific data. It is used to encapsulate all the service specific data. As of now, it contains net_cidr_map, which contains the CIDR map for all the networks. Additional data will be added as and when required.

  • ServiceNetMap: Mapping of service_name -> network name. Default mappings for service to network names are defined in ../network/service_net_map.j2.yaml, which may be overridden via ServiceNetMap values added to a user environment file via parameter_defaults.

  • EndpointMap: Mapping of service endpoint -> protocol. Contains a mapping of endpoint data generated for all services, based on the data included in ../network/endpoints/endpoint_data.yaml.

  • DefaultPasswords: Mapping of service -> default password. Used to pass some passwords from the parent templates, this is a legacy interface and should not be used by new services.

  • RoleName: Name of the role on which this service is deployed. A service can be deployed in multiple roles. This is an internal parameter (should not be set via environment file), which is fetched from the name attribute of the roles_data.yaml template.

  • RoleParameters: Parameter specific to a role on which the service is applied. Using the format "<RoleName>Parameters" in the parameter_defaults of user environment file, parameters can be provided for a specific role. For example, in order to provide a parameter specific to "Compute" role, below is the format:

    parameter_defaults:
      ComputeParameters:
        Param1: value

Update Steps

Each service template may optionally define a update_tasks key, which is a list of ansible tasks to be performed during the minor update process. These are executed in a rolling manner node-by-node.

We allow a series of steps for the per-service update sequence via conditionals referencing a step variable e.g when: step|int == 2.

Pre-upgrade Rolling Steps

Each service template may optionally define a pre_upgrade_rolling_tasks key, which is a list of ansible tasks to be performed before the main upgrade phase, and these tasks are executed in a node-by-node rolling manner on the overcloud, similarly as update_tasks.

Upgrade Steps

Each service template may optionally define a upgrade_tasks key, which is a list of ansible tasks to be performed during the upgrade process.

Similar to the update_tasks, we allow a series of steps for the per-service upgrade sequence, defined as ansible tasks with a "when: step|int == 1" for the first step, "== 2" for the second, etc.

Steps correlate to the following:

  1. Perform any pre-upgrade validations.
  2. Stop the control-plane services, e.g disable LoadBalancer, stop pacemaker cluster and stop any managed resources. The exact order is controlled by the cluster constraints.
  3. Perform a package update and install new packages: A general upgrade is done, and only new package should go into service ansible tasks.
  4. Start services needed for migration tasks (e.g DB)
  5. Perform any migration tasks, e.g DB sync commands

Note that the services are not started in the upgrade tasks - we instead re-run puppet which does any reconfiguration required for the new version, then starts the services.

When running an OS upgrade via the tags system_upgrade_prepare and system_upgrade_run, or the combined tag system_upgrade, the steps corellate to the following:

  1. Any pre-service-stop actions. (system_upgrade_prepare)
  2. Stop all services. (system_upgrade_prepare)
  3. Post-service-stop actions like removing packages before the upgrade. (system_upgrade_prepare)
  4. Step reserved for the tripleo-packages service. Only package download for upgrade (under system_upgrade_prepare tag), and reboot for performing the offline upgrade (under system_upgrade_run tag) happens here.
  5. Any post-upgrade tasks (system_upgrade_run).

Nova Server Metadata Settings

One can use the hook of type OS::TripleO::ServiceServerMetadataHook to pass entries to the nova instances' metadata. It is, however, disabled by default. In order to overwrite it one needs to define it in the resource registry. An implementation of this hook needs to conform to the following:

  • It needs to define an input called RoleData of json type. This gets as input the contents of the role_data for each role's ServiceChain.
  • This needs to define an output called metadata which will be given to the Nova Server resource as the instance's metadata.

Keystone resources management

Keystone resources, such as users, roles, domains, endpoints, services, role assignments, are now managed by tripleo-keystone-resources Ansible role.


  1. See the override file which can be used to build Kolla packages that work with TripleO.↩︎