A set of tools for Upgrading a TripleO environment
Go to file
mciecier 0fd72e2100 Skip ceph update run when update is not required
Improve ceph update run tasks by detecting if ceph update is required
to not start ceph update to the same version. Also exclude ceph images
from container check when update of those containers is not happening.

Change-Id: Idd4132c75208b635c9bf06e28457e02aeb314708
2023-02-28 15:05:39 +01:00
ci-scripts Migrate linting to pre-commit 2018-12-19 19:25:13 +01:00
defaults Check podman containers update during logs collection 2023-02-16 18:19:41 +01:00
doc Add doc/requirements 2021-01-05 11:05:38 +01:00
files RHOP 11 to 12 upgrade 2017-06-12 13:33:21 +02:00
filter_plugins Create entry for empty groups in inventory_rolemap 2022-10-28 13:03:35 +00:00
infrared_plugin Bump cirros verision to 0.6.0 in plugin.spec file 2022-10-17 10:33:31 +05:30
meta Remove ansible-core and molecule pins 2022-11-24 21:43:48 +05:30
molecule Remove ansible-core and molecule pins 2022-11-24 21:43:48 +05:30
releasenotes Update master for stable/zed 2022-11-02 16:31:08 +00:00
tasks Skip ceph update run when update is not required 2023-02-28 15:05:39 +01:00
templates Skip ceph update run when update is not required 2023-02-28 15:05:39 +01:00
tests Fix update tasks for Ephemeral Heat 2021-09-23 16:53:21 +02:00
zuul.d Moves the shared zuul queue tripleo to the project level 2022-06-23 14:54:27 +03:00
.ansible-lint Migrate linting to pre-commit 2018-12-19 19:25:13 +01:00
.gitignore Linters fix-refresh 2020-02-27 10:40:26 +00:00
.gitreview Linters fix-refresh 2020-02-27 10:40:26 +00:00
.pre-commit-config.yaml Remove additional dependency from pre-commit 2021-06-24 14:50:22 +02:00
.yamllint Migrate linting to pre-commit 2018-12-19 19:25:13 +01:00
LICENSE Initial commit 2017-05-31 15:43:01 +02:00
README.rst Add log collection during update. 2023-02-14 12:03:09 +01:00
ansible-requirements.txt Remove ansible-core and molecule pins 2022-11-24 21:43:48 +05:30
ansible.cfg Stop assuming a specific inventory structure 2020-04-01 00:32:10 +01:00
requirements.txt Bump test-requirements.txt 2021-06-18 16:13:57 +02:00
setup.cfg setup.cfg: Replace dashes with underscores 2021-04-27 15:47:22 +08:00
setup.py Disable auto discovery 2022-03-28 15:01:00 +05:30
test-requirements.txt Bump test-requirements.txt 2021-06-18 16:13:57 +02:00
tox.ini Changed minversion in tox to 3.18.0. 2022-08-01 12:44:40 +02:00



This role aims to provide a unified tool for upgrading TripleO based deploments.


This role requires:

  • An ansible inventory file containing reacheable undercloud and overcloud nodes

  • Nodes in the inventory file are placed in groups based on their roles (e.g compute nodes are part of the 'compute' group)

  • Repositories containing packages to be upgraded are already installed on undercloud and overcloud nodes (or, for overcloud, define an upgrade_init_command variable)

  • The initial overcloud deploy command is placed in a script file located in the path set by the overcloud_deploy_script var. Each option/environment file should be placed on a separate new line, e.g:

    source ~/stackrc
    export THT=/usr/share/openstack-tripleo-heat-templates/
    openstack overcloud deploy --templates $THT \
    -r ~/openstack_deployment/roles/roles_data.yaml \
    -e $THT/environments/network-isolation.yaml \
    -e $THT/environments/network-management.yaml \
    -e $THT/environments/storage-environment.yaml \
    -e ~/openstack_deployment/environments/nodes.yaml \
    -e ~/openstack_deployment/environments/network-environment.yaml \
    -e ~/openstack_deployment/environments/disk-layout.yaml \
    -e ~/openstack_deployment/environments/neutron-settings.yaml \
    --log-file overcloud_deployment.log &> overcloud_install.log

Role Variables

Available variables are listed below.

Only create upgrade scripts without running them:

upgrade_noop: false

Only create update scripts without running them:

update_noop: false

Run undercloud upgrade:

undercloud_upgrade: false

Run containerized undercloud upgrade:

containerized_undercloud_upgrade: false

Run overcloud upgrade:

overcloud_upgrade: false

Run undercloud update:

undercloud_update: false

Run overcloud update:

overcloud_update: false

Validate overcloud after update:

overcloud_images_validate: false

Location of the initial overcloud deploy script:

overcloud_deploy_script: "~/overcloud_deploy.sh"

Location of the undercloud upgrade script which is going to be generated by this role:

undercloud_upgrade_script: "~/undercloud_upgrade.sh"

Location of the upgrade script used in the composable docker upgrade step which is going to be generated by this role:

overcloud_composable_upgrade_script: "~/composable_docker_upgrade.sh"

Location of the upgrade script used in the converge docker upgrade step which is going to be generated by this role:

overcloud_converge_upgrade_script: "~/converge_docker_upgrade.sh"

Location of the undercloud credentials file:

undercloud_rc: "~/stackrc"

Location of the overcloud credentials file:

overcloud_rc: "~/overcloudrc"

Allows the user to apply known issues workarounds during the upgrade process. The list of patches/commands used for workarounds should be passed via --extra-vars and it should include dictionaries for undercloud/overcloud workarounds:

upgrade_workarounds: false

Set to true when the deployment has been done by tripleo quickstart:

use_oooq: false

Set to true to launch an instance before starting upgrade. This can be useful for running tests during upgrade such as live migration or floating IP connectivity checks:

workload_launch: false

Set to true to cleanup previously launched workload when update/upgrade finishes:

workload_cleanup: false

Name of the external network providing floating IPs for instance connectivity. This provides external connectivity and needs to exist beforehand, created by the user:

external_network_name: "public"

URL of the image used for the workload instance:

workload_image_url: "https://download.cirros-cloud.net/0.6.0/cirros-0.6.0-x86_64-disk.img"

Amount of memory assigned for the workload instance:

workload_memory: "512"

Set to true to use an SRIOV PF port when workload is created. Notice this will not work with cirros images:

workload_sriov: false

Set to true when running the role in the TripleO CI jobs. It avoids losing connectivity to the undercloud by skipping reboot and ssh kill tasks:

tripleo_ci: false

Bash commands, defines a custom upgrade init to be taken into account during overcloud upgrade:

upgrade_init_command: |
    sudo tripleo-repos -b pike current

Set it to true to get a multi-cell update. It changes the way the oc_roles_hosts is calculated:

update_cell: false

When set to true add a vm with attached fip and monitor ping from the undercloud. If ping loss time is higher than loss_threshold seconds or loss_threshold_percent in percentage we fail:

l3_agent_connectivity_check: false

For update run tasks we set a 0 seconds loss threshold by default:

update_loss_threshold: 0

Default time is 60 seconds for ping loss:

loss_threshold: 60

Failsafe percentage check for loss threashold in percentage:

loss_threshold_percent: 1

Set to true to enable validations:

updates_validations: true

Enable extra logs during update. Default to true. It collects in /var/log/extras/ the output of the commands from collect_logs.yaml for every stages of the update run:

log_stages: true



Example Playbook

An example playbook is provided in tests/test.yml:

- hosts: undercloud
  gather_facts: true
  become: true
  become_method: sudo
  become_user: stack
    - tripleo-upgrade

Usage with tripleo Quickstart

After a successful deployment with OOOQ, you can create the necessary scripts using this example playbook (duplicate from ./tests/oooq-test.yaml):

- hosts: undercloud
  gather_facts: true
  become: true
  become_method: sudo
  become_user: stack
  - { role: tripleo-upgrade, use_oooq: 'true'}

And then you run it like this (adjust the paths to your oooq specific one):

ANSIBLE_SSH_ARGS="-F $(pwd)/ssh.config.ansible" \
  ANSIBLE_CONFIG=$PWD/ansible.cfg \
  ansible-playbook -i hosts -vvv tripleo-upgrade/tests/oooq-test.yaml
This will only create the file (without running the actual upgrade):
  • undercloud_upgrade.sh
  • composable_docker_upgrade.sh
  • overcloud-compute-*_upgrade_pre.sh
  • overcloud-compute-*_upgrade.sh
  • overcloud-compute-*_upgrade_post.sh
  • converge_docker_upgrade.sh

with the correct parameters.

Usage with InfraRed

tripleo-upgrade comes preinstalled as an InfraRed plugin. In order to install it manually, the following InfraRed command should be used:

infrared plugin add tripleo-upgrade
# add with a specific revision / branch
infrared plugin add --revision stable/rocky tripleo-upgrade

After a successful InfraRed overcloud deployment you need to run the following steps to upgrade the deployment:

Symlink roles path:

ln -s $(pwd)/plugins $(pwd)/plugins/tripleo-upgrade/infrared_plugin/roles

Set up undercloud upgrade repositories:

infrared tripleo-undercloud \
    --upgrade yes \
    --mirror ${mirror_location} \

Set up undercloud update repositories:

infrared tripleo-undercloud \
    --update-undercloud yes \
    --mirror ${mirror_location} \
    --build latest \
    --version 12 \

Upgrade undercloud:

infrared tripleo-upgrade \
    --undercloud-upgrade yes

Update undercloud:

infrared tripleo-upgrade \
    --undercloud-update yes

Set up overcloud upgrade repositories:

infrared tripleo-overcloud \
    --deployment-files virt \
    --upgrade yes \
    --mirror ${mirror_location} \

Set up overcloud update repositories/containers:

infrared tripleo-overcloud \
    --deployment-files virt \
    --ocupdate True \
    --build latest \

Upgrade overcloud:

infrared tripleo-upgrade \
    --overcloud-upgrade yes

Update overcloud:

infrared tripleo-upgrade \
    --overcloud-update yes

Advanced upgrade options

Operator can now specify order of roles to upgrade by using roles_upgrade_order variable.

It's the responsibility of operator to specify Controller role first followed by all other roles.

roles_upgrade_order variable expects roles being separated by ;(semicolon), for e.g.:

infrared tripleo-upgrade \
    --overcloud-upgrade yes \
    -e 'roles_upgrade_order=ControllerOpenstack;Database;Messaging'

will upgrade ControllerOpenstack group, then Database and finally Messaging.

Multiple roles could be upgraded in parallel, to achieve this they should be separated by ,(comma), for e.g:

infrared tripleo-upgrade \
    --overcloud-upgrade yes \
    -e 'roles_upgrade_order=ControllerOpenstack,Database;Messaging'

will upgrade Controller and Database groups in parallel and then continue with Messaging.

Running the role manually from the undercloud

This role can be run manually from the undercloud by doing the following steps:

Note: before starting the upgrade process make sure that both the undercloud and overcloud nodes have the repositories with upgraded packages set up

Clone this repository

git clone https://opendev.org/openstack/tripleo-upgrade

Set ansible roles path::


Create inventory file::

printf "[undercloud]nlocalhost ansible_connection=local" > hosts

Run the playbook including this role::

ansible-playbook -i hosts tripleo-upgrade/tests/test.yml


Apache License 2.0

Author Information

An optional section for the role authors to include contact information, or a website (HTML is not allowed).