openstack-ansible/doc/source/upgrade-guide/manual-upgrade.rst

Upgrading manually

Deployers can run the upgrade steps manually. Manual upgrades are useful for scoping the changes in the upgrade process (for example, in very large deployments with strict SLA requirements), or performing other upgrade automations beyond what is provided by OpenStack-Ansible.

The steps detailed here match those performed by the run-upgrade.sh script. You can safely run these steps multiple times.

Check out the release

Ensure that your OpenStack-Ansible code is on the latest tagged release.

# git checkout

Prepare the shell variables

Define these variables to reduce typing when running the remaining upgrade tasks. Because these environments variables are shortcuts, this step is optional. If you prefer, you can reference the files directly during the upgrade.

From the openstack-ansible root directory, run the following commands:

# export MAIN_PATH="$(pwd)"
# export SCRIPTS_PATH="${MAIN_PATH}/scripts"
# export UPGRADE_PLAYBOOKS="${SCRIPTS_PATH}/upgrade-utilities/playbooks"

Bootstrap Ansible again

Bootstrap Ansible again to ensure that all OpenStack-Ansible role dependencies are in place before you run playbooks from the release.

# ${SCRIPTS_PATH}/bootstrap-ansible.sh

Change to the playbooks directory

Change to the playbooks directory so that the OpenStack-Ansible dynamic inventory is found automatically.

# cd playbooks

Preflight checks

Before starting with the upgraded version, perform preflight checks to ensure your environment is stable. If any of those checks fail, the upgrade should stop to let the deployer chose what to do.

Clean up old facts

Some configurations have changed, so purge old facts before the upgrade. For more information, see fact-cleanup-playbook.

# openstack-ansible "${UPGRADE_PLAYBOOKS}/ansible_fact_cleanup.yml"

Update configuration and environment files

The user configuration files in the /etc/openstack_deploy/ directory and the environment layout in the /etc/openstack_deploy/env.d directory have new name values added in . Update the files as follows. For more information, see config-change-playbook.

# openstack-ansible "${UPGRADE_PLAYBOOKS}/deploy-config-changes.yml"

Update user secrets file

introduces new user secrets to the stack. These secrets are populated automatically when you run the following playbook. For more information, see user-secrets-playbook.

# openstack-ansible "${UPGRADE_PLAYBOOKS}/user-secrets-adjustment.yml"

Clean up the pip.conf file

The presence of the pip.conf file can cause build failures during the upgrade to . This playbook removes the pip.conf file on all the physical servers and on the repo containers. For more information, see pip-conf-removal.

# openstack-ansible "${UPGRADE_PLAYBOOKS}/pip-conf-removal.yml"

Upgrade hosts

Before installing the infrastructure and OpenStack, update the host machines.

# openstack-ansible setup-hosts.yml --limit '!galera_all'

This command is the same setting up hosts on a new installation. The galera_all host group is excluded to prevent reconfiguration and restarting of any Galera containers.

Update Galera LXC container configuration

Update the Galera container configuration independently.

# openstack-ansible lxc-containers-create.yml -e \
'lxc_container_allow_restarts=false' --limit galera_all

This command is a subset of the host setup playbook, limited to the galera_all host group. The configuration of those containers is updated but a restart for any changes to take effect is deferred to another playbook (see the next section).

Perform a controlled rolling restart of the Galera containers

Restart containers one at a time, ensuring that each is started, responding, and synchronized with the other nodes in the cluster before moving on to the next. This step allows the LXC container configuration that you applied earlier to take effect, ensuring that the containers are restarted in a controlled fashion.

# openstack-ansible "${UPGRADE_PLAYBOOKS}/galera-cluster-rolling-restart.yml"

Update HAProxy configuration

Install and update any new or changed HAProxy service configurations.

# openstack-ansible haproxy-install.yml

Update repository servers

Update the configuration of the repository servers and build new packages required by the release.

# openstack-ansible repo-install.yml

Upgrade the MariaDB version

Upgrade MariaDB to the most recent 10.x minor release across the cluster.

# openstack-ansible galera-install.yml -e 'galera_upgrade=true'

Upgrade the infrastructure

The following commands perform all of the steps from the setup-infrastructure playbook, except for repo-install.yml, haproxyinstall.yml, and galera-install.yml which you ran earlier. Running these playbook applies the relevant settings and packages.

For certain versions of , you must upgrade the RabbitMQ service.

For more information, see setup-infra-playbook.

# openstack-ansible unbound-install.yml
# openstack-ansible memcached-install.yml
# openstack-ansible rabbitmq-install.yml -e 'rabbitmq_upgrade=true'
# openstack-ansible etcd-install.yml
# openstack-ansible utility-install.yml
# openstack-ansible rsyslog-install.yml

Flush Memcached cache

Flush all of the caches in Memcached. For more information, see memcached-flush.

# openstack-ansible "${UPGRADE_PLAYBOOKS}/memcached-flush.yml"

Upgrade OpenStack

Upgrade the OpenStack components with the same installation playbook, without any additional options.

# openstack-ansible setup-openstack.yml