Commit Graph

49 Commits (master)

Author SHA1 Message Date
Jonathan Rosser a831e4b6c1 Fix ansible_ssh_extra_args extra newline
This breaks the use of the ansible synchronize module
when the parameter use_ssh_args is true with an error
from ssh via rsync that there is an unknown parameter.

Removing the newline makes the synchronize module
work correctly.

Change-Id: Ib7fc3068ecc339e02d641196513c1b676a9a9f69
2023-08-31 15:58:02 +00:00
Damian Dabrowski b75a9d0dd0 Implement support for haproxy_accept_both_protocols
Enabling TLS on the internal VIP for existing deployments will cause
downtime until each client is configured to use HTTPS instead of HTTP.

To avoid downtime, it is recommended to enable
`openstack_service_accept_both_protocols` until all services are
configured correctly.
It allows haproxy frontends to accept both HTTP and HTTPS.

Change-Id: Ie6f5b73c54b0a6d1f661a9d4f33b8a301d8c4170
2023-05-26 00:33:18 +02:00
Dmitriy Rabotyagov 86d1bdff55 Rename container_address to management_address
This patch aims to reduce confusion caused by a variable
`container_address` that's applicable for bare metal hosts. With that
it renames  `is_container_address` to `is_management_address`
to be aligned with the purpose of the variable, as `container` part
raised confusion.

Change-Id: I314224f3376cf91e05680b11d225fdaf81ec32ab
2023-05-22 09:57:17 +02:00
Damian Dabrowski e9445504f4 Add support for TLS backends
This patch allows haproxy to communicate with service backends over TLS.

It's disabled by default and each service role needs to have TLS backend
support implemented to get it working.

For example, TLS support for glance was added in [1]



Change-Id: I5fc507f4031dcf63ed95dae307c30d9f436ef3da
2023-04-25 15:24:24 +02:00
Jonathan Rosser df4758ab1b Allow git servers for openstack services and tempest to be overridden
This will allow the services to be cloned from github or a local
mirror by setting a small number of variables rather than overriding
every git repo URL individually.

Change-Id: I750d897e9e3c8ca161c0740c73cdc4c6e42b6440
2023-02-01 14:57:14 +00:00
Dmitriy Rabotyagov bc5428b21d Remove usage of rsyslog roles
We've switched all services to store logs to journald by default and
rsyslog roles are not used except really small amount of usecases that
also hardly valid as of today. With that we deprecate repos and remove
their usega to reduce maintenance load.

Change-Id: Iefd4143f83f4df44b917180000a1aa57161b2811
2022-10-19 15:10:59 +02:00
Andrew Bonney 0cc8e039ea rabbitmq: default to using TLS for management user interface
The RabbitMQ role defaults change in Yoga to enable the management
UI with TLS/HTTPS. This implicitly disables the HTTP port.

This commit adjusts the HAProxy config to take account of this
change and switch the port used accordingly. The
rabbitmq_management_ssl variable is also set explicitly to ensure
it is defined with appropriate scope.

Change-Id: I5a9f9855aa701d12bc3c9e2e7e9c651ff606c319
2022-07-27 13:54:38 +01:00
Dmitriy Rabotyagov b515d895e1 Cleanup distro vars
Since we have introduced service_install_method variable across all
repos we can leverage it instead of defining dozen of other variables
for same purpose.

Change-Id: Ie2e797de51a826b774e796a8e3fdc234e712f71b
2022-04-20 14:34:45 +00:00
Jonathan Rosser f1ed3f5dc9 Remove references to unsupported operating systems
All references to Gentoo, SUSE, Debian stretch and Centos-7  are removed.
Conditional tasks, ternary operators and variables are simplified where possible
OS specific variables files are generalised where possible

Change-Id: I42e40adc20a122ea5bba292b1f6fea9e79b3c365
2022-01-13 19:45:26 +02:00
Zuul 8b57e091f8 Merge "Globally define systemd_lock_dir" 2021-12-01 13:33:00 +00:00
Dmitriy Rabotyagov fbbf1e275d Globally define systemd_lock_dir
Change-Id: I4659c1afcc0b783fdb9ba65b61d7d37b795db9a5
2021-11-25 14:51:04 +00:00
Dmitriy Rabotyagov 678b14c21a Do not upgrade packages without upgrades
Package upgrade during normal operations might be risky since often
involve service restarts. It's better to avoid that when possible and
perform package upgrade only during major/minor upgrades. We change
default value of package_state and adjust doc and upgrade script to
reflect this change.

Change-Id: I9971a259a207b263df48a77d5ac74752f044873a
2021-11-23 17:09:25 +02:00
Andrew Bonney a332bda378 Fix permissions for files created on repo server
If wheels or requirements files are created on a secondary
repo server these cannot be modified/deleted by the primary
repo server when it comes back online as this uses rsync as the
'nginx' user. This patch allows the owner of the created files
to be set explicitly to avoid this issue.

Change-Id: Ifdbac876013068855cd4405ca1bbacd262ec4d74
2021-08-10 13:25:40 +01:00
Zuul 90d16a7325 Merge "Cleanup after service variables merged" 2021-06-03 19:39:35 +00:00
Dmitriy Rabotyagov aab510e1fc Cleanup after service variables merged
Change-Id: If1278c7cc167305c148d01b3db972acb3afa4fd1
2021-06-02 08:17:30 +00:00
Dmitriy Rabotyagov f89d87c4b5 Gather additional required facts to min
With minimal facts gathering we will experience failures while trying to
evaluate amount of threads for apis or mounts for hardening. So to avoid
gathering full hardware subset, we add common-task which will gather
only specific subset that can be additionally filtered.

Gathering processor or mounts subsent simply does not work with ansible
and result in full hardware subset as well.

Change-Id: Ia5802b4ec0b18271b8c5fbcc5574b484c5233a01
2021-06-01 09:29:22 +00:00
Zuul 0adc8fcf1c Merge "Use ansible_facts[] instead of fact variables in group_vars" 2021-04-06 21:01:06 +00:00
Jonathan Rosser 6c94ac8515 Use ansible_facts[] instead of fact variables in group_vars

Change-Id: I06ab9cee6539762d7ba3e25e07b5661a8fa485f3
2021-03-27 11:48:56 +00:00
Jonathan Rosser 63fdf7a41e Remove group var overriding uwsgi_python_executable
This now defaults to python3 in the uwsgi ansible role so does not
need to be overridden

Change-Id: I511987b8d0026bd6737fdb1a0a4af680b137e902
2021-03-22 17:32:37 +00:00
Jonathan Rosser a05e5de565 Remove Centos-7 support
Ussuri is the last release that openstack-ansible will support Centos-7.
Remove this for the Victoria release.

Change-Id: Ief6cbb2d02bfdc4e6c3c0fbb0cc9ab5a3eab98c3
2020-10-16 15:51:59 +00:00
Jonathan Rosser f4510b6baa Bind services to mgmt network addresses
These addresses are given defaults of in the role defaults
but in a deployment we know which address each service should bind to.

The variable container_address should hold the local mgmt network IP
address for either containerised or metal deployments.

This patch defines a new variable management_address which removes
confusion about container_address for metal deployments, and overrides
the role default bind addresses for all roles.

Change-Id: I7471ff1da9602f67134e217f5427e492fa7a7814
2020-09-24 10:03:22 +01:00
Logan V e9c33c1c00 Fix default openstack_deploy dir evaluation
The default is broken because lookup from a missing env var returns
an empty string. Unless the 2nd parameter 'true' is passed to
default, it only defaults undefined variables. An empty string is
not undefined, so the default never works.

In group_vars/all/all.yml, it doesn't matter that the default never
works, because openstack-ansible.rc always sets OSA_CONFIG_DIR.

In get-ansible-role-requirements.yml, it makes a difference, because
the first time this playbook runs during bootstrap,
openstack-ansible.rc will not be loaded yet. This means the default
user roles path is always going to be '/user-role-requirements.yml'
instead of '/etc/openstack_deploy/user-role-requirements.yml'.

Change-Id: Id9e53958daf9494e5baf9cd1e34ed6f83eefa5f4
2020-05-17 12:52:15 -05:00
Dmitriy Rabotyagov d7a05389c3 Use py3 for venvs for all OS
Change-Id: Iab09ec3b79cd2a82c8887aa70a49b82a89ea184c
2019-12-20 19:01:46 +00:00
Dmitriy Rabotyagov be89da6a13 Make upgrade script respect OSA_CONFIG_DIR
This also adds variable `openstack_clone_root` which equals to
environment variable $OSA_CLONE_ROOT and defaults to `/opt/openstack-ansible`.

Change-Id: I9e0c980f80980c389e27d5b54077b08a596a33f4
2019-12-09 13:41:29 +02:00
Jonathan Rosser 3c076ed645 Use python3 source installs of OpenStack services where possible
Detect CentOS and deploy from source as python2, otherwise default to
python3 on all other distros.

Change-Id: Ic68dc0923a26ece68b1971648c62bb2e0c05c50b
2019-09-11 20:53:19 +00:00
Logan V c06e6b5885 Move hosts file management toggle to all group vars
In Queens, the hosts file management began occurring on all hosts
and containers because openstack_hosts role now runs on all inventory
items (prior to Queens, it only ran on 'hosts'). The flag for managing
the /etc/hosts management should now be moved to the 'all' group

Change-Id: I0a9e8e86cd1eb40f55d40eb291f49563a433fd82
2019-05-29 12:25:21 -05:00
Jesse Pretorius 0635dfbccc Ensure that global-requirement-pins.txt is applied
In the previous repo build process, we had global constraints which
override upper constraints and anything set in the roles. This was
essential for two purposes:

1. To enable us to pin things that were not in upper constraints. eg: pip,
   setuptools, wheel
2. To enable us to pin things which were in upper constraints, but broken.
   This would usually be a temporary measure until upper constraints was

This patch extracts the global pins from global-requirement-pins.txt
into a list and sets 'venv_build_global_constraints' to the resulting
list so that it is applied to all venvs built.

In order to reliably find this file without using a hard-coded path, we
implement a change to the wrapper script to set the path in it in a
similar manner to that implemented for the inventory path.

Depends-On: I9ae3ef19c863b9237a51d2fcd6f4ebce1a9ebad7
Change-Id: I138fe1c8ea80fe71244ab0dc6497cfc6d7bdf953
2019-04-17 15:15:22 +01:00
Mohammed Naser acae8b5c0f config: Enable OSA_CONFIG_DIR to customize configuration folder
We have some basics wired up in order to use a different config.
directory for running OpenStack Ansible.  However, a lot of those
are implemented as configuration argument to the dynamic inventory
script but that's impossible to pass normally when running Ansible.

This patch creates a new environment variable called OSA_CONFIG_DIR
which allows you to customize the directoy of your openstack_deploy
folder which means that you can use a deploy host for several diff.

Change-Id: Ia528ff449330fbb04be6cd0d03353fa3158e6694
2019-04-03 15:43:41 -04:00
Jean-Philippe Evrard 7e0d2e5e5c Use an env lookup to determine the OSA version
This is about as fast as the current static code, and doesn't
require bumping at every release.

Change-Id: I75657c6dae2c6246ec2513f4ec452a4c354d638b
2019-02-12 10:16:02 +00:00
Jesse Pretorius 43a5c874ef Remove apt-cacher-ng
The repo container's package cache causes quite a bit of confusion
given that it's a 'hidden' feature which catches deployers off-guard
when they already have their own cache configured. This is really
the kind of service which people should manage outside of OSA. It
also makes no sense if the deployer is using their own local mirror
which is a fairly common practise. Adding to that, it seems that it
is broken in bionic, causing massive delays in package installs.
Finally, it also adds to quite a bit of complexity due to the fact
that it's in a container - so in the playbooks prior to the container's
existence we have to detect whether it's there and add/remove the config

Let's just remove it and let deployers managing their own caching
infrastructure if they want it.

Change-Id: Ia0fb41266a6d62073b02c8ad6fa97b8b7d408a67
2018-10-08 17:14:10 +00:00
Jean-Philippe Evrard dc421891ff Prepare master branch for M1
With branching done earlier than the rc1 freeze, we needed
to bring the release notes from the updated roles into the
integrated repo.

On top of the changes needed for M1 is the version change.

The openstack SHAs will be updated in a separate commit.

Change-Id: I4d5211062966bc4fdb9f5d36eec46d9911012acd
2018-08-10 12:55:12 +00:00
Markos Chandras 4603188934 Add support for using distribution packages for OpenStack services
Add new 'aio_distro_basekit' jobs to test the minimal basekit deployment
using distribution packages for the OpenStack services.

We can skip all repo-* related playbooks and roles since we are not
building pip packages for OpenStack services anymore. Finally, we can
populate the utility container using the distribution packages for the
OpenStack client instead of using the wheel packages.

Change-Id: Ia8c394123b5588fff8c4acbe1532ed5a6dc7e8ec
Implements: blueprint openstack-distribution-packages
2018-07-20 08:14:32 +01:00
Zuul 4613c2d070 Merge "Remove not needed glance variables" 2018-07-12 14:09:00 +00:00
Jean-Philippe Evrard 1e4121f5a4 Allow inventories with no "properties"
Without this patch, inventories with hosts without the field
"properties" cannot be used. Currently, the group variable
resolution will fail when parsing properties.is_metal, as
properties wouldn't be defined.

This is a problem, because it forces deployers with an external
inventory to add a useless "properties" field for them.

This patch solves the problem by simplifying the conditions,
making them valid if the deployer has defined is_metal in its
inventory or not.

Change-Id: Ic09b455016dd3d4f7945900d13d37fdc86f80134
2018-07-06 09:56:47 +00:00
Jean-Philippe Evrard f253b6c3b7 Unfreeze master
Now that master has a patch for releasing milestone 2, we can
unfreeze the ansible-role-requirements to allow role iterations.

The version of OSA has been bumped at the same time to be ready
for the next freeze.

Change-Id: Ibfd14d2113ff41d8548551881846cdfaa927d6d6
2018-06-07 18:18:31 +00:00
Albert Mikaelyan f6a61909f0 Remove not needed glance variables
As part of cleaning inventory variables, we are
moving some of the variables to be created in
roles instead of the inventory, and removing
variables used only to create other variables and
nowhere else.

Change-Id: I8dfd1904de36ca2b9f163bd5447bfa40ddf4ca3a
Depends-On: I9a00e3bd5dca4581cb43bf298c4d792375b5252f
Depends-On: Icc0606ed948e3596d037b9159602e7b7b06311d8
Depends-On: I983a20d923384bf54cb0af924ec0a0f8ef4db191
2018-05-28 16:49:08 +03:00
Kevin Carter 03956a9274 Convert rsyslog to an include_task
The rsyslog role has served us well however there's now a better way
given the ability to remotely journal. This change disables the use of
the `rsyslog_client` and `server` roles unless journal remote is
unavailable. In this change the `rsyslog_client` interaction has been
moved into an include_tasks which is conditionally loaded when the
variable `rsyslog_client_enabled` is set to true. Additionally the
`rsyslog_server` role is disabled unless `rsyslog_server_enabled`
is set true. Using the new variables legacy functionality can be
enabled. By disabling the general rsyslog roles we'll lessen the
overall IO on the cloud and improve the speed of the deployment.

> NOTE: At this time there's no suitable package to install
        "systemd-journal-remote" on opensuse so a conditional has been
        added to ensure distributed log syncing remains functional on
        all of our supported distros. Should a package be made
        available for journal remote we can globally disable the
        legacy rsyslog roles entirely by default.

Change-Id: Ice21667c6999d0ac86b2d7bde648a0375f890210
Signed-off-by: Kevin Carter <>
2018-05-17 07:50:17 +00:00
Jesse Pretorius 2539195e88 Revert role freeze and update openstack_release for Rocky m2
In order to continue testing from the head of the role repositories
to prepare for milestone 2, we revert the role freeze and set the
release to

Change-Id: I35b2eef362a0a53f2236cd2dab11778e44c85d48
2018-04-30 17:14:51 +01:00
Jean-Philippe Evrard 0eb50f8ce3 Update all SHAs for master
This patch updates all the roles to the latest available stable
SHA's, copies the release notes from the updated roles into the
integrated repo, updates all the OpenStack Service SHA's, and
updates the appropriate python requirements pins.

Change-Id: I76277f47d9216dde91dac7248bb1adbf3805af20
2018-04-13 07:57:21 +00:00
Kevin Carter 717462996a Add playbook to ship journals from hosts
The journal within systemd is able to be shipped from a physical hosts
to a centralized location. This change introduces
`systemd-journal-remote` which will ship all journals on the physical
host to the log host and store the journals under
"/var/log/journal/remote". This change gives deployers greater
visability into the cloud using the systemd built-ins.

> NOTE: This change is all accomplished in a playbook using our common
        roles. While this could be moved into a role by itself, it would
        be a waist of effort given how small this change is.

Given all services are inherently logging to the journal, this change
may allow us to one day deprecate or minimize the usage of our
rsyslog roles. If we were to remove the requirement for rsyslog to run
everywhere we could reduce overall internal cluster IO (CPU, network and
block) and remove the requirement for all services to ship log files from
all containers and hosts. This change is NOT modifying the integrated
logging architecture. At this time we're simply ensuring that the
journals on the physical host are co-located on the logging machines.

At this time there's no suitable package available for
systemd-journal-remote on suse so the playbook to install and setup
remote journalling is being omitted when the suse is detected. When a
suitable package is found the playbook omission should be removed.

Change-Id: I254d52df6303b7cc4d4071b4beaf347922b2616e
Signed-off-by: Kevin Carter <>
2018-04-06 00:12:21 +00:00
Markos Chandras a8b7f7c8c9 inventory: all: Switch package state to 'present' on openSUSE
Similar to the CentOS case, we should use 'present' instead of 'latest'
for the package state so we don't spend time resolving dependencies,
querying repos etc. The 'present' state can be further improved either
in the zypper itself or in the Ansible module.


Change-Id: I8a6598013714e468d19f4984288d2c746515b5ca
2018-03-14 16:06:42 +00:00
Jesse Pretorius c068952a5b Switch openstack_version back to 'master'
Change-Id: I30c56ae696ec29db243a25959523b829a6c76600
2018-03-01 10:28:50 +00:00
Major Hayden f4bc81cff9 CentOS 7 integrated gate optimization
The CentOS 7 AIO jobs time out fairly frequently in the gates and
this patch switches the jobs to use the new aio_basekit scenario in
the hopes that jobs will complete more often.

This patch also sets SELinux to Permissive mode for CentOS 7 gate
testing. This can be changed back to Enforcing when CentOS 7's
SELinux policies are ready.

This patch also sets package_state to present for CentOS 7 jobs. This
is a temporary measure to ensure more of the gate jobs can finish on
time and we can expose deployment issues in the gates.

Change-Id: Ic478a7402fee3df0667f0d6fd9a714688808b2ec
2018-02-23 13:02:26 +00:00
Kevin Carter fd9cda8df9
Add nspawn container driver
This change adds an nspawn container driver which will enable deployers
to run clouds with systemd-nspawn instead of LXC. This adds "nspawn" to
as an option to the `container_tech` variable. To support this change,
The inventory generation tools have been updated to allow for a
new group named `nspawn_hosts`. All of the container connectivity and
setup are stored within the integrated repo under the new templates

The addition of "nspawn" container driver enables the ability for
deployers to change, or mix container technologies within a single
deployment without needing to change our well defined network
topology or storage layout.

Depends-On: I13d05ba8bcfe785257a9cf98dbdb6024ec937816
Change-Id: I41cfec63c423cd56a91c25dabae9aa1031c27e03
Signed-off-by: Kevin Carter <>
2018-02-11 19:02:24 -06:00
Jean-Philippe Evrard 68fa3f4597 Update all SHAs for 17.0.0
This patch updates all the roles to the latest available stable
SHA's, copies the release notes from the updated roles into the
integrated repo.

The OpenStack services SHA's will be manually updated in another

Change-Id: I78d85f33926bc979f7d0df0adc8e8245bf1223ad
2018-02-10 17:50:34 +00:00
Jean-Philippe Evrard 8f50c671a7 Unfreeze roles after milestone 3
Now that milestone 3 is release, we can move back to using master
for the roles, to see fast changes.

Change-Id: Idce198cf45a49857b64cc13a0a7476b2d4e632ae
2018-01-30 08:05:02 +00:00
Jean-Philippe Evrard ee6ff67765 Update all SHAs for
This patch updates all the roles to the latest available stable
SHA's, copies the release notes from the updated roles into the
integrated repo, updates all the OpenStack Service SHA's, and
updates the appropriate python requirements pins.

Change-Id: I5fea16a84c881d0e66b713cdb55c8dcabcb80491
2018-01-29 08:03:39 +00:00
Jesse Pretorius (odyssey4me) cf9dcbcdef Revert "Freeze all SHAs for"
In order to continue Queens development, the role
SHA pins are removed.

Change-Id: I77945703fc8ed511a7d6dafe4f37ef004897f6b9
2017-12-18 10:38:04 +00:00
Jimmy McCrory c5551f2c8c Move inventory files to folder in root of repo
Move the playbooks/inventory folder, group_vars, and host_vars to
inventory/ in the root of the OpenStack-Ansible repo. This helps better
organize the repo structure since playbooks/ will now only contain
playbooks, shared task files, and included repo package var files.

group_vars and host_vars are moved alongside the inventory since that's
the default place that Ansible expects those folders and to help better
prepare for Ansible 2.4 where multiple inventories can be loaded,
automatically including relative group and host var files.

Effected docs, scripts, and variables have been updated with the new

Change-Id: If50e2412c3fd6575d7041deb8ecc9480b04184cc
2017-12-16 02:34:33 -08:00