- Implemented new variable ``connection_recycle_time`` responsible for SQLAlchemy's connection recycling
- Set new default values for db pooling variables which are inherited from the global ones.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/819424
Change-Id: I28c64b44eadfd726e07cb7159e5d3d94fde106ed
With PKI role in place in most cases you don't need to explicitly
provide path to the CA file because PKI role ensures that CA is trusted
by the system overall. In the meanwhile in PyMySQL [1] you must either
provide CA file or cert/key or enable verify.
Since current behaviour is to provide path to the custom CA we expect
certificate being trusted overall. Thus we enable cert verification when
galera_use_ssl is True.
[1] 78f0cf99e5/pymysql/connections.py (L267)
Change-Id: Ic5b072d983c6d553d996a0a3bd708eec4c2137e5
Instead of overriding each service separatelly it might make
sense for deployers to define some higher level variable that
will be used first or fallback to default variable.
Change-Id: If3627990a028b4c28c85609f5bf1341b836b0ccd
Move it to the service setup host (defaults to utility[0]) instead
of the galera[0] host, and use galera_address (defaults to internal VIP)
as the endpoint instead of a local connection on the db host.
Change-Id: Ia09330b907387f70db5f553f83d5a23a4f7f8a98
This patch aims to add a prefix for memcached_server
on each role to give the ability for deployers to
override the location of memcached cluster. I.e users
wants to create a single memcached cluster with k8s
for each service.
We also add pymemcache based on [1]
[1] https://review.opendev.org/711429
Change-Id: I152220d4c440202de1a61b1aee891bdb659e5577
Install the python3-designate package and rely on the OSA provided
service configs instead of installing each of the individual designate
service packages. The 'Mask any conflicting units' task can also be
removed since the installation of unwanted services will no longer be an
issue.
Change-Id: I40a596803df656f5e6f75996660fe9ace685c2ed
The variables barbican_developer_mode and barbican_venv_download
no longer carry any meaning. This review changes glance to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Change-Id: Ifd7d4d0a314834165141019dbacb82bbc1c4176d
This change broke the admin URL because there is no variable
with the value designate_service_adminurl.
This reverts commit 2950db130db08b4c6662551d0d6e24ac8ba0bb1a.
Change-Id: I8acb48e8a3fe9721b97be8a87c7be28974642e72
This patch add the conditional inclusion of the notification
section of the service configuration. This ensures that oslo.messaging
notifications use the correct transport for deployments that have
separate rpc and notify messaging backends. For example, if the
transport_url is not provided in the notification section of the
service configuration, the transport_url specified in the default
section will be used instead.
This patch conditionally selects the notifier driver. The noop
driver will be selected when notification publishing is disabled.
The messagingv2 driver is selected when notification publishing is
enabled.
Change-Id: Ie5cc5499980f986f1a7e530adf42f4dcc43fbaca
Closes-Bug: #1794320
This change allow deployer to set project that will be owner of managed
resources like auto-created records and zones.
The owner is specified using project name and defaults to service
tenant.
Depends-On: https://review.openstack.org/628979
Change-Id: I620be82d890aaa547decc59f81f55345f7177900
In order to enable the service setup host python interpreter to
be changed easily, we make it a variable. This will be useful
when someone sets the service setup host to be the utility
container, because we'll be able to set this var by default.
Change-Id: Id70abddc0ba2a4086406d35162d9aca0b526f196
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: I29833c41eba9b3ff80200b7f567a120f589e170e
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed.
The systemd journal would normally be populated with the standard out of
a service however with the use of uwsgi this is not actually happening
resulting in us only capturing the logs from the uwsgi process instead
of the service itself. This change implements journal logging in the
service config, which is part of OSLO logging.
OSLO logging docs found here: <https://docs.openstack.org/oslo.log/3.28.1/journal.html>
Change-Id: I9764f557007d97cfcbe02abf7166cce423b39a31
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When the RPC and Notify service are the same, the credentials
must match - otherwise the tasks to create the user/password
will overwrite with each other.
If the two clusters are different, then the matching credentials
and vhost will not be a problem. However, if the deployer really
wishes to make sure they're different, then the vars can be
overridden.
Also, to ensure that the SSL value is consistently set in the
conf file, we apply the bool filter. We also use the 'notify'
SSL setting as the messaging system for Notifications is more
likely to remain rabbitmq in our default deployment with qrouterd
becoming the default for RPC messaging.
Change-Id: Id94e6337032807ee5baa79945e043c9282e17263
There is no record for why we implement the MQ vhost/user creation
outside of the role in the playbook, when we could do it inside the
role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement two new variables:
- designate_oslomsg_rpc_setup_host
- designate_oslomsg_notify_setup_host
These are used in the role to allow delegation of the MQ vhost/user
setup for each type to any host, but they default to using the first
member of the applicable oslomsg host group.
We also adjust some of the defaults to automatically inherit existing
vars set in group_vars form the integrated build so that we do not
need to do the wiring in the integrated build's group vars. We still
default them in the role too for independent role usage.
Change-Id: I65203372e551d5ab3ef17a823f2089c23a2af8fe
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. As there are no required packages left, the task
to install them is also removed.
With the dependent patches, the openstack_openrc role is now executed once
on the designated host, so it is no longer required as a meta-dependency for
the role.
Depends-On: https://review.openstack.org/579233
Depends-On: https://review.openstack.org/579959
Change-Id: I155b5036dd98f80a0d31d8bb691beae147a97bc3
There is no record for why we implement the database creation outside
of the role in the playbook, when we could do it inside the role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement a new variable called 'designate_db_setup_host'
which is used in the role to allow delegation of the database setup
task to any host, but defaults to the first member of the galera_all
host group. We also document the variable 'designate_galera_address' which
has been used for a long time, but never documented. A bunch of unused
variables have also been removed.
Change-Id: Ie0d09401adaa66142fe12d1b6454641dd76104b7
The following packages are required in-order to run osprofiler.
these packages will provide deployers the ability to profile
a service on demand should they choose to enable the profile
functionality.
Depends-On: I3df2c670beeb78baaa1515bcd27e8f2b0d95b3a9
Change-Id: Ibe97e34f8e65948882f8c93f0852460cc53eb231
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This introduces oslo.messaging variables that define the RPC and Notify
transports for the OpenStack services. These parameters replace the
rabbitmq values and are used to generate the messaging transport_url for
the service. The association of the messaging backend server to the
oslo.messaging service will be transparent to the designate service.
This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Update transport_url generation (add for notification)
* Add oslo.messaging to tests inventory
* Update tests
* Add releaes note
* Update README and example playbook
Change-Id: I620a13e1ea3c24c8bd31c02206613d37d769dd30
Distributions provide packages for the OpenStack services so we add
support for using these instead of the pip ones.
Change-Id: Iab155254fa3600e1b6b3e8992f7c1fdc6f710ed6
Implements: blueprint openstack-distribution-packages
In order to clean variables in designate, the uri variable is removed,
and we filter url variable as needed
Change-Id: I537c963722553d798356287de3daefcad99ac62f
virtualenv-tools has a bug which gets triggered in gates: it can't
change the shebang of a virtualenv python bin/ files if they
were generated with a virtualenv script whose shebang ends with
python2 instead of python.
Because we can't modify virtualenv-tools, we use shell scripts
instead.
Change-Id: I7351e9e7e842b6de4b3e841ee9e44b6b6c514522
Partial-Bug: #1741634
When 'designate_galera_use_ssl' is True, use an encrypted connection to
the database using either a self-signed or user-provided CA certificate.
A new non-voting test has been added to verify that the role remains
functional when enabling SSL features.
Change-Id: I0d8e3b685faa8d394fd56f8fbfd9b492d2c2cb60
Partial-Bug: 1667789
Use openstack service proto variable in defaults for designate endpoints
This will align the designate role with other roles method
Partial-Bug: 1717273
Change-Id: I033940c29e91945a96bf9a4a9c9058badac37c5e
The keystonemiddleware library recently switched to using the
cryptography library over pycrypto, which was unmaintained. See
Iced7f5115e49ccf4f7f5bf6813cb5988b95c248b
Change-Id: I7cb31ead9146e8abf02c8ff01188dc69c069607d
Co-Authored-By: Nolan Brubaker <nolan.brubaker@rackspace.com>
This creates a specific slice which all OpenStack services will operate
from. By creating an independent slice these components will be governed
away from the system slice allowing us to better optimise resource
consumption.
See the following for more information on slices:
* https://www.freedesktop.org/software/systemd/man/systemd.slice.html
See for following for more information on resource controls:
* https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html
Tools like ``systemd-cgtop`` and ``systemd-cgls`` will now give us
insight into specific processes, process groups, and resouce consumption
in ways that we've not had access to before. To enable some of this reporting
the accounting options have been added to the [Service] section of the unit
file.
Change-Id: I5885643199db3ef618fc86f0cd80c14f1d7c89c4
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
designate-pool-manager and designate-zone-manager are deprecated
in favor of designate-worker and designate-producer. This enables
those services.
This does not functionally change the way Designate works, so the
associated config changes are minimal. This does remove some
cumbersome pool manager cache configuration that is no longer
needed, but wasn't being used anyway. It also simplifies the
Designate architecture by making the separation of duties easier
to grok, and enables simple horizontal scaling by starting more
``designate-worker`` processes.
Change-Id: I7adb2cea21136c18f36e0ed6404989d4e5de8e4d
This adds the ability for a user to configure the Designate
pools.yaml file inside of the role by specifying an attribute.
Because the data required is yaml, it's a nice mapping to specify
the yaml attribute and have it dumped directly to the pools.yaml
file.
This allows users to use attributes from other plays (perhaps setting
up some complex DNS infrastructure in their cloud) and insert them
into Designate without having to write their own template or supply
their own file.
This also invokes the `designate-manage` command to load the pools.yaml
file into the Designate database, and simplifies the tests that
do the pools.yaml needful.
Change-Id: I11a849898bf33aa6b8aa6605296ac7fd733d7c01
Fixes the ability to deploy a venv in cases where:
1) developer_mode is not enabled
2) A cached venv is not downloaded from the repo server
Additional cleanup to the developer_mode venv deployment
logic is implemented by adding a *_venv_download var
which is used to decouple developer_mode from the
cached venv extraction process so that a deployer
can force venv builds in-place (disable cached
venv usage) without enabling developer mode
constraints.
Change-Id: I3e8d45aae6dbfb7fc3627f3dfb7e7352cf7b426a
The current constraints generation for the
installation involves multiple tasks and multiple
variables.
Using multiple tasks extends the installation time
unnecessarily and the additional variables are
unnecessary.
This patch aims to simplify the mechanism and
hopes to speed it up a little.
Change-Id: I5e97ed325ba7971a1b483ac1278d0450026bdf18
Change the 'designate_service_names' from a list to a dictionary mapping
of services, groups that install those services. This brings the
method into line with that used in the os_neutron role in order to
implement a more standardised method.
The init tasks have been updated to run once and loop through this
mapping rather than being included multiple times and re-run against
each host. This may potentially reduce role run times.
Currently the reload of upstart/systemd scripts may not happen if
only one script changes as the task uses a loop with only one result
register. This patch implements handlers to reload upstart/systemd
scripts to ensure that this happens when any one of the scripts
change.
The handler to reload the services now only tries to restart the
service if the host is in the group for the service according to the
service group mapping. This allows us to ensure that handler
failures are no longer ignored and that no execution time is wasted
trying to restart services which do not exist on the host.
Finally:
- Common variables shared by each service's template files have
been updated to use the service namespaced variables.
- Unused handlers have been removed.
- Unused variables have been removed.
Change-Id: I8b3df067d5e27711d9f962d74932c818a506e77a