In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: Iecb64d28afe3acfbae7060af55c1a891310e5ef4
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
This removes the systemd service templates and tasks from this role and
leverages a common systemd service role instead. This change removes a
lot of code duplication across all roles all without sacrificing features
or functionality. The intention of this change is to ensure uniformity and
reduce the maintenance burden on the community when sweeping changes are
needed. The exterior role is built to be OSA compatible and may be pulled
into tree should we deem it necessary.
Change-Id: Icb7ca523cb19c560de5c84b0d60a06305029192c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The systemd module doesn't support the sleep option and this patch
removes it from the handlers.
Closes-Bug: 1735786
Change-Id: Ida1d83903dc4d3cd85029735a0882bf342d21dca
Swift should mirror other roles and use a list instead of a dictionary
for it's filtered_services.
This patch makes that change.
Change-Id: Ie6bf282a36ed63d73996447a88c3c5f6056465a7
This patch implements the use of dynamic includes,
the filtered service list and the elimination of
redundant tasks in order to optimise the role
execution.
Change-Id: Ia957bd80ec6a97a29b4b3a1c28bf37dfc9226ab1
As part of the Trusty removal, the "pattern:" option for service
restarts can be removed. This prevents the following warning message
from appearing in the Ansible runs:
[WARNING]: Ignoring "pattern" as it is not used in "systemd"
Change-Id: I38e04f48360ef558fa7d99e90fb0e73a9cb887be
As a part of removing Trusty from OpenStack-Ansible we shall aim to
remove this from all the roles.
Testing has already been removed for Trusty in the integrated build and
all individual repositories on master (Ocata), as such we can now go
ahead and remove the support within the roles.
Change-Id: I89ba35fd15703aba2a05d11d4550690704bdf272
Implements: blueprint trusty-removal
This PR Adds CentOS 7 support for Swift.
The following was required to get CentOS 7 to work:
* Add yum install path + packages
* Variablize rsync service name
* Gather network interface facts prior to setting storage/repl IPs
* Ensure /etc/defaults/rsync is only set for apt installations.
* Ensure the rsyslog service is started and enabled.
Change-Id: Ibaf8bc8d54b55820e8b527b52940c61c05c732d8
When not using dedicated replication systemd still puts init scripts
down, which take a long time to restart/start. upstart scripts get
around this by setting blank scripts.
Now that we are using a service dict we can do better by defining an
"service_en" flag and not setting up scripts when the service isn't
enabled.
Additionally, the systemd tempfiles and init files were not using the
appropriate "program_binary" variable at all, this has been fixed.
Change-Id: Iae569bfe38a440fb09e56658b3a934799a8821e8
Change the 'swift_x_program_names' from a list to a dictionary
mapping of services, groups that install those services. This
brings the method into line with that used in the os_neutron role
in order to implement a more standardised method.
The init tasks have been updated to run once and loop through this
mapping rather than being included multiple times and re-run against
each host. This may potentially reduce role run times.
Currently the reload of upstart/systemd scripts may not happen if
only one script changes as the task uses a loop with only one result
register. This patch implements handlers to reload upstart/systemd
scripts to ensure that this happens when any one of the scripts
change.
The handler to reload the services now only tries to restart the
service if the host is in the group for the service according to the
service group mapping. This allows us to ensure that handler
failures are no longer ignored and that no execution time is wasted
trying to restart services which do not exist on the host.
Finally:
- Common variables shared by each service's template files have
been updated to use the service namespaced variables.
- Unused handlers have been removed.
- Unused variables have been removed.
Change-Id: Id35de501acf6b3164221085f8f9e142234ea0d73
When executing the role with Ansible 2.1, the following
deprecation warning is issued in the output for some tasks.
[DEPRECATION WARNING]: Using bare variables is deprecated.
This patch addresses the tasks to fix the behaviour appropriately.
Also removed a single usage of with_items that contained one item
in favor of a simple non-looping task.
Change-Id: Ief1a53bb0804dfdbd742bfe919438d80428287da
The rsync service is currently restarted using two handlers, one to stop
the service and a second to start it. There is not a sufficient delay
between the two task and so the rsync pid has not been removed before
the attempt is made to start the service.
This commit replaces the two handlers with a single one that will do the
restart in one go.
Change-Id: I8ed4630da1add7205552b6ec731a143dbe45112b
Closes-bug: 1538649
Existing rsync stop/start handlers were relying on the pattern
parameter to the Ansible service module which relies on the results
of ps to determine if the service is running. This is unnecessary
because the rsync service script is well-behaved and responds
appropriately to start stop and restart commands. Removal of the
pattern param ensures that the response from the service command is
used instead.
Root cause of the bug is that when Keystone was changed to share
fernet secrets via rsync over ssh tunnel, an rsync process was
introduced in AIOs, Swift stand-alones, and other deployment
configurations that contain Keystone containers on the storage hosts.
The resulting rsync processes within Keystone containers pollute the
results of ps commands on the host, fooling Ansible into thinking
that an rsync service is running on the standard port when it is not.
Secondly, the handler responsible for stopping rsync was not causing
the notice for "Ensure rsync service running" to trigger cleanly in
my testing, so the tasks were changed to trigger both notices in an
ordered list.
Change-Id: I5ed47f7c1974d6b22eeb2ff5816ee6fa30ee9309
Closes-Bug: 1481121
* Adjust the handler to include a "restart" handler for each of account,
container, object and proxy service groups.
* Add a variable in defaults listing program names for each swift
service group.
* Remove the over-arching "all swift program_names" variable.
* Change the storage and proxy host tasks to call the appropriate
* handler.
Change-Id: I25adfa152fc7a3da83ca7c12d57977eec8b51d7b
Closes-Bug: #1427601
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
simplistic approach. This change duplicates code within the roles but
ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
anyone who may want or need to dive into the JSON blob that is created.
In the inventory a properties field is used for items that customize containers
within the inventory.
* The environment map has been modified to support additional host groups to
enable the seperation of infrastructure pieces. While the old infra_hosts group
will still work this change allows for groups to be divided up into seperate
chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
variables extracted into the separate file
etc/openstack_deploy/user_secrets.yml in order to allow seperate
security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
should allow roles to be consumed outside of the `os-ansible-deployment`
reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
containing a deprecation warning instructing the user to move to the standard
playbooks directory.
* While all of the rackspace specific components and variables have been removed
and or were refactored the repository still relies on an upstream mirror of
Openstack built python files and container images. This upstream mirror is hosted
at rackspace at "http://rpc-repo.rackspace.com" though this is
not locked to and or tied to rackspace specific installations. This repository
contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e