Debian-specific vars and logic have been moved to tasks
that will execute only on those distributions.
Change-Id: I370621eda9e87bd19532e4e37e46097607bf4166
Implements: blueprint multi-platform-host
The intention of this change is to enable an option to increase swift
performance without changing any of the swift internals. This change is
in-line with documentation published by the good folks at swiftstack and
intel. This change will only effect swift when its installed in a
virtual environment which is a self-imposed limitation such that we're
not messing with the system's default Python.
New Options:
- A new option `swift_pypy_enabled` has been added to enable or
disable the pypy interpreter for swift. The default is "false".
- A new option `swift_pypy_archive` has been added to allow a
pre-built pypy archive to be downloaded and moved into place. This
option is a dictionary and contains the URL and SHA256 as keys.
Reference: [ https://software.intel.com/en-us/blogs/2016/05/06/doubling-the-performance-of-openstack-swift-with-no-code-changes ].
Change-Id: I045e8b97d29b96e2c2450761efa1c2668de12a75
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This PR sets up rsync module per drive for swift object servers. This
improves resource distribution, and also allows us to utilise rsync
module changes to prevent disk writes to specific drives when they begin
filling up.
Enable this option by setting "swift_rsync_module_per_drive: True", this
will still default to False to match upstream defaults. Additionally the
rsync max connections has been increased to match upstream defaults.
Additionally we include rsync.d/*.conf files by default and set this
directory up, so that we can add individual configuration to disable
specific drives.
Change-Id: I2019cade5bf5f2878497d30ce738dff07786fa64
Currently all pip install tasks only require the package to be
present. This means that when an environment undergoes a minor
upgrade the package is not upgraded to the same version that
was tested with. This ultimately results in a deployed
environment that does not match the tested environment.
While for the services installed into venvs this is not an
issue, it does affect those which do not use venvs and any
packages which are installed outside of a venv or on top
of a venv.
This patch changes the behaviour to ensure that the install
task will always use the latest available package. In
developer_mode this will mean using the version specified
in upper-constraints, and in an integrated build this will
mean the version which is available in the wheel repo's
folder for the tag.
Change-Id: Id04b2f74831e3422b036308c638be5428509e57a
This fix removes the XFS filesystem from the daily cron job.
It will help to remove unnecessary disk IO due to updatedb/mlocate
swift object indexing inside the /srv/node folders.
Change-Id: I8bfa92003ce06ee4f065663e054cd2d04f458ec6
Closes-Bug: #1572307
This PR fixes the following minor test issues:
* The swift ring and swift-service addresses are now consistent.
* Memcache now listens on the appropriate IP.
* The openrc file is now populated correctly.
* Rsyslog restarts appropriately, meaning logs are generated.
Change-Id: Ie4f9672eba5936232ccfc613bdcc03a7fdeb090b
This will allow the specification of container-sync realms via the
swift_container_sync_realms variable (documented in the
defaults/main.yml within the os_swift role).
Creating a conf file that is then used to enable and utilise
container-sync Realms within Swift.
Change-Id: Icf71d008765ff83743f6ab28ef0cea29943e362e
We want to improve the gate speed by running the swift-role once on AIO,
since we don't need to split the sync tasks out when not using MR swift.
This means for MR swift we need to differentiate between the tasks that
should be run on remote hosts and those that should be run on local
hosts.
Change-Id: I0a8436ebbab0226f6c4fb499c938f760429aad39
This commit changes the pip_install_options fact name in
swift_install.yml to pip_install_options_fact. This allows us to
maintain the existing pip_install_options variable without overwriting
it with options when in developer mode, which ultimately means we can
have multiple services running in a container use a combination of
*_developer_mode: true and *_developer_mode: false. At the moment,
if a service writes pip_install_options fact with the constraints
options, those options will persist to other services running in the
container even if *_developer_mode: false.
Change-Id: I248aeecafe9ac4e06e3e24796dd279f67c6589dd
Now that auth token usage is deprecated, prefer the admin
user and password for all swift setup tasks run against
keystone.
Change-Id: I1c87dc6976c26e0c00e740ce1e4105bcb30cc32f
The repo split out has meant that having multiple roles for swift has
become inconvenient. We can merge the os-swift-sync role back into
os-swift and utilise variables to ensure that the swift-install,
swift-setup and swift-sync playbooks work in the same way.
This will avoid code duplication and the requirement for duplicated
changes to happen to both repositories.
Change-Id: If01d8a3aa29fc7637aecb92e61929523af08103d
Pre-empting a situation where we run into issues installing pip packages
due to unspecified upper constraints files.
We ran into this issue with the os_keystone repo which was patched in
this way: https://review.openstack.org/#/c/290446
Change-Id: I0ee1471a65ad92323e4a274f5ec2968969e291fe
Adds a functional/convergence tests for os-swift including keystone,
galera, memcache and rabbit.
Swift testing infrastructure consists of:
1 service container /w keystone, galera, rabbit, memcached.
4 storage hosts (each with 2 disks)
1 proxy container.
This fixes a few issues to ensure the functional tests will run, such as
delegating sysctl net.ipv4.tcp_tw_reuse task to physical host, since we
are running swift storage inside containers for the purposes of this
test, and the containers won't have access to make this change.
Fixes the developer_mode settings which were incorrect.
Change-Id: Ie1dfaf666cbbac9fe78edd0a9218cec790c8c4d2
Closes-Bug: #1553967
This patch adds configuration for swift's drive-audit script with
a cron job to run it hourly at 15 minutes past the hour.
Closes-Bug: 1433619
Change-Id: I1a5cfbfffef007a0d8e9b56dd5936aad7c38916d
This commit adds the ability to install cinder without a repo server.
This pattern is lifted from the os_keystone role and allows us to
further develop functional testing for this role.
Change-Id: I85ba753a946b22ee3e9b9403977501a1804f9d86
Partial-Bug: #1553967
This change makes it so that all services are expecting SSL termination
at the load balancer by default. This is more indicative of how a real
world deployment will be setup and is being added such that we can test
a more production like deployment system by default.
The AIO will now terminate SSL in HAProxy using a self-signed cert.
Change-Id: I6273ffa453b4e5eb8a33767974d390a126296c47
Re-Implementation-Of: https://review.openstack.org/#/c/277199/9
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The sudoers file was being created in the pre-install tasks
which causes an incorrect configuration variable to be dropped
when the venv env is not turned on. To correct this issue the
sudoers template is now dropped in the post install task file
after the bin_path fact has been set.
This change also removes the directory create task for heat, keystone,
glance, and swift because no sudoers files are needed for these services.
Re-Implementation-Of: https://review.openstack.org/#/c/277674/1
Change-Id: I609c9c12579dc1897787d19a1f58fe3e919b5e35
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This PR sets the ownership for the swift log directories to the syslog
user instead of the swift user. Since swift uses syslog, no logs were
being created/logged to before this change.
Change-Id: I44768d4cd04108a7163169dfec2f0de774a2cf83
The memcache configuration was only setup for the proxy-server.conf
within Swift, and was not set for the object and container reconcilers
which both use memcache.
This patch moves the memcache settings into a separate memcache.conf
file which is then configured on all swift hosts, removing the specific
conf from the proxy-server.conf file.
Change-Id: I047b2d1178de43c694c30280f6ed9fe8511341fd
Closes-Bug: #1542121
Workarounding the upstream ansible apt module bug
documented here:
https://github.com/ansible/ansible-modules-core/pull/1517
For the next versions of ansible we'll be using, we should
check if the apt bug is fixed. When it's fixed, we could
abandon this change and use the standard apt module
with correct cache handling.
Change-Id: I2aaf00da175f31d0157bbc4ae30a4e176b055078
We currently have two issues with venvs:
- if you update your venv on the repo server, it is not possible for
that updated venv to land on the service's container as the get_url
task always skips if the file exists (even if the file is different)
- if you have an updated venv on the repo server and forcefully delete
the cached venv tarball on the service's container, the new tarball
will get unarchived over top of the existing venv
This commit does the following:
- gets the checksum of the /var/cache tarball and downloads checksum
file from repo server
- updates "Attempt venv download" to only download the venv if the
cache doesn't exist or if the local and remote checksums differ
- adds a "force: true" to "Attempt venv download" task so that the venv
tarball will get re-downloaded when the when condition is true (this
is necessary otherwise the download will get skipped since the
destination already exists)
- adds a new task "Remove existing venv" so we can first remove the
venv before we unarchive the potentially new venv from the repo
server
- updates "Create swift venv dir" and "Unarchive pre-built venv"
tasks to only proceed if "swift_get_venv | changed", which
prevents these tasks from running when they the venv tarball hasn't
changed
- adds multiple service restarts to
os_swift/tasks/swift_install.yml so that swift will restart
correctly should the venv/packages update without any associated
config changes
NOTE: The reason why we compare local and remote checksum is to avoid
unnecessarily downloading the venv when the checksums are in fact
the same. On small deploys this is more or less a non-issue but
if a deploy w/ thousands of compute nodes re-runs playbooks we
want to limit the venv downloads when it's unnecessary.
Change-Id: I4b028f6e4ca59eceac010d2bbc10a8d79f6f3937
The rsync service is currently restarted using two handlers, one to stop
the service and a second to start it. There is not a sufficient delay
between the two task and so the rsync pid has not been removed before
the attempt is made to start the service.
This commit replaces the two handlers with a single one that will do the
restart in one go.
Change-Id: I8ed4630da1add7205552b6ec731a143dbe45112b
Closes-bug: 1538649
It's possible to remove keystoneauth from the middleware pipeline,
but the play will fail because the keystone swift User/roles/perms
tasks will not be able to succeed.
This patch skips those tasks when not using keystoneauth in the
middleware pipeline.
Change-Id: I87143b5c220dc312e2cb5d7e3dd3e9e01609ff91
Closes-Bug: #1523581
When using an LDAP backend the plabooks fail when "ensuring.*"
which is a keystone client action. The reason for the failure is
related to how ldap backend, and is triggered when the service
users are within the ldap and not SQL. To resolve the issue a boolean
conditional was created on the various OS_.* roles to skip specific
tasks when the service users have already been added into LDAP.
Change-Id: I64a8d1e926c54b821f8bfb561a8b6f755bc1ed93
Closes-Bug: #1518351
Closes-Bug: #1519174
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The container-reconciler and object-expirer were missing from the os-swift
role. The reconciler makes sure incorrectly placed objets live in the
correct storage policy. The expirer is the service that deletes expired objects.
This change also adds the abilty to optionally specify a reclaim_age in the swift
section of the configuration, which is now set in all the locations required,
still with the default of 604800 seconds (7 days).
Change-Id: Ic56a714c3fb3c84b9bb5ed8e2ae3c86dad474161
Closes-Bug: #1516877
The change builds venvs in a single repo container and then
ships them to to all targets. The built venvs will be within
the repo servers and will allow for faster deployments,
upgrades, and more consistent deployments for the life cycle
of the deployment.
This will create a versioned tarball that will allow for
greater visablility into the build process as well as giving
deployers/developers the ability to compair a release in
place.
Change-Id: Ieef0b89ebc009d1453c99e19e53a36eb2d70edae
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This commit conditionally allows the os_swift role to
install build and deploy within a venv. This is the new
default behavior of the role however the functionality
can be disabled.
In this PR, like all of the other venv related PRs, the
`is_metal` flag was removed from the role however unlike
some of the other PRs this removal required moving some
of the `is_metal` logic out of the role and into the
play. This was done for consistency as well as making
the role more standalone. The only thing that the role
should care about, in terms of installation, is whether
or not to install in a venv.
Change-Id: I6f5b883a853611659567bd12e8bcf572189854b7
Implements: blueprint enable-venv-support-within-the-roles
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch adds the variable 'pip_install_options' which is passed to the pip
install module as extra arguments in order to allow the use of options like
'--force-reinstall' when executing playbooks.
eg: openstack-ansible -e pip_install_options="--force-reinstall" \
setup-openstack.yml
This is required due to constant upstream changes in dependencies which
result in python wheel version upgrades and downgrades between tagged
versions of openstack-ansible.
The intention is that this can be used whenever a deployer switches between
tags for both upgrades and downgrades.
DocImpact
Closes-Bug: #1489251
Closes-Bug: #1499451
Related-Bug: #1501114
Change-Id: I996185e009a4c4af4f23798619bdbd0d490360c9
The Swift configuration item [filter:recon]/recon_lock_path is set to
'/var/lock/swift' by openstack-ansible in the appropriate configuration
files. The playbooks also create the directory if it does not exist. If
the host is rebooted the directory '/var/lock/swift' is missing and must
be recreated.
/var/lock (/run/lock) is a tmpfs and so the directory /var/lock/swift
will not persist between reboots.
Swift attempts to create a directory in the directory specified by
recon_lock_path however it does not recursively create any missing
parent directories.
This commit changes the value of [filter:recon]/recon_lock_path to that
set by Swift, '/var/lock'. This allows it to create the directory
'/var/lock/swift-recon-object-cron'. The creation of '/var/lock/swift'
is also removed from the playbooks.
Change-Id: I714367b02c7cf961e9e0bdee4e41f9e4e105b088
Closes-bug: #1496117
The change modifies the swift template tasks such that it's now
using the config_template action plugin. This change will make so that
config files can be dynamically updated, by a deployer, at run time,
without requiring the need to modify the in tree templates or defaults.
Partially implements: blueprint tunable-openstack-configuration
Change-Id: Id992937f35afa0549f9f0d0fbcf0be5e6978df57
Existing rsync stop/start handlers were relying on the pattern
parameter to the Ansible service module which relies on the results
of ps to determine if the service is running. This is unnecessary
because the rsync service script is well-behaved and responds
appropriately to start stop and restart commands. Removal of the
pattern param ensures that the response from the service command is
used instead.
Root cause of the bug is that when Keystone was changed to share
fernet secrets via rsync over ssh tunnel, an rsync process was
introduced in AIOs, Swift stand-alones, and other deployment
configurations that contain Keystone containers on the storage hosts.
The resulting rsync processes within Keystone containers pollute the
results of ps commands on the host, fooling Ansible into thinking
that an rsync service is running on the standard port when it is not.
Secondly, the handler responsible for stopping rsync was not causing
the notice for "Ensure rsync service running" to trigger cleanly in
my testing, so the tasks were changed to trigger both notices in an
ordered list.
Change-Id: I5ed47f7c1974d6b22eeb2ff5816ee6fa30ee9309
Closes-Bug: 1481121
Add the swift-remote host group and environment file.
Add an os_swift_sync role which will sync the swift ring and ssh keys
for swift hosts (remote and not-remote). Which has the following:
* Moves the key and ring tasks out of os_swift role to os_swift_sync.
* This adds the use of the "-r" flag that was added to the
swift_rings.py and swift_rings_check.py.
* Adds a ring.builder vs contents file consistency check.
* Adjusts the rsync process to use the built-in synchronize module
* Ensure services have started post ring/ssh key sync.
Adds environment file and sample configuration file for swift-remote
hosts (conf.d).
Move appropriate default vars to the os_swift_sync role, and remove them
from the os_swift role.
Rename the "os-swift-install.yml" playbook to "os-swift-setup.yml" as
this handles only the setup, and add a playbook to for both
os-swift-sync.yml and an overarching playbook (os-swift-install.yml)
that will call both the os-swift-sync.yml and os-swift-setup.yml
playbooks. This means the funcitonality of "os-swift-install.yml"
remains unchanged.
Adjust the run-playbooks.sh so that it calls the new overarching swift
playbook.
Change-Id: Ie2d8041b4bc46f092a96882fe3ca430be92195ed
Partially-Implements: blueprint multi-region-swift
Moving towards multi-region swift there is a chance that 2 regions will
attempt to update the ring at the same time. Whilst measures are in
place to ensure a region only updates its own region entries in the
ring it would still be possible, if the 2 runs happened simultaneously,
that some ring inconsistencies could happen. For example, if a region A
updates at the same time as region B but the sync order is different
some nodes could have region A's "updated" ring and some with region
B's "updated" ring.
To ensure this hasn't happened (without our knowledge) this patch adds
another md5sum check which will report if the rings are inconsistent
across the nodes.
Change-Id: Id88dfebcaa0553437953f92235bf63363f750797
Partially-Implements: blueprint multi-region-swift
