This patch updates all the roles to the latest available SHA's,
updates all the OpenStack Service SHA's and also updates the
appropriate python requirements pins.
Change-Id: Ifc77c02d456500651e8adcaf9338f81601e2c148
This change allows Ansible to be installed and executed from within
a virtual environment.
Closes-Bug: 1431324
Implements: blueprint ansible-2-1-support
Change-Id: I2fb19dd8218753f75bd7550ed7ba9b79f777083d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch updates all the roles to the latest available stable
SHA's, updates all the OpenStack Service SHA's and also updates
the appropriate python requirements pins.
Note that 'ceilometer_api' has been removed from the standard
tempest testing in the integrated build as the tests have been
removed from the tempest repository with
I0775bcc15dc9cbae6e075fe92f44b5f6c9b9d5d2
The ceilometer testing will have to be re-introduced using the
tempest plugin framework.
Change-Id: Iea7e4213893fb331c410c2d09e3258493f4643f3
Upstream has removed the paramiko restriction of <2 and now has a
dependency on paramiko >2. This is the commit where the requirement
changed: [0]
[0] - e379813e9c
Related-Change-Id: I2369638282b4fefccd8484a5039fcfa9795069a7
Change-Id: I06ba8a9da100a3b02e8eb7f15042f9573b71ed59
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
* Corrected the repo server log directory. This change updates the repos
server log directory which was being incorrectly indexed. The old container
build process would create a service log directory based on "properties".
The "properties" log directory is unused or is re-defined by the various
service roles. This change simply corrects the log paths which the
`rsyslog_client` role uses to ensure all logs are shipped to the right
places.
* Add a log directory creation task to the utility container. This play
incorrectly assumed that the log directory based on "properties" would
be automatically created. This update simply makes that more explicit.
* paramiko has been pinned to match global requirements and use a version
<2. This will ressolve runtime issues in Newton with an unbound
requirements.
Closes-Bug: #1576755
Change-Id: I15d841106ec9a13555b9737c9388f40557f5bec5
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch includes updates of all SHA's and pip, wheel and setuptools
pins.
Due to Bug #1561947, the backend glance store has been changed from swift
to file in the AIO gate. This has been done becuase the glance store project
has an issue with session that causes it to be incompatible with swift. Being
that the master integration gate is depending on this change to go it is
needed to temporarily disable the swift image store in glance until this issue
is fixed.
The branch updater sciprt was modified to search for and update roles if they're
found within the default or user provided role path. This change is resolving
bash errors that happen when the script is run in mitaka+.
Related-Bug: 1561947
Change-Id: I2e09e3e0abb61f5e97e2af7b283f0aed6dadd853
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch provides a convenience script to check for the current
version of any package on PyPI, then output it in various ways.
This script is used in the SHA updating script in order to provide
a current set of critical packages to ensure that each SHA bump
includes an update to a current version of pip, setuptools and
wheel but also to ensure that they are pinned to a specific version
with this particular set of packages.
This ensures that we keep current with these packages as they
change, but also ensures that the versions tested for each tag are
the versions used forever.
The patch also ensures that any package installed by pip is upgraded
to the expected versions.
Depends-On: I0a78135737c40a505d77df6ed67da0ef6695bfcb
Change-Id: I61795b3afb4804060d494a08975c10adcf52f468
This change sets the requirement for pip to be pip>7,<8.
This is being done because the global pip wheel building
producing a built wheel for pip8 which causing downstream
breakage in the most current releases of master and liberty.
Change-Id: Iddcbba94c62dc87dac6fc25efb17b99ee8ec37d6
Closes-Bug: #1541193
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch removes an unused script which performed actions that
ansible-galaxy could not previously perform, but now can. The
associated requirements.txt entries are also removed.
Change-Id: Ice29a8ba255b60870fd42ab51af93f2d683b22ca
We're seeing some neutron db sync failures in master branch (liberty).
It looks like there may be a few issues with the way we're handling
migrations:
1. We're not capturing the correct information in the 'Check last DB
revision' task as the first line returned by 'neutron-db-manage
history' is a header (this also means that the 'Inspect on disk
neutron DB revision' task will never do what we intend).
2. We delegate 'Check last DB revision' to 'neutron_all' group, but in
an ideal world this is probably only necessary on 'neutron_server'.
3. When stopping neutron server to run 'Perform a Neutron DB Upgrade',
we're only stopping it on groups['neutron_server'][0], while all
other neutron server containers are up servicing requests.
4. Perform a db stamp, which doesn't appear to be necessary.
This change makes the following changes:
1. Bumps neutron SHA to include 43c00a9, which introduces new
--expand / --contract upgrade options (otherwise, you have to
specify liberty_expand@head / liberty_contract@head which may no
longer work when neutron gets bumped in the future)
2. Checks if migrations have previously run, and if not runs a
'neutron-db-manage upgrade heads'
3. If migrations have previously run:
a) it runs an online migration against expand alembic branch using
'neutron-db-manage upgrade --expand'.
b) it stops all neutron-server services
c) it runs an offline migration against contract alembic branch
using 'neutron-db-manage upgrade --contract'
d) it starts all neutron-server instances
4. It removes the temporary pin introduced in
https://review.openstack.org/218572 as the SHA bump includes the
upstream fix https://review.openstack.org/218723
TODO: Currently, we upgrade expand and contract branches (shutting down
neutron-server in the proceses), even if there are no pending
migrations. We need to find a clean way to check what migration
we are on and compare to alembic HEADS file to see if we're up to
date. Unfortunately, 'neutron-db-manage current' doesn't indicate
which migration is in which alembic branch, so you would have to
further grep for each migration in the neutron migrations code to
determine the branch.
NOTE: Liberty introduces the split alembic branches for online/offline
migrations, see [1] for more information.
[1] http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html
Change-Id: I1176b5fe12cad1ee732486ae179e76deea5623e1
Closes-Bug: #1486593
This patch adds a wait for the container's sshd to be available
after the container's apparmor profile is updated. When the
profile is updated the container is restarted, so this wait is
essential to the success of the playbook's completion.
It also includes 3 retries which has been found to improve the
rate of success.
Due to an upstream change in behaviour with netaddr 0.7.16 we
need to pin the package to a lower version until Neutron is
adjusted and we bump the Neutron SHA.
Change-Id: I30575ee31929b0c9af6353b7255cdfb6cebd2104
Closes-Bug: #1490142
While this should be backported this purpose of this commit
is to unblock gating and provide for the always changing
requirements within tempest while also allowing us to ensure
that we are defcore complaint.
Tempest is not an integrated service within OpenStack and does
not adhere to any given package or requirement set that would
in a real work function with the rest of OpenStack. Because
tempest is intended to be a standalone system that is not
installed along side of the rest of OpenStack it general will
break and or introduce new requirements that break the services
that depend on various versions of packages as found within
global requirements. To fix this issue tempest is now being
installed within a VENV. The virtual environment will ensure
that tempest is installed in a location where it can resolve
its own dependencies without general impact to the rest of the
system.
Additionally, we removed the heat_contrib_extraroute heat
plugin from the build process because its presently
incompatible with PBR >= 0.11.0 which is related to issue
https://bugs.launchpad.net/openstack-ansible/+bug/1450733 .
However we have already built wheels in our repo which will
still allow this contributing plugin to be installed as an
integrated part of the system. Currently, we git clone heat
source onto heat_engine containers and install selected plugins
via a 'python setup.py install'. This change removes the tasks
that do that and simply adds heat-contrib-extraroute
to heat_pip_packages so it gets installed on all heat-related
containers. This is actually only required on the heat_engine
containers but the package is tiny and should not cause any issues
being present on the heat_api containers.
Change-Id: Ib972704084ead5748b19362b142fb161fea4a734
This change was made to ensure that PBR is using version < 0.11.0.
This is because changes in upstream PBR have made it impossible to
build contributing packages in heat and will likely impact other
services until the switch to liberty is in full effect. To ensure
consistent building of OpenStack packages PBR is now being installed
in the repo server containers which will ensure there are no
assumptions being made when creating python packages from OpenStack
contributing sources that may not be as well maintained as others.
Change-Id: I20c216543e63ef142119c2625fb219e030a708dd
Partial-bug: 1450733
This commit brings o-a-d's requirements.txt in line w/
global-requirements.txt (master). Also, we remove any packages that
are no longer required (a number were installed previously for tools
in scripts/ which no longer exist). Additionally, we add some comments
indicating what requires a specific package.
NOTE: wheel requirement was originally added as pip < 6.0 required
wheels package in order to be able to install wheels themselves. Since
we also bump pip to 6.0, wheel is no longer required.
Change-Id: I0de858283157cfb22db935bfe81a7ec23c9a7b9c
Closes-Bug: #1428106
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
simplistic approach. This change duplicates code within the roles but
ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
anyone who may want or need to dive into the JSON blob that is created.
In the inventory a properties field is used for items that customize containers
within the inventory.
* The environment map has been modified to support additional host groups to
enable the seperation of infrastructure pieces. While the old infra_hosts group
will still work this change allows for groups to be divided up into seperate
chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
variables extracted into the separate file
etc/openstack_deploy/user_secrets.yml in order to allow seperate
security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
should allow roles to be consumed outside of the `os-ansible-deployment`
reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
containing a deprecation warning instructing the user to move to the standard
playbooks directory.
* While all of the rackspace specific components and variables have been removed
and or were refactored the repository still relies on an upstream mirror of
Openstack built python files and container images. This upstream mirror is hosted
at rackspace at "http://rpc-repo.rackspace.com" though this is
not locked to and or tied to rackspace specific installations. This repository
contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
setuptools 8.0 (and subsequently 8.0.1, 8.0.2) were released over the
weekend. This introduces a breaking change whereby python packages
installed from SHA, rather than with a proper release version, do not
have their version numbers parsed properly due to the appended sha.
Eg python-glanceclient 0.14.2.2.gb126351
This commit:
- pins setuptools to 7.0, to workaround this issue until a fix
is made upstream, inside the global requirements.txt
- removes package installations of python-setuptools and
python-pip which had potential to further conflict.
- removes the list pip_common_packages from the all_common vars file
Closes-Bug: 1402679
Closes-Bug: 1402672
Change-Id: Idb75f816f47740ee28ec5d7d8afb118acf4a71bb
The rpc-slushee.rackspace.com has numerous issues, so
we have created mirror.rackspace.com/rackspaceprivatecloud
to address those issues. All instances of rpc-slushee
have been replaced.