Commit Graph

24 Commits

Author SHA1 Message Date
Jesse Pretorius
bb69b667f0 Update all SHAs for Newton-1 2016-06-02
This patch updates all the roles to the latest available SHA's,
updates all the OpenStack Service SHA's and also updates the
appropriate python requirements pins.

Change-Id: Ifc77c02d456500651e8adcaf9338f81601e2c148
2016-06-02 21:05:45 +00:00
Kevin Carter
c904de2db5 Isolate Ansible from the deployment host
This change allows Ansible to be installed and executed from within
a virtual environment.

Closes-Bug: 1431324
Implements: blueprint ansible-2-1-support
Change-Id: I2fb19dd8218753f75bd7550ed7ba9b79f777083d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-05-26 10:41:27 +00:00
Jesse Pretorius
30c59b297d Updates all SHAs for Newton 2016-05-19
This patch updates all the roles to the latest available stable
SHA's, updates all the OpenStack Service SHA's and also updates
the appropriate python requirements pins.

Note that 'ceilometer_api' has been removed from the standard
tempest testing in the integrated build as the tests have been
removed from the tempest repository with
I0775bcc15dc9cbae6e075fe92f44b5f6c9b9d5d2

The ceilometer testing will have to be re-introduced using the
tempest plugin framework.

Change-Id: Iea7e4213893fb331c410c2d09e3258493f4643f3
2016-05-23 14:52:36 +01:00
Kevin Carter
b15363c5fa Remove paramiko restriction
Upstream has removed the paramiko restriction of <2 and now has a
dependency on paramiko >2. This is the commit where the requirement
changed: [0]

[0] - e379813e9c
Related-Change-Id: I2369638282b4fefccd8484a5039fcfa9795069a7

Change-Id: I06ba8a9da100a3b02e8eb7f15042f9573b71ed59
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-05-20 12:03:39 -05:00
Kevin Carter
e971e15997
Integrated updates after the multi-distro changes
* Corrected the repo server log directory. This change updates the repos
  server log directory which was being incorrectly indexed. The old container
  build process would create a service log directory based on "properties".
  The "properties" log directory is unused or is re-defined by the various
  service roles. This change simply corrects the log paths which the
  `rsyslog_client` role uses to ensure all logs are shipped to the right
  places.
* Add a log directory creation task to the utility container. This play
  incorrectly assumed that the log directory based on "properties" would
  be automatically created. This update simply makes that more explicit.
* paramiko has been pinned to match global requirements and use a version
  <2. This will ressolve runtime issues in Newton with an unbound
  requirements.

Closes-Bug: #1576755
Change-Id: I15d841106ec9a13555b9737c9388f40557f5bec5
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-05-03 22:51:43 -05:00
Jesse Pretorius
2fc728daa7 Update Newton SHA's 2016-04-22
Change-Id: Icab701a3ee2594f9f126582dcfe8eb99ebc79d0c
2016-04-22 17:26:03 +01:00
Jesse Pretorius
2f99139da8 Set SHA's for Mitaka RC1
This sets all pins for OpenStack-Ansible Mitaka RC1.

Change-Id: I9edd36801487aff08356149270b8bc761a3d7880
2016-04-01 01:48:19 +01:00
Jesse Pretorius
f635e090bd
Updates all repo SHAs to prepare for Mitaka release
This patch includes updates of all SHA's and pip, wheel and setuptools
pins.

Due to Bug #1561947, the backend glance store has been changed from swift
to file in the AIO gate. This has been done becuase the glance store project
has an issue with session that causes it to be incompatible with swift. Being
that the master integration gate is depending on this change to go it is
needed to temporarily disable the swift image store in glance until this issue
is fixed.

The branch updater sciprt was modified to search for and update roles if they're
found within the default or user provided role path. This change is resolving
bash errors that happen when the script is run in mitaka+.

Related-Bug: 1561947
Change-Id: I2e09e3e0abb61f5e97e2af7b283f0aed6dadd853
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-03-28 10:46:02 -05:00
Jesse Pretorius
ab2834ef8e Use current, but pinned versions of pip, setuptools and wheel
This patch provides a convenience script to check for the current
version of any package on PyPI, then output it in various ways.

This script is used in the SHA updating script in order to provide
a current set of critical packages to ensure that each SHA bump
includes an update to a current version of pip, setuptools and
wheel but also to ensure that they are pinned to a specific version
with this particular set of packages.

This ensures that we keep current with these packages as they
change, but also ensures that the versions tested for each tag are
the versions used forever.

The patch also ensures that any package installed by pip is upgraded
to the expected versions.

Depends-On: I0a78135737c40a505d77df6ed67da0ef6695bfcb
Change-Id: I61795b3afb4804060d494a08975c10adcf52f468
2016-03-02 20:06:24 +00:00
Kevin Carter
4a9b7b848a
Set hard requirements for pip
This change sets the requirement for pip to be pip>7,<8.
This is being done because the global pip wheel building
producing a built wheel for pip8 which causing downstream
breakage in the most current releases of master and liberty.

Change-Id: Iddcbba94c62dc87dac6fc25efb17b99ee8ec37d6
Closes-Bug: #1541193
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-02-02 22:20:48 -06:00
Jesse Pretorius
f6d7c1fa35 Remove unused openstack-ansible-role-requirements.py script
This patch removes an unused script which performed actions that
ansible-galaxy could not previously perform, but now can. The
associated requirements.txt entries are also removed.

Change-Id: Ice29a8ba255b60870fd42ab51af93f2d683b22ca
2016-01-05 17:52:49 +00:00
Jesse Pretorius
81a750da5e Block/cap incompatible libraries
This updates the global requirements to block requests 2.8.0 due to:
  https://launchpad.net/bugs/1476770 and
  https://launchpad.net/bugs/1503768 and
  https://launchpad.net/bugs/1505326

And also blocks oslo.messaging 2.6.0 temporarily due to:
  https://launchpad.net/bugs/1505295

And also blocks oslo.versionedobjects 0.11.0 temporarily due to:
  https://launchpad.net/bugs/1505677

And also blocks WebOb<1.5.0 temporarily due to:
  https://launchpad.net/bugs/1505153

Related-Bug: #1476770
Related-Bug: #1503768
Related-Bug: #1505326
Related-Bug: #1505295
Related-Bug: #1505153
Related-Bug: #1505677
Change-Id: I3aabbf717ef21a41c7bb9d21957df838642926f0
2015-10-13 15:46:30 +01:00
Matt Thompson
092f606adc Update how neutron migrations are handled
We're seeing some neutron db sync failures in master branch (liberty).
It looks like there may be a few issues with the way we're handling
migrations:

1. We're not capturing the correct information in the 'Check last DB
   revision' task as the first line returned by 'neutron-db-manage
   history' is a header (this also means that the 'Inspect on disk
   neutron DB revision' task will never do what we intend).
2. We delegate 'Check last DB revision' to 'neutron_all' group, but in
   an ideal world this is probably only necessary on 'neutron_server'.
3. When stopping neutron server to run 'Perform a Neutron DB Upgrade',
   we're only stopping it on groups['neutron_server'][0], while all
   other neutron server containers are up servicing requests.
4. Perform a db stamp, which doesn't appear to be necessary.

This change makes the following changes:

1. Bumps neutron SHA to include 43c00a9, which introduces new
   --expand / --contract upgrade options (otherwise, you have to
   specify liberty_expand@head / liberty_contract@head which may no
   longer work when neutron gets bumped in the future)
2. Checks if migrations have previously run, and if not runs a
   'neutron-db-manage upgrade heads'
3. If migrations have previously run:
   a) it runs an online migration against expand alembic branch using
      'neutron-db-manage upgrade --expand'.
   b) it stops all neutron-server services
   c) it runs an offline migration against contract alembic branch
      using 'neutron-db-manage upgrade --contract'
   d) it starts all neutron-server instances
4. It removes the temporary pin introduced in
   https://review.openstack.org/218572 as the SHA bump includes the
   upstream fix https://review.openstack.org/218723

TODO: Currently, we upgrade expand and contract branches (shutting down
      neutron-server in the proceses), even if there are no pending
      migrations.  We need to find a clean way to check what migration
      we are on and compare to alembic HEADS file to see if we're up to
      date.  Unfortunately, 'neutron-db-manage current' doesn't indicate
      which migration is in which alembic branch, so you would have to
      further grep for each migration in the neutron migrations code to
      determine the branch.

NOTE: Liberty introduces the split alembic branches for online/offline
      migrations, see [1] for more information.

[1] http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html

Change-Id: I1176b5fe12cad1ee732486ae179e76deea5623e1
Closes-Bug: #1486593
2015-09-03 05:45:35 +00:00
Jesse Pretorius
a40cb58118 Wait for container ssh after apparmor profile update
This patch adds a wait for the container's sshd to be available
after the container's apparmor profile is updated. When the
profile is updated the container is restarted, so this wait is
essential to the success of the playbook's completion.

It also includes 3 retries which has been found to improve the
rate of success.

Due to an upstream change in behaviour with netaddr 0.7.16 we
need to pin the package to a lower version until Neutron is
adjusted and we bump the Neutron SHA.

Change-Id: I30575ee31929b0c9af6353b7255cdfb6cebd2104
Closes-Bug: #1490142
2015-09-02 09:21:55 +01:00
Kevin Carter
b59c08dd5f Update the tempest install environment
While this should be backported this purpose of this commit
is to unblock gating and provide for the always changing
requirements within tempest while also allowing us to ensure
that we are defcore complaint.

Tempest is not an integrated service within OpenStack and does
not adhere to any given package or requirement set that would
in a real work function with the rest of OpenStack. Because
tempest is intended to be a standalone system that is not
installed along side of the rest of OpenStack it general will
break and or introduce new requirements that break the services
that depend on various versions of packages as found within
global requirements. To fix this issue tempest is now being
installed within a VENV. The virtual environment will ensure
that tempest is installed in a location where it can resolve
its own dependencies without general impact to the rest of the
system.

Additionally, we removed the heat_contrib_extraroute heat
plugin from the build process because its presently
incompatible with PBR >= 0.11.0 which is related to issue
https://bugs.launchpad.net/openstack-ansible/+bug/1450733 .
However we have already built wheels in our repo which will
still allow this contributing plugin to be installed as an
integrated part of the system. Currently, we git clone heat
source onto heat_engine containers and install selected plugins
via a 'python setup.py install'.  This change removes the tasks
that do that and simply adds heat-contrib-extraroute
to heat_pip_packages so it gets installed on all heat-related
containers.  This is actually only required on the heat_engine
containers but the package is tiny and should not cause any issues
being present on the heat_api containers.

Change-Id: Ib972704084ead5748b19362b142fb161fea4a734
2015-05-06 07:46:39 -05:00
Kevin Carter
688516acf0 Lock down PBR to < version 0.11.0
This change was made to ensure that PBR is using version < 0.11.0.
This is because changes in upstream PBR have made it impossible to
build contributing packages in heat and will likely impact other
services until the switch to liberty is in full effect. To ensure
consistent building of OpenStack packages PBR is now being installed
in the repo server containers which will ensure there are no
assumptions being made when creating python packages from OpenStack
contributing sources that may not be as well maintained as others.

Change-Id: I20c216543e63ef142119c2625fb219e030a708dd
Partial-bug: 1450733
2015-05-01 10:53:11 -05:00
Matt Thompson
0452c5d81b Bump requirements in preparation for kilo
This commit brings o-a-d's requirements.txt in line w/
global-requirements.txt (master).  Also, we remove any packages that
are no longer required (a number were installed previously for tools
in scripts/ which no longer exist).  Additionally, we add some comments
indicating what requires a specific package.

NOTE: wheel requirement was originally added as pip < 6.0 required
wheels package in order to be able to install wheels themselves.  Since
we also bump pip to 6.0, wheel is no longer required.

Change-Id: I0de858283157cfb22db935bfe81a7ec23c9a7b9c
Closes-Bug: #1428106
2015-04-03 18:02:48 +00:00
Kevin Carter
8e6dbd01c9 Convert existing roles into galaxy roles
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.

Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
  simplistic approach. This change duplicates code within the roles but
  ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
  Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
  anyone who may want or need to dive into the JSON blob that is created.
  In the inventory a properties field is used for items that customize containers
  within the inventory.
* The environment map has been modified to support additional host groups to
  enable the seperation of infrastructure pieces. While the old infra_hosts group
  will still work this change allows for groups to be divided up into seperate
  chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
  variables extracted into the separate file
  etc/openstack_deploy/user_secrets.yml in order to allow seperate
  security settings on that file.

Items Excised:
* All of the roles have had the LXC logic removed from within them which
  should allow roles to be consumed outside of the `os-ansible-deployment`
  reference architecture.

Note:
* the directory rpc_deployment still exists and is presently pointed at plays
  containing a deprecation warning instructing the user to move to the standard
  playbooks directory.
* While all of the rackspace specific components and variables have been removed
  and or were refactored the repository still relies on an upstream mirror of
  Openstack built python files and container images. This upstream mirror is hosted
  at rackspace at "http://rpc-repo.rackspace.com" though this is
  not locked to and or tied to rackspace specific installations. This repository
  contains all of the needed code to create and/or clone your own mirror.

DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-18 10:56:25 +00:00
Ian Cordasco
28b6c2da9c Use HTTPS when installing from mirror.rackspace.com
SecurityImpact

Closes-bug: 1411331

Change-Id: I78716b737b37feab8b21d6e765a349e4e6724bf6
2015-01-15 11:32:18 -06:00
Darren Birkett
2793a1dbcb pin setuptools to 7.0 to workaround brokenness in 8.x
setuptools 8.0 (and subsequently 8.0.1, 8.0.2) were released over the
weekend.  This introduces a breaking change whereby python packages
installed from SHA, rather than with a proper release version, do not
have their version numbers parsed properly due to the appended sha.
Eg python-glanceclient 0.14.2.2.gb126351

This commit:

- pins setuptools to 7.0, to workaround this issue until a fix
is made upstream, inside the global requirements.txt

- removes package installations of python-setuptools and
python-pip which had potential to further conflict.

- removes the list pip_common_packages from the all_common vars file

Closes-Bug: 1402679

Closes-Bug: 1402672

Change-Id: Idb75f816f47740ee28ec5d7d8afb118acf4a71bb
2014-12-16 13:45:09 +00:00
d34dh0r53
95770070b2 Changes frozen repo to point to mirror.rackspace.com
The rpc-slushee.rackspace.com has numerous issues, so
we have created mirror.rackspace.com/rackspaceprivatecloud
to address those issues.  All instances of rpc-slushee
have been replaced.
2014-11-13 09:48:20 -06:00
Kevin Carter
e87efc7a79 changed ansible install from the package name to a URL as a tarball
updated ansible version to 1.6.10

This should resolve Issue: https://github.com/rcbops/ansible-lxc-rpc/issues/184
2014-09-30 10:45:25 -05:00
Kevin Carter
c6e5c9a74e Adjusted pip install config and repos
This PR ensures that the environment is always pointing at a frozen repository of PIP packages which consists of prebuilt wheels. A script was created to allow for packages and apps to be easily updated.

Resolves:
* https://github.com/rcbops/ansible-lxc-rpc/issues/113
* https://github.com/rcbops/ansible-lxc-rpc/issues/120

Begins resolving:
* https://github.com/rcbops/ansible-lxc-rpc/issues/31
* https://github.com/rcbops/ansible-lxc-rpc/issues/53

set all project group_vars to a given sha

added retry

added tempest
2014-09-15 18:48:12 -05:00
d34dh0r53
6f6e75f549 Initial Commit 2014-08-26 18:08:15 -05:00