1342 Commits

Author SHA1 Message Date
Ian Cordasco
58d0daef9b Use playbook status to report upgrade failure
Previously, we simply checked $? which at that point would be the exit
status of echo, not openstack-ansible. By recording the actual
openstack-ansible exit status, we can properly report failures of the
upgrade script.

Closes-bug: 1480342
Change-Id: Icf43bea84660e4160a2dfcdb4ac93055340b3573
(cherry picked from commit 8a106d184a566f1baed616aee7747d5f2d581c1b)
2015-08-13 18:18:20 +00:00
Jenkins
4c9d94880f Merge "Verify lxc cache file after download" 2015-08-13 17:59:47 +00:00
Jenkins
6c81e82edc Merge "Cinder_volumes_containers are automatically tagged with is_metal" 2015-08-13 15:12:51 +00:00
Jenkins
788f742179 Merge "Document required repository hosts config info" 2015-08-13 11:58:00 +00:00
Jenkins
0be330058f Merge "Documentation of the static-route feature" 2015-08-13 11:54:32 +00:00
Nolan Brubaker
7431aafa86 Document required repository hosts config info
Installs must have either the repo-infra_hosts list, or the
openstack_repo_url defined. Otherwise, greenfield installations
following our installation docs will fail.

Change-Id: I116040302e846530895836dd8aab9d4136b110af
Closes-Bug: #1475000
2015-08-13 12:45:57 +01:00
Jean-Philippe Evrard
e7036b8e8d Documentation of the static-route feature
This documents the change Id5a74db2399166af2d6ac289b71ebb0de04f5679.

DocImpact
Change-Id: I0977fb5c8372b42fbd62c52c62f6a5162fc6962b
2015-08-13 12:43:22 +01:00
Jean-Philippe Evrard
4bf2a1f37c Cinder_volumes_containers are automatically tagged with is_metal
In env.d/cinder.yml there is is_metal:true. 
But there was no mention of it in the documentation.
Therefore, if an user wants to use a cinder volume container with 
netapp/ceph/whatever, the container will be (by default) 
considered as metal.
This should be documented somewhere.

Change-Id: I65e9d0654d50d8c8825f858e89fdf4595134dddb
2015-08-13 12:00:49 +01:00
kevin
e65066cf12 Remove hardcoded config drive enforcement
This change removes the forced use of config drive to ensure that a user
can choose to use config drive as needed. This adds ability to
disable/enable config drive and allows libvirt to listen for connections
on tcp as needed for live migrations (prohibited otherwise by config drive).

The following new variables were added to os_nova role:

nova_force_config_drive
nova_libvirtd_listen_tls: 1
nova_libvirtd_listen_tcp: 0
nova_libvirtd_auth_tcp: sasl

Change-Id: I1de35a4b3611b8bc33a21930dae3fd38f9aaa151
Closes-Bug: #1468514
DocImpact
2015-08-13 10:03:37 +01:00
Jenkins
a19a6a7a48 Merge "Updated master for new dev work - Liberty-2" 2015-08-12 23:58:15 +00:00
kevin
dac24618c0 Updated master for new dev work - Liberty-2
Update all branches to Liberty-2.

Also, as of Change ID I3823900bc5aaf7757c37edb804027cf4d9c757ab
the new neutron releases have a new db upgrade and stamp process
in order for these version to be rev'd we need to incorporate
those change. As such the neutron_db_setup.yml has been updated
along with the neutron `neutron_db_revision` default variable.

Change-Id: Icfb75d377498e288e67be1a8bc049b42d8aa57b1
2015-08-12 21:57:38 +00:00
Major Hayden
8dee2959fb Small readme fix
Change-Id: If70fad4160dcbc3c3af197b3329e6a1d7bc34ee4
2015-08-12 21:57:20 +00:00
David Stanek
ed6ebcca03 Adds ceilometer link in configure.rst
This gets rid of the warning message saying that nothing is actually
linking to the document.

To get rid of the chicken-egg problem:

The -infra templated job for docs requires that a venv be created using
tox. It will actually run this command to build the documentation:

  tox -evenv -- python setup.py build_sphinx

Change-Id: I0f03ad6efe2a997c9cecac6240e1e8be8e85ccf6
2015-08-12 17:20:52 +00:00
Jenkins
169432e10b Merge "Add ebtables to neutron agent configuration" 2015-08-12 13:21:07 +00:00
Jenkins
9062b468c6 Merge "Improve Keystone Apache configuration" 2015-08-12 13:14:43 +00:00
David Stanek
0ec3ce3d31 Uses tox for automating documentation builds
Change-Id: I8012889aa6938378b5e5e85fb443cf13be460d61
Closes-Bug: #1469870
2015-08-12 06:32:40 +01:00
Jenkins
eaf58659c5 Merge "Move ansible logging to gate-check script only" 2015-08-11 19:54:36 +00:00
Jenkins
8376133b48 Merge "Add resume_guests_state_on_host_boot to nova.conf" 2015-08-11 19:19:09 +00:00
Jenkins
a5575eeb72 Merge "Re-order setup-openstack play to match the gate check order" 2015-08-11 19:17:28 +00:00
Jenkins
ede3e48e5b Merge "Add swift service region to group vars" 2015-08-11 19:17:25 +00:00
Christopher H. Laco
203e6dfb42 Add resume_guests_state_on_host_boot to nova.conf
Add the ability to enable the resume_guests_state_on_host_boot flag in
nova.conf to start guests that were running before the host rebooted.

Change-Id: I7365d972dc7e41a46b340396a73518b1da918f05
Closes-Bug: 1483246
2015-08-11 10:41:30 -04:00
Jesse Pretorius
a1eebe6afd Add ebtables to neutron agent configuration
Neutron now uses ebtables as an extra security layer for ARP
spoof filtering. This patch adds the ebtables package and
rootwrap to the neutron role to ensure that the agent is able
to use this subsystem. Without it the networking from the
instances to the L3 router will fail.

Co-Authored-By: Evan Callicoat <diopter@gmail.com>
Closes-Bug: #1482756
Change-Id: Ibc960564a3acfbb10cfbc3cfe0ad60d3366d2443
2015-08-09 19:51:10 +01:00
Jesse Pretorius
7f91978a46 Move ansible logging to gate-check script only
Enacting the log link creation and the ansible.cfg change has
resulted in polluted patch reviews by developers making use of
AIO's for dev/test purposes.

This patch moves the Ansible logging changes to the
gate-check-script only as that's the only time that it's
actually required.

Change-Id: I4a1accad94ae153bf363b53fda0905e814c15173
Closes-Bug: #1479824
2015-08-09 09:58:46 +00:00
Jesse Pretorius
98c9768497 Improve Keystone Apache configuration
This patch does the following:

1. Introduces two new Keystone variables which are useful for
   debugging the Keystone service. The values are defaulted
   to the same values as before the patch.
   - keystone_wsgi_processes: number of wsgi processes to run
   - keystone_wsgi_threads: number of wsgi threads to run

4. Moves the keystone service and admin processes into their
   own wsgi groups for better isolation.

5. Sets each wsgi process to run under the keystone group.

6. Bring the configuration file in line with the upstream
   recommended configuration as at 4 Aug 2015 in order to
   overcome import race conditions.

Change-Id: I861d1ef233dd6121452dc0e9e590d2d9f9b7973e
Closes-Bug: #1481339
2015-08-09 09:02:07 +01:00
Miguel Grinberg
27831a026f Wrapper script to perform K2K federated login
This patch adds a small script that automates the process of accessing a
service provider (SP) cloud using credentials from a identity provider
cloud (IdP), where both clouds use Keystone based authentication. The
script performs the complete authentication flow and displays the token
and endpoints to use with the openstack command line client.

Implements: blueprint keystone-federation
Change-Id: I4b8113d0aef9c754fb55497d44138df660332bb8
2015-08-08 14:17:51 +00:00
Jenkins
126b2e7d8e Merge "Add sample Keystone Federation SP configuration for ADFS" 2015-08-08 01:43:54 +00:00
Jenkins
e29295cde9 Merge "Reduce neutron configuration" 2015-08-07 23:47:03 +00:00
Jenkins
de0f5eeaf3 Merge "Keystone Federation Service Provider Configuration" 2015-08-07 21:44:33 +00:00
Jenkins
da3cfda0b1 Merge "Add swift-sync role and environment" 2015-08-07 21:33:44 +00:00
Jesse Pretorius
b86ad26f7d Add sample Keystone Federation SP configuration for ADFS
An ADFS v3.0 (Windows 2012 R2) Identity Provider is capable of
interacting via SAML2 to the Service provider, so there is no
special configuration over and above the same as required from
the TestShib/Keystone IdP.

This patch adds a sample configuration to the defaults file.

DocImpact
Implements: blueprint keystone-sp-adfs-idp
Change-Id: I37728e618d4624699a00f4ecfbb8cab0745e9e52
2015-08-07 20:37:04 +00:00
Jesse Pretorius
e5440cfcbe Verify lxc cache file after download
This patch adds a sha256sum verification to the lxc cache file
download task and also sets the task to retry.

Change-Id: Ie6342c1ee004a3d2de2256408361259d2fb47f1b
Closes-Bug: #1482091
(cherry picked from commit 0ccf11eeddaad8b8f4b53e3a7cf3f33f81d208ee)
2015-08-07 17:24:32 +00:00
Jenkins
f23e8e0c7d Merge "Properly parse and quote arguments in upgrade script" 2015-08-07 15:16:30 +00:00
Jenkins
e61aee7d41 Merge "Allow Swift middleware to be set via a variable" 2015-08-07 10:09:26 +00:00
Andy McCrae
de8b857668 Add swift-sync role and environment
Add the swift-remote host group and environment file.

Add an os_swift_sync role which will sync the swift ring and ssh keys
for swift hosts (remote and not-remote). Which has the following:
 * Moves the key and ring tasks out of os_swift role to os_swift_sync.
 * This adds the use of the "-r" flag that was added to the
   swift_rings.py and swift_rings_check.py.
 * Adds a ring.builder vs contents file consistency check.
 * Adjusts the rsync process to use the built-in synchronize module
 * Ensure services have started post ring/ssh key sync.

Adds environment file and sample configuration file for swift-remote
hosts (conf.d).

Move appropriate default vars to the os_swift_sync role, and remove them
from the os_swift role.

Rename the "os-swift-install.yml" playbook to "os-swift-setup.yml" as
this handles only the setup, and add a playbook to for both
os-swift-sync.yml and an overarching playbook (os-swift-install.yml)
that will call both the os-swift-sync.yml and os-swift-setup.yml
playbooks. This means the funcitonality of "os-swift-install.yml"
remains unchanged.

Adjust the run-playbooks.sh so that it calls the new overarching swift
playbook.

Change-Id: Ie2d8041b4bc46f092a96882fe3ca430be92195ed
Partially-Implements: blueprint multi-region-swift
2015-08-07 08:45:18 +00:00
Miguel Grinberg
23da164fe4 Keystone Federation Service Provider Configuration
This patch adds the ability to configure Keystone as a Service
Provider (SP) for a Federated Identity Provider (IdP).

* New variables to configure Keystone as a service provider are now
  supported under a root `keystone_sp` variable. Example configurations
  can be seen in Keystone's defaults file. This configuration includes
  the list of identity providers and trusted dashboards. (At this time
  only one identity provider is supported).

* Identity provider configuration includes the remote-to-local user
  mapping and the list of remote attributes the SP can obtain from the
  IdP.

* Shibboleth is installed and configured in the Keystone containers when
  SP configuration is present.

* Horizon is configured for SSO login

DocImpact
UpgradeImpact
Implements: blueprint keystone-federation
Change-Id: I78b3d740434ea4b3ca0bd9f144e4a07026be23c6
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
2015-08-07 08:44:51 +00:00
Ian Cordasco
ef3e743113 Properly parse and quote arguments in upgrade script
This patch fixes the following:

1. Properly quote arguments to run_lock function
2. Properly parse out the playbook filename in run_lock

Specifically the upgrade steps where we were using

   "-e 'rabbitmq_upgrade=true' setup-infrastructure.yml"
   "/tmp/fix_container_interfaces.yml || true"

Were causing issues and this patch resolves them.

Closes-bug: 1479916
Change-Id: I809085d6da493f7f7d545547a0d984c0e7b1bf45
(cherry picked from commit 560fbbdb077c0f8d6f8bfa9b48e967ccef86664a)
2015-08-07 08:41:32 +00:00
Matthew Kassawara
4f4d81acb5 Reduce neutron configuration
Reduce neutron configuration as follows:

1) Limit [ml2*] sections to neutron server containers [1].
2) Remove the [vlan] section everywhere because it only
   pertains to the defunct Linux bridge monolithic
   plug-in [2].
3) Explicitly disable VXLAN if deployment only includes flat
   or VLAN networks [3].
4) Limit Linux bridge agent configuration options to neutron
   agent containers.
5) Remove [agent] tunnel_type option because the Linux bridge
   agent does not use it.
6) Move some options to correct locations.
7) Reorder some options to improve readability.
8) Annotate groups of options or specific options.

[1] https://review.openstack.org/#/c/196759/
[2] https://review.openstack.org/#/c/196765/
[3] https://review.openstack.org/#/c/160826/

Change-Id: I275fb600360530534f7673e6eb2a3d397b10fb8e
Closes-Bug: #1473230
2015-08-07 08:08:44 +00:00
Miguel Grinberg
453ba074c2 Add swift service region to group vars
The swift service region is currently only set in the swift playbook
defaults file. It needs to be exposed in group vars and controlled by
the global `service_region` variable.

Closes-Bug: #1481700
Change-Id: I458be04582c590d58d1e7130878a9dc1f914c02a
2015-08-07 08:00:56 +00:00
Jesse Pretorius
5a5012996a Re-order setup-openstack play to match the gate check order
This patch re-orders the setup-openstack playbook to match
the order in which the gate check is executed.

Change-Id: I9e83a3b276fb99674cce7ee19a6e1ad860924492
Closes-Bug: #1481355
2015-08-06 21:21:42 +00:00
Jesse Pretorius
d82bbb4336 Remove unused python clients from repo & tempest
This patch removes the unused python clients from the tempest
role and the openstack_clients in the repo as these projects
may introduce incompatible requirements to the projects we
deploy and support.

Change-Id: I2d412dea8d91c94fc4ff9a5f64c19ae9c44fed8e
Closes-Bug: #1482260
2015-08-06 19:43:27 +01:00
Jenkins
6bf8a7b0f4 Merge "Change ansible forks used" 2015-08-03 17:59:49 +00:00
Jenkins
0c25d33422 Merge "Add Ceph/RBD support to playbooks" 2015-08-03 13:16:36 +00:00
Serge van Ginderachter
b878370a0b Add Ceph/RBD support to playbooks
Currently the playbooks do not allow Ceph to be configured as a backend
for Cinder, Glance or Nova. This commit adds a new role called
ceph_client to do the required configuration of the hosts and updates
the service roles to include the required configuration file changes.
This commit requires that a Ceph cluster already exists and does not
make any changes to that cluster.

ceph_client role, run on the OpenStack service hosts
  - configures the Ceph apt repo
  - installs any required Ceph dependencies
  - copies the ceph.conf file and appropriate keyring file to /etc/ceph
  - creates the necessary libvirt secrets

os_glance role
glance-api.conf will set the following variables for Ceph:
  - [DEFAULT]/show_image_direct_url
  - [glance_store]/stores
  - [glance_store]/rbd_store_pool
  - [glance_store]/rbd_store_user
  - [glance_store]/rbd_store_ceph_conf
  - [glance_store]/rbd_store_chunk_size

os_nova role
nova.conf will set the following variables for Ceph:
  - [libvirt]/rbd_user
  - [libvirt]/rbd_secret_uuid
  - [libvirt]/images_type
  - [libvirt]/images_rbd_pool
  - [libvirt]/images_rbd_ceph_conf
  - [libvirt]/inject_password
  - [libvirt]/inject_key
  - [libvirt]/inject_partition
  - [libvirt]/live_migration_flag

os_cinder is not updated because ceph is defined as a backend and that
is generated from a dictionary of the config, for an example backend
config, see etc/openstack_deploy/openstack_user_config.yml.example

pw-token-gen.py is updated so that variables ending in uuid are assigned
a UUID.

DocImpact
Implements: blueprint ceph-block-devices
Closes-Bug: #1455238
Change-Id: Ie484ce0bbb93adc53c30be32f291aa5058b20028
2015-08-01 19:49:00 +01:00
Jesse Pretorius
fb6438e8d8 Enable Horizon to consume a Keystone v3 API endpoint
This patch enables Horizon to consume a Keystone v3 API endpoint.

This patch also introduces two variables to allow the endpoint to be
specified independently if required:
 - horizon_keystone_host: this defaults to the internal LB IP address
 - horizon_keystone_endpoint: this defaults to the internal Keystone
   endpoint

This patch also does the following:
 - properly consumes the horizon_ssl_no_verify role setting;
 - includes a little comment cleanup which does nothing but clutter
   the local_settings configuration file.

Closes-Bug: #1478996
Change-Id: I5b7ceeecab072ead6fd380dcef7a48f1978a56f2
2015-07-31 08:25:09 +00:00
Jesse Pretorius
7b10c64007 Change ansible forks used
This patch changes the number of forks used by ansible when
using any of the convenience (and thus gate check) scripts
to the number of processors available on the deployment
system.

The previous values used were found to cause ssh connection
errors and it was found that reducing the number improved
the chances of success.

This patch also removes the forks setting from ansible.conf
so that ansible will use the default value when run in any
other way. This leaves the decision of setting the number
of forks to the deployer, as it should be.

Change-Id: I31ad7353344f7994063127ecfce8f4733769234c
Closes-Bug: #1479812
2015-07-30 15:11:14 +01:00
Jenkins
98153efac1 Merge "remove conntrackd package" 2015-07-30 12:40:26 +00:00
Jenkins
4e17a53952 Merge "Keystone Federation Identity Provider Configuration" 2015-07-29 23:41:12 +00:00
Jenkins
cf93a8de54 Merge "Remove reference to missing user_group_vars file" 2015-07-29 22:52:28 +00:00
Nolan Brubaker
928af8a9ac Remove reference to missing user_group_vars file
The in-tree version of user_group_vars.yml was removed in
30f9443c5d2f3a3bbb51bf75ad5743ef46c9b0ef, but the corresponding
reference in the upgrade script was not also updated.

This commit changes the behavior to remove the file from /etc/ if found.

Change-Id: I9f5b061289c5f43e32983845469f5123cc9f209d
Closes-Bug: #1479501
2015-07-29 17:15:24 -04:00
Jenkins
9290e8871e Merge "update teardown.sh to remove mongodb" 2015-07-29 20:15:20 +00:00