406 Commits

Author SHA1 Message Date
Jenkins
111a9fb344 Merge "Fix Horizon SSL certificate management and distribution" 2015-07-21 20:14:13 +00:00
Jenkins
037829465e Merge "Adjust the network rings are syncd on" 2015-07-20 11:15:37 +00:00
Jenkins
808f24b265 Merge "Allow configuration of all default quota options" 2015-07-20 09:54:06 +00:00
Jesse Pretorius
ba7587267a Set haproxy install to use latest packages
This patch changes the apt task for installing haproxy packages
from only checking for presence to always checking for the
latest package version.

This is essential to allow a deployer to switch from a
configuration that does not implement SSL to one that does.

Change-Id: Iaf6eaedba835a332920336b1cb66190924537301
Closes-Bug: #1475597
2015-07-18 17:52:01 +00:00
Jenkins
b44dfc18a6 Merge "Container create/system tuning" 2015-07-17 23:51:08 +00:00
Jenkins
47db0fc106 Merge "Adjust key distribution mechanism for Swift" 2015-07-17 17:26:35 +00:00
kevin
4361ed12e0
Container create/system tuning
This change has shown to improve general performance and system stability.
Ubuntu 14.04 has some systemD bits within it and there are a few items that can
be tuned. These changes add some additional in container tuning the ensure that
the pseudo systemD availability within our containers is not impeding container
performance. Additionally the netfilter for bridged interfaces was set to 0 which
should also improve bridge performance in general.

Change-Id: Iad4f0472cf3e38c95c04b29cee8433c0781ff9ff
2015-07-17 09:59:10 -05:00
Jesse Pretorius
3a14a988b9 Fix Horizon SSL certificate management and distribution
This patch revises the SSL certificate management and
distribution with something that is more consistent with how
it's done everywhere else in the project. It also repairs the
current user provided certificate distribution which was broken.

* The server key/certificate (and optionally a CA cert) are
  distributed to all horizon containers.

* Two new variables have been implemented for a user-provided
  server key and certificate:
  - horizon_user_ssl_cert: <path to cert on deployment host>
  - horizon_user_ssl_key: <path to cert on deployment host>
  If either of these is not defined, then the missing cert/key
  will be self generated on the first Horizon container and
  distributed to the other containers.

* A new variable has been implemented for a user-provided CA
  certificate:
  - horizon_user_ssl_ca_cert: <path to cert on deployment host>

* A new variable called 'horizon_ssl_self_signed_subject' has
  been implemented to allow the user to override the self-signed
  certificate properties, such as the CN and subjectAltName.

Upgrade notes:

* The Apache configuration appropriately implements the
  'SSLCACertificateFile' instead of the 'SSLCACertificatePath'
  directive in order to ensure that the appropriate signing
  certificate is provided to the browser.

* The variable 'horizon_self_signed' (which defaulted to true)
  has been removed. The decision of whether to generate a
  self-signed certificate has been made based on whether a
  user provided key/cert pair has been provided.

* The 'horizon_self_signed_regen' variable has been renamed
  to 'horizon_ssl_self_signed_regen'.

* The default names for the deployed keys/certificates have been
  changed:
  - /etc/ssl/certs/apache.cert  > /etc/ssl/certs/horizon.pem
  - /etc/ssl/private/apache.key > /etc/ssl/private/horizon.key

DocImpact
UpgradeImpact
Closes-Bug: #1475578

Change-Id: I7089abbd81ce422b21ce65488e8bc32053ba32ca
2015-07-17 13:19:13 +01:00
Jenkins
e1d6751472 Merge "Fix haproxy service config when ssl is enabled" 2015-07-17 07:40:04 +00:00
Andy McCrae
57577bb369 Remove {{ from "with_items" and "when" statements
This patch resolves a minor issue that was generating a "warning"
message when running the galera-install.yml playbook.

The "{{" and "}}" are not required for conditionals, and loop
expressions, so these can and should safely be removed from "when:"
statements and "with_items:" statements.

Functionality does not change but the warning message will go away after
this patch.

Change-Id: I3e9d3a4afb0e3d1284c78a37dba6d1534154a063
Closes-Bug: #1475328
2015-07-16 15:09:41 +01:00
Jesse Pretorius
7984844008 Fix haproxy service config when ssl is enabled
This patch fixes the formatting for an haproxy service which has
ssl enabled.

Without this patch the 'reqadd X-Forwarded-Proto' line ends up
on the same line as the 'set request_option' line.

Change-Id: I0567797304646d7da9badfd193a1368b8a97b0db
Closes-Bug: #1475242
2015-07-16 11:24:15 +01:00
Jimmy McCrory
8f593e92a6 Allow configuration of all default quota options
Add all remaining configurable default quota options for nova, neutron,
and cinder.

Change-Id: I3bbbba2f2ca9495c7060ffa5ac5d70e865758518
Closes-Bug: #1446431
2015-07-15 23:47:52 -07:00
Jenkins
76680e9abd Merge "Adjust swift_rings.py to work on specified regions" 2015-07-16 01:37:39 +00:00
Jenkins
f59b88867a Merge "Cleanup unused functions in swift_rings.py" 2015-07-15 22:19:05 +00:00
Jenkins
68f9f73bf5 Merge "Add md5sum check for swift rings after ring-sync" 2015-07-15 21:04:42 +00:00
Jenkins
a85e50f693 Merge "Add openstackclient to the keystone containers" 2015-07-15 16:03:23 +00:00
Jenkins
6aad682f00 Merge "Restart mysql when config changed" 2015-07-15 15:38:48 +00:00
Andy McCrae
e1603af4f5 Add md5sum check for swift rings after ring-sync
Moving towards multi-region swift there is a chance that 2 regions will
attempt to update the ring at the same time. Whilst measures are in
place to ensure a region only updates its own region entries in the
ring it would still be possible, if the 2 runs happened simultaneously,
that some ring inconsistencies could happen. For example, if a region A
updates at the same time as region B but the sync order is different
some nodes could have region A's "updated" ring and some with region
B's "updated" ring.

To ensure this hasn't happened (without our knowledge) this patch adds
another md5sum check which will report if the rings are inconsistent
across the nodes.

Change-Id: Id88dfebcaa0553437953f92235bf63363f750797
Partially-Implements: blueprint multi-region-swift
2015-07-14 14:05:49 +01:00
Andy McCrae
66ccc1a260 Adjust the network rings are syncd on
Swift nodes don't have to be able to talk to each other on the
ansible_ssh_host, but will always have to talk on the storage_network
specified.

This will allow us to let remote or local hosts, that can't connect to
each other on the ansible_ssh_host address to still be able to sync
their rings.

In order to achieve this we set a swift_storage_address fact which then
simplifies the "ring_contents" file, to avoid performing the same logic
twice.

Change-Id: Ic1f2a915244101ad4fbbe52496dd2b991915d01d
Partially-Implements: blueprint multi-region-swift
2015-07-14 13:29:34 +01:00
Andy McCrae
8d65120e34 Cleanup unused functions in swift_rings.py
Removes 2 unused functions in swift_rings.py (check_section &
has_section), these are not called at all and should be removed for
clarity.

Change-Id: Id56654df92834f7a48ce21e70b372f04e920653e
Closes-Bug: #1474334
2015-07-14 13:06:31 +01:00
Ian Cordasco
8887318ee3 Add openstackclient to the keystone containers
Implements: blueprint keystone-federation
Closes-Bug: #1472694
Change-Id: Ia0a802e8203f49360b26aeb1be73fc8715e16fc7
2015-07-14 09:17:37 +00:00
Hugh Saunders
db43734df6 Add v3 calls for federation to keystone module
Add ensure functions for v3 resources that are needed for federation:
 * domain
 * group
 * identity provider
 * service provider
 * protocol
 * mapping

These are implemented using _ensure_generic to reduce duplicated code.

Partially-implements: blueprint keystone-federation
Change-Id: Ibf7fdd868d01f87414b59e541cc9b877be541639
2015-07-13 19:41:56 -07:00
Ian Cordasco
c1fdbab4de Upgrade the Keystone library to use v3
In order to enable and deploy federated Keystone, we need to use version
3 of the Keystone API and the v3 Keystone Client. This work begins that
transition by having a set of backwards compatible library commands.

Specifically, this commit updates the keystone library to use v3
Keystone Client and the usage of ensure_tenant in the os_keystone tasks
to use the v3 admin url.

In version 3 of Keystone's Endpoints (Catalog) API each endpoint only
has one URL and has separate interface types (public, internal, admin).
This change updates all uses of ensure_endpoint to structure the
endpoint data in a better way for the ensure_endpoint command in the
keystone module. As a result, some incidents where internalurl and
adminurl were swapped have been fixed.

Note:
In new deployments the endpoints will be created using the v3 API and
will therefore not be available via the v2 API. This will be a breaking
change to legacy CLI clients. The openstack CLI should be used instead.

DocImpact
Related-Bug: #1470635
Partially-implements: blueprint keystone-federation
Change-Id: I2cd4f505e850b4b113452abc25ee00d486b1637d
2015-07-13 19:41:44 -07:00
Andy McCrae
c927f4205e Adjust swift_rings.py to work on specified regions
This patch allows the swift_ring.py to only adjust/add/remove nodes from
a specified region, leaving the other regions that are already in the
ring unmodified.

This will allow multi-region swift to be managed by separate locations
each managing their own region's nodes and leaving other regions to
handle their own nodes.

The default is to manage all regions, so not specifying a region will
work the same as it does now and the script's functionality remains
unchanged.

Change-Id: I1cf73be20f27c437450c0181bb247c86e0f46bc6
Partially-Implements: blueprint multi-region-swift
2015-07-13 10:36:28 -07:00
Miguel Grinberg
36640a8f43 SSL support for haproxy
This change adds support for SSL to the haproxy role. When
enabled, this implements/upgrades haproxy to v1.5.x from a PPA.

* A new boolean variable called 'haproxy_ssl' enables/disables
  the configuration of SSL for the haproxy service.

* A new variable called 'haproxy_ssl_self_signed_subject' has
  been implemented to allow the user to override the certificate
  properties, such as the CN and subjectAltName.

* A new variable called 'haproxy_cert_regen' has been
  implemented to allow the user to regenerate the self-signed
  certificate used for the SSL endpoint.

* SSL will only be enabled for a load balanced service if
  haproxy_ssl is true in the service vars. This has only been
  implemented for the Keystone service endpoints in this patch.

* The keystone admin service endpoint will only have SSL enabled
  if keystone_service_adminuri_proto == 'https'.

* The keystone internal/public service endpoint will only have
  SSL enabled if keystone_service_publicuri_proto == 'https'.

Implements: blueprint keystone-federation
Change-Id: I069f1a0f928feb754816b7d450929fb62df66244
2015-07-13 16:05:40 +00:00
git-harry
4a860f715c Restart mysql when config changed
Currently mysql is only (re)started when creating a cluster, adding a
node or the service isn't running. This means changes to configuration
files are not used.

This commit moves the restart functionality to a handler and sets
notifies on the the appropriate tasks so that when the config is changes
mysql is restarted.

This commit modifies the galera play to run in serial so that only one
instance of mysql can be restarted at a time. This is to prevent the
possibility of them all being restarted in one go. An additional play
'Check galera cluster status' has been added, this makes an attempt to
check that the cluster is healthy before proceeding with any possible
config changes. The checks can be overridden by setting
'galera_ignore_cluster_state=true' when running the playbook.

A facts module has been added to gather information about the status of
the cluster for use by the galera-install playbook.

DocImpact
Closes-bug: #1449958

Change-Id: Id83ba642c114d1f5cac04cc219be151a82a1023f
2015-07-12 18:04:10 +01:00
Andy McCrae
7407cb9f97 Adjust key distribution mechanism for Swift
In preparation for Swift global cluster we need a solution that moves
away from a centralised "memcache" server as a distribution mechanism,
since we can't guarantee that all swift hosts will have access to one
memcache server.

This patch uses ansible variables (which were already being set) as the
distribution mechanism so that the authorized_keys file can be generated
by using the ansible variables set for each host as part of the tasks.

Additionally this moves away from using "shell" commands to generate the
key and adjust the authorized_keys file and instead uses the built in
ansible "authorized_keys" module as well as the built in
"generate_ssh_key" option for the "user" module.

Finally this adds a "swift_recreate_keys" option which can be set to
"True" in order to recreate all the swift keys, and recreate the
authorized_keys file. This was happening on every run and will now not
happen by default, unless the variable is set to "True".

Change-Id: Ic4eb05042244c32050233e1445633d5731f9603b
Partially-Implements: blueprint multi-region-swift
2015-07-12 10:25:22 +01:00
Jenkins
7d2a0231d8 Merge "Added openstack_kernel options for gc_thresh" 2015-07-11 04:55:38 +00:00
Jenkins
19775d4131 Merge "Parameterize galera slow/unindexed logging options" 2015-07-10 21:10:57 +00:00
Jenkins
f5388b61e4 Merge "Enable all services to use Keystone 'insecurely'" 2015-07-10 16:53:24 +00:00
Jesse Pretorius
4341b79b3a Enable all services to use Keystone 'insecurely'
This patch introduces an insecure flag for the Keystone internal
 and admin endpoints:

* keystone_service_adminuri_insecure
* keystone_service_internaluri_insecure

Both values default to false. If you have setup SSL endpoints
for Keystone using an untrusted certificate then you should
set the appropriate flag to true in your user_variables.

This patch is used to enable testing and development with
Keystone SSL endpoints without having to make use of SSL
certificates signed by a trusted, public CA.

The patch introduces a new optional argument (insecure) to the
keystone, glance and neutron Ansible libraries. This is a
boolean value which, when true, enables these libraries to
access Keystone endpoints 'insecurely'. When these libraries
are used in plays, the appropriate value is set automatically
as per the above conditions.

Implements: blueprint keystone-federation
Change-Id: Ia07e7e201f901042dd06a86efe5c6f6725e9ce13
2015-07-10 14:06:25 +01:00
Dave Wilde
70edf52fcd Parameterize galera slow/unindexed logging options
This change creates variables for Galera slow logging and logging
of unindexed queries.  The change also defaults these variables to
off.

DocImpact
Closes-Bug: #1463956

Change-Id: I5f786e2090c631627f3b44ada2baf05fd9699b97
2015-07-09 14:58:59 -07:00
Jenkins
3d1fade039 Merge "Implement Ceilometer" 2015-07-09 17:50:10 +00:00
kevin
b95827f351
Added openstack_kernel options for gc_thresh
This commit adds to the openstack_kernel_options to set the
"net.ipv4/6.neigh.default.gc_thresh*" values according to how much ram
is available on the box.

How this is being defined:
The change brings with it a filter to find the closest power of 2 from the
amount of ram discovered on the target host. If facts are disabled when the
role is called a default value of 1024 will be used. The `set_gc_val`, when
computed, has a max value of 8192. For both ipv4/6 thresh1 is half the
`set_gc_val` thresh2 is computed value, and finally both
thresh3/router.gc_thresh are double the `set_gc_val`.

The changes here should provide for a more scalable neutron networking
environment by default while also ensuring that the values computed are sane.
Additionallyi, should the user want to define their own values they can do so by
simply overriding the `set_gc_val`.

Change-Id: Ic5fd7ebdac009fa1472aeb0b0666f9b2611a31d7
Closes-Bug: #1427893
2015-07-09 10:20:31 -05:00
Jesse Pretorius
f827d8d015 Ensure that flake8 runs against all python files
This patch ensures that all python files (including the
Ansible modules) are included in the flake8 test.

This patch also fixes the current flake8 failures found in the
Ansible modules we're carrying.

Change-Id: I9ebef6ef7bbd2e424ad1be26d593471fd6fd3056
Closes-Bug: #1473002
2015-07-09 13:06:23 +01:00
Bjoern Teipel
94e399e351 Fixing protocol for UDP in horizon for master
Closes-Bug: #1472646
Change-Id: Ib5777fcd0f172aa68a99260539187ed9f50f7c98
2015-07-08 15:49:20 +00:00
Miguel Cantu
653f184733 Implement Ceilometer
This patch implements the implement-ceilometer blueprint.
It addes the necessary role/variables to deploy ceilometer
with a Mongodb backend. The Monogdb backend is assumed to
be up and configured and the playbooks only require a few
values to be set in user_variables to establish a connection.

Change-Id: I2164a1f27f632ce254cc2711ada2c449a9961fed
Implements: blueprint implement-ceilometer
2015-07-07 19:31:04 +00:00
kevin
980938ab01 Fixes RabbitMQ guest user creation
This change removes the "guest" user. This patch fixes an issues
where the guest user was not being cared by default which could
leave a deployment vulnerable to should someone gain access to the
internal network that the messaging system runs within. To fix this
issue the guest user is simply being removed as its not needed by the
rest of the stack.

Closes-Bug: #1466982
Change-Id: I81f9295daf7923b828ad884b6707bb3f74b0684a
2015-07-06 10:59:56 -05:00
Jenkins
8b5f0703c4 Merge "Use the correct upstream repos." 2015-07-03 16:06:07 +00:00
Jenkins
0fec0d63ba Merge "Rabbit config changes dont initiate a restart" 2015-07-03 03:31:55 +00:00
Tom Cameron
90bef6a3c4 Rabbit config changes dont initiate a restart
Restart rabbit-server when the config files change. This required
the templates to be rendered from the post_install playbook to
prevent attempting to restart a service that had not yet been
installed.

Change-Id: I7c9ba030aa0e58d3fb5fa3cb6f17dd013f36471c
Closes-Bug: #1460115
2015-07-02 17:21:13 -04:00
Jenkins
cdab4a5ef9 Merge "Updated default fernet key usage" 2015-07-02 20:45:51 +00:00
Jenkins
16f0944f1a Merge "Add role system-crontab-coordination" 2015-07-02 17:39:53 +00:00
Jenkins
0086ac33f6 Merge "Updated the clone process to exclude unneeded files" 2015-07-02 16:12:57 +00:00
Jenkins
935be2c473 Merge "Default to use host/port for glance_api_servers" 2015-07-02 15:28:33 +00:00
kevin
955c851cdd Updated the clone process to exclude unneeded files
This commit changes the clone process to properly exclude files from the repo
clone play/task. This also ensures that when this play is used that all local
repos are an exact match to what is upstream.

Change-Id: I840997d6778f40a9007bce18df792c6b0c69c5eb
Closes-Bug: #1468567
2015-07-02 08:29:36 -05:00
git-harry
e2f3662408 Remove unused swift vars
The following variables are defined as os_swift role defaults but are
unused:

    swift_service_admin_tenant_name
    swift_service_admin_username

This commit removes these unused variables.

Change-Id: I8272f4e398030e04c69a7092d4a770675e3c6df0
Closes-bug: #1460497
2015-07-01 17:54:48 +00:00
Jenkins
501a627c76 Merge "Configurable memcached connections limit & threads" 2015-07-01 13:34:59 +00:00
Andy McCrae
d8b4cf9e79 Default to use host/port for glance_api_servers
The glance_api_servers points to a list of glance_api_servers for both
cinder and nova. This causes "nova image-list" to fail when glance api
servers are unavailable. Pointing to the LB VIP works as intended, so
removing this var in favour of the "host/port" vars ensures that only
the glance servers that are available are used.

glance_api_servers is still available and if specified will be used in
favour of host/port, but default it is commented out and the host/port
will be used - which uses the internal_lb_vip_address and default
glance_api_service_port.

Change-Id: I6794a1a266d22944be8d5634ee0c0ce6cd9f2c59
Closes-Bug: #1461245
2015-07-01 11:17:10 +01:00
Jenkins
c0ce11cc6a Merge "Add support for specifying custom static routes" 2015-06-30 21:17:15 +00:00