156 Commits

Author SHA1 Message Date
Major Hayden
4a1d412f8c Add SSL/TLS listener to RabbitMQ
This patch adds a SSL/TLS listener to RabbitMQ without disrupting the existing
plaintext TCP listener.  Various services that use RabbitMQ will have the
option to encrypt messaging traffic with this change.  Documentation is
included for this change.

By default, it will create a self-signed certificate for the user, but users
have the option to specify their own existing certificates as well.

This makes it easier to bring RabbitMQ (and the services which talk to it)
into compliance with PCI DSS 3.1's Requirement 2.2.3.

In addition, this change is recommended within the OpenStack Security Guide.

Closes-bug: 1496001

Change-Id: I0d29cbb6e963b24f77a8375eba8a8c6a558aaf81
2015-09-25 08:11:54 -05:00
venkatamahesh
1b51320d5d Change the network from management to container
According to Line 60 of the modified file, the
network name is changed from management to
container

Change-Id: I819e99910a0276f9e9cca36dbdaf2b2d14e312fd
Closes-Bug: #1495704
2015-09-21 11:10:12 +00:00
Jenkins
235649dc78 Merge "Compartmentalizing RabbitMQ" 2015-09-12 23:24:07 +00:00
Kevin Carter
35ec780515 Rename everything for the osad to osa name change
This change updates all fo the names that we were using to the post
openstack migration name for openstack-ansible.

Change-Id: I6524af53ed02e19a0f56908e42a65d2dae8b71e3
2015-09-12 02:31:21 +01:00
kevin
b2624d4a26
Compartmentalizing RabbitMQ
Presently all services use the single root virtual host within RabbitMQ
and while this is “OK” for small to mid sized deployments however it
would be better to divide services into logical resource groups within
RabbitMQ which will bring with it additional security. This change set
provides OSAD better compartmentalization of consumer services that use
RabbitMQ.

UpgradeImpact
DocImpact
Change-Id: I6f9d07522faf133f3c1c84a5b9046a55d5789e52
Implements: blueprint compartmentalize-rabbitmq
2015-09-11 11:08:50 -05:00
Jean-Philippe Evrard
c667b2e164 Be more clear about used_ips, mostly in the example file
Cf. also bug description.

DocImpact
Closes-Bug: #1493820

Change-Id: If3621ecc115d5064b36e5a4f72df4929f99f4510
2015-09-09 16:35:25 +00:00
Jean-Philippe Evrard
ea0309e505 Updated the configuration to current dynamic inventory standards
documentation is outdated for management network (cidr section)
in openstack_user_config.

cidr_networks:
  management: 172.29.236.0/22

moved to:

cidr_networks:
  container: 172.29.236.0/22

This fixes the documentation

DocImpact
Closes-Bug: #1493451

Change-Id: I1669b58408ed2eb297209bb4b2e4171008b08e0f
2015-09-08 18:49:45 +02:00
Matthew Kassawara
6f6a37fce8 Disable scatter-gather offload on host bridges
Disable scatter-gather offload on host bridges to eliminate
kernel traces that may impact container connectivity. Only
addressing AIO interfaces for now as host configuration for
actual deployments resides in documentation.

Change-Id: Ia66b2bb64b9ace66f5fa3ca8edcc9909af54a4f2
Partial-Bug: #1488815
Co-Authored-By: Evan Callicoat <apsu@propter.net>
2015-09-04 13:57:02 +00:00
Jenkins
f770167e76 Merge "Add sorting_method to swift proxy config as needed" 2015-09-03 14:32:40 +00:00
Matt Thompson
44d3f25de6 Allow cinder-backup to use ceph
This change updates ceph_client and os_cinder roles to allow
cinder-backup to use ceph.  We also create a new group called
'cinder_backup' which allows us to only retreive the cinder backup key
if cinder-backup is actually in use.

To use, you would simply need to set cinder_service_backup_driver to
cinder.backup.drivers.ceph in your user_variables.yml file.

NOTE: You will need to update your
      /etc/openstack_deploy/env.d/cinder.yml in order for this change
      to execute successfully.

Change-Id: Ib94effa40208bbc8de0f78c5487316be007adcf1
Closes-Bug: #1481787
Implements: blueprint ceph-block-devices
DocImpact
2015-09-03 07:37:21 +00:00
Steve Lewis
39b0e5d443 Add sorting_method to swift proxy config as needed
When read_affinity is used and sorting_method is not used warnings
are generated in the swift proxy log indicating that the
read_affinity is not being respected. When read_affinity is specified
this change sets the sorting_method to affinity automatically, and
otherwise uses a configured value which defaults to shuffle.

Note that write_affinity does not respect sorting_method and follows
a different code path and does not issue warnings in logs when used
without sorting_method.

Closes-bug: 1480581
Co-Authored-By: Andy McCrae <andy.mccrae@gmail.com>
Change-Id: I3cab89c95f288b4a59f4dd3c7360daca7a4f47bf
2015-09-03 05:46:56 +00:00
george paraskevas
79e55616aa Enable HAProxy Stats Web UI
This patch enables the HAProxy webstats for all the configures
backends and frontends.

A password entry is added to user_secrets.yml for the webstats
password.

It also adds variables for port number, username and password
which can be overridden in user_variables.yml appropriately.

Change-Id: Iec866ad124bec6fb0b8524a966adf64e22422035
Closes-Bug: #1446432
2015-08-24 21:23:43 +00:00
Stanley Karunditu
57c5f2c77e Add configurable ssh_delay
This patch adds a configurable delay time for retrying the
ssh connection when waiting for the containers to restart.

This is useful for environments where resources are constrained
and containers may take longer to restart.

Change-Id: I0383e34a273b93e1b2651460c853cf1ceba89029
Closes-Bug: #1476885
2015-08-22 11:43:02 +01:00
Jenkins
25ef1c46a8 Merge "Update documentation for multiple VLAN ranges" 2015-08-22 00:02:42 +00:00
Jenkins
ced90e9a5d Merge "Update the documented ceph user variables" 2015-08-21 21:37:35 +00:00
Matthew Kassawara
c3025ed093 Update documentation for multiple VLAN ranges
Update example configuration file and installation guide to
mention support for multiple VLAN ranges on a particular
network and provide examples. Also address other minor
issues with networking content in the installation guide.

Change-Id: Idca97092f326cfbd83a018e853d3e7655d20e491
Partial-Bug: #1475436
2015-08-21 16:42:24 +00:00
Jenkins
980866d19f Merge "Ensure that nova and neutron provide consistent DHCP domain values" 2015-08-21 13:57:27 +00:00
Jesse Pretorius
7427775201 Ensure that nova and neutron provide consistent DHCP domain values
This patch adds the dhcp_domain config entry to nova.conf and
implements group_vars to default both nova and neutron's dhcp
domain values to be the same.

The individual values can still be overridden in user_variables
by using nova_dhcp_domain or neutron_dhcp_domain, but it's expected
that deployers would like these to be consistent.

Change-Id: I97beb78f62aeca2665ff72805056d36ead2adaaa
Closes-Bug: #1482045
2015-08-21 08:59:59 +00:00
Andy McCrae
f77cc7a861 Allow a log dir that isn't the "service_name"
Galera logs by default to /var/log/mysql_logs these logs are being
ignored and not mounted properly on the host server, because the
service_name that is used to mount the log directory is "galera" and not
"mysql_logs".

This patch adds an option to set the log_directory specifically inside
the env.d yml files, but defaults to "service_name" when the
log_directory option isn't set. This will mean functionality remains
unchanged for all services.

Additionally, the log_directory has changed to "mysql_logs" for galera
containers, to resolve the galera log issue.

Change-Id: I18400248da455156dbba31b9c2ff254b58c51673
Closes-Bug: #1486955
2015-08-20 18:19:02 +01:00
Serge van Ginderachter
0fafe4f8c3 Update the documented ceph user variables
The ceph patch missed an update to reflect the proper variables as presented in

    playbooks/roles/ceph_client/default/main.yml

into

    etc/openstack_deploy/user_variables.yml

DocImpact
Implements: blueprint ceph-block-devices
Closes-Bug: #1455238

Change-Id: Id450f8eb03493ed8413d748c893637173dbaf8f2
2015-08-19 12:04:55 +00:00
Andy McCrae
de8b857668 Add swift-sync role and environment
Add the swift-remote host group and environment file.

Add an os_swift_sync role which will sync the swift ring and ssh keys
for swift hosts (remote and not-remote). Which has the following:
 * Moves the key and ring tasks out of os_swift role to os_swift_sync.
 * This adds the use of the "-r" flag that was added to the
   swift_rings.py and swift_rings_check.py.
 * Adds a ring.builder vs contents file consistency check.
 * Adjusts the rsync process to use the built-in synchronize module
 * Ensure services have started post ring/ssh key sync.

Adds environment file and sample configuration file for swift-remote
hosts (conf.d).

Move appropriate default vars to the os_swift_sync role, and remove them
from the os_swift role.

Rename the "os-swift-install.yml" playbook to "os-swift-setup.yml" as
this handles only the setup, and add a playbook to for both
os-swift-sync.yml and an overarching playbook (os-swift-install.yml)
that will call both the os-swift-sync.yml and os-swift-setup.yml
playbooks. This means the funcitonality of "os-swift-install.yml"
remains unchanged.

Adjust the run-playbooks.sh so that it calls the new overarching swift
playbook.

Change-Id: Ie2d8041b4bc46f092a96882fe3ca430be92195ed
Partially-Implements: blueprint multi-region-swift
2015-08-07 08:45:18 +00:00
Serge van Ginderachter
b878370a0b Add Ceph/RBD support to playbooks
Currently the playbooks do not allow Ceph to be configured as a backend
for Cinder, Glance or Nova. This commit adds a new role called
ceph_client to do the required configuration of the hosts and updates
the service roles to include the required configuration file changes.
This commit requires that a Ceph cluster already exists and does not
make any changes to that cluster.

ceph_client role, run on the OpenStack service hosts
  - configures the Ceph apt repo
  - installs any required Ceph dependencies
  - copies the ceph.conf file and appropriate keyring file to /etc/ceph
  - creates the necessary libvirt secrets

os_glance role
glance-api.conf will set the following variables for Ceph:
  - [DEFAULT]/show_image_direct_url
  - [glance_store]/stores
  - [glance_store]/rbd_store_pool
  - [glance_store]/rbd_store_user
  - [glance_store]/rbd_store_ceph_conf
  - [glance_store]/rbd_store_chunk_size

os_nova role
nova.conf will set the following variables for Ceph:
  - [libvirt]/rbd_user
  - [libvirt]/rbd_secret_uuid
  - [libvirt]/images_type
  - [libvirt]/images_rbd_pool
  - [libvirt]/images_rbd_ceph_conf
  - [libvirt]/inject_password
  - [libvirt]/inject_key
  - [libvirt]/inject_partition
  - [libvirt]/live_migration_flag

os_cinder is not updated because ceph is defined as a backend and that
is generated from a dictionary of the config, for an example backend
config, see etc/openstack_deploy/openstack_user_config.yml.example

pw-token-gen.py is updated so that variables ending in uuid are assigned
a UUID.

DocImpact
Implements: blueprint ceph-block-devices
Closes-Bug: #1455238
Change-Id: Ie484ce0bbb93adc53c30be32f291aa5058b20028
2015-08-01 19:49:00 +01:00
kevin
a72d22e203
Removed inaccurate documentation
The basic user_variables.yml file was referencing a variable that no longer
exists. As such, it has been removed.

Change-Id: I11bfd8d9af0b94a57a49043cc595a13addd4c986
2015-07-27 09:44:04 -05:00
kevin
8cf1f44c9e
Change to set the container network MTU
This change adds the container network MTU option within the container
network LXC config file. This will allow a deployer to set the MTU within
a provider networks entry in openstack_user_config.yml.

Example:

  ....
  provider_networks:
    - network:
        container_bridge: "br-storage"
        container_type: "veth"
        container_interface: "eth2"
        ip_from_q: "storage"
        type: "raw"
        container_mtu: "9000"
        group_binds:
          - glance_api
          - cinder_api
          - cinder_volume
          - nova_compute
          - swift_proxy

This changes gives the deployer the ability to selectively set the mtu as
needed.

The dynamic_inventory.py script has been updated to allow for the MTU entry.

Example file documentation has been added to show how to use this new setting.

BackportPotential
DocImpact
Closes-Bug: #1477346

Change-Id: If8c0ee042d2f1322f8322ea6c8ee33606070d880
2015-07-22 19:25:01 -05:00
Jenkins
4f405f4704 Merge "Target AIO swift vars at specific containers" 2015-07-22 19:24:44 +00:00
Jenkins
3f6334c453 Merge "Moved user_group_vars to defaults" 2015-07-22 00:16:01 +00:00
git-harry
1ef256a308 Target AIO swift vars at specific containers
The variables defined in etc/openstack_deploy/conf.d/swift.yml are
added as hostvars by the dynamic inventory script. Currently the vars
are being added to hosts that do not require them.

There is a variable called limit_container_types that is used by the
dynamic inventory to filter the hosts to which variables are assigned.
This script makes use of that variable so that only swift hosts have
these variables set.

The dynamic inventory script checks to see if the limit_container_types
string is in the container name as the method of filtering. This means
that the AIO host, aio1, does not get swift_vars assigned to it. This
commit alters the filter so that the component is checked instead of the
container name.

Change-Id: I945594a55ce058d822b941d08eb0ad9c5614db15
Closes-bug: #1472261
2015-07-16 10:53:21 +01:00
Matthew Kassawara
cd1a6b28d3 Fix repo section in example config file
Fix repository target host section in the
openstack_user_config.yml.example to use repo-infra_hosts
instead of repo_hosts.

Change-Id: I5d51d5ed75be702fb25a2911094c528ddd1f632f
Closes-Bug: #1475116
2015-07-15 22:53:18 -05:00
kevin
30f9443c5d Moved user_group_vars to defaults
The change moves the user_group_vars.yml from openstack_deploy/ to
a standard default file. This will allow a deployer to override items
in the file while also allowing development to make changes to the file
that may impact its contents. When the file was moved out it was short
sighted to move it into 100% userspace and should have simply become the
main default file.

Change-Id: Icfc3efea0a43d55b5a23bca2f64d4d6d73542d10
Related-Bug: #1460516
2015-07-15 09:48:03 -05:00
Ian Cordasco
c1fdbab4de Upgrade the Keystone library to use v3
In order to enable and deploy federated Keystone, we need to use version
3 of the Keystone API and the v3 Keystone Client. This work begins that
transition by having a set of backwards compatible library commands.

Specifically, this commit updates the keystone library to use v3
Keystone Client and the usage of ensure_tenant in the os_keystone tasks
to use the v3 admin url.

In version 3 of Keystone's Endpoints (Catalog) API each endpoint only
has one URL and has separate interface types (public, internal, admin).
This change updates all uses of ensure_endpoint to structure the
endpoint data in a better way for the ensure_endpoint command in the
keystone module. As a result, some incidents where internalurl and
adminurl were swapped have been fixed.

Note:
In new deployments the endpoints will be created using the v3 API and
will therefore not be available via the v2 API. This will be a breaking
change to legacy CLI clients. The openstack CLI should be used instead.

DocImpact
Related-Bug: #1470635
Partially-implements: blueprint keystone-federation
Change-Id: I2cd4f505e850b4b113452abc25ee00d486b1637d
2015-07-13 19:41:44 -07:00
Miguel Grinberg
36640a8f43 SSL support for haproxy
This change adds support for SSL to the haproxy role. When
enabled, this implements/upgrades haproxy to v1.5.x from a PPA.

* A new boolean variable called 'haproxy_ssl' enables/disables
  the configuration of SSL for the haproxy service.

* A new variable called 'haproxy_ssl_self_signed_subject' has
  been implemented to allow the user to override the certificate
  properties, such as the CN and subjectAltName.

* A new variable called 'haproxy_cert_regen' has been
  implemented to allow the user to regenerate the self-signed
  certificate used for the SSL endpoint.

* SSL will only be enabled for a load balanced service if
  haproxy_ssl is true in the service vars. This has only been
  implemented for the Keystone service endpoints in this patch.

* The keystone admin service endpoint will only have SSL enabled
  if keystone_service_adminuri_proto == 'https'.

* The keystone internal/public service endpoint will only have
  SSL enabled if keystone_service_publicuri_proto == 'https'.

Implements: blueprint keystone-federation
Change-Id: I069f1a0f928feb754816b7d450929fb62df66244
2015-07-13 16:05:40 +00:00
Jenkins
f5388b61e4 Merge "Enable all services to use Keystone 'insecurely'" 2015-07-10 16:53:24 +00:00
Jesse Pretorius
4341b79b3a Enable all services to use Keystone 'insecurely'
This patch introduces an insecure flag for the Keystone internal
 and admin endpoints:

* keystone_service_adminuri_insecure
* keystone_service_internaluri_insecure

Both values default to false. If you have setup SSL endpoints
for Keystone using an untrusted certificate then you should
set the appropriate flag to true in your user_variables.

This patch is used to enable testing and development with
Keystone SSL endpoints without having to make use of SSL
certificates signed by a trusted, public CA.

The patch introduces a new optional argument (insecure) to the
keystone, glance and neutron Ansible libraries. This is a
boolean value which, when true, enables these libraries to
access Keystone endpoints 'insecurely'. When these libraries
are used in plays, the appropriate value is set automatically
as per the above conditions.

Implements: blueprint keystone-federation
Change-Id: Ia07e7e201f901042dd06a86efe5c6f6725e9ce13
2015-07-10 14:06:25 +01:00
Jenkins
e0b3dd163b Merge "Fixed minor typo in etc/README.rst "Global Options" section" 2015-07-10 10:53:17 +00:00
Jenkins
3d1fade039 Merge "Implement Ceilometer" 2015-07-09 17:50:10 +00:00
Charles Farquhar
054c6e6d1c Fixed minor typo in etc/README.rst "Global Options" section
Change-Id: I5f43e580f55c1b16e0a009602d8856e659782594
Closes-Bug: 1472884
2015-07-08 23:46:38 -05:00
Miguel Cantu
653f184733 Implement Ceilometer
This patch implements the implement-ceilometer blueprint.
It addes the necessary role/variables to deploy ceilometer
with a Mongodb backend. The Monogdb backend is assumed to
be up and configured and the playbooks only require a few
values to be set in user_variables to establish a connection.

Change-Id: I2164a1f27f632ce254cc2711ada2c449a9961fed
Implements: blueprint implement-ceilometer
2015-07-07 19:31:04 +00:00
Jenkins
935be2c473 Merge "Default to use host/port for glance_api_servers" 2015-07-02 15:28:33 +00:00
Andy McCrae
d8b4cf9e79 Default to use host/port for glance_api_servers
The glance_api_servers points to a list of glance_api_servers for both
cinder and nova. This causes "nova image-list" to fail when glance api
servers are unavailable. Pointing to the LB VIP works as intended, so
removing this var in favour of the "host/port" vars ensures that only
the glance servers that are available are used.

glance_api_servers is still available and if specified will be used in
favour of host/port, but default it is commented out and the host/port
will be used - which uses the internal_lb_vip_address and default
glance_api_service_port.

Change-Id: I6794a1a266d22944be8d5634ee0c0ce6cd9f2c59
Closes-Bug: #1461245
2015-07-01 11:17:10 +01:00
Jenkins
b06d3fb455 Merge "Move Cinder-volumes to "on metal"" 2015-06-27 03:49:37 +00:00
kevin
2086f6edb1 Updated tempest isolation options
This change updates the way that the packages are installed in an
isolated deployment. This ensures that our build packages are used
within the deployment while also allowing temptest to go online and
consume upstream packages from pypi. Additionally this updates
tempest-lib which is needed to make keystone w/ fernet tokens go as
there is an upstream issues with date timestamps with all versions
of tempest-lib that will prohibit the use of fernet tokens.

Change-Id: Ia479610b9ab69d64d1de756099d840b2ab331378
Related-Bug: #1466010
2015-06-24 23:29:42 +00:00
Andy McCrae
91bd96b9e1 Move Cinder-volumes to "on metal"
Cinder-volumes is tightly coupled to the storage host and doesn't
gain from the benefits of containerization. This patch moves the default
location for cinder-scheduler into it's own container on the
"storage-infra_hosts", and the default location for cinder-volumes to be
"on metal" in the openstack_environment cinder.yml file.

Added documentation to suggest that you may want to remove "is_metal:
true" from cinder-volumes service when not using lvm.

Change-Id: I1879b1aa6dbe69a5a7e769a43b849441a38feeb8
Closes-Bug: #1433240
2015-06-24 17:33:16 +01:00
Jenkins
0f1e9a8ad9 Merge "Add global endpoint_type_proto options" 2015-06-22 15:02:08 +00:00
Jenkins
8352c967ed Merge "Add read/write_affinity settings for Swift" 2015-06-21 01:34:52 +00:00
Andy McCrae
2b6b896105 Add global endpoint_type_proto options
This allows you to set the endpoint-type protocol globally for all
services, e.g. internaluri can be http, and publicuri can be https.

You will no longer have to specify it per service, although those
settings already exist and have not changed.

This patch changes no functionality for existing installs or deployments
and the values are defaulted to be the same as before, but allows these
values to be adjusted on a per-endpoint type basis.

Change-Id: I4854216726491f6ea4e265694e702f980fddc5a6
Closes-Bug: #1399383
2015-06-19 15:29:30 +01:00
Andy McCrae
5b9b49f52b Add read/write_affinity settings for Swift
Allow the setting of read/write_affinity and write_affinity_node_count
on a per proxy_host basis.

This allows the deployer to set preferences for which region to
read/write to, which can increase the efficiency of a multi-region
swift cluster.

Sample swift.yml has been updated, as well as the aio swift.yml to
ensure these settings are setup as part of the gate, but this shouldn't
change the functionality of swift at all (since there is only 1 region).

Change-Id: I95b456672f419fcc331d6739ce259b022d350472
Closes-Bug: #1415172
2015-06-19 12:30:38 +01:00
Jesse Pretorius
337f9949df Implement service_region user_group_var
Currently each service has an individual service_region var
which needs to be set when setting up a region that is
different to the default.

This patch implements a user_group_var called 'service_region'
which can be used to set the region for all services.

The AIO bootstrap script is also altered to set the
service_region in the user config.

Change-Id: I5ed216916f667506ffed900a01744dad973221bf
Closes-Bug: #1463366
2015-06-17 17:11:43 +01:00
Andy McCrae
1a2907c6a1 Allow protocol to be set per endpoint-type
If services are running behind an SSL terminating LB you will want to
differentiate between protocol on internalURL and publicURL endpoints.
This patch allows you to set the values of protocol per endpoint type,
but doesn't change the default behaviour which is to have it set in one
var.

Change-Id: I7a74c85a8841499623746586ae27103a71c6fec0
Partial-Bug: #1399383
2015-06-16 08:57:17 +00:00
Jesse Pretorius
ecf7729795 Split environment file into component parts
This patch splits the monolithic openstack_environment file
into component parts, making them easier to understand and
manipulate.

Change-Id: I45887599bd887a272798047f9ad650d4811ab380
Closes-Bug: #1463862
2015-06-15 19:26:18 +00:00
Matt Thompson
495f8b31d6 Move swift_allow_all_users to role defaults
We currently default swift_allow_all_users in
etc/openstack_deploy/user_variables.yml instead of
playbooks/roles/os_swift/defaults/main.yml.  If a deployer removes this
variable from etc/openstack_deploy/user_variables.yml, the swift
playbook will fail.  This commit moves the variable default to
playbooks/roles/os_swift/defaults/main.yml.

Change-Id: I9a73eda990327bf427f40a13965484fde00cbe21
Closes-Bug: #1424981
2015-06-12 10:16:16 +01:00