148 Commits

Author SHA1 Message Date
Rob Crittenden
4b109297d5 Configure glance CA certificate path in cinder
A new configuration option is available in cinder for setting
the CA path. Configure this option in devstack when native SSL
or TLS proxy is configured.

Change-Id: I2cf9a3b8ba8e4896bd98efe0fb6f7d62fb279d90
Closes-Bug: 1384267
2014-10-21 18:17:48 -04:00
Rob Crittenden
18d4778cf7 Configure endpoints to use SSL natively or via proxy
Configure nova, cinder, glance, swift and neutron to use SSL
on the endpoints using either SSL natively or via a TLS proxy
using stud.

To enable SSL via proxy, in local.conf add

ENABLED_SERVICES+=,tls-proxy

This will create a new test root CA, a subordinate CA and an SSL
server cert. It uses the value of hostname -f for the certificate
subject. The CA certicates are also added to the system CA bundle.

To enable SSL natively, in local.conf add:

USE_SSL=True

Native SSL by default will also use the devstack-generate root and
subordinate CA.

You can override this on a per-service basis by setting

<SERVICE>_SSL_CERT=/path/to/cert
<SERVICE>_SSL_KEY=/path/to/key
<SERVICE>_SSL_PATH=/path/to/ca

You should also set SERVICE_HOST to the FQDN of the host. This
value defaults to the host IP address.

Change-Id: I36fe56c063ca921131ad98439bd452cb135916ac
Closes-Bug: 1328226
2014-09-24 18:36:37 -04:00
Brant Knudson
0595237e8a Function for auth_token middleware config
Each project was configuring the auth_token middleware using several
lines of inisets. Since all the projects should configure the
auth_token middleware in the same way create a function and call it.

Change-Id: I3b6727d5a3bdc0ca600d8faa23bc6db32bb32260
2014-09-21 11:18:01 -05:00
Dean Troyer
05bd7b803d Set default API_WORKERS
Set the API_WORKERS default to control memory usage. Maximum is nproc / 2 and
minimum is 2.

* Also updates https://review.openstack.org/#/c/117517/ to remove the
  conditional test as API_WORKERS should always be set.
* Update https://review.openstack.org/#/c/109058/ for ceilometer to use API_WORKERS

The following reviews can move forward either as-is or with minor tweaks:
* Keystone: https://review.openstack.org/#/c/121384/ - remove the if check
* Swift: https://review.openstack.org/#/c/121456/ - unabandon, the default
  to 1 is fine, or remove it to match the others.
* Trove: https://review.openstack.org/#/c/121438/ - remove the if check

https://etherpad.openstack.org/p/devstack-workers has the details

Change-Id: Id28d72ebf01c88b7df301edf7d1dd7ec23fcd0d6
2014-09-19 09:06:21 -05:00
Jenkins
efa18c73ab Merge "Replace screen_it() with run_process() throughout" 2014-09-13 12:38:34 +00:00
Jenkins
d577fdc794 Merge "Allow setting the number of workers to be used." 2014-09-13 07:18:05 +00:00
Chris Dent
2f27a0ed3c Replace screen_it() with run_process() throughout
run_process will use screen if USE_SCREEN=True (the default),
otherwise it will simply start the requested service. Therefore
wherever screen_it used, run_process can be instead.

Where stop_screen was found it has been replaced with stop_process.

A tail_log function has been added which will tail a logfile in a
screen if USE_SCREEN is True.

lib/template has been updated to reflect the use of the new
functions.

When using sg the quoting in run_process gets very complicated.
To get around this run_process and the functions it calls accepts
an optional third argument. If set it is a group to be used with sg.

Change-Id: Ia3843818014f7c6c7526ef3aa9676bbddb8a85ca
2014-09-11 18:59:39 +01:00
Jenkins
f94854811c Merge "Run processes without screen" 2014-09-10 12:00:40 +00:00
Bob Ball
2f72050ace Allow setting the number of workers to be used.
Environments with large numbers of CPUs will create a large
number of workers which can have an unnecessarily large
impact on memory usage, particular where you know how many
workers are needed.

Change-Id: Ie4bb075310a61a0873c9e56e4974600dbb4794a1
2014-09-09 15:54:36 +01:00
Dean Troyer
3159a821c2 Run processes without screen
This introduces new run_process() and screen_service() functions and sets the
groundwork to change how DevStack starts services.  screen_service() is simply a
direct call to the screen portion of the old screen_it() function and is intended
to run commands that only need to run under screen, such as log file watchers.

run_process() is a replacement for screen_it() (which remains until all of the
services are updated).  The usage is similar but requires updates to every current
screen_it() call to remove everything that requires the command to be interpreted
by a shell.

The old run_process() and _run_process() functions are still present as
old_run_process() and _old_run_process() to support the deprecated screen_it()
function.  These will all go away in the future once all services have been
confirmed to have been changed over.

There is a similar new set of stop process functions stop_process() and
screen_stop_service().  The old screen_stop() will also remain for the deprecation
period.

As an initial test/demostration this review also includes the changes for
lib/cinder to demonstrate what is required for every service.

I included the scripts I used to test this; tests/fake-service.sh and
tests/run-process.sh are quite rough around the edges and may bite.  They should
mature into productive members of the testing ecosystem someday.

Change-Id: I03322bf0208353ebd267811735c66f13a516637b
2014-09-09 13:54:01 +01:00
Dean Troyer
a25922b961 Fix regression in Cinder volume type creation
The local-vars cleanup (Iedeafa15def1fa26d4ddcbc5f670129f66e38b9d)
spanned (and missed) this fix (Ib35ffc09f6582354010ac8ca35da581ea508877a)
for volume type creation.

Change-Id: I2b95cbc946727d7708283eca0c07ba1eaddf5db8
2014-08-28 09:29:51 -05:00
Dean Troyer
e8a35acd1d Clean up local variable usage - cinder
Change-Id: Iedeafa15def1fa26d4ddcbc5f670129f66e38b9d
2014-08-19 19:12:27 -05:00
Ramy Asselin
57f8b52326 Allow cinder types to be created without be file
3rd party ci tests have not whitelisted errors:
"Not Whitelisted Volume type with name * could not be found."

These are creating noise. Instead of requiring a backend file
for each vendor, remove the if statement and just create the
necessary type for each defined back-end.

Change-Id: Ic88efa20eb53864e4f3d22d3306841797dcd118d
2014-08-15 16:21:40 -07:00
Matt Riedemann
6a4aa78fa3 Fix cinder volume type creation and rename variable
Commit 7062b12b9b91a1eb405d7079c62b62b0de59109f introduced
a bug where the default_volume_type is set to 'lvm' but
that is not the volume-type that's actually created, that
is 'lvmdriver-1' in gate runs, so we see a ton of
VolumeTypeNotFoundByName for the 'lvm' volume type in the
cinder-api logs.

This simply sets the default_volume_type to the volume
type that's created in the create_volume_types function.

Also renames the default_type variable to default_name to
be consistent with variable naming in the rest of the
cinder script.

Change-Id: Ib35ffc09f6582354010ac8ca35da581ea508877a
Closes-Bug: #1348786
2014-07-25 14:46:12 -07:00
Gael Chamoulaud
6dd8a8bee4 Users in service group should not have email addresses
Change-Id: Ieed9dffce5cf1e735e482dd3494ac1e103b50955
Closes-Bug: 1185201
Signed-off-by: Gael Chamoulaud <gchamoul@redhat.com>
2014-07-22 17:29:04 +02:00
Sébastien Han
7062b12b9b Fix default_volume_type for Cinder multi backend
The default_volume_type option doesn't support comma separated
arguments. We can only select one default volume type.
We have a variable that contains the default volume type, so we simply use it.

Closes-Bug: 1342569

Change-Id: Ibd080442f00f3d446ec374fc5906f0778d449007
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
2014-07-16 15:36:49 +02:00
Dean Troyer
097183356e Support multiple Cinder backend types
This is the first step in supporting multiple Cinder backend types at
once.  It initially converts the existing hard-coded multi-lvm support
to a new cinder_backends driver form.  Eventually the cinder_plugins
will be converted to this form so they can be enabled more than just
one at a time using CINDER_ENABLED_BACKENDS.

The default configuration should be identical to the previous defaults,
including for both True and False values of CINDER_MULTI_LVM_BACKEND.

The existing cinder_plugins are expected to be removed when this is
complete. They should continue to work until they have been converted.

Add wait for c-api to ensure it is started before continuing.

Change-Id: I93b8ef32832269d730c76a6dc24ddb4f20c6d9df
2014-07-15 16:03:42 +02:00
Bartosz Górski
0abde393c5 Adds support for multi-region
Change-Id: Ib85fe7cb375692b04aca4c46f61ba7e1fbfa501b
Implements: blueprint multi-region
2014-07-01 14:58:35 +00:00
Mike Perez
f64f43b533 Default cinder enable_v1_api to true
As of Juno, Cinder will be deprecating the Cinder V1 API and setting it
to not be enabled by default. We'll make it enabled in devstack to
continue tests while it's supported.

Change-Id: Ia09eed0375a73c69a29b1aa379b5bd9fd3a1f5b7
2014-06-25 08:53:55 -07:00
Jenkins
cab1eb12c6 Merge "cinder: use non-deprecated notification_driver" 2014-06-20 16:20:16 +00:00
Mark McLoughlin
dc0938e1c9 cinder: use non-deprecated notification_driver
Cinder has been using oslo.messaging since Icehouse so it should be
using 'messaging' as its notification driver and not the now deprecated
full python path to the oslo-incubator notification driver.

Change-Id: I3edca1ff54e51b464cc1b23935c8530e2e242fee
2014-06-17 06:37:43 +01:00
Jamie Lennox
3561d7f9ed Use identity_uri instead of auth fragments
auth_token middleware now accepts a standard URL string as the parameter
identity_uri instead of specifying protocol etc individually. Change the
services over to use this.

Also changes over some other places in which the auth fragments are used
individually to the new variables and fixes up some misconfigurations of
auth_token.

identity_uri option was release in keystoneclient 0.8.0

Change-Id: Iac13bc3d08c524a6a0f39cdfbc1009e2f5c45c2a
2014-06-16 15:16:48 +10:00
John Griffith
55c0b225e2 Remove copy of cinder.conf.sample file
Rather than continue to be bit by conf updates
from external sources, Cinder is going to join
the party and cease generation of the sample.conf
as part of the unit tests.

First step however is to not attempt to copy it
from cinder/etc/cinder in the devstack setup.

This just removes that cp operation, nothing
in that sample conf was modfied from defaults
so it wasn't really doing anything anyway.

Change-Id: I727b1747ff36d775ef881e93e3b9a8528a17a4df
2014-05-30 15:46:13 +00:00
Attila Fazekas
2983474e37 Use the $SERVICE_HOST in backup_swift_url
The $SERVICE_HOST is used to specify the swift proxy endpoint,
the c-bak should use the same endpoint.

Change-Id: Ia815f514839b0d1ec3fb9bb40992637c4f123e06
2014-03-10 19:52:49 +01:00
Sean Dague
d5b52ca755 fix tgt to use 'service' instead of upstart calls
the comments in here were largely about oneric, which we don't
support any more. service is installed in a precise environment,
and will support debian and the upcoming transition to systemd
better, so use that instead.

Change-Id: If15493549a8c93a7387df9b3bba31443aed46995
2014-03-04 09:23:07 -05:00
Ian Wienand
aee18c749b Enforce function declaration format in bash8
Check that function calls look like ^function foo {$ in bash8, and fix
all existing failures of that check.  Add a note to HACKING.rst

Change-Id: Ic19eecb39e0b20273d1bcd551a42fe400d54e938
2014-02-28 07:59:03 +11:00
Ian Wienand
3b57829ece Don't use --skip-redirect for cinder restart on rhel6
RHEL6 doesn't support this flag so the restart fails.  Not exactly
sure why it is required, seems unchagned from the initial
commit 67787e6b4c6f31388cbee6d83b67371b31c443d4

(found running stack.sh with -e per [1])

[1] https://review.openstack.org/#/c/71996/

Change-Id: Ib34c3663409d7b96b932286cb5a6974e940075d3
2014-02-26 14:55:20 +11:00
Steve Martinelli
19685428e3 Change most keystoneclient commands to openstacklient in libs
migrated most keystoneclient commands from the following libs:
 ceilometer
 cinder
 ironic
 keystone
 marconi
 neutron
 nova
 savanna
 swift
 trove

Also need to set and unset openstackclient specific environment
variables from stack.sh

Change-Id: I725f30bc08e1df5a4c5770576c19ad1ddaeb843a
2014-02-14 21:30:57 -06:00
Jenkins
1d081ebc2b Merge "Cleanup cinder-rootwrap support" 2014-02-15 02:21:13 +00:00
Jenkins
f0029532f8 Merge "Replace custom cinder driver configs" 2014-02-12 22:38:53 +00:00
john-griffith
d0860cc26d Replace custom cinder driver configs
The devstack/lib/cinder file has a number of third party
driver config options hard-coded in it.  Rather than add
yet another if driver== statement here let's use plugin
files and do something similar to what's already
in place for nova_hypervisors and neutron plugins.

This works the same way folks were implementing their
drivers already, the key is to use a CINDER_DRIVER
variable in your localrc file that matches the name
of the lib/cinder_plugin file to use.

The existing third party driver entries that were
in lib/cinder have been migrated to cooresponding
plugin files.

Change-Id: I4ee51ea542d5aa63879afd5297311a9df727c57f
2014-02-11 11:30:04 -08:00
Dean Troyer
e4fa721322 Begin is_service_enabled() cleanup
This converts the special cases in the is_service_enabled() function to call
individual functions declared by the projects.  This allows projects that
are not in the DevStack repo and called via the extras.d plugin to handle
an equivalent service alias.

* Ceilometer
* Cinder
* Glance
* Neutron
* Nova
* Swift

TODO: remove the tests from is_service_enabled() after a transition period

Patch Set 2: Rebased

Change-Id: Ic78be433f93a9dd5f46be548bdbd4c984e0da6e7
2014-02-07 10:06:21 -06:00
Thierry Carrez
d5d4974cb7 Cleanup cinder-rootwrap support
cinder_rootwrap support in devstack handled a number of now-abandoned
use cases:

- no $CINDER_DIR/etc/cinder/rootwrap.d (old-style rootwrap)
- using oslo-rootwrap instead of cinder-rootwrap (abandoned experiment)

This change removes unused code paths and aligns
configure_cinder_rootwrap() with configure_nova_rootwrap().

Change-Id: I387808dae0e064cc9c894c74ab78e86124f08dd2
2014-02-06 16:00:08 +01:00
Dean Troyer
4237f590b7 Generate Tempest service list rather than hard-code it
The list of services that Tempest used to set its 'service_available'
config values was hard-coded. To be plugin-friendly have each
service (project) add its name to the TEMPEST_SERVICES variable
and use that for setting the 'service_avilable' values.

Change-Id: I208efd7fd0798b18ac2e6353ee70b773e84a2683
2014-01-29 17:25:45 -06:00
Jenkins
adfaf0cef0 Merge "clean.sh updates" 2014-01-26 19:21:43 +00:00
Dean Troyer
53ffc713b1 clean.sh updates
* Clean out data, log and state dirs
* Include lib/apache to clear is_apache_enabled_service not found error
* Clean errors removing tgt config files

* Clean errors removing VG backing file in lib/cinder

Change-Id: I33dfde17eb8daaaed7f7e76337fe6a8085a266bf
2014-01-25 13:26:58 -06:00
Jenkins
e5a5fe33e0 Merge "Remove duplicated name services" 2014-01-13 18:50:59 +00:00
Jenkins
3e98388d07 Merge "Robustify service shutdown" 2014-01-13 14:09:44 +00:00
Jenkins
1d8abb95b9 Merge "Put cinder rootwrap config in separate function" 2014-01-12 13:51:27 +00:00
Jenkins
ec528ecae2 Merge "Add a flexible API version choice for Cinder, Glance and Heat" 2014-01-12 00:16:11 +00:00
Dean Troyer
9fc8792b0a Robustify service shutdown
* Save PID when using screen in screen_it()
* Add screen_stop()
* Call out service stop_*() in unstack.sh functions so screen_stop()
  can do its thing

Closes-bug: 1183449
Change-Id: Iac84231cfda960c4197de5b6e8ba6eb19225169a
2014-01-11 11:46:19 -06:00
Thierry Carrez
63e1784354 Put cinder rootwrap config in separate function
Separate out Cinder's rootwrap configuration so that it can be called
from Grenade's upgrade scripts. This follows the same model as Nova uses
with configure_nova_rootwrap() which can be called from Grenade to
refresh rootwrap config.

Change-Id: Id808abc2b5754443362b3de4b3453e305d3720f3
2014-01-10 14:23:03 +01:00
Dirk Mueller
25049cd23d Use --tenant-id, not --tenant_id
Change-Id: I0e3d65d5b69ac82cbf7ee6ffc41ead369af8c126
2014-01-09 13:55:28 +01:00
Juan Manuel Olle
4ad37630a2 Remove duplicated name services
Due to the fact that keystone will not allow services with
duplicated names, cinder and nova services names were
changed

Closes-Bug: #1259425
Change-Id: I988aef477b418a289426e02e5e108aa57dd1076b
2014-01-06 15:25:31 -03:00
Vincent Hou
21fe4e76d5 Add a flexible API version choice for Cinder, Glance and Heat
The version of the authentication url is set to v1.0 for some
projects by default. We can make it configurable via the parameter
"$IDENTITY_API_VERSION".

Closes-Bug: #1253539
Change-Id: I6640e345d1317b1308403c95b13f8a998320241b
2014-01-06 01:22:57 -05:00
Thierry Carrez
0718568b12 Support oslo-rootwrap in lib/cinder
Make lib/cinder support both cinder-rootwrap (current case) and
oslo-rootwrap (future case) to handle the Cinder transition towards
oslo-rootwrap usage peacefully.

Related blueprint:
https://blueprints.launchpad.net/cinder/+spec/cinder-oslo-rootwrap

Change-Id: I663986304bd74cb6d72d51c553540fb5f9db1d1d
2013-12-13 15:20:26 +01:00
Matt Riedemann
7103a84e39 Workaround Cinder "wishlist" bug 1255593 for multi-backend volume_clear
Cinder currently only applies the volume_clear setting from the DEFAULT
section of cinder.conf if you're using a single backend.

The Cinder team has determined this is a 'wishlist' item to propagate
volume_clear to each backend, but it does impact usability and
performance.

To improve the performance of running Tempest with multi-backends in the
gate, workaround the bug in devstack.

Related-Bug: #1255593

Change-Id: Ia0ff5422f53eeda9a3ac4336eefec3b9bdea6da2
2013-12-07 13:53:33 -08:00
Jamie Lennox
bd24a8d0f8 Allow deploying keystone with SSL certificates
Allow providing certificates through environment variables to be used
for keystone, and provide the basis for doing this for other services.
It cannot be used in conjunction with tls-proxy as the service provides
it's own encrypted endpoint.

Impletmenting: blueprint devstack-https
Change-Id: I8cf4c9c8c8a6911ae56ebcd14600a9d24cca99a0
2013-11-25 22:27:51 +00:00
Stephan Renatus
e578effb33 Make use of STACK_USER instead of relying on USER
Quite easily one ends up calling ./stack.sh in an environment that,
albeit being user "stack" (for example), doesn't quite meet the
expectations of devstack.  The errors that follow can be rather hard to
track down, as the dependency on `USER` is not mentioned.

To remedy this situation, this commit

- uses STACK_USER instead of USER and
- mentions that dependency in the script headers of lib/*

Change-Id: If4cdc39b922ea64b4c0893a0e695ec06349fccc5
2013-11-19 13:39:58 +01:00
Jenkins
b7c11a0f98 Merge "Fix some Markdown formatting issues" 2013-10-31 14:19:15 +00:00