As part of Queens Goal "Split Tempest Plugins into Separate
Repos/Projects",the cinder tempest plugin needs to be split
into a separate project.
https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html
This patch removes the in-tree tempest plugin as well as the
setuptools entry point.
Change-Id: I5ff95ca13f02ed6e0afb35f0d6bb5c36d4111f5a
Depends-On: Ib309cb1f87211e82eae0d27fa2464ec13a9deca6
Implements: blueprint goal-split-tempest-plugin
This completes the "policy-in-code" work for Cinder by removing
the old policy.json handling. It also adds new in-code checks to
the legacy consistency group code for completeness.
Change-Id: I810b6cb6bba2d95cc5bb477d6e2968ac1734c96b
Depends-on: I364e401227fe43e2bacf8a799e10286ee445f835
Implements: bp policy-in-code
This is the basic patch which consits of the framework
code for default policy in code feature as well as
support in attachment resource.
Change-Id: Ie3ff068e61ea8e0e8fff78deb732e183e036a10c
Partial-Implements: blueprint policy-in-code
In order to make it simpler to use the default
configuration files when deploying services
from source, the files are added to pbr's
data_files section so that the files are
included in the built wheels and therefore
deployed with the code. Packaging and deployment
tools can then more easily use the default files
if they wish to.
This pattern is already established with similar
files for neutron and the glance metadefs as has
been mentioned in the related bug report.
Change-Id: Idbc616e9ca23b1b7187f39a1e157f56862851f6c
Closes-Bug: #1718356
We should be treating warnings from Sphinx as
errors as they usually result in bad HTML output.
Now that we have all of the existing warnings resolved we
want to turn on this option so that more bad docstrings, etc.
do not get merged.
One last bad docstring appears to have snuck in. This patch
also fixes that docstring problem in cinder/volume/qos_specs.py .
We have a requirement on graphviz that was never handled and is
actually causing one of our web pages to be improperly rendered.
The patch adds the dependency to bindep.txt so that we do not
see a check failure and so the documentation builds properly.
Change-Id: I66a9282b63d884a390534b0c777be8433a7e9bb3
Two years ago the translation files have been split into several
files, separating the log messages of different log levels from each
other, like X.pot, X-log-warning.pot, X-log-info.pot, and so on.
However, the setup.py command `compile_catalogs`, that comes from the
babel package and compiles the corresponding .po files into .mo
files, only supported one file per python package. This means that
during packaging `compile_catalogs` never compiled the X-log-*.po
files, so the corresponding translations were always missing.
Since babel 2.3 the domain can be set to a space separated list of
domains. This change adds the the additional log level files to the
domain list.
The obsolete check that .po and .pot files are valid is removed from
tox.ini.
Change-Id: I149c2254cb04297e598cfd3ca73b24efd0c8ef18
Closes-Bug: #1536226
To avoid conflicts, it needs to be done and
the subdirs 'api' and 'scenario' directly under
the cinder/tests/tempest directory
Closes-Bug: #1577016
Change-Id: I5b8d1411e0917ecb7e70eb5a6b139d75437ba032
To give more flexibility for tempest test coverage, use the tempest plugin
interface to add tempest tests to the Cinder repo.
Just the plumbing for now. A temporary tempest test was added just for
validation. This test should be removed once additional tests are added.
Third party CIs should change their command line from:
tox -e all -- volume
To something like:
tox -e all-plugin -- volume
Change-Id: I679b1cee6dc9187fd2b11c47a14be9f7acdad938
Implements: blueprint add-intree-tempest-tests
The keystonemiddleware entry_point and the oslo_concurrency,
oslo_db_concurrency and oslo_messaging entry_points are not
owned by Cinder so it is possible that Keystone and Oslo could
move or rename the list_opts methods causing our sample
configuration generator to break.
This patch removes the lines that register the entry_points since
they aren't actually adding anything to the sample list of
configuration options because the options are being added to the
sample file elsewhere.
Change-Id: I186d4d9e8e565dad836543b5f6a96af1a53d4a83
No config generator hooks should ever be registered with a name that
belongs to another project. In this case, using oslo.middleware.cors
means that *every other project* that loads the middleware gets this
application's defaults when the generator is run on a system with
everything installed (such as a dev box with devstack). Use the name
of the app instead, to ensure that the defaults are only set when this
app's sample config and documentation are being generated.
Change-Id: I6a8c7d44b9db9325003ff2fdb667b0ced7739e96
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
The default values needed for cinder's implementation of cors
middleware have been moved from paste.ini into the configuration
hooks provided by oslo.config. Furthermore, these values have been
added to cinder's default configuration parsing. This ensures
that if a value remains unset in cinder.conf, it will be set
to use sane defaults, and that an operator modifying the
configuration file will be presented with a default set of
necessary sane headers.
Depends-on: I658e54966c390b41e3b551dd9827606c2e013511
Change-Id: Ia8735d5952d7e03b6948f748afead13e6f890271
Closes-Bug: 1551836
oslo-incubator is ending its life and we should move remaining
dependencies from there to cinder namespace. This commit does so with
openstack.common.scheduler. Apart from that tests from oslo-incubator
repository are added.
Change-Id: I10d88c120c9c847826986483065f5493e91f89d6
Closes-Bug: 1519337
pbr 1.4.0 added support for wsgi_scripts that creates scripts for wsgi
servers on install.
Change-Id: Ifbab059001d1567b1f7b394c0411a9ca4629f846
Closes-Bug: #1500367
Bump preversion to mark the start of the Mitaka development branch.
The liberty release branch will be cut from the previous commit.
Change-Id: Icb57e22104f70a9b5d8c1c0b56b3156e51c60b4a
As it stands, the opts.py file that is passed into
oslo-config-generator isn't being generated dynamically
and the old way of generating the cinder.conf.sample is
dependent on oslo-incubator which Cinder is trying to
move away from. oslo-config-generator works differently
than oslo-incubator so a number of changes had to be made
in order to make this switch.
This patch adds the config directory to Cinder and in it
are two files:
-generate_cinder_opts.py that will take the
results of a grep command to create the opts.py
file to be passed into oslo-config-generator.
-cinder.conf which is the new configuration for
oslo-config-generator. The file is inside the config
directory to be consistent with other projects.
Some changes were made to the generate_sample.sh file in
order to give the base directories and target directories
to the generate_cinder_opts.py program.
tox.ini was edited to remove the checkonly option because
all that needs to happen in check_uptodate.sh is a check to
ensure that the cinder.conf.sample is actually being
generated with no issues.
All options were removed from the check_uptodate.sh
because they were unnecessary given the new, more simple
way of generating the cinder.conf.sample.
setup.cfg was also edited in order to add information
oslo-config-generator needs to run.
Co-Authored By: Jay Bryant <jsbryant@us.ibm.com>
Co-Authored By: Jacob Gregor <jgregor@us.ibm.com>
Change-Id: I643dbe5675ae9280e204f691781e617266f570d5
Closes-Bug: 1473768
Closes-Bug: 1437904
Closes-Bug: 1381563
This reverts commit e67adc385ce6997249fbf70e9402264f758fc6f4.
Adding optional dependency tracking would be nice,
but Cinder is not yet in a place to do this.
- Cinder supports tox 1.6 (minversion in tox.ini)
- This functionality does not work on 1.6, requires 1.7. Tox
fails to build a test environment if using tox 1.6.
- We have decided we can't move to tox 1.7 for now.
( https://review.openstack.org/#/c/211614/ )
This reverts commit 478e8e970e54aaf3a034dc6347605c1b18291d0d.
- Need to remove oslo.vmware from test-requirements.txt now, as
well.
Closes-Bug: #1484035
Change-Id: Icff52508ff1cd67e8c17964c840d3a5cdcfbbce3
Driver-specific requirements are not hard requirements, since the
choice of drivers is up to the operator. The oslo.vmware module is
herein moved out of requirements.txt and instead listed as an optional
dependency using the extras functionality in setup.cfg.
A check is added to the vmdk driver to gracefully handle import errors
if the vmdk driver is used and oslo.vmware is not found.
Change-Id: I9b00edc38f0700304a1a164f0679a734f8701ebe
Closes-Bug: #1475739
Bump pre-version in setup.cfg to formally open Liberty development.
Kilo release branch will be cut from the previous commit.
Depends-On: I868118528d831a9f49081a6393e64eb7a414b36f
Change-Id: I2934522ec7e7a4485f9a3a07e0b375587c976bd9
This patch moves Cinder to using olso.middleware, updates us so
we are using the oslo_middleware namespace and syncs the latest
middleware code from oslo-incubator to support grenade jobs.
The details for the middleware sync from oslo-incubator are as follows:
Current HEAD in OSLO:
---------------------
commit e589dde0721a0a67e4030813e582afec6e70d042
Date: Wed Feb 18 03:08:12 2015 +0000
Merge "Have a little fun with release notes"
Changes merged with this patch:
---------------------
__init__.py
4ffc4c87 - Add middleware.request_id shim for Kilo
4504e4f4 - Remove middleware
catch_errors.py
a01a8527 - Use oslo_middleware instead of deprecated oslo.middleware
ce8f8fa4 - Add middleware.catch_errors shim for Kilo
4504e4f4 - Remove middleware
5d40e143 - Remove code that moved to oslo.i18n
76183592 - add deprecation note to middleware
463e6916 - remove oslo log from middleware
fcf517d7 - Update oslo log messages with translation domains
request_id.py
a01a8527 - Use oslo_middleware instead of deprecated oslo.middleware
66d8d613 - Fix oslo.middleware deprecation error
4ffc4c87 - Add middleware.request_id shim for Kilo
4504e4f4 - Remove middleware
76183592 - add deprecation note to middleware
d7bd9dc3 - Don't store the request ID value in middleware as class variable
Some notes on this change. It is based on the change made in Nova:
https://review.openstack.org/#/c/130771 and is the recommended method
for cleaning up the unused portions of middleware from oslo-incubator,
moving to the oslo.middleware library and not breaking grenade in the
gate.
Change-Id: Ia99ab479cb8ef63a0db1a1208cc2501abba6132c
This is the sixth in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.
There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away. Thus we need patches like this one to
move to the new namespace.
This patch also updates our hacking check to ensure that no
instances of oslo.messaging sneak back in.
Change-Id: Id028f8968717bed49386864081cf150397a6ec5f
Partial-bug: 1409733
Change oslo.db to oslo_db
Change oslo.config to olso_config
Change oslo.rootwrap to oslo_rootwrap
The oslo libraries are moving away from namespace packages.
Partial-bug: 1409733
Change-Id: I3429282afa2db4c0b3aa630eb3a0eaece04405ed
Having an instance and an attached volume on the same physical host
(i.e. data locality) can be desirable in some configurations, in order
to achieve high-performance disk I/O.
This patch adds an InstanceLocalityFilter filter that allow users to
request creation of volumes 'local' to an existing instance, without
specifying the hypervisor's hostname, and without any knowledge of the
underlying back-ends.
In order to work:
- At least one physical host should run both nova-compute and
cinder-volume services.
- The Extended Server Attributes extension needs to be active in Nova
(this is by default), so that the 'OS-EXT-SRV-ATTR:host' property is
returned when requesting instance info.
- The user making the call needs to have sufficient rights for the
property to be returned by Nova. This can be achieved either by
changing Nova's policy.json (the 'extended_server_attributes' option),
or by setting an account with privileged rights in Cinder conf.
For example:
Instance 01234567-89ab-cdef is running in a hypervisor on the physical
host 'my-host'.
To create a 42 GB volume in a back-end hosted by 'my-host':
cinder create --hint local_to_instance=01234567-89ab-cdef 42
Note:
Currently it is not recommended to allow instance migrations for
hypervisors where this hint will be used. In case of instance
migration, a previously locally-created volume will not be
automatically migrated. Also in case of instance migration during the
volume's scheduling, the result is unpredictable.
DocImpact: New Cinder scheduler filter
Change-Id: Id428fa2132c1afed424443083645787ee3cb0399
This patch adds a new filter for the cinder scheduler that
can interpret two new properties provided by backends,
'filter_function' and 'goodness_function'. A driver can rely
on cinder.conf entries to define these properties for a backend
or the driver can generate them some other way. An evaluator is
used by the filter to parse the properties. The 'goodness_function'
property is used to weigh qualified backends in case multiple ones
pass the filter. More details can be found in the spec:
https://review.openstack.org/#/c/129330/
Implements: blueprint filtering-weighing-with-driver-supplied-functions
DocImpact: New optional backend properties in cinder.conf.
New filter and weigher available for scheduler.
Change-Id: I38408ab49b6ed869c1faae746ee64a3bae86be58
Refactored migration tests to use OpportunisticTestCase, removed
unused code and ``test_migrations.conf`` file.
The main feature of this approach is to create a new database with
random name for each migration test. This will avoid migration tests of
race conditions and reduce tests intersection. After this change, database
``openstack_citest`` will be used only for initial connection to the database.
``test_migrations.conf`` file not required anymore, because we create test
database for migration test, so we no longer need to keep database credentials.
Implements blueprint: db-migration-tests
Related-bug: #1266595
Change-Id: I4febd485ff53936b636947c86773a23724e24c65
The following patch changes how cinder scripts are
installed and unit tested. This patch moves all the
cinder scripts from bin into cinder/cmd and creates
entry_points for those scripts in setup.cfg. When
cinder is installed, these scripts will be installed
under /usr/local/bin by pbr.
DocImpact: 'host' config option for multiple-storage backends
in cinder.conf is renamed to 'backend_host' in order to avoid
a naming conflict with the 'host' to locate redis
Change-Id: If82cf0c58d765bf79dbf721ca95c10a468940cab
Implements: blueprint unit-test-cases-for-cinder-scripts
Cinder does not support Python 2.6 anymore starting with Kilo and might
not work correctly with it, so remove the classifier.
Change-Id: I47f5e3ba30c7d8950862bc0a0d67d33d90c7eecf
cinder-clear-rabbit-queues should be removed because it does not
appear to be widely used and there is no user documentation for
it. Also, RabbitMQ now includes a management plugin and CLI,
rabbitmqadmin, that allows you to delete queues and exchanges.
This script was also removed in nova as part of
https://review.openstack.org/#/c/40355/.
DocImpact: The RabbitMQ management plugin ships with rabbitmqadmin.
It supports RabbitMQ version 2.8.x and above.
Closes-Bug: #1348859
Change-Id: I77bb54065280771e5cc64835af9c584626ecf8b8
cinder-rpc-zmq-receiver should be removed as part of the switch to
oslo.messaging. It is now replaced with oslo-messaging-zmq-receiver.
Change-Id: I1fd7d7571bb5660bf77e7664f35052db29ebea57
Closes-Bug: #1348856
Cinder has done a good job hiding the details of storage backends from end
users by using volume types. However there are use cases where users who
build their application on top of volumes would like to be able to 'choose'
where a volume be created on. How can Cinder provide such capability without
hurting the simplicity we have been keeping? Affinity/anti-affinity is one
of the flexibility we can provide without exposing details to backends.
The term affinity/anti-affinity here is to to describe the relationship
between two sets of volumes in terms of location. To limit the scope, we
describe one volume is affinity with the other one only when they reside in
the same volume back-end (this notion can be extended to volume pools if
volume pool support lands in Cinder); on the contrary, 'anti-affinity'
relation between two sets of volumes simply implies they are on different
Cinder back-ends (pools).
This affinity/anti-affinity filter filters Cinder backend based on hint
specified by end user. The hint expresses the affinity or anti-affinity
relation between new volumes and existing volume(s). This allows end
users to provide hints like 'please put this volume to a place that is
different from where Volume-XYZ resides in'.
This change adds two new filters to Cinder - SameBackendFilter and
DifferentBackendFilter. These two filters will look at the scheduler hint
provided by end users (via scheduler hint extension) and filter backends by
checking the 'host' of old and new volumes see if a backend meets the
requirement (being on the same backend as existing volume or not being on
the same backend(s) as existing volume(s)).
For example:
Volume A is on 'backend 1', to create Volume B on the same backend as A,
use:
cinder create --hint same_host=VolA-UUID SIZE
To create Volume C on different backend than that of A, use:
cinder create --hint different_host=VolA-UUID SIZE
Now, to create Volume D on different backend other than those of A and C,
use:
cinder create --hint different_host=VolA-UUID --hint
different_host=VolC-UUID SIZE
or:
cinder create --hint different_host="[VolA-UUID, VolC-UUID]" SIZE
implements bp: affinity-antiaffinity-filter
DocImpact
Change-Id: I19f298bd87b0069c0d1bb133202188d3bf65b770
Currently cinder support choosing volume backend according to
free_capacity and allocated_capacity. Volume Num Weighter is
that scheduler could choose volume backend based on volume number
in volume backend, which could provide another mean to help
improve volume-backends' IO balance and volumes' IO performance,
see details in ref [1] and [2].
blueprint support-volume-num-weighter
Ref:
[1]https://blueprints.launchpad.net/cinder/+spec/support-volume-num-weighter
[2]https://review.openstack.org/#/c/99683/
Change-Id: Id9275ed954f1b35ee8d7bee0f0b61cc3a7c48f63
Bump preversion to 2014.2 to formally open Juno development.
The Icehouse release branch (milestone-proposed) will be cut
from the last commit before this one.
Change-Id: I0296da7e564db58d164d8d867c45959d425dea37
The oslo.messaging library takes the existing RPC code from oslo and
wraps it in a sane API with well defined semantics around which we can
make a commitment to retain compatibility in future.
The patch is large, but the changes can be summarized as:
* oslo.messaging>=1.3.0a4 is required; a proper 1.3.0 release will be
pushed before the icehouse release candidates.
* The new rpc module has init() and cleanup() methods which manage the
global oslo.messaging transport state. The TRANSPORT and NOTIFIER
globals are conceptually similar to the current RPCIMPL global,
except we're free to create and use alternate Transport objects
in e.g. the cells code.
* The rpc.get_{client,server,notifier}() methods are just helpers
which wrap the global messaging state, specifiy serializers and
specify the use of the eventlet executor.
* In oslo.messaging, a request context is expected to be a dict so
we add a RequestContextSerializer which can serialize to and from
dicts using RequestContext.{to,from}_dict()
* The allowed_rpc_exception_modules configuration option is replaced
by an allowed_remote_exmods get_transport() parameter. This is not
something that users ever need to configure, but it is something
each project using oslo.messaging needs to be able to customize.
* We maintain a global NOTIFIER object and create specializations of
it with specific publisher IDs in order to avoid notification driver
loading overhead.
* rpc.py contains transport aliases for backwards compatibility
purposes. setup.cfg also contains notification driver aliases for
backwards compat.
* The messaging options are moved about in cinder.conf.sample because
the options are advertised via a oslo.config.opts entry point and
picked up by the generator.
* We use messaging.ConfFixture in tests to override oslo.messaging
config options, rather than making assumptions about the options
registered by the library.
Implements blueprint: oslo-messaging
Change-Id: Ib912809428d92e788558439e2d85b51272ebefdd
Remove rootwrap code copied from oslo-incubator, make
cinder-rootwrap a console_script entrypoint pointing in
oslo.rootwrap instead.
Change-Id: I4e924bc9fccd7f5a8522ac38b7c6d29683990ea8
Implements: blueprint cinder-oslo-rootwrap
The RetryFilter is now part of Oslo, although it's been renamed to
IgnoreAttemptedFilter. Thanks to entry_point, we are able to maintain
backwards compatibility after pulling that commit from Oslo.
Commits in this change:
* 66fe978 2013-12-10 | Change IgnoreAttemptedHostFilter to expect 'retry' key (attempt-retry)
* 135dd00 2013-12-10 | Remove start index 0 in range()
* 45658e2 2013-12-09 | Fix violations of H302:import only modules
* 70004c6 2013-12-04 | Add IgnoreAttemptedHostsFilter to oslo
* 880acf7 2013-11-14 | Change capabilities filters to use resource type (capfilter_message)
* 06e9d98 2013-11-10 | Add some log messages to capabilities_filter.py
* 3970d46 2013-11-02 | Fix typos in oslo
* 8718763 2013-08-19 | Replace list with dict in AvailabilityZoneFilter.host_passes
* c0d052a 2013-07-12 | python3: Add basic compatibility support.
* e3545f8 2013-06-02 | Enable hacking H402 test
* 484a1df 2013-05-30 | Enable hacking H403 test
* 35660da 2013-05-30 | Enable hacking H401 test
* 5dcc43b 2013-05-05 | Break out common functionality for filters and weights
* 1f2aba5 2013-05-03 | Renames filter to base_filter and weight to base_weight
Change-Id: Ibeb685ef60e44cb6388fc460ee6a78255ed3dbae
AllocatedCapacityWeigher is a weigher that weigh hosts by their
allocated capacity. The main purpose of this weigher is to simulate
the SimpleScheduler's behavior, which sorts hosts by the size of
all volumes on them. So by allocated capacity, it equals to the
sum of size of all volumes on target host.
In order to keep track of 'allocated' capacity, host state is updated
to add a 'allocated_capacity_gb' attribute to record the value, which
means each back-end must report one extra stats to scheduler.
Fortunately, the 'allocated' capacity we are interested in here is
pure Cinder level capacity, the volume manager can take all the burden
to calculate this value without having to query back-ends. The volume
manager does the initial calculation in init_host() by the time when it
has to query all existing volumes from DB for ensure_export(). After
initial calculation, volume manager/scheduler will keep track of every
new request that changes 'allocated_capacity' and make sure this value
is up to date.
!DriverImpact! Cinder driver developers, please read on:
This patch contains a change that might IMPACT volume drivers: volume
manager now uses 'stats' attribute to save 'allocated_capacity_gb'.
And this information will be merged with those stats drivers provide
as a whole for scheduler to consume. If you plan to report any form
of allocated space other than the apparent Cinder level value, (e.g.
actual capacity allocated), Please choose a key name other than
'allocated_capacity_gb', otherwise it will *OVERWRITE* the value volume
manager has calculated and confuse scheduler.
Partially implements bp: deprecate-chance-and-simple-schedulers
Change-Id: I306230b8973c2d1ad77bcab14ccde68e997ea816