Commit Graph

61 Commits (master)

Author SHA1 Message Date
Gorka Eguileor 6df1839bdf Reject unsafe delete attachment calls
Due to how the Linux SCSI kernel driver works there are some storage
systems, such as iSCSI with shared targets, where a normal user can
access other projects' volume data connected to the same compute host
using the attachments REST API.

This affects both single and multi-pathed connections.

To prevent users from doing this, unintentionally or maliciously,
cinder-api will now reject some delete attachment requests that are
deemed unsafe.

Cinder will process the delete attachment request normally in the
following cases:

- The request comes from an OpenStack service that is sending the
  service token that has one of the roles in `service_token_roles`.
- Attachment doesn't have an instance_uuid value
- The instance for the attachment doesn't exist in Nova
- According to Nova the volume is not connected to the instance
- Nova is not using this attachment record

There are 3 operations in the actions REST API endpoint that can be used
for an attack:

- `os-terminate_connection`: Terminate volume attachment
- `os-detach`: Detach a volume
- `os-force_detach`: Force detach a volume

In this endpoint we just won't allow most requests not coming from a
service. The rules we apply are the same as for attachment delete
explained earlier, but in this case we may not have the attachment id
and be more restrictive.  This should not be a problem for normal
operations because:

- Cinder backup doesn't use the REST API but RPC calls via RabbitMQ
- Glance doesn't use this interface anymore

Checking whether it's a service or not is done at the cinder-api level
by checking that the service user that made the call has at least one of
the roles in the `service_token_roles` configuration. These roles are
retrieved from keystone by the keystone middleware using the value of
the "X-Service-Token" header.

If Cinder is configured with `service_token_roles_required = true` and
an attacker provides non-service valid credentials the service will
return a 401 error, otherwise it'll return 409 as if a normal user had
made the call without the service token.

Closes-Bug: #2004555
Change-Id: I612905a1bf4a1706cce913c0d8a6df7a240d599a
1 month ago
whoami-rajat 6eb2f4fb77 Update nova microversion for volume backed instance rebuild
The cinder side change for reimaging a volume merged in yoga[1].
However, it assume the nova side change has microversion 2.91
which was correct during that time but since then many new
features have merged in nova causing the nova side microversion
to bump to 2.93 (as per current upstream change[2]).
This patch updates the nova microversion (which we use) to send
'volume-reimaged' external event to nova notifying about
completion of the operation from cinder side.

[1] d69e89ea3b

Change-Id: I8fae9e2b138545e97cc60e32bbfb527c2e2072ea
9 months ago
Zhang Fan 998a654d7e Add REIMAGE_VOLUME message action
message_field.Action.REIMAGE_VOLUME doesn't exist. This patch adds the REIMAGE_VOLUME.

The content of message_field.Detail.NOTIFY_COMPUTE_SERVICE_FAILED is "Compute service failed to extend volume".
This message should be "Compute service failed to reimage volume". This patch adds the new REIMAGE_VOLUME_FAILED detail parameter for this message.

Closes-Bug: #1968170
Change-Id: I3e17c6a2a7f252a9d324a1fea616e22869e55702
1 year ago
Yikun Jiang d69e89ea3b Support volume re-image
This patch adds volume re-image API to enable the ability to
re-image a specific volume.

Implements: blueprint add-volume-re-image-api

Co-Authored-by: Rajat Dhasmana <>

Change-Id: I031aae50ee82198648f46c503bba04c6e231bbe5
1 year ago
Stephen Finucane 4b246564ac db: Remove 'db' argument from various managers
This is no longer used for anything.

Change-Id: Idb1492012487625772528d957ff65e2070e7248d
Signed-off-by: Stephen Finucane <>
2 years ago
Pavlo Shchelokovskyy 738f81feeb Fix instance locality scheduler filter
the filter tries to use list_extensions API of novaclient,
however it was removed in Ussuri release

All the former extensions are long ago rolled into Nova "core" code,
so we may take the "extension" this filter depends on as given.

Change-Id: Idc03d997cfedfc2fcc967cca519997d157df115a
2 years ago
Matt Riedemann 3cef55907b nova: use EndpointNotFound from keystoneauth1
I16e0b6e55a9c9da04c4582f9be672018d37bf368 in python-novaclient 15.0.0
removed the EndpointNotFound class which breaks the cinder unit
test "test_novaclient_exceptions". That test was added with change
Iea3ff0405ef8cf9c5222a489d85f9d135ebd3652 in Ocata after novaclient
7.0.0 removed its service catalog code. The cinder novaclient module
code was updated in I55613793c8f525a36ac74636f47d7ab76f5c7e39 (Pike)
and Ie27f3b528dbfaa57fe354a84a93787e1618182a3 (Pike) to remove the
hacked service catalog code but still checks the service catalog for
an identity endpoint if cinder isn't configured with an auth_url for
talking to nova. That code was raising novaclient.EndpointNotFound
which is now gone, so this change swaps it to use EndpointNotFound
from the keystoneauth1 library instead.

Needed by:

Change-Id: I4b03beba0f847d779b6f3031be7ac68925cc79f4
Closes-Bug: #1842440
4 years ago
TommyLike 3aec5274da Add user messages for extend volume operation
Add user messages for these cases below when performing
extend volume operation.
1. When no valid host is found.
2. When nova failed to perform extend volume event.
3. When backend failed to extend block device.

Change-Id: I7d2061ac13b2c74745da56feecfe6ed53b284fc4
Closes-Bug: #1773833
5 years ago
Sean McGinnis 50e6b7af36 Clean out config deprecation information
Nova settings were deprecated, then removed in change
Ic112a68101c0dcb14da0db06c62ae190474a37d9. Some legacy
info was left behind in one of the new options referring
to the old config setting that should now be removed.

Change-Id: I031e5ffbc11c4b044f3baf808d852ce871a2d6c5
5 years ago
nirajsingh 0787710c24 Add service_token for cinder-nova interaction
Service token will be passed along with user token to communicate
with services when dealing with long running tasks like
Create volume snapshot.

Partial-Implements: blueprint use-service-tokens

Change-Id: Id95beae0a46ab492756e0108039fefb28f4f0b69
6 years ago
Sean McGinnis 82447a26b5 Update oslo.context RequestContext args
Commit I9272f71e0e68268ad9f558ddd1e1183e3ea69806 fixed a few
deprecation warnings due to properties being renamed in oslo
context, but there were a few other name changes as well.

This updates domain_id, project_domain_id, and user_domain_id
and gets rid of ~130 DeprecationWarning messages from our unit
test runs.

Change-Id: Ic7e8e6bc5c68e73a2c26c3105885e5c6ea06a4a8
6 years ago
Gyorgy Szombathelyi c463c6f50c Remove deprecated nova config options
In Pike, the nova client converted to use the [nova] section.
Remove the obsolete config options now.

Change-Id: Ic112a68101c0dcb14da0db06c62ae190474a37d9
6 years ago
Jay S. Bryant b5fe8f6eb1 [DOC BLD FIX] Fix formatting in cinder.compute.nova
The docstring for novaclient was not properly
formatted causing Sphinx to generate warnings and the
resulting document was not bolded properly.

This change corrects the formatting to remove the
warning and enable proper doc generation.

Change-Id: Ibe204e8ed2cc2d0557cb9ad4a0f2bf3a61744387
6 years ago
Mathieu Gagné 3dd842de82 Add ability to extend 'in-use' volume
This change adds the ability to extend 'in-use' volume.

Once the volume size is extended, Nova is informed of the size change
through the external-event extension so the virt driver can perform
the appropriate actions for the host and guest to detect the new volume size.

Tempest related patches:


Depends-On: If10cffd0dc4c9879f6754ce39bee5fae1d04f474

Blueprint: extend-attached-volume
Co-Authored-By: TommyLike <>
Change-Id: I60c8ea9eb0bbcfe41f5f0a30ed8dc67bdcab3ebc
6 years ago
zhangdaolong b1a6784799 Fix the inappropriate parameter name
Parameter names are easily misleading.

Change-Id: I87e8b17aa425160339a98418c86f9151f06435df
6 years ago
Sean Dague 72800e6b22 Send global_request_id to nova/glance when calls are made
This makes cinder add the global_request_id to the constructors for
nova and glance clients, which will pass the global_request_id into
those services on all API calls.

Adjust unit tests when needed to support the new parameter.

oslo spec I65de8261746b25d45e105394f4eeb95b9cb3bd42

(requirements bump)
Depends-On: I5b247f75edeea9da50fe524eadf5f9a2c626d665

Change-Id: I6366ca6bd8286858093b76579571b35b062f97d7
6 years ago
Gyorgy Szombathelyi f11b51b55e Use the deprecated os_privileged_xxx settings
Correctly deprecate the os_privileged_xxx settings, and use them
if still configured.

Change-Id: Ie27f3b528dbfaa57fe354a84a93787e1618182a3
Closes-bug: #1691362
6 years ago
Gyorgy Szombathelyi da93e11794 Modernize the nova client in cinder
The nova client (used by the InstanceLocalityFilter for example)
seems to have obscure config options (compared to other projects),
and seems it is buggy too. Fix this by introducing a [nova] section,
where the usual auth parameters can be put (auth_type, auth_url,
username, password, etc...), and deprecate the old options.
Also doesn't play with the Service Catalog, the authentication plugins
can handle it, use a Token authentication plugin when using
the user context, and removed the separate usage of the admin endpoint.

Change-Id: I55613793c8f525a36ac74636f47d7ab76f5c7e39
Closes-bug: #1686616
DocImpact: to use a Nova connection (e.g. for InstanceLocalityFilter),
one has to configure the [nova] section.
6 years ago
Ken'ichi Ohmichi de3979f184 Remove TODO related to Nova API version
Nova v2.1 API is used in Cinder over 1 year since the commit
Iadd3363265be6c5a8ed46704a712da6c15e2b046 and Nova v2.1 API has
been available since Kilo. So it seems unnecessary to make the
used Nova API version configuable.

Change-Id: Ib76b52199ee23997cd54eadbc27b13d46eb4a3f8
6 years ago
Sean Dague 95550a5095 Change nova_catalog_admin_info to default to publicURL
The only services in OpenStack that have dedicated admin REST APIs
that actually enforce different permissions at the API level are
keystone and swift. Assuming there is a dedicated admin endpoint is an
artifical construct.

Change the default for the admin_info to be the normal public url. For
sites that really want 2 different values here, they can have it, but
default to the simpler thing.

Change-Id: I92efc4639d37cb2736bae7e40707a853f1e72be6
6 years ago
Jenkins f152819b01 Merge "Improve compatibility with novaclient 7.0.0" 6 years ago
Sean McGinnis 53f2e4cca4 Replace deprecated pylint disable-msg with disable
The "disable-msg" flag has been deprecated for pylint and "disable"
should now be used. This replaces the two instances we had in the
code base where we were disabling pylint errors.

Change-Id: Id9f223c95cd8563ffcf545ab8c8dee0bf0354ecd
6 years ago
Dirk Mueller 5b1c206b8f Improve compatibility with novaclient 7.0.0
novaclient 7.0.0 removed the service_catalog module, which we still
depend on. Add a deep copy until this can be properly cleaned up.

Change-Id: Iea3ff0405ef8cf9c5222a489d85f9d135ebd3652
Closes-Bug: 1651468
6 years ago
Clenimar Filemon 4fe3f4cfda Read domain info from context when contacting nova
When we make a keystoneauth Session instance we are not looking at domain
info. This way we're always creating v2Password auth plugin, which will
obviously fail in an Identity v3-only scenario.

This patch makes domain information (i.e user_domain_id and project_domain_id)
be read from the context so that we can issue v3Password plugin as well.

Also, the logic behind auth plugin creation has been changed, as long as the
loading mechanism is designed to read options from config files, which is not
the case. Creating the plugin directly seems to fit better here.

This change is backwards compatible, since identity.Password can figure
out what Identity version is in use and will ignore domain info if v2.0

Change-Id: I687a73cce4230e1d98d5c42fdc81549d5bc6ab1a
Closes-Bug: #1599168
7 years ago
lisali 2cd599ac6b Fix the interface to call update_server_volume
Currently when calling update_server_volume through nova client,
it gives the user id of the used context, which leads failure.
As this user may can't access the nova service catalog.

This patch sets privileged_user as True and calls nova.

Change-Id: I5861b2ca9cc1d10ba83eb088aa9fc073973aee93
Closes-bug: #1614344
7 years ago
liusheng c614308563 Bump to Nova v2.1
The Nova v2 API code has been completely removed from Nova by
change[1]. Although Nova still supports v2 compatible API with the
legacy endpoint, it is better to switch to use Nova v2.1 in Cinder.


Change-Id: Iadd3363265be6c5a8ed46704a712da6c15e2b046
Closes-bug: #1588171
7 years ago
Jordan Pittier 7570482c53 Assisted snapshot: use Keystone Session with Nova Client
The Nova client has trouble dealing with Keystone V3 or unversioned
Keystone endpoint, when instantiated without a Keystone Session. Thus
we can't instantiate Nova client directly as cinder.compute.nova
does currently.

Keystone V3 and unversioned service catalog is the future (see [1]
and [2]) so we should adapt.

Please read comments #5, #6 and #11 of the related bug report.


Change-Id: Ic790955e677c5dfa47680bf619c57fba3deeee20
Related-Bug: #1522402
7 years ago
Ivan Kolodyazhny 48ca2ca67c Fix InstanceLocalityFilter scheduler filter
Commit I51254e20151fe1d6771897e64a8d0bdf2d674d03 introduces new error:
We use novaclient extensions instead of Nova API extensions. In such
case, InstanceLocalityFilter won't find Nova ExtendedServerAttributes
extension and will raise CinderException.

This patch fixes novaclient usage in a right way.

Change-Id: I36fb67d597b7848dd4796ea672bc573a570677e8
Closes-Bug: #1517431
8 years ago
Nate Potter db8076be96 Don't use default=None for config options
In the config module default=None is set as the default value,
so it isn't necessary to set it again when doing config options.

Change-Id: I09989ab679d249a9f7dea6af5d53c14dd1726e09
Closes-Bug: #1323975
8 years ago
Ivan Kolodyazhny c131767a19 Fix usage of novaclient
We should use 'client interface' for initialization of novaclient
instead of direct import of some specific client.

Change-Id: I51254e20151fe1d6771897e64a8d0bdf2d674d03
Closes-Bug: #1493576
8 years ago
Kendall Nelson 86197acd36 Cleaning up CONF.register_opts() in compute/
This patch makes the import of oslo_config and registration
of _compute_opts consistent with all other imports and
registration statements in other files. In a future patch
that dynamically generates Cinder config options, it will now
be possible to get the options being registered in this file.

Change-Id: Ifaae49dd6d0163b5cde44df7b3ca904fb550f211
8 years ago
Yusuke Hayashi 000aeb9df7 Update deprecated version of novaclient
Update novaclient v1.1 to v2
because warning messages are shown as below when executing

UserWarning: Module novaclient.v1_1 is deprecated
(taken as a basis for novaclient.v2).
The preferable way to get client class or object
you can find in novaclient.client module.

Change-Id: I26602aceadac9a43d4fbaa14791bf6756eef9874
Partial-Bug: #1482873
8 years ago
Mathieu Gagné c228066002 Add ability to override OpenStack privileged user auth url
Introduce os_privileged_user_auth_url config to give the ability
to override the auth_url used when authenticating
the OpenStack privileged user and bypass use of catalog found in token.

DocImpact: New os_privileged_user_auth_url config
Closes-bug: #1473206
Change-Id: I4ffca8df0eb38fe41264439ae5bb93e025c808ff
8 years ago
Sean McGinnis 3578d9e341 Logging not using oslo.i18n guidelines
Part of multi-patch set for easier chunks.

There have been quite a few instances found where the
i18n guidelines are not being followed. I believe this
has helped lead to some of the confusion around how to
correctly do this. Other developers see this code and
assume it is an example of the correct usage.

This patch attempts to clean up most of those violations
in the existing codebase to hopefully help avoid some of
that confusion in reviews.

Some issues address:
* Correct log translation markers for different log levels
* Passing format values as arguments to call, not preformatting
* Not forcing translation via six.text_type and others

Guidelines can be found here:

Hacking checks will not be able to identify all violations of
the guidelines, but it could be useful for catching obvious ones
such as"No markers!").

Change-Id: I38f52c6408b47ccb59ec2064b360f7d4427d6830
Partial-bug: 1433216
8 years ago
Ivan Kolodyazhny 5980da5d88 Use oslo.log instead of oslo-incubator
log module was removed from oslo-incubator after oslo.log released.

Change-Id: I205d0625c502cb462919edc76d12091edcc21b1b
Implements: blueprint port-oslo-incubator-to-oslo-log
Partial-Bug: #1381563
8 years ago
Jenkins 4adb20773f Merge "Passing privileged user to create nova assisted snapshots" 8 years ago
Adrien Vergé 43374438bd Pass region name to Nova client
When using a Novaclient method on a multi-region infrastructure,
requests sometimes fail with an AmbiguousEndpoints Exception,
because more than one Nova endpoint is returned in the catalog.

This patch passes `region_name` and `endpoint_type` from conf to
Novaclient, so this error case is avoided.

Change-Id: Ic4810a1c5223575c6d6300156491437ea67c4a7a
Closes-Bug: #1421314
8 years ago
Bharat Kumar Kobagana 5370563250 Passing privileged user to create nova assisted snapshots
This patch passes privileged user to create nova assisted
snapshots. We can pass privileged user credentials through
cinder.conf as below:
  os_privileged_user_name = <user>
  os_privileged_user_password = <password>
  os_privileged_user_tenant = <tenant>

Closes-Bug: 1308736
Change-Id: I004f3962f7c83113bed6bd9e3ccd002f017bd260
8 years ago
Anish Bhatt cbcda7c7e8 Change oslo.* to oslo_*
Change oslo.db to oslo_db
Change oslo.config to olso_config
Change oslo.rootwrap to oslo_rootwrap

The oslo libraries are moving away from namespace packages.

Partial-bug: 1409733

Change-Id: I3429282afa2db4c0b3aa630eb3a0eaece04405ed
8 years ago
Jay S. Bryant b6ec77e6d4 Move oslo.utils to oslo_utils namespace
This is the fourth in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.

There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away.  Thus, we need patches like this one to
move to the new namespace.

This patch also updates our hacking check to ensure that no instances
of oslo.utils sneak back in.

Change-Id: I19c331a9b54c5f42c278577ecc59fe1b65e61af0
9 years ago
Jay S. Bryant 6a2df86bba Move oslo.config to oslo_config namespace
This is the third in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.

There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away.  Thus, we need patches like this one to
move to the new namespace.

This patch also updates our hacking check to ensure that no instances
of oslo.config sneak back in.

Change-Id: I45d24c4330f7c23fc5ad0a0527f1840d09710512
9 years ago
Adrien Vergé 0269a26f13 Add an instance-locality filter
Having an instance and an attached volume on the same physical host
(i.e. data locality) can be desirable in some configurations, in order
to achieve high-performance disk I/O.

This patch adds an InstanceLocalityFilter filter that allow users to
request creation of volumes 'local' to an existing instance, without
specifying the hypervisor's hostname, and without any knowledge of the
underlying back-ends.

In order to work:
- At least one physical host should run both nova-compute and
  cinder-volume services.
- The Extended Server Attributes extension needs to be active in Nova
  (this is by default), so that the 'OS-EXT-SRV-ATTR:host' property is
  returned when requesting instance info.
- The user making the call needs to have sufficient rights for the
  property to be returned by Nova. This can be achieved either by
  changing Nova's policy.json (the 'extended_server_attributes' option),
  or by setting an account with privileged rights in Cinder conf.

For example:
  Instance 01234567-89ab-cdef is running in a hypervisor on the physical
  host 'my-host'.

  To create a 42 GB volume in a back-end hosted by 'my-host':
    cinder create --hint local_to_instance=01234567-89ab-cdef 42

  Currently it is not recommended to allow instance migrations for
  hypervisors where this hint will be used. In case of instance
  migration, a previously locally-created volume will not be
  automatically migrated. Also in case of instance migration during the
  volume's scheduling, the result is unpredictable.

DocImpact: New Cinder scheduler filter
Change-Id: Id428fa2132c1afed424443083645787ee3cb0399
9 years ago
Adrien Vergé 04003d7c51 Add a privileged user for OpenStack services
Currently Cinder makes all requests to other services (Nova, Swift,
etc.) with current user context. Sometimes Cinder needs privileged
rights for external queries (e.g. asking Nova where an instance is
hosted); there is no way to do it yet.

This patch adds to ability to configure an account with special rights
in the configuration ('os_privileged_user_name',
'os_privileged_user_password' and 'os_privileged_user_tenant' options).
Then, requests that need special permissions can be achieved by creating
a client(privileged_user=True).

Note: This user does not necessarily need to have an admin role
associated with it. For instance, policies can be changed to allow a
specific user (without any roles) to perform special actions.

DocImpact: New configuration options to set a privileged user account
Change-Id: I61d8a6de1c5db5ee2ecce124997f9b6447b04e47
9 years ago
Trung Trinh 5ad15c040f Correct default service_name for nova_catalog*_info config option
The default service_name in config option (nova_catalog_info and 
nova_catalog_admin_info) was mistakenly put as 'nova'. Consequently,
it is impossible to invoke novaclient APIs from Cinder code without
changing the default value of those config options.

The correct service_name defined in Keystone's config file
"/etc/keystone/default_catalog.templates" is: = Compute Service.

This change replaces 'nova' with 'Compute Service' for default value
of 'nova_catalog*_info' config options.

Change-Id: Idee3585a08efa1d6af1f4ee2b74e8cefb05cc54f
Closes-Bug: 1401800
9 years ago
ChangBo Guo(gcb) 7d341e267f Use oslo.utils
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some of these outdated modules will still be
around for a while until all other oslo modules that use them have been
updated in future commits.

Change-Id: Idee8600dfe42e5977b8fb824e91afff7e9119981
9 years ago
git-harry 72a3092bc6 Remove cinder/compute/
This file is from the initial fork from nova. It defines the states a
host aggregate can take. It is not used anywhere in the cinder code
and is therefore unnecessary.

The commit removes the file from the codebase.

Change-Id: I9ba12aabc5009020950b746f312372fa4bfa156d
9 years ago
Christian Berendt 5061ab9586 debug level logs should not be translated
According to the OpenStack translation policy available at debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.

This patch adds a new hacking check - N319 - that ensures all
debug log messages don't have translations.

Change-Id: Id9c2715f25c8f2ea52235aba4bd1583655391584
Implements: blueprint debug-translation-removal
Closes-Bug: #1318713
9 years ago
Andreas Jaeger 464277220d Improve consistency of help strings
Fixes include rework of text, missing or extra spaces, wrong
capitalization and no sentence style capitalization.

Co-Authored-By: Diane Fleming <>
Change-Id: Iddab947ce9597f4a1de885f1f6c0fb0e5ba9dc46
9 years ago
ruichen 09b20b1b84 Fix wrong example of "nova_endpoint_template"
In cinder.conf

e.g. http://localhost:8774/v2/%(tenant_id)s

should be

e.g. http://localhost:8774/v2/%(project_id)s

Change-Id: I7dd3be9bc9d2e8f26619b00b5bdbc66505a984b2
Closes-Bug: #1279747
9 years ago
DennyZhang ff3f2e5735 Remove vim header
No need to set tabstop tons of times, this can be set in your vimrc
file instead.

More disucssion:

Partial-Bug: #1229324
Change-Id: Ib3a31c8819ee1950d72f7da5627593623d351560
10 years ago