Due to how the Linux SCSI kernel driver works there are some storage
systems, such as iSCSI with shared targets, where a normal user can
access other projects' volume data connected to the same compute host
using the attachments REST API.
This affects both single and multi-pathed connections.
To prevent users from doing this, unintentionally or maliciously,
cinder-api will now reject some delete attachment requests that are
deemed unsafe.
Cinder will process the delete attachment request normally in the
following cases:
- The request comes from an OpenStack service that is sending the
service token that has one of the roles in `service_token_roles`.
- Attachment doesn't have an instance_uuid value
- The instance for the attachment doesn't exist in Nova
- According to Nova the volume is not connected to the instance
- Nova is not using this attachment record
There are 3 operations in the actions REST API endpoint that can be used
for an attack:
- `os-terminate_connection`: Terminate volume attachment
- `os-detach`: Detach a volume
- `os-force_detach`: Force detach a volume
In this endpoint we just won't allow most requests not coming from a
service. The rules we apply are the same as for attachment delete
explained earlier, but in this case we may not have the attachment id
and be more restrictive. This should not be a problem for normal
operations because:
- Cinder backup doesn't use the REST API but RPC calls via RabbitMQ
- Glance doesn't use this interface anymore
Checking whether it's a service or not is done at the cinder-api level
by checking that the service user that made the call has at least one of
the roles in the `service_token_roles` configuration. These roles are
retrieved from keystone by the keystone middleware using the value of
the "X-Service-Token" header.
If Cinder is configured with `service_token_roles_required = true` and
an attacker provides non-service valid credentials the service will
return a 401 error, otherwise it'll return 409 as if a normal user had
made the call without the service token.
Closes-Bug: #2004555
Change-Id: I612905a1bf4a1706cce913c0d8a6df7a240d599a
The cinder side change for reimaging a volume merged in yoga[1].
However, it assume the nova side change has microversion 2.91
which was correct during that time but since then many new
features have merged in nova causing the nova side microversion
to bump to 2.93 (as per current upstream change[2]).
This patch updates the nova microversion (which we use) to send
'volume-reimaged' external event to nova notifying about
completion of the operation from cinder side.
[1] d69e89ea3b
[2] https://review.opendev.org/c/openstack/nova/+/830883
Change-Id: I8fae9e2b138545e97cc60e32bbfb527c2e2072ea
message_field.Action.REIMAGE_VOLUME doesn't exist. This patch adds the REIMAGE_VOLUME.
The content of message_field.Detail.NOTIFY_COMPUTE_SERVICE_FAILED is "Compute service failed to extend volume".
This message should be "Compute service failed to reimage volume". This patch adds the new REIMAGE_VOLUME_FAILED detail parameter for this message.
Closes-Bug: #1968170
Change-Id: I3e17c6a2a7f252a9d324a1fea616e22869e55702
This patch adds volume re-image API to enable the ability to
re-image a specific volume.
Implements: blueprint add-volume-re-image-api
Co-Authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>
Change-Id: I031aae50ee82198648f46c503bba04c6e231bbe5
the filter tries to use list_extensions API of novaclient,
however it was removed in Ussuri release
https://review.opendev.org/c/openstack/python-novaclient/+/686516
All the former extensions are long ago rolled into Nova "core" code,
so we may take the "extension" this filter depends on as given.
Change-Id: Idc03d997cfedfc2fcc967cca519997d157df115a
I16e0b6e55a9c9da04c4582f9be672018d37bf368 in python-novaclient 15.0.0
removed the EndpointNotFound class which breaks the cinder unit
test "test_novaclient_exceptions". That test was added with change
Iea3ff0405ef8cf9c5222a489d85f9d135ebd3652 in Ocata after novaclient
7.0.0 removed its service catalog code. The cinder novaclient module
code was updated in I55613793c8f525a36ac74636f47d7ab76f5c7e39 (Pike)
and Ie27f3b528dbfaa57fe354a84a93787e1618182a3 (Pike) to remove the
hacked service catalog code but still checks the service catalog for
an identity endpoint if cinder isn't configured with an auth_url for
talking to nova. That code was raising novaclient.EndpointNotFound
which is now gone, so this change swaps it to use EndpointNotFound
from the keystoneauth1 library instead.
Needed by: https://review.opendev.org/679295/
Change-Id: I4b03beba0f847d779b6f3031be7ac68925cc79f4
Closes-Bug: #1842440
Add user messages for these cases below when performing
extend volume operation.
1. When no valid host is found.
2. When nova failed to perform extend volume event.
3. When backend failed to extend block device.
Change-Id: I7d2061ac13b2c74745da56feecfe6ed53b284fc4
Closes-Bug: #1773833
Nova settings were deprecated, then removed in change
Ic112a68101c0dcb14da0db06c62ae190474a37d9. Some legacy
info was left behind in one of the new options referring
to the old config setting that should now be removed.
Change-Id: I031e5ffbc11c4b044f3baf808d852ce871a2d6c5
Service token will be passed along with user token to communicate
with services when dealing with long running tasks like
Create volume snapshot.
Partial-Implements: blueprint use-service-tokens
Change-Id: Id95beae0a46ab492756e0108039fefb28f4f0b69
Commit I9272f71e0e68268ad9f558ddd1e1183e3ea69806 fixed a few
deprecation warnings due to properties being renamed in oslo
context, but there were a few other name changes as well.
This updates domain_id, project_domain_id, and user_domain_id
and gets rid of ~130 DeprecationWarning messages from our unit
test runs.
Change-Id: Ic7e8e6bc5c68e73a2c26c3105885e5c6ea06a4a8
In Pike, the nova client converted to use the [nova] section.
Remove the obsolete config options now.
Change-Id: Ic112a68101c0dcb14da0db06c62ae190474a37d9
The docstring for novaclient was not properly
formatted causing Sphinx to generate warnings and the
resulting document was not bolded properly.
This change corrects the formatting to remove the
warning and enable proper doc generation.
Change-Id: Ibe204e8ed2cc2d0557cb9ad4a0f2bf3a61744387
This change adds the ability to extend 'in-use' volume.
Once the volume size is extended, Nova is informed of the size change
through the external-event extension so the virt driver can perform
the appropriate actions for the host and guest to detect the new volume size.
Tempest related patches:
1. https://review.openstack.org/#/c/480746/
2. https://review.openstack.org/#/c/480778/
Depends-On: If10cffd0dc4c9879f6754ce39bee5fae1d04f474
Blueprint: extend-attached-volume
Co-Authored-By: TommyLike <tommylikehu@gmail.com>
APIImpact
Change-Id: I60c8ea9eb0bbcfe41f5f0a30ed8dc67bdcab3ebc
This makes cinder add the global_request_id to the constructors for
nova and glance clients, which will pass the global_request_id into
those services on all API calls.
Adjust unit tests when needed to support the new parameter.
oslo spec I65de8261746b25d45e105394f4eeb95b9cb3bd42
(requirements bump)
Depends-On: I5b247f75edeea9da50fe524eadf5f9a2c626d665
Change-Id: I6366ca6bd8286858093b76579571b35b062f97d7
Correctly deprecate the os_privileged_xxx settings, and use them
if still configured.
Change-Id: Ie27f3b528dbfaa57fe354a84a93787e1618182a3
Closes-bug: #1691362
The nova client (used by the InstanceLocalityFilter for example)
seems to have obscure config options (compared to other projects),
and seems it is buggy too. Fix this by introducing a [nova] section,
where the usual auth parameters can be put (auth_type, auth_url,
username, password, etc...), and deprecate the old options.
Also doesn't play with the Service Catalog, the authentication plugins
can handle it, use a Token authentication plugin when using
the user context, and removed the separate usage of the admin endpoint.
Change-Id: I55613793c8f525a36ac74636f47d7ab76f5c7e39
Closes-bug: #1686616
DocImpact: to use a Nova connection (e.g. for InstanceLocalityFilter),
one has to configure the [nova] section.
Nova v2.1 API is used in Cinder over 1 year since the commit
Iadd3363265be6c5a8ed46704a712da6c15e2b046 and Nova v2.1 API has
been available since Kilo. So it seems unnecessary to make the
used Nova API version configuable.
Change-Id: Ib76b52199ee23997cd54eadbc27b13d46eb4a3f8
The only services in OpenStack that have dedicated admin REST APIs
that actually enforce different permissions at the API level are
keystone and swift. Assuming there is a dedicated admin endpoint is an
artifical construct.
Change the default for the admin_info to be the normal public url. For
sites that really want 2 different values here, they can have it, but
default to the simpler thing.
Change-Id: I92efc4639d37cb2736bae7e40707a853f1e72be6
The "disable-msg" flag has been deprecated for pylint and "disable"
should now be used. This replaces the two instances we had in the
code base where we were disabling pylint errors.
Change-Id: Id9f223c95cd8563ffcf545ab8c8dee0bf0354ecd
novaclient 7.0.0 removed the service_catalog module, which we still
depend on. Add a deep copy until this can be properly cleaned up.
Change-Id: Iea3ff0405ef8cf9c5222a489d85f9d135ebd3652
Closes-Bug: 1651468
When we make a keystoneauth Session instance we are not looking at domain
info. This way we're always creating v2Password auth plugin, which will
obviously fail in an Identity v3-only scenario.
This patch makes domain information (i.e user_domain_id and project_domain_id)
be read from the context so that we can issue v3Password plugin as well.
Also, the logic behind auth plugin creation has been changed, as long as the
loading mechanism is designed to read options from config files, which is not
the case. Creating the plugin directly seems to fit better here.
This change is backwards compatible, since identity.Password can figure
out what Identity version is in use and will ignore domain info if v2.0
Change-Id: I687a73cce4230e1d98d5c42fdc81549d5bc6ab1a
Closes-Bug: #1599168
Currently when calling update_server_volume through nova client,
it gives the user id of the used context, which leads failure.
As this user may can't access the nova service catalog.
This patch sets privileged_user as True and calls nova.
Change-Id: I5861b2ca9cc1d10ba83eb088aa9fc073973aee93
Closes-bug: #1614344
The Nova v2 API code has been completely removed from Nova by
change[1]. Although Nova still supports v2 compatible API with the
legacy endpoint, it is better to switch to use Nova v2.1 in Cinder.
[1] https://review.openstack.org/#/c/311653/
Change-Id: Iadd3363265be6c5a8ed46704a712da6c15e2b046
Closes-bug: #1588171
The Nova client has trouble dealing with Keystone V3 or unversioned
Keystone endpoint, when instantiated without a Keystone Session. Thus
we can't instantiate Nova client directly as cinder.compute.nova
does currently.
Keystone V3 and unversioned service catalog is the future (see [1]
and [2]) so we should adapt.
Please read comments #5, #6 and #11 of the related bug report.
[1] https://review.openstack.org/#/c/271508/
[2] https://review.openstack.org/#/c/302480/
Change-Id: Ic790955e677c5dfa47680bf619c57fba3deeee20
Related-Bug: #1522402
Commit I51254e20151fe1d6771897e64a8d0bdf2d674d03 introduces new error:
We use novaclient extensions instead of Nova API extensions. In such
case, InstanceLocalityFilter won't find Nova ExtendedServerAttributes
extension and will raise CinderException.
This patch fixes novaclient usage in a right way.
Change-Id: I36fb67d597b7848dd4796ea672bc573a570677e8
Closes-Bug: #1517431
In the config module default=None is set as the default value,
so it isn't necessary to set it again when doing config options.
Change-Id: I09989ab679d249a9f7dea6af5d53c14dd1726e09
Closes-Bug: #1323975
We should use 'client interface' for initialization of novaclient
instead of direct import of some specific client.
Change-Id: I51254e20151fe1d6771897e64a8d0bdf2d674d03
Closes-Bug: #1493576
This patch makes the import of oslo_config and registration
of _compute_opts consistent with all other imports and
registration statements in other files. In a future patch
that dynamically generates Cinder config options, it will now
be possible to get the options being registered in this file.
Change-Id: Ifaae49dd6d0163b5cde44df7b3ca904fb550f211
Update novaclient v1.1 to v2
because warning messages are shown as below when executing run_test.sh:
UserWarning: Module novaclient.v1_1 is deprecated
(taken as a basis for novaclient.v2).
The preferable way to get client class or object
you can find in novaclient.client module.
Change-Id: I26602aceadac9a43d4fbaa14791bf6756eef9874
Partial-Bug: #1482873
Introduce os_privileged_user_auth_url config to give the ability
to override the auth_url used when authenticating
the OpenStack privileged user and bypass use of catalog found in token.
DocImpact: New os_privileged_user_auth_url config
Closes-bug: #1473206
Change-Id: I4ffca8df0eb38fe41264439ae5bb93e025c808ff
Part of multi-patch set for easier chunks.
There have been quite a few instances found where the
i18n guidelines are not being followed. I believe this
has helped lead to some of the confusion around how to
correctly do this. Other developers see this code and
assume it is an example of the correct usage.
This patch attempts to clean up most of those violations
in the existing codebase to hopefully help avoid some of
that confusion in reviews.
Some issues address:
* Correct log translation markers for different log levels
* Passing format values as arguments to call, not preformatting
* Not forcing translation via six.text_type and others
Guidelines can be found here:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html
Hacking checks will not be able to identify all violations of
the guidelines, but it could be useful for catching obvious ones
such as LOG.info("No markers!").
Change-Id: I38f52c6408b47ccb59ec2064b360f7d4427d6830
Partial-bug: 1433216
When using a Novaclient method on a multi-region infrastructure,
requests sometimes fail with an AmbiguousEndpoints Exception,
because more than one Nova endpoint is returned in the catalog.
This patch passes `region_name` and `endpoint_type` from conf to
Novaclient, so this error case is avoided.
Change-Id: Ic4810a1c5223575c6d6300156491437ea67c4a7a
Closes-Bug: #1421314
This patch passes privileged user to create nova assisted
snapshots. We can pass privileged user credentials through
cinder.conf as below:
os_privileged_user_name = <user>
os_privileged_user_password = <password>
os_privileged_user_tenant = <tenant>
Closes-Bug: 1308736
Change-Id: I004f3962f7c83113bed6bd9e3ccd002f017bd260
Change oslo.db to oslo_db
Change oslo.config to olso_config
Change oslo.rootwrap to oslo_rootwrap
The oslo libraries are moving away from namespace packages.
Partial-bug: 1409733
Change-Id: I3429282afa2db4c0b3aa630eb3a0eaece04405ed
This is the fourth in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.
There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away. Thus, we need patches like this one to
move to the new namespace.
This patch also updates our hacking check to ensure that no instances
of oslo.utils sneak back in.
Change-Id: I19c331a9b54c5f42c278577ecc59fe1b65e61af0
This is the third in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.
There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away. Thus, we need patches like this one to
move to the new namespace.
This patch also updates our hacking check to ensure that no instances
of oslo.config sneak back in.
Change-Id: I45d24c4330f7c23fc5ad0a0527f1840d09710512
Having an instance and an attached volume on the same physical host
(i.e. data locality) can be desirable in some configurations, in order
to achieve high-performance disk I/O.
This patch adds an InstanceLocalityFilter filter that allow users to
request creation of volumes 'local' to an existing instance, without
specifying the hypervisor's hostname, and without any knowledge of the
underlying back-ends.
In order to work:
- At least one physical host should run both nova-compute and
cinder-volume services.
- The Extended Server Attributes extension needs to be active in Nova
(this is by default), so that the 'OS-EXT-SRV-ATTR:host' property is
returned when requesting instance info.
- The user making the call needs to have sufficient rights for the
property to be returned by Nova. This can be achieved either by
changing Nova's policy.json (the 'extended_server_attributes' option),
or by setting an account with privileged rights in Cinder conf.
For example:
Instance 01234567-89ab-cdef is running in a hypervisor on the physical
host 'my-host'.
To create a 42 GB volume in a back-end hosted by 'my-host':
cinder create --hint local_to_instance=01234567-89ab-cdef 42
Note:
Currently it is not recommended to allow instance migrations for
hypervisors where this hint will be used. In case of instance
migration, a previously locally-created volume will not be
automatically migrated. Also in case of instance migration during the
volume's scheduling, the result is unpredictable.
DocImpact: New Cinder scheduler filter
Change-Id: Id428fa2132c1afed424443083645787ee3cb0399
Currently Cinder makes all requests to other services (Nova, Swift,
etc.) with current user context. Sometimes Cinder needs privileged
rights for external queries (e.g. asking Nova where an instance is
hosted); there is no way to do it yet.
This patch adds to ability to configure an account with special rights
in the configuration ('os_privileged_user_name',
'os_privileged_user_password' and 'os_privileged_user_tenant' options).
Then, requests that need special permissions can be achieved by creating
a client(privileged_user=True).
Note: This user does not necessarily need to have an admin role
associated with it. For instance, policies can be changed to allow a
specific user (without any roles) to perform special actions.
DocImpact: New configuration options to set a privileged user account
Change-Id: I61d8a6de1c5db5ee2ecce124997f9b6447b04e47
The default service_name in config option (nova_catalog_info and
nova_catalog_admin_info) was mistakenly put as 'nova'. Consequently,
it is impossible to invoke novaclient APIs from Cinder code without
changing the default value of those config options.
The correct service_name defined in Keystone's config file
"/etc/keystone/default_catalog.templates" is:
catalog.RegionOne.compute.name = Compute Service.
This change replaces 'nova' with 'Compute Service' for default value
of 'nova_catalog*_info' config options.
Change-Id: Idee3585a08efa1d6af1f4ee2b74e8cefb05cc54f
Closes-Bug: 1401800
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some of these outdated modules will still be
around for a while until all other oslo modules that use them have been
updated in future commits.
Change-Id: Idee8600dfe42e5977b8fb824e91afff7e9119981
This file is from the initial fork from nova. It defines the states a
host aggregate can take. It is not used anywhere in the cinder code
and is therefore unnecessary.
The commit removes the file from the codebase.
Change-Id: I9ba12aabc5009020950b746f312372fa4bfa156d
According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.
This patch adds a new hacking check - N319 - that ensures all
debug log messages don't have translations.
Change-Id: Id9c2715f25c8f2ea52235aba4bd1583655391584
Implements: blueprint debug-translation-removal
Closes-Bug: #1318713
Fixes include rework of text, missing or extra spaces, wrong
capitalization and no sentence style capitalization.
Co-Authored-By: Diane Fleming <diane.fleming@rackspace.com>
Change-Id: Iddab947ce9597f4a1de885f1f6c0fb0e5ba9dc46
Gorka Eguileor