This change adds a new hw:locked_memory extra spec and hw_locked_memory
image property to contol preventing guest memory from swapping.
This change adds docs and extend the flavor
validators for the new extra spec.
Also add new image property.
Blueprint: libvirt-viommu-device
Change-Id: Id3779594f0078a5045031aded2ed68ee4301abbd
This change starts the process of wiring up the new ephemeral encryption
control mechanisims in the compute layer. This initial step being to
ensure the BlockDeviceMapping objects are correctly updated with the
required ephemeral encryption details when requested through the
instance flavor extra specs or image metadata properties.
Change-Id: Id49cb238f7bbf2b97f018ddbe090ebdc08d762dc
The PowerVM driver was deprecated in November 2021 as part of change
Icdef0a03c3c6f56b08ec9685c6958d6917bc88cb. As noted there, all
indications suggest that this driver is no longer maintained and may be
abandonware. It's been some time and there's still no activity here so
it's time to abandon this for real.
This isn't as tied into the codebase as the old XenAPI driver was, so
removal is mostly a case of deleting large swathes of code. Lovely.
Change-Id: Ibf4f36136f2c65adad64f75d665c00cf2de4b400
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
As agreed in the spec, we will both drop the generation support for a keypair
but we'll also accept @ (at) and . (dot) chars in the keyname, all of them in
the same API microversion.
Rebased the work from I5de15935e83823afa545a250cf84f6a7a37036b4
APIImpact
Implements: blueprint keypair-generation-removal
Co-Authored-By: Nicolas Parquet <nicolas.parquet@gandi.net>
Change-Id: I6a7c71fb4385348c87067543d0454f302907395e
This mirrors the 'hw_vif_multiqueue_enabled' image metadata property.
Providing a way to set this via flavor extra specs allows admins to
enable this by default and easily enable it for existing instances
without the need to rebuild (a destructive operation).
Note that, in theory at least, the image import workflow provided by
glance should allows admins to enable this by default, but the legacy
image create workflow does not allow this and admins cannot really
control which API end users use when uploading their own images.
Also note that we could provide this behavior using a host-level
configuration option. This would be similar to what we do for other
attributes such as machine type ('hw_machine_type' image meta prop or
'[libvirt] hw_machine_type' config option) or pointer model
('hw_pointer_model' image meta prop or '[compute] pointer_model' config
option) and would be well suited to things that we don't expect to
change, such as enabling multiqueue (it's a sensible default). However,
we would need to start storing this information in system_metadata, like
we do for machine type (since Wallaby) to prevent things changing over
live migration. We have also started avoiding host-level config options
for things like this since one must ensure that the value configured are
consistent across deployments to behavior that varies depending on the
host the guest is initially created on.
Change-Id: I405d0324abe32b31a434105cf2c104876fe9c127
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Add microversion 2.90, which allows allows users to configure the
hostname that will be exposed via the nova metadata service when
creating their instance.
Change-Id: I95047c1689ac14fa73eba48e19dc438988b78aad
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This better reflects its actual meaning.
The 'hostname_or_ip_address' parameter type is removed as it has been
unused since support for support for the 'os-cells' API was removed in
change Iddb519008515f591cf1d884872a5887afbe766f2.
Change-Id: Ia99aeb37785284e129b01d336da17ea907db95b7
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Correct a variety of gaps and other issues seen while improving the
flavor docs.
Change-Id: I8d68016cecb0269a5f9af88b0a08578f85403e23
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The penultimate step in our journey from the secure boot wilderness.
Start configuring the relevant attribute of the guest if and when secure
boot is enabled.
Blueprint: allow-secure-boot-for-qemu-kvm-guests
Change-Id: Ic38ab840f59619bf921e5387cd7a11c88a77b2a5
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This patch adds the 'socket' value to the allowed PCI NUMA affinity
policies, both to the 'hw:pci_numa_affinity_policy' flavor extra spec,
and the 'hw_pci_numa_affinity_policy' image property.
For now the new value is a no-op and remains undocumented. It will be
wired-in in a subsequent patch.
Implements: blueprint pci-socket-affinity
Change-Id: I0680d4e21f3e317ac702b55afef4c87e8acbfc3a
Replace six.text_type with str.
A subsequent patch will replace other six.text_type.
Change-Id: I23bb9e539d08f5c6202909054c2dd49b6c7a7a0e
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Enable the 'hw:cpu_dedicated_mask' flavor extra spec interface, user
can create CPU mixing instance through a flavor with following
extra spec settings:
openstack flavor set <flavor_id> \
--property hw:cpu_policy=mixed \
--property hw:cpu_dedicated_mask=0-3,7
In a topic coming later, we'll introduce another way to create a
mixed instance through the real-time interface.
Part of blueprint use-pcpu-and-vcpu-in-one-instance
Change-Id: I2a3311c08a52eb11859c68ef940a0bd755a94c6b
Signed-off-by: Wang Huaqiang <huaqiang.wang@intel.com>
Add support for the 'hw:tpm_version' and 'hw:tpm_model' flavor extra
specs along with the equivalent image metadata properties. These are
picked up by the scheduler and transformed into trait requests. This is
effectively a no-op for now since we don't yet have a driver that
reports these traits.
Part of blueprint add-emulated-virtual-tpm
Change-Id: I8645c31b4ecb18afea592b2a5b360b0165626009
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Introduce a 'mixed' instance CPU allocation policy and
will be worked with upcoming patches, for purpose of
creating an instance combined shared CPUs with dedicated
or realtime CPUs.
In an instance mixed with different type of CPUs, the shared CPU
shared CPU time slots with other instances, and also might be a
CPU with less or un-guaranteed hardware resources, which implies
to have no guarantee for the behavior of the workload running on
it. If we call the shared CPU as 'low priority' CPU, then the
realtime or dedicated CPU could be called as 'high priority' CPU,
user could assign more hardware CPU resources or place some
guaranteed resource to it to let the workload to entail high
performance or stable service quality.
Based on https://review.opendev.org/714704
Part of blueprint use-pcpu-and-vcpu-in-one-instance
Change-Id: I99cfee14bb105a8792651129426c0c5a3749796d
Signed-off-by: Wang Huaqiang <huaqiang.wang@intel.com>
If the end-user specifies a cpu_realtime_mask that does not begin
with a carat (i.e. it is not a purely-exclusion mask) it's likely
that they're expecting us to use the exact mask that they have
specified, not realizing that we default to all-vCPUs-are-RT.
Let's make nova's behaviour a bit more friendly by correctly
handling this scenario.
Note that the end-user impact of this is minimal/non-existent. As
discussed in bug #1884231, the only way a user could have used this
before would be if they'd configured an emulator thread and purposefully
set an invalid 'hw:cpu_realtime_mask' set. In fact, they wouldn't have
been able to use this value at all if they used API microversion 2.86
(extra spec validation).
Part of blueprint use-pcpu-and-vcpu-in-one-instance
Change-Id: Id81859186de6fb6b728ad566a532244008fe77d0
Closes-Bug: #1688673
When we use the flavor extra_specs "hide_hypervisor_id" in
AggregateInstanceExtraSpecsFilter, then will retrun False.
So we need correct the extra_specs.
Change-Id: I9d8d8c3a30cf6da7e8fb48374347e069ab075df2
Closes-Bug: 1841932
This was missed in change Ib64a1348cce1dca995746214616c4f33d9d664bd. We
also correct the allowed syntax for resource group and values.
Part of blueprint flavor-extra-spec-validators
Change-Id: I6a46556aaa1e7cab03286f92b716841759c63e3d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Clarify the impact of this microversion in the release note and improve
some tests.
Part of blueprint flavor-extra-spec-validators
Change-Id: Idfccfa9831f1df6f261a247489154492a231d832
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Now that we have a registry of all extra specs known by stock nova, we
can start documenting these. We choose the configuration guide to do
this since configuration of flavor extra specs is traditionally an
admin-only operation.
Part of blueprint flavor-extra-spec-validators
Change-Id: I5ad6576e0d31a29822d1c7b47751ea81828630cf
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We had planned to introduce a 'validation' parameter to the various
extra spec APIs in change If67f0d924ea372746a6dc440ea7bdc655e4f0bea to
configure a policy for extra spec validation. Following reviews, we've
decided to instead go with a strict policy for all known namespaces and
no validation for any unknown namespaces. Update the tests we had in
place for this.
Part of blueprint flavor-extra-spec-validators
Change-Id: If30990ec1c43177b7d13bd8ee1c5dc481265e47b
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Add the validation framework necessary to verify extra specs along with
the definitions for every extra spec we currently recognize in-tree.
None of this is currently used since we don't have the API microversions
wired up, but that will come in a future patch.
Note that we must add the H238 hacking check to the ignore list here,
since this includes our first use of Python 3-type classes without the
explicit 'object' subclass. This can be removed when that check is
removed from hacking.
Part of blueprint flavor-extra-spec-validators
Change-Id: Ib64a1348cce1dca995746214616c4f33d9d664bd
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Allow PUT /servers/{server_id}/os-volume_attachments/{volume_id}``
to support specifying ``delete_on_termination`` field in the request
body. This allows updating the attached volume's flag that controls
whether or not it is automatically deleted when the instance is deleted.
When we request 'volumeId' and 'delete_on_termination' in the requst
body to swap volume, since the new microversion it will be support
updating the swapping volume's delete flag.
Co-Authored-By: Dan Smith <dansmith@redhat.com>
Blueprint: destroy-instance-with-datavolume
Change-Id: I6ccac4e17f56b40e67c79d40f32558ef414685ea
Picked up with vulture [1].
[1] https://pypi.org/project/vulture/
Change-Id: I0b318ea5309d201499f0a9ef6f205425c89e6946
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Like '/os-networks', we can't remove this in its entirety due to the
fact that some of these are proxy URLs that also work with neutron.
These are retained but everything else is nuked.
Note that this highlights a bug in the API, whereby a missing
'objects.Network.cidr' value results in a value of 'None' being output
over the API. Clearly this is incorrect, but it's probably not worth
fixing for this deprecated API.
Change-Id: I31cb0891144bdd7945479bb6692b0a533de4c5d0
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This microversion implements below API cleanups:
1. 400 for unknown param for query param and for request body.
2. Making server representation always consistent among all APIs
returning the complete server representation.
3. Change the default return value of ``swap`` field from the empty string
to 0 (integer) in flavor APIs.
4. Return ``servers`` field always in the response of GET
hypervisors API even there are no servers on hypervisor
Details: https://specs.openstack.org/openstack/nova-specs/specs/train/approved/api-consistency-cleanup.html
Partial-Implements: blueprint api-consistency-cleanup
Change-Id: I9d257a003d315b84b937dcef91f3cb41f3e24b53
This one's actually important since it will be an error in future
versions of Python.
Change-Id: Ib9f735216773224f91ac7f49fbe2eee119670872
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Drop support for the os-cells REST APIs, which are part of the cells v1
feature which has been deprecated since Pike.
This API now returns a 410 response for all routes.
Unit tests are removed and the functional API sample tests are just
asserting the 410 response now. The latter are also expanded to cover
APIs that weren't previously tested.
The API sample docs are left intact since the API reference still builds
from those and can be considered more or less branchless, so people
looking at the API reference can apply it to older deployments of nova
before os-cells was removed.
A release note added for previous cells v1 removals is amended to note
this additional change.
Part of blueprint remove-cells-v1
Change-Id: Iddb519008515f591cf1d884872a5887afbe766f2
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This brings in a couple of new checks which must be addressed, many of
which involve a rather large amount of changes, so these are ignored for
now. A series of follow-up changes will resolved these.
'pycodestyle' is added as a dependency rather than it being pulled in
transitively. This is necessary since we're using it in tests.
Change-Id: I35c654bd39f343417e0a1124263ff31dcd0b05c9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The rfc3986.is_valid_uri has been deprecated in 1.1.0.
It generates the following warnings.
DeprecationWarning: Please use rfc3986.validators.Validator instead.
This method will be eventually removed.
So this patch replaces rfc3986.is_valid_uri with
rfc3986.validators.Validator.
Even after applying this patch, the warnings are still output
because it is caused by oslo.config.
The fix for oslo.config will be done in another patch.
Change-Id: I70aebad6c6bd384dbd11ef732226356922bf1913
Closes-Bug: #1809755
Add a new microversion 2.67 to support specify ``volume_type``
when boot instances.
Part of bp boot-instance-specific-storage-backend
Change-Id: I13102243f7ce36a5d44c1790f3a633703373ebf7
- _build_regex_range is called 17 times on
import of nova.api.validation.parameters_types.
_build_regex_range internally calls re.escape
and valid_char on every char returned
from _get_all_chars.
_get_all_chars yields all chars up to 0xffff.
As a result re.escape and valid_char are called
1.1 million times when
nova.api.validation.parameters_types is imported.
- This change add a memorize decorator and uses
it to cache _build_regex_range
- This change does not cache valid_char,
_is_printable or re.escape as hashing and
caching them for each invocation would
be far more costly both in time and memory
than computing the result.
Change-Id: Ic1f2c560a6da815b26fdf770450bbe439d18d4f9
Closes-Bug: #1790195
API request schema should be using hard coded values
than reference of other nova code.
If refference code is changed for any reason and without
knowledge of that is being used in schema then, it can
change the schema unintentionally which leads to API backward
incompatible change.
It is always better to use raw and hard coded values in schema
to avoid unintentional API changes.
This commit removes the db value reference from few schema.
Change-Id: Ie1431b7f08172292d77d72fd8d0471a2c1b392b0
nova/db/__init__.py was importing * from nova.db.api. This meant that
any time any code anywhere within the nova.db package was imported
then nova.db.api was too, leading to a cascade of imports that may
not have been desired. Also, in general, code in __init__.py is a pain.
Therefore, this change adjusts code that so that either:
* nova.db.api is used directly
* nova.db.api is imported as 'db'
In either case, the functionality remains the same.
The primary goal of this change was to make it possible to import the
model files without having to import the db api. Moving the model files
to a different place in the directory hierarchy was considered, but
given that "code in __init__.py is a pain" this mode was chosen.
This looks like a very large change, but it is essentially adjusting
package names, many in mocks.
Change-Id: Ic1fd7c87ceda05eeb96735da2a415ef37060bb1a
This change adds support for the trusted_image_certificates parameter,
which is used to define a list of trusted certificate IDs that can be
used during image signature verification and certificate validation. The
parameter may contain a list of strings, each string representing the ID
of a trusted certificate. The list is restricted to a maximum of 50 IDs.
The list of certificate IDs will be stored in the trusted_certs field of
the instance InstanceExtra and will be used to verify the validity of
the signing certificate of a signed instance image.
The trusted_image_certificates request parameter can be passed to
the server create and rebuild APIs (if allowed by policy):
* POST /servers
* POST /servers/{server_id}/action (rebuild)
The following policy rules were added to restrict the usage of the
``trusted_image_certificates`` request parameter in the server create
and rebuild APIs:
* os_compute_api:servers:create:trusted_certs
* os_compute_api:servers:rebuild:trusted_certs
The trusted_image_certificates parameter will be in the response
body of the following APIs (not restricted by policy):
* GET /servers/detail
* GET /servers/{server_id}
* PUT /servers/{server_id}
* POST /servers/{server_id}/action (rebuild)
APIImpact
Implements blueprint: nova-validate-certificates
Change-Id: Iedd3fea0e86648fae364f075915555dcb2c4f199
Invalid query parameter could lead to HTTP 500, although
Nova used JSON Schema verification to check input query
params, but query like: GET /servers?limit=%88 will still
lead to HTTP 500, as it failed to parse at webob which is
pre JSON Schema check.
Partial-Bug: #1746202
Change-Id: I11b94a1aaeb67dc1a5abdcf0af5961ee8942a50a
This patch adds `key_name` param to instance rebuild
API. Then the user could reset the instance keypair
when rebuilding. If set key_name to None, the API
will unset the keypair of the instance.
APIImpact
Implements blueprint: rebuild-keypair-reset
Change-Id: I23886a89c25f811cfbe7e2500ce7ff52f9162966
Nova has a legacy hack to allow admins to specify hosts via an
availability zone using az:host:node. That means ':' cannot be
included in the name of an availability zone itself.
However, the aggregate API accepts requests which have
availability zone names including ':'.
This patch checks the availabilty zone name when aggregate is
created or updated and raises an error if it contains ':'.
Change-Id: I9b0d8e8d4b3ab2cb3d578c22fa259e0e7c0d325b
Closes-Bug: #1695861
If nova list API is called with empty regex pattern like below:
http://<host-ip>/compute/v2.1/servers?name=""
it fails at db layer and returns 500 InternalServerError.
Empty string('') is a valid regex for re.compile [1] so nova-api fails
to catch it at schema layer and it is passed to databse for searching.
Database fails to search it with "REGEXP %(display_name_1)s)" and
'display_name_1' as u'' in sql query which leads to below error:
InternalError: (1139, u"Got error 'empty (sub)expression' from regexp")
This issue is there for every query parameter which is using below
regex parameter types defined in parameter_types.py:
common_query_regex_param
This patch fixes this issue by rejecting the request with 400 if the
provided filter regex is empty string. If user is intending to filter
something, user must pass something to the filter, it cannot be a empty
string.
[1] https://github.com/openstack/nova/blob/16.0.0/nova/api/validation/validators.py#L40
Closes-Bug: #1718877
Change-Id: I3f6fa04dc7267279964e8e5dd2a790b997a40e4e
move common definition into common layer, as dependency patches
like I64ad387ec7b532e9dbc791f683a005587825ff61 plan to use the
definitions of the schema, this patch moves these definitions
as common ones
Change-Id: I17c2937e6b8ac3d09fabe273c50c81dd9b0babb6
The JSON-Schema of integer string is without min length limits. That
leads to the user can input empty string, and nova API code can't
handle that case. Finally the user will get 500 returned.
The create_backup and multi_create API are fixed. The networks and
tenant_networks API have same problem on few parameters, but due
to the nova-network is deprecated, so just keep those nova-network
specific API behaviour as before.
Change-Id: I521e914fc48b7a221431f41c567f2cb4b9b4ab99
Closes-bug: #1652719
Danylo Vodopianov