This is the first stage of the power management series.
In order to be able to switch the CPU state or change the
governor, we need a framework to access sysfs.
As some bits can be reused, let's create a nova.filesystem helper module
that will define read-write mechanisms for accessing sysfs-specific commands.
Partially-Implements: blueprint libvirt-cpu-state-mgmt
Change-Id: Icb913ed9be8d508de35e755a9c650ba25e45aca2
A new PCI resource handler is added to the update_available_resources
code path update the ProviderTree with PCI device RPs, inventories and
traits.
It is a bit different than the other Placement inventory reporter. It
does not run in the virt driver level as PCI is tracked in a generic way
in the PCI tracker in the resource tracker. So the virt specific
information is already parsed and abstracted by the resource tracker.
Another difference is that to support rolling upgrade the PCI handler
code needs to be prepared for situations where the scheduler does not
create PCI allocations even after some of the compute already started
reporting inventories and started healing PCI allocations. So the code
is not prepared to do a single, one shot, reshape at startup, but
instead to do a continuous healing of the allocations. We can remove
this continuous healing after the PCI prefilter will be made mandatory
in a future release.
The whole PCI placement reporting behavior is disabled by default while
it is incomplete. When it is functionally complete a new
[pci]report_in_placement config option will be added to allow enabling
the feature. This config is intentionally not added by this patch as we
don't want to allow enabling this logic yet.
blueprint: pci-device-tracking-in-placement
Change-Id: If975c3ec09ffa95f647eb4419874aa8417a59721
This addresses remaining comments from the unified limits series to add
type hints to new code and add a docstring to the is_qfd_populated()
method in nova/quota.py.
Related to blueprint unified-limits-nova
Change-Id: I948647b04b260e888a4c71c1fa3c2a7be5d140c5
Local API and DB limits are limits on resources that are counted either
as an API request parameter (example: server metadata items) or as
records in the database (example: server key pairs).
Future patches will make use of this logic, and actually enforce the
limits. This patch just adds the infrastructure to allow for the
enforcement of the limits.
We are moving all existing quotas to be managed via Keystone's
unified limits.
To stop confusion between injected_file_path_length and
injected_file_path_bytes, the unified limit in Keystone will use the
latter name to match the name used the API.
These local limits are all about preventing excessive load on the API
and database and have little to do with resource usage. These limits
are represented by keystone registered limits only, accordingly.
Local limits include things that just limit items in an API request:
* metadata_items
* injected_files
* injected_file_content_bytes
* injected_file_path_bytes
Local limits also include things that are stored in the database:
* key_pairs
* server_groups
* server_group_members
Some resource names have been changed to prepend a prefix of "server_"
in order to disambiguate them from other potential unified limits in
keystone:
* metadata_items => server_metadata_items
* injected_files => server_injected_files
* injected_file_content_bytes => server_injected_file_content_bytes
* injected_file_path_bytes => server_injected_file_path_bytes
* key_pairs => server_key_pairs
Note that each of the above are counted via a different scope. This new
code ensures that key_pairs are counted per user, server_groups are
counted per project and server_group_members are counted per
server_group.
Note: Previously server_group_member were counted per user inside each
server_group, which has proved very confusing, as adding more users into
a project increases the maximum size of allowed for a server_group.
blueprint unified-limits-nova
Change-Id: I0b6f4d29aaee1d71541a95cbecfd0708aac325d2
The 'nova.pci' module is poorly understood and difficult to debug.
Start adding type hints to make this a little easier to parse and catch
dumb errors. Some code needs to be reworked to make 'mypy' happy, but
it's mostly just type annotations.
Note that because of how the 'nova.objects' module works, we need to
delay interpolation by using forward references, or expressing the type
as string literals to be resolved later [1].
[1] https://www.python.org/dev/peps/pep-0484/#forward-references
Change-Id: I2a609606806c6cabdf95d53339335f61743fc5b0
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This patch fixes couple of thing that is needed to run mypy
* fixed a wrong type hint in LibvirtDriver._add_vtpm_device,
_configure_guest_by_virt_type, and _conf_non_lxc signature
* fixed a local variable type hint in LibvirtDriver._create_guest_with_network
* added an assert to _create_guest_with_network as the guest local
variable can be None if we get eventlet.timeout.Timeout and
CONF.vif_plugging_is_fatal is False.
Change-Id: I42c579531bac61063a381598094720271364ec92
This change introduces the first machine_type command to nova-manage to
fetch and display the current machine type if set in the system metadata
of the instance.
blueprint: libvirt-default-machine-type
Change-Id: Idc035671892e4668141a93763f8f2bed7a630812
There are already type hints in this file so now we enable checking
those hints. To make mypy happy some fields needed an explicit type
declaration.
Change-Id: Id1df976f067777d4115e3cb2c72d54764bb31c6a
This patch turns on type checking for our neturon interface code. To
make mypy pass this patch adds the minimum set of type hints and does
some necessary refactoring to make the types unambiguous.
Change-Id: I82705e035c49fc88ed0dc0df4db135f51c4c7fbf
This patch registers for VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED and
VIR_DOMAIN_EVENT_ID_DEVICE_REMOVAL_FAILED libvirt events and transforms
them to nova virt events.
This patch also extends the libvirt driver to have a driver specific
event handling function for these events instead of using the generic
virt driver event handler that passes all the existing lifecycle events
up to the compute manager.
This is part of the longer series trying to transform the existing
device detach handling to use libvirt events.
Co-Authored-By: Lee Yarwood <lyarwood@redhat.com>
Related-Bug: #1882521
Change-Id: I92eb27b710f16d69cf003712431fe225a014c3a8
As explained in the spec, in order to support routed networks, we need
to add a new scheduler pre-filter with a specific conf option
so we can then look at the nova aggregates related to the segments.
Since this pre-filter is called every time we verify the scheduler,
that means that when we move instances, we should be also be able
to only support hosts that are in the same related aggregate.
NOTE(sbauza) : We're just setting admin_api variable in the
integrated helpers as it's used by _get_provider_uuid_by_host()
Implements: blueprint routed-networks-scheduling
Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>
Co-Authored-By: Balazs Gibizer <balazs.gibizer@est.tech>
Change-Id: I667f56612d7f63834863476694cb1f4c71a58302
A couple of extra hints was needed. A later patch will add type hints
for some functions that is being refactored
Change-Id: I9886270dea092e28a8123245a2c70c8d8c7127f3
Blueprint: support-interface-attach-with-qos-ports
The ROI for these util modules is pretty high, given how they are used.
Part of blueprint add-emulated-virtual-tpm
Change-Id: I46846627311beb21946fd4d24048e0e3ea7ac942
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We'll need these shortly to handle shuffling of vTPM files during cold
migrations and resizes.
Part of blueprint add-emulated-virtual-tpm
Change-Id: I7c8dedc9e74117838db5d418dda1dcbe7143ad08
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
As with previous changes, we're going to be doing some surgery to this
file shortly so enable type hints now. These are *super* incomplete but
at least we have a starting point.
Part of blueprint add-emulated-virtual-tpm
Change-Id: Iee44ea525deb0b43ae43df3ba08c95ea8a4e317c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
In some environments the libvirt nodedev list can become out of sync
with the current MAC address assigned to a netdev, As a result the
nodedev lookup can fail. This results in an uncaught libvirt exception
which breaks the update_available_resource function resultingin an
incorrect resource view in the database.
e.g. libvirt.libvirtError: Node device not found:
no node device with matching name 'net_enp7s0f3v1_ea_60_77_1f_21_50'
This change removes the dependency on the nodedev name when looking up
nic feature flags.
Change-Id: Ibf8dca4bd57b3bddb39955b53cc03564506f5754
Closes-Bug: #1883671
Nothing to complicated here, other than working around mypy's dislike of
changing variable types and inability to process 'if six.PY3'
conditional blocks because it doesn't run the code.
Part of blueprint add-emulated-virtual-tpm
Change-Id: I805eaa8b6fb55ce9cbc8f6b8b777af48302ba2ba
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
In a future change, we'll want access to this so that it's possible to
retrieve vTPM data during the unrescue operation. While we're here, it
seems nothing is using the 'network_info' argument anymore, presumably
since the demise of nova-network, and this can and should be dropped.
Resolve both issues in one go, adding the 'context' parameter, dropping
the 'network_info' one, and updating the various callers and tests for
same.
Maintainers of out-of-tree drivers have been notified of these changes
[1].
[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-July/015824.html
Part of blueprint add-emulated-virtual-tpm
Change-Id: Id5e4b0f26d5a2a93db6a7d96555a2cff29d9a2cf
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
As with the 'nova.virt.hardware' module, add typing information here now
so that we can use it during development of later features. This
requires some minor tweaks of code that mypy found confusing.
Part of blueprint use-pcpu-and-vcpu-in-one-instance
Change-Id: Icc7b3d250bb9dd3d162731959185d9e962727247
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This is going to be used extensively in forthcoming patches. Lay the
groundwork now. This requires some minor tweaks of code that mypy found
confusing along with unit tests for coverage gaps it exposed.
Part of blueprint use-pcpu-and-vcpu-in-one-instance
Change-Id: Ied35762c353a084398ab8032a8efe6eada69dd9b
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
mypy is an experimental optional static type checker for Python that
aims to combine the benefits of dynamic (or "duck") typing and static
typing. While still in development, most features are supported and it's
already being used by real world projects like Sphinx. Let's start small
by integrating it into some of the interfaces that nova exposes. We can
further build upon this if it works out.
This change sets up the boilerplate necessary to use mypy in nova. Type
annotations are not included for any module - these will be added
separately. We're calling mypy by way of a script, as this allows us to
store a list of files that we have converted while we're in the process
of adding type annotations where necessary.
Change-Id: I75ab46a6768c4ca2050fdde2b7f8eeb90724c8c6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
