A later patch in the pci-device-tracking-in-placement work
will extend the existing [pci]passthrough_whitelist config syntax.
So we take the opportunity here to deprecate the old non inclusive
passthrough_whitelist name and introduce a better one.
All the usage of CONF.pci.passthrough_whitelist is now changed over to
the new device_spec config. Also the in tree documentation is updated
accordinly.
However the nova code still has a bunch of references to the
"whitelist" terminology. That will be handled in subsequent patches.
blueprint: pci-device-tracking-in-placement
Change-Id: I843032e113642416114f169069eebf6a56ed78dd
The existing generic restart_compute_service() call in the nova test
base class is not appropriate for the libvirt functional test that needs
to reconfigure the libvirt connection as it is not aware of the libvirt
specific mocking needed when a compute service is started.
So this patch adds a specific restart_compute_service() call
to nova.tests.functional.libvirt.base.ServersTestBase. This will be used
by a later patch testing [pci]device_spec reconfiguration scenarios.
This change showed that some of the existing libvirt functional test
used the incomplete restart_compute_service from the base class. Others
used local mocking to inject new pci config to the restart. I moved all
these to the new function and removed the local mocking.
Change-Id: Ic717dc42ac6b6cace59d344acaf12f9d1ee35564
Unit test should never depend on the file system of the machine it runs
on. So this patch injects a poison to various std lib calls to catch
these cases. We only selectively poison access to /sys fs as there are
legitimate cases to access temp files.
When I isolated the poisoned test cases I tried to add a meaningful mocking
around the sysfs reads based on the meaning of the test.
Change-Id: I66636ebfe627cba8c4470cd84faa78f69249aa05
Nova uses the RequestSpec.pci_request in the PciPassthroughFilter to
decide if the PCI devicesm, requested via the pci_alias in the flavor
extra_spec, are available on a potential target host. During resize the
new flavor might contain different pci_alias request than the old flavor
of the instance. In this case Nova should use the pci_alias from the new
flavor to scheduler the destination host of the resize. However this
logic was missing and Nova used the old pci_request value based on the
old flavor. This patch adds the missing logic.
Closes-Bug: #1983753
Closes-Bug: #1941005
Change-Id: I73c9ae27e9c42ee211a53bed3d849650b65f08be
I thought we fixed all the double mocking issues with
I3998d0d49583806ac1c3ae64f1b1fe343cefd20d but I was wrong.
While we used both mock and unittest.mock the fixtures.MockPatch
used the mock lib instead of the unittest.mock lib.
The path Ibf4f36136f2c65adad64f75d665c00cf2de4b400 (Remove the PowerVM driver)
removed the last user of mock lib from nova. So it is also
removed the mock from test-requirements. This triggered that
fixtures.MockPatch athat started using unittest.mock too.
Before Ibf4f36136f2c65adad64f75d665c00cf2de4b400 a function can be mocked
twice once with unittest.mock and once with fixtures.MockPatch (still
using mock). However after that patch both path of such double
mocking goes through unittest.mock and the second one fails.
So this patch fixes double mocking so far hidden behind
fixtures.MockPatch.
Also this patch makes the py310 and functional-py310 jobs voting at
least in the check queue to prevent future changes adding double mocks.
Change-Id: Ic1352ec31996577a5d0ad18a057339df3e49de25
libvirt has a set of enlightenments in domain XML that make it
friendlier for Windows guests. We already enabled a few of these, this
patch just completes the list. All of them are available in libvirt
4.7.0 (QEMU 3.0) [1], which is way below our current minimum libvirt
and QEMU versions, so we don't need any extra checks.
[1] https://libvirt.org/formatdomain.html#hypervisor-features
Implements: bp/libvirt-update-windows-englightenments
Change-Id: I008841988547573878c4e06e82f0fa55084e51b5
The recent release of openstacksdk 0.100 showed that our testing is not
fully mocking underlying behavior. Such behavior changed in 0.100 and
broke our unit test. This patch fixes the mocking to avoid the breakage.
Change-Id: Ife4b8b42a414af281ac462faaa42c456ced13bb7
The PowerVM driver was deprecated in November 2021 as part of change
Icdef0a03c3c6f56b08ec9685c6958d6917bc88cb. As noted there, all
indications suggest that this driver is no longer maintained and may be
abandonware. It's been some time and there's still no activity here so
it's time to abandon this for real.
This isn't as tied into the codebase as the old XenAPI driver was, so
removal is mostly a case of deleting large swathes of code. Lovely.
Change-Id: Ibf4f36136f2c65adad64f75d665c00cf2de4b400
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Since we dropped support for python 2 [1], we no longer need to use the
mock library, which existed to backport py3 functionality into py2.
Change Ib44b5bff657c8e76c4f701e14d51a4efda3f6d32 cut over to importing
the stock mock, which must be done by saying::
from unittest import mock
...because if you say::
import mock
...you will be using the third party mock library instead, which may or
may not be installed.
This commit adds hacking check N371 to enforce the former.
[1] https://review.opendev.org/#/c/687954/
Change-Id: I71439580e80d33cff62aba807df2b35164a47cbe
In py310 unittest.mock does not allow to mock the same function twice as
the second mocking will fail to autospec the Mock object created by the
first mocking.
This patch manually fixes the double mocking.
Fixed cases:
1) one of the mock was totally unnecessary so it was removed
2) the second mock specialized the behavior of the first generic mock.
In this case the second mock is replaced with the configuration of
the first mock
3) a test case with two test steps mocked the same function for each
step with overlapping mocks. Here the overlap was removed to have
the two mock exists independently
The get_connection injection in the libvirt functional test needed a
further tweak (yeah I know it has many already) to act like a single
mock (basically case #2) instead of a temporary re-mocking. Still the
globalness of the get_connection mocking warrant the special set / reset
logic there.
Change-Id: I3998d0d49583806ac1c3ae64f1b1fe343cefd20d
Custom filters are still supported. Custom schedulers / scheduler
drivers are what is not. This patch fixes the wording.
Change-Id: Icf9576b75bd006ba65484b9232e85d8b748458f1
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib. Most of this
is autogenerated, as described below, but there is one manual change
necessary:
nova/tests/functional/regressions/test_bug_1781286.py
We need to avoid using 'fixtures.MockPatch' since fixtures is using
'mock' (the library) under the hood and a call to 'mock.patch.stop'
found in that test will now "stop" mocks from the wrong library. We
have discussed making this configurable but the option proposed isn't
that pretty [1] so this is better.
The remainder was auto-generated with the following (hacky) script, with
one or two manual tweaks after the fact:
import glob
for path in glob.glob('nova/tests/**/*.py', recursive=True):
with open(path) as fh:
lines = fh.readlines()
if 'import mock\n' not in lines:
continue
import_group_found = False
create_first_party_group = False
for num, line in enumerate(lines):
line = line.strip()
if line.startswith('import ') or line.startswith('from '):
tokens = line.split()
for lib in (
'ddt', 'six', 'webob', 'fixtures', 'testtools'
'neutron', 'cinder', 'ironic', 'keystone', 'oslo',
):
if lib in tokens[1]:
create_first_party_group = True
break
if create_first_party_group:
break
import_group_found = True
if not import_group_found:
continue
if line.startswith('import ') or line.startswith('from '):
tokens = line.split()
if tokens[1] > 'unittest':
break
elif tokens[1] == 'unittest' and (
len(tokens) == 2 or tokens[4] > 'mock'
):
break
elif not line:
break
if create_first_party_group:
lines.insert(num, 'from unittest import mock\n\n')
else:
lines.insert(num, 'from unittest import mock\n')
del lines[lines.index('import mock\n')]
with open(path, 'w+') as fh:
fh.writelines(lines)
Note that we cannot remove mock from our requirements files yet due to
importing pypowervm unit test code in nova unit tests. This library
still uses the mock lib, and since we are importing test code and that
lib (correctly) only declares mock in its test-requirements.txt, mock
would not otherwise be installed and would cause errors while loading
nova unit test code.
[1] https://github.com/testing-cabal/fixtures/pull/49
Change-Id: Id5b04cf2f6ca24af8e366d23f15cf0e5cac8e1cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
As agreed in the spec, we will both drop the generation support for a keypair
but we'll also accept @ (at) and . (dot) chars in the keyname, all of them in
the same API microversion.
Rebased the work from I5de15935e83823afa545a250cf84f6a7a37036b4
APIImpact
Implements: blueprint keypair-generation-removal
Co-Authored-By: Nicolas Parquet <nicolas.parquet@gandi.net>
Change-Id: I6a7c71fb4385348c87067543d0454f302907395e
When turned on, this will disable the version-checking of hypervisors
during live-migration. This can be useful for operators in certain
scenarios when upgrading. E.g. if you want to relocate all instances
off a compute node due to an emergency hardware issue, and you only have
another old compute node ready at the time.
Note, though: libvirt will do its own internal compatibility checks, and
might still reject live migration if the destination is incompatible.
Closes-Bug: #1982853
Change-Id: Iec387dcbc49ddb91ebf5cfd188224eaf6021c0e1
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
This change add a repoducer test for evacuating
a vm in the powering-off state
Related-Bug: #1978983
Change-Id: I5540df6c7497956219c06cff6f15b51c2c8bc299
The default value of the dictionary get method is None,
so remove the default value of None in the get method
Change-Id: Ic1b69047b6edd4987df3191ce84658393a1cb968
This adds support to the REST API, in a new microversion, for specifying
a destination host to unshelve server action when the server
is shelved offloaded.
This patch also supports the ability to unpin the availability_zone of an
instance that is bound to it.
Note that the functional test changes are due to those tests using the
"latest" microversion 2.91.
Implements: blueprint unshelve-to-host
Change-Id: I9e95428c208582741e6cd99bd3260d6742fcc6b7
This patch introduce changes to the compute API that will allow
PROJECT_ADMIN to unshelve an shelved offloaded server to a specific host.
This patch also supports the ability to unpin the availability_zone of an
instance that is bound to it.
Implements: blueprint unshelve-to-host
Change-Id: Ieb4766fdd88c469574fad823e05fe401537cdc30
We no longer support Python 3.6 so there's no need for Python 3.6-only
dependencies.
Change-Id: I139e454a2c6e49a1d5229c90a56d518b0d96ca57
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The 'guest.os_mach_type' property is set to the value returned by
'libvirt.utils.get_machine_type'. This will attempt to pull a machine
type from image metadata and fallback to a sensible default. As such, we
never need to worry about 'guest.os_mach_type' being unset on x86 hosts.
Change-Id: Ibc6d3dcaaa704599b7963bf97bdd901788f5af45
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We were mostly ignoring these except for a few limited examples. Do it
for everything.
Change-Id: Id55e1a8f3559386e27c5cb6a0fde94a4a69e7ea9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
What we have there is not a schema URI. Having a proper URI allows
auto-detection of the schema version (draft-07 in this case).
Change-Id: I208a05c7ba64a21ed8cde7bc1ebae2b5878109e0
Previously, the libvirt driver defaulted to 1024 * (# of CPUs) for the
value of domain/cputune/shares in the libvirt XML. This value is then
passed directly by libvirt to the cgroups API. Cgroups v2 imposes a
maximum value of 10000 that can be passed in. This makes Nova
unable to launch instances with more than 9 CPUs on hosts that run
cgroups v2, like Ubuntu Jammy or RHEL 9.
Fix this by just removing the default entirely. Because there is no
longer a guarantee that domain/cputune will contain at least a shares
element, we can stop always generating the former, and only generate
it if it will actually contain something.
We can also make operators's lives easier by leveraging the fact that
we update the XML during live migration, so this patch also adds a
method to remove the shares value from the live migration XML if one
was not set as the quota:cpu_shares flavor extra spec.
For operators that *have* set this extra spec to something greater
than 10000, their flavors will have to get updates, and their
instances resized.
Partial-bug: 1978489
Change-Id: I49d757f5f261b3562ada27e6cf57284f615ca395