Commit Graph

59804 Commits

Author SHA1 Message Date
Balazs Gibizer
14e68ac6e9 Rename [pci]passthrough_whitelist to device_spec
A later patch in the pci-device-tracking-in-placement work
will extend the existing [pci]passthrough_whitelist config syntax.
So we take the opportunity here to deprecate the old non inclusive
passthrough_whitelist name and introduce a better one.

All the usage of CONF.pci.passthrough_whitelist is now changed over to
the new device_spec config. Also the in tree documentation is updated
accordinly.

However the nova code still has a bunch of references to the
"whitelist" terminology. That will be handled in subsequent patches.

blueprint: pci-device-tracking-in-placement
Change-Id: I843032e113642416114f169069eebf6a56ed78dd
2022-08-10 17:08:35 +02:00
Balazs Gibizer
57c253a609 Add compute restart capability for libvirt func tests
The existing generic restart_compute_service() call in the nova test
base class is not appropriate for the libvirt functional test that needs
to reconfigure the libvirt connection as it is not aware of the libvirt
specific mocking needed when a compute service is started.

So this patch adds a specific restart_compute_service() call
to nova.tests.functional.libvirt.base.ServersTestBase. This will be used
by a later patch testing [pci]device_spec reconfiguration scenarios.

This change showed that some of the existing libvirt functional test
used the incomplete restart_compute_service from the base class. Others
used local mocking to inject new pci config to the restart. I moved all
these to the new function and removed the local mocking.

Change-Id: Ic717dc42ac6b6cace59d344acaf12f9d1ee35564
2022-08-10 17:08:35 +02:00
Balazs Gibizer
ebae3c2081 Poison /sys access via various calls in test
Unit test should never depend on the file system of the machine it runs
on. So this patch injects a poison to various std lib calls to catch
these cases. We only selectively poison access to /sys fs as there are
legitimate cases to access temp files.

When I isolated the poisoned test cases I tried to add a meaningful mocking
around the sysfs reads based on the meaning of the test.

Change-Id: I66636ebfe627cba8c4470cd84faa78f69249aa05
2022-08-10 17:08:34 +02:00
Balazs Gibizer
a93092e0d5 Update RequestSpec.pci_request for resize
Nova uses the RequestSpec.pci_request in the PciPassthroughFilter to
decide if the PCI devicesm, requested via the pci_alias in the flavor
extra_spec, are available on a potential target host. During resize the
new flavor might contain different pci_alias request than the old flavor
of the instance. In this case Nova should use the pci_alias from the new
flavor to scheduler the destination host of the resize. However this
logic was missing and Nova used the old pci_request value based on the
old flavor. This patch adds the missing logic.

Closes-Bug: #1983753
Closes-Bug: #1941005
Change-Id: I73c9ae27e9c42ee211a53bed3d849650b65f08be
2022-08-10 17:08:34 +02:00
Balazs Gibizer
6d602c6b73 Reproducer for bug 1983753
Related-Bug: #1983753
Change-Id: Ic6566272b8c5af57cd1c73c73b3cba6cd265bebe
2022-08-10 17:08:34 +02:00
Zuul
8bfff29cbc Merge "Imported Translations from Zanata" 2022-08-10 14:48:58 +00:00
Zuul
ea0526d959 Merge "Remove double mocking... again" 2022-08-10 13:37:02 +00:00
OpenStack Proposal Bot
d57bb972a2 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I7ec951c32577af0e0a1ba99659792a9eb75ae210
2022-08-10 03:57:11 +00:00
Zuul
1220d53c2f Merge "Update libvirt enlightenments for Windows" 2022-08-09 16:47:16 +00:00
Balazs Gibizer
bf654e3a4a Remove double mocking... again
I thought we fixed all the double mocking issues with
I3998d0d49583806ac1c3ae64f1b1fe343cefd20d but I was wrong.

While we used both mock and unittest.mock the fixtures.MockPatch
used the mock lib instead of the unittest.mock lib.
The path Ibf4f36136f2c65adad64f75d665c00cf2de4b400 (Remove the PowerVM driver)
removed the last user of mock lib from nova. So it is also
removed the mock from test-requirements. This triggered that
fixtures.MockPatch athat started using unittest.mock too.

Before Ibf4f36136f2c65adad64f75d665c00cf2de4b400 a function can be mocked
twice once with unittest.mock and once with fixtures.MockPatch (still
using mock). However after that patch both path of such double
mocking goes through unittest.mock and the second one fails.

So this patch fixes double mocking so far hidden behind
fixtures.MockPatch.

Also this patch makes the py310 and functional-py310 jobs voting at
least in the check queue to prevent future changes adding double mocks.

Change-Id: Ic1352ec31996577a5d0ad18a057339df3e49de25
2022-08-08 19:50:02 +02:00
Amit Uniyal
de830f87b0 Adds img_config_drive in nova users docs
Added img_config_drive property usuage in nova users
config drive docs

Closes-Bug: #1879066
Change-Id: I56be0b6cda51447813c0a97b01ccb48d0c898c20
2022-08-05 13:01:46 +00:00
Zuul
adeea3d5e7 Merge "Make test_wait_for_instance_event_* test time independent" 2022-08-04 14:02:47 +00:00
Zuul
7f5279edc9 Merge "For evacuation, ignore if task_state is not None" 2022-08-04 14:02:38 +00:00
Zuul
3b4378c189 Merge "Fix mocking SafeConnectedTestCase" 2022-08-03 17:44:37 +00:00
Zuul
8fe928156a Merge "[trivial] Simplify dict get call by removing unused default" 2022-08-03 11:30:46 +00:00
Zuul
ad7249b3fc Merge "Remove the PowerVM driver" 2022-08-03 10:47:24 +00:00
Zuul
c838e678a2 Merge "Updated Suspend definition in server concepts doc" 2022-08-03 07:33:15 +00:00
Amit Uniyal
db919aa15f For evacuation, ignore if task_state is not None
ignore instance task state and continue with vm evacutaion

Closes-Bug: #1978983
Change-Id: I5540df6c7497956219c06cff6f15b51c2c8bc29d
2022-08-03 04:52:10 +00:00
Artom Lifshitz
57ab45323c Update libvirt enlightenments for Windows
libvirt has a set of enlightenments in domain XML that make it
friendlier for Windows guests. We already enabled a few of these, this
patch just completes the list. All of them are available in libvirt
4.7.0 (QEMU 3.0) [1], which is way below our current minimum libvirt
and QEMU versions, so we don't need any extra checks.

[1] https://libvirt.org/formatdomain.html#hypervisor-features

Implements: bp/libvirt-update-windows-englightenments
Change-Id: I008841988547573878c4e06e82f0fa55084e51b5
2022-08-02 16:47:09 -04:00
Zuul
4ca7955365 Merge "hacking: force explicit import of python's mock" 2022-08-02 20:25:02 +00:00
Zuul
cc6045a4ac Merge "Remove double mocking" 2022-08-02 20:24:55 +00:00
Zuul
a0d520d221 Merge "add regression test case for bug 1978983" 2022-08-02 20:24:47 +00:00
Balazs Gibizer
6069592bcf Fix mocking SafeConnectedTestCase
The recent release of openstacksdk 0.100 showed that our testing is not
fully mocking underlying behavior. Such behavior changed in 0.100 and
broke our unit test. This patch fixes the mocking to avoid the breakage.

Change-Id: Ife4b8b42a414af281ac462faaa42c456ced13bb7
2022-08-02 18:17:05 +02:00
Stephen Finucane
deae814611 Remove the PowerVM driver
The PowerVM driver was deprecated in November 2021 as part of change
Icdef0a03c3c6f56b08ec9685c6958d6917bc88cb. As noted there, all
indications suggest that this driver is no longer maintained and may be
abandonware. It's been some time and there's still no activity here so
it's time to abandon this for real.

This isn't as tied into the codebase as the old XenAPI driver was, so
removal is mostly a case of deleting large swathes of code. Lovely.

Change-Id: Ibf4f36136f2c65adad64f75d665c00cf2de4b400
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2022-08-02 15:31:19 +02:00
Eric Fried
c36782a96a hacking: force explicit import of python's mock
Since we dropped support for python 2 [1], we no longer need to use the
mock library, which existed to backport py3 functionality into py2.
Change Ib44b5bff657c8e76c4f701e14d51a4efda3f6d32 cut over to importing
the stock mock, which must be done by saying::

    from unittest import mock

...because if you say::

    import mock

...you will be using the third party mock library instead, which may or
may not be installed.

This commit adds hacking check N371 to enforce the former.

[1] https://review.opendev.org/#/c/687954/

Change-Id: I71439580e80d33cff62aba807df2b35164a47cbe
2022-08-02 15:31:19 +02:00
Balazs Gibizer
f8cf050a13 Remove double mocking
In py310 unittest.mock does not allow to mock the same function twice as
the second mocking will fail to autospec the Mock object created by the
first mocking.

This patch manually fixes the double mocking.

Fixed cases:
1) one of the mock was totally unnecessary so it was removed
2) the second mock specialized the behavior of the first generic mock.
   In this case the second mock is replaced with the configuration of
   the first mock
3) a test case with two test steps mocked the same function for each
   step with overlapping mocks. Here the overlap was removed to have
   the two mock exists independently

The get_connection injection in the libvirt functional test needed a
further tweak (yeah I know it has many already) to act like a single
mock (basically case #2) instead of a temporary re-mocking. Still the
globalness of the get_connection mocking warrant the special set / reset
logic there.

Change-Id: I3998d0d49583806ac1c3ae64f1b1fe343cefd20d
2022-08-02 15:31:15 +02:00
Radosław Piliszek
b351f0b848 [docs] Fix mention of custom scheduling after Wallaby
Custom filters are still supported. Custom schedulers / scheduler
drivers are what is not. This patch fixes the wording.

Change-Id: Icf9576b75bd006ba65484b9232e85d8b748458f1
2022-08-02 10:15:51 +02:00
Amit Uniyal
1495d802c6 Updated Suspend definition in server concepts doc
Closes-Bug: #1943881
Change-Id: Icb8de13f665c340cd5d863654f8ad981de21293d
2022-08-02 04:18:10 +00:00
Stephen Finucane
89ef050b8c Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib. Most of this
is autogenerated, as described below, but there is one manual change
necessary:

nova/tests/functional/regressions/test_bug_1781286.py
  We need to avoid using 'fixtures.MockPatch' since fixtures is using
  'mock' (the library) under the hood and a call to 'mock.patch.stop'
  found in that test will now "stop" mocks from the wrong library. We
  have discussed making this configurable but the option proposed isn't
  that pretty [1] so this is better.

The remainder was auto-generated with the following (hacky) script, with
one or two manual tweaks after the fact:

  import glob

  for path in glob.glob('nova/tests/**/*.py', recursive=True):
      with open(path) as fh:
          lines = fh.readlines()
      if 'import mock\n' not in lines:
          continue
      import_group_found = False
      create_first_party_group = False
      for num, line in enumerate(lines):
          line = line.strip()
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              for lib in (
                  'ddt', 'six', 'webob', 'fixtures', 'testtools'
                  'neutron', 'cinder', 'ironic', 'keystone', 'oslo',
              ):
                  if lib in tokens[1]:
                      create_first_party_group = True
                      break
              if create_first_party_group:
                  break
              import_group_found = True
          if not import_group_found:
              continue
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              if tokens[1] > 'unittest':
                  break
              elif tokens[1] == 'unittest' and (
                  len(tokens) == 2 or tokens[4] > 'mock'
              ):
                  break
          elif not line:
              break
      if create_first_party_group:
          lines.insert(num, 'from unittest import mock\n\n')
      else:
          lines.insert(num, 'from unittest import mock\n')
      del lines[lines.index('import mock\n')]
      with open(path, 'w+') as fh:
          fh.writelines(lines)

Note that we cannot remove mock from our requirements files yet due to
importing pypowervm unit test code in nova unit tests. This library
still uses the mock lib, and since we are importing test code and that
lib (correctly) only declares mock in its test-requirements.txt, mock
would not otherwise be installed and would cause errors while loading
nova unit test code.

[1] https://github.com/testing-cabal/fixtures/pull/49

Change-Id: Id5b04cf2f6ca24af8e366d23f15cf0e5cac8e1cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2022-08-01 17:46:26 +02:00
Zuul
1a32196074 Merge "api: Drop generating a keypair and add special chars to naming" 2022-07-28 19:09:10 +00:00
Sylvain Bauza
a755e5d9f2 api: Drop generating a keypair and add special chars to naming
As agreed in the spec, we will both drop the generation support for a keypair
but we'll also accept @ (at) and . (dot) chars in the keyname, all of them in
the same API microversion.

Rebased the work from I5de15935e83823afa545a250cf84f6a7a37036b4

APIImpact

Implements: blueprint keypair-generation-removal
Co-Authored-By: Nicolas Parquet <nicolas.parquet@gandi.net>

Change-Id: I6a7c71fb4385348c87067543d0454f302907395e
2022-07-28 11:05:50 +02:00
Zuul
0bea7f6b6b Merge "Add a workaround to skip hypervisor version check on LM" 2022-07-27 13:46:29 +00:00
Kashyap Chamarthy
00ed8a232b Add a workaround to skip hypervisor version check on LM
When turned on, this will disable the version-checking of hypervisors
during live-migration.  This can be useful for operators in certain
scenarios when upgrading.  E.g. if you want to relocate all instances
off a compute node due to an emergency hardware issue, and you only have
another old compute node ready at the time.

Note, though: libvirt will do its own internal compatibility checks, and
might still reject live migration if the destination is incompatible.

Closes-Bug: #1982853

Change-Id: Iec387dcbc49ddb91ebf5cfd188224eaf6021c0e1
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
2022-07-27 12:20:03 +02:00
Amit Uniyal
5904c7f993 add regression test case for bug 1978983
This change add a repoducer test for evacuating
a vm in the powering-off state

Related-Bug: #1978983
Change-Id: I5540df6c7497956219c06cff6f15b51c2c8bc299
2022-07-25 09:00:15 +00:00
liuhuajie
ff7d9d7b7e [trivial] Simplify dict get call by removing unused default
The default value of the dictionary get method is None,
so remove the default value of None in the get method

Change-Id: Ic1b69047b6edd4987df3191ce84658393a1cb968
2022-07-22 15:18:45 +00:00
René Ribaud
09239fc2ea Allow unshelve to a specific host (REST API part)
This adds support to the REST API, in a new microversion, for specifying
a destination host to unshelve server action when the server
is shelved offloaded.
This patch also supports the ability to unpin the availability_zone of an
instance that is bound to it.

Note that the functional test changes are due to those tests using the
"latest" microversion 2.91.

Implements: blueprint unshelve-to-host
Change-Id: I9e95428c208582741e6cd99bd3260d6742fcc6b7
2022-07-22 10:22:34 +02:00
René Ribaud
a263fa46f8 Allow unshelve to a specific host (Compute API part)
This patch introduce changes to the compute API that will allow
PROJECT_ADMIN to unshelve an shelved offloaded server to a specific host.
This patch also supports the ability to unpin the availability_zone of an
instance that is bound to it.

Implements: blueprint unshelve-to-host
Change-Id: Ieb4766fdd88c469574fad823e05fe401537cdc30
2022-07-22 10:22:24 +02:00
Zuul
bcb96f362a Merge "libvirt: Remove unnecessary TODO" 2022-07-19 11:47:39 +00:00
Zuul
9d8a8d8ed8 Merge "libvirt: Ignore LibvirtConfigObject kwargs" 2022-07-19 11:27:26 +00:00
Zuul
bf347a17c0 Merge "Add a proper schema version to network_data.json" 2022-07-19 09:22:42 +00:00
Zuul
f90fd95f5e Merge "Adds link in releasenotes for hw machine type bug" 2022-07-19 08:48:54 +00:00
Stephen Finucane
2709e30956 Fix compatibility with jsonschema 4.x
This changed one of the error messages we depend on [1].

[1] https://github.com/python-jsonschema/jsonschema/commit/641e9b8c

Change-Id: I643ec568ee2eb2ec1a555f813fd2f1acff915afa
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2022-07-18 12:30:25 +01:00
Stephen Finucane
e1a925772b Remove unused requirement
We no longer support Python 3.6 so there's no need for Python 3.6-only
dependencies.

Change-Id: I139e454a2c6e49a1d5229c90a56d518b0d96ca57
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2022-07-18 12:30:25 +01:00
Zuul
ef18d36bf8 Merge "etc: Highlight absence of packages from config gen" 2022-07-18 05:12:09 +00:00
Zuul
700db274c6 Merge "Update the file for IPv4-only or IPv6-only network" 2022-07-18 05:00:34 +00:00
Zuul
e413ac015f Merge "libvirt: remove default cputune shares value" 2022-07-15 11:25:40 +00:00
Stephen Finucane
f08b71f3ab libvirt: Remove unnecessary TODO
The 'guest.os_mach_type' property is set to the value returned by
'libvirt.utils.get_machine_type'. This will attempt to pull a machine
type from image metadata and fallback to a sensible default. As such, we
never need to worry about 'guest.os_mach_type' being unset on x86 hosts.

Change-Id: Ibc6d3dcaaa704599b7963bf97bdd901788f5af45
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2022-07-15 04:10:58 +08:00
Stephen Finucane
f708feee35 libvirt: Ignore LibvirtConfigObject kwargs
We were mostly ignoring these except for a few limited examples. Do it
for everything.

Change-Id: Id55e1a8f3559386e27c5cb6a0fde94a4a69e7ea9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2022-07-15 04:10:58 +08:00
Dmitry Tantsur
4fdc0807b4 Add a proper schema version to network_data.json
What we have there is not a schema URI. Having a proper URI allows
auto-detection of the schema version (draft-07 in this case).

Change-Id: I208a05c7ba64a21ed8cde7bc1ebae2b5878109e0
2022-07-14 17:59:21 +02:00
Artom Lifshitz
f77a9fee5b libvirt: remove default cputune shares value
Previously, the libvirt driver defaulted to 1024 * (# of CPUs) for the
value of domain/cputune/shares in the libvirt XML. This value is then
passed directly by libvirt to the cgroups API. Cgroups v2 imposes a
maximum value of 10000 that can be passed in. This makes Nova
unable to launch instances with more than 9 CPUs on hosts that run
cgroups v2, like Ubuntu Jammy or RHEL 9.

Fix this by just removing the default entirely. Because there is no
longer a guarantee that domain/cputune will contain at least a shares
element, we can stop always generating the former, and only generate
it if it will actually contain something.

We can also make operators's lives easier by leveraging the fact that
we update the XML during live migration, so this patch also adds a
method to remove the shares value from the live migration XML if one
was not set as the quota:cpu_shares flavor extra spec.

For operators that *have* set this extra spec to something greater
than 10000, their flavors will have to get updates, and their
instances resized.

Partial-bug: 1978489
Change-Id: I49d757f5f261b3562ada27e6cf57284f615ca395
2022-07-14 11:55:40 -04:00