55151 Commits

Author SHA1 Message Date
Eric Fried
73f1fda7e9 Bump min for oslo.service & .privsep to fix SIGHUP
The combined fixes for the two related bugs resolve the problem where
SIGHUP breaks the nova-compute service. Bump the minimum requirements
for oslo.privsep and oslo.service to make sure these fixes are in place,
and add a reno to advertise resolution of the issue.

This also bumps oslo.utils to match the lower constraint from
oslo.service.

Change-Id: I39ead744b21a4423352a88573f327273e4d09630
Related-Bug: #1794708
Related-Bug: #1715374
2019-09-05 18:16:43 -05:00
Zuul
cbaea3bd69 Merge "re-calculate provider mapping during migration" 2019-09-04 01:00:07 +00:00
Zuul
a496bb9397 Merge "libvirt: Fold in argument to '_update_provider_tree_for_vgpu'" 2019-09-03 22:12:56 +00:00
Zuul
de6b5dd87d Merge "Use SDK for setting instance id" 2019-09-03 18:30:18 +00:00
Zuul
8670b0b9cb Merge "Use SDK for validating instance and node" 2019-09-03 18:17:46 +00:00
Zuul
5e6446bb2b Merge "Use SDK for node.list" 2019-09-03 17:50:31 +00:00
Zuul
27f2eb9cb6 Merge "Remove dead code" 2019-09-03 16:56:29 +00:00
Zuul
3d9344a88b Merge "Nice to have test coverage for If1f465112b8e9b0304b8b5b864b985f72168d839" 2019-09-03 16:56:20 +00:00
Zuul
34a052f04e Merge "objects: Rename 'fields' import to 'obj_fields'" 2019-09-03 16:56:13 +00:00
Zuul
40e7fdbaec Merge "libvirt: Start checking compute usage in functional tests" 2019-09-03 16:01:59 +00:00
Zuul
2821908a07 Merge "libvirt: Simplify 'fakelibvirt.HostInfo' object" 2019-09-03 16:01:49 +00:00
Zuul
4318bfdd18 Merge "Add <launchSecurity> and <driver iommu='on' /> to config.py" 2019-09-03 15:21:03 +00:00
Stephen Finucane
30c5ba30dc libvirt: Fold in argument to '_update_provider_tree_for_vgpu'
The '_update_provider_tree_for_vgpu' argument took an 'inventories_dict'
argument, but that argument was generated immediately before the sole
call to the function. Just generate the argument inside the function
itself.

Change-Id: Id026855c06e047023b8092a45a0c3e02364c3dbb
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-09-03 09:42:14 +01:00
Stephen Finucane
726a35b345 objects: Rename 'fields' import to 'obj_fields'
Otherwise we overload the import with the 'fields' variable defined on
the class.

Change-Id: Ife4646eb8da78cc25baddddfd3eb8c7d360352b3
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-09-03 09:42:14 +01:00
Stephen Finucane
64f09a4a1f libvirt: Start checking compute usage in functional tests
We're using placement. Let's make sure everything is working in concert
and requesting the right amount and types of things.

Change-Id: Ieea576b700327ba5a5300e512d42e51a255abbba
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Co-authored-by: Bhagyashri Shewale <bhagyashri.shewale@nttdata.com>
2019-09-03 09:42:14 +01:00
Stephen Finucane
ea5bb8e1ec libvirt: Simplify 'fakelibvirt.HostInfo' object
Nothing was overriding a number of these fields so make things less
complex by removing the customizability. We can easily re-add it if we
need to in the future. We can do more here (most callers of this use the
exact same values), but that's a change for later.

Change-Id: Id3e0668acb71b3c7350b73b4afff7e940c6dfb1d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-09-03 09:42:14 +01:00
Zuul
cc46495d5e Merge "Extract SEV-specific bits on host detection" 2019-09-02 23:57:10 +00:00
Dustin Cowles
ce4c60cf4b Use SDK for setting instance id
We would like nova not to use ironicclient, but instead to invoke the
ironic API directly through an openstacksdk connection.

The parent commits set up the framework, and this commit uses it
for the _set_instance_id function.

Blueprint: openstacksdk-in-nova
Change-Id: I95f3414d29a5ff5cc49994bac1cda917edc4f292
2019-09-02 14:21:25 -07:00
Dustin Cowles
3440c3d37a Use SDK for validating instance and node
We would like nova not to use ironicclient, but instead to invoke the
ironic API directly through an openstacksdk connection.

The parent commits set up the framework, and this commit uses it
for the instance_exists function.

Blueprint: openstacksdk-in-nova
Change-Id: I764b0086d0bf39d4ca8cb308696169282b12b42b
2019-09-02 14:21:23 -07:00
Zuul
81853e2783 Merge "Add request_spec to server move RPC calls" 2019-09-02 20:02:07 +00:00
Zuul
375e80f9a5 Merge "Pass network API to the conducor's MigrationTask" 2019-09-02 18:07:06 +00:00
Zuul
7bc8f24dc9 Merge "allow getting resource request of every bound ports of an instance" 2019-09-02 16:34:58 +00:00
Stephen Finucane
1635be335e Remove dead code
The '_test_simple_call' test is unused since change
Ibcb6bf912b3fb69c8631665fef2832906ba338aa.

Change-Id: If9e36cb4a21ced44f2ee7dede480a0719fcccdf1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-09-02 17:31:25 +01:00
Zuul
ef6e49d5bc Merge "trivial: Rewrap definitions of 'NUMACell'" 2019-09-02 12:47:25 +00:00
Balazs Gibizer
84b1a98efe Nice to have test coverage for If1f465112b8e9b0304b8b5b864b985f72168d839
Matt pointed out in [1] that there is missing test coverage for some of
the changes in If1f465112b8e9b0304b8b5b864b985f72168d839 for the rpc api
behavior. So this follow up adds the extra unit tests.

[1] https://review.opendev.org/#/c/655721/17/nova/compute/rpcapi.py@615

Change-Id: I5e837f30c888ed2fa63856670695d2836cf48e00
blueprint: support-move-ops-with-qos-ports
2019-09-02 13:46:26 +02:00
Stephen Finucane
dc3a8de1f8 trivial: Rewrap definitions of 'NUMACell'
In a future change, I492803eaacc34c69af073689f9159449557919db, we're
going to be adding a new field to all definitions of this object. That
is going to result in a lot of line wrapping which will make the change
a lot harder to grok. Do that wrapping now so that we can make the
functional changes in that future change easier to identify.

Part of blueprint cpu-resources

Change-Id: Id3441073adde563a568c9550df53690d1e6c998a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-09-02 11:31:27 +01:00
Boris Bobrov
3941679aac Add <launchSecurity> and <driver iommu='on' /> to config.py
<launchSecurity> enables use of SEV, so add the required config class
for it.  For now the class just describes the new element; it will be
used in further commits.

Furthermore, AMD SEV requires enabling DMA APIs for virtio devices
via <driver iommu='on' />:

  http://specs.openstack.org/openstack/nova-specs/specs/train/approved/amd-sev-libvirt-support.html#proposed-change

So also add the necessary bits for that to domain configuration.

blueprint: amd-sev-libvirt-support
Change-Id: Ia78458ac698f66d297cb91bcab663fcf123c5442
2019-09-01 15:17:36 +01:00
Boris Bobrov
44dc83d473 Extract SEV-specific bits on host detection
Extract reducedPhysBits and cbitpos when detecting that host is
SEV-enabled.

These parameters will later be used to configure the guest.
http://specs.openstack.org/openstack/nova-specs/specs/train/approved/amd-sev-libvirt-support.html#sev-launch-time-configuration

Change-Id: I1d9308147a389fc94b7e872f9bd0621bdb25b193
blueprint: amd-sev-libvirt-support
2019-09-01 15:17:34 +01:00
Adam Spiers
b4905467db Add extra spec parameter and image property for memory encryption
Add a new "hw:mem_encryption" extra spec parameter, and a new
"hw_mem_encryption" image property, which indicate that any guest
booted with that extra spec parameter or image property respectively
needs to be booted with its memory hardware-encrypted.

This is achieved by converting the requirement stated in the extra
spec parameter and/or image property into an additional extra spec
parameter which requests resources for one slot of the inventory of
the new MEM_ENCRYPTION_CONTEXT resource class (introduced in
os-resource-classes 0.4.0).  The inventory will be provided by the
follow-up commit I659cb77f12a38a4d2fb118530ebb9de88d2ed30d.

Since future commits adding support for SEV to guest XML config will
also need to know at launch-time whether memory encryption has been
requested, add a reusable mem_encryption_requested() function to the
nova.virt.hardware library for detecting which of the extra spec /
image property (if either) have requested encrypted memory.

If both the extra spec parameter and the image property are explicitly
specified and they contradict each other, or if either request memory
encryption but the image does not have hw_firmware_type set to UEFI,
then log an error and raise a new generic FlavorImageConflict
exception.  This exception can also be useful in the future for
handling other similar conflicts.  In this particular use case,
FlavorImageConflict is raised by mem_encryption_requested(), and then
if caught during API call validation, it's re-raised as
HTTPBadRequest.

In order to test this code, we need to construct various ImageMeta
objects containing fake data and a ImageMetaProps instance for each.
This is a slightly fiddly task which future patches in the SEV series
will also need to perform, so add a helper to nova.tests.unit.image.fake
for this.

blueprint: amd-sev-libvirt-support
Change-Id: I8c63b5cc5ad97ce831adb2eb96a995ebc798ecb7
2019-09-01 15:17:31 +01:00
Zuul
160d1042b4 Merge "Provide HW_CPU_X86_AMD_SEV trait when SEV is supported" 2019-08-31 16:07:44 +00:00
Zuul
f8969bc44f Merge "Add cold migrate and resize to nova-grenade-multinode" 2019-08-31 02:58:23 +00:00
Zuul
c477f3654e Merge "Rename the nova-grenade-live-migration job to nova-grenade-multinode" 2019-08-31 02:49:39 +00:00
Zuul
2f5b578f99 Merge "Delete InstanceMapping in conductor if BuildRequest is already deleted" 2019-08-31 00:50:13 +00:00
Zuul
5005ff9086 Merge "libvirt: use native AIO mode for StorPool Cinder volumes." 2019-08-31 00:50:04 +00:00
Balazs Gibizer
2e9fd01e7a re-calculate provider mapping during migration
Nova intentionally does not persist the resoruce request of the neturon
ports. Therefore during migration nova needs to query neturon about the
resource requests to include them to the allocation_candidates query
sent to placement during scheduling. Also when the allocation is made by
the scheduler nova needs to re-calculate request group - resource
provider mapping. A subsequent patch will use this mapping to update the
binding profile when the port is bound to the destination host of the
migration.

blueprint: support-move-ops-with-qos-ports

Change-Id: I8e5a0480c81ba548bc1f50a8098eabac52b11453
2019-08-30 17:48:42 -04:00
Balazs Gibizer
a0e60feb3e Add request_spec to server move RPC calls
To be able to fill the allocation key in the port binding:profile during
the move operations the nova-compute needs to get the RequestSpec object
to have access to the port - resource provider mapping calculated in the
conductor.

This patch bumps the compute RPC api version and adds a new request_spec
parameter to multiple calls. Also it makes sure that the request_spec is
passed by the sender.

Change-Id: If1f465112b8e9b0304b8b5b864b985f72168d839
blueprint: support-move-ops-with-qos-ports
2019-08-30 15:51:14 -04:00
Balazs Gibizer
a413150b20 Pass network API to the conducor's MigrationTask
During migration conductor needs to heal the content of the
RequestSpec.requested_resources field based on the resource requests of
the ports attached to the instance being migrated.

This patch makes sure that the MigrationTask has access the networking
API to do such healing.

blueprint: support-move-ops-with-qos-ports

Change-Id: Idf38568c3c237687c54fbbfcc6c5792c49c95161
2019-08-30 15:51:14 -04:00
Balazs Gibizer
fb2ec18477 allow getting resource request of every bound ports of an instance
This patch adds a new network api method
get_requested_resource_for_instance() to query neutron about the
resource needs of the ports currently bound to an instance. A later
patch will use this to properly allocate resource for the ports during
migration.

blueprint: support-move-ops-with-qos-ports

Change-Id: I56ea5ee94139dfa2cdf6bb76656bca2902e7ea9c
2019-08-30 15:51:10 -04:00
Zuul
5ab30d5510 Merge "fix lxml compatibility issues" 2019-08-30 19:42:55 +00:00
Zuul
c3dadbdd9c Merge "libvirt/host.py: remove unnecessary temporary variable" 2019-08-30 19:42:33 +00:00
Zuul
7f501b571b Merge "Introduces SDK to IronicDriver and uses for node.get" 2019-08-30 19:42:20 +00:00
Balazs Gibizer
2cf9a5f9fa Add cold migrate and resize to nova-grenade-multinode
Changes in [1] could potentially break a mixed-compute-version
environment as we don't have grenade coverage for cold migrate and
resize. This adds that coverage to the nova-grenade-multinode
job.

[1]https://review.opendev.org/#/c/655721/10

Change-Id: I81372d610ddf8abb473621deb6e7cb68eb000fee
2019-08-30 15:35:46 -04:00
Balazs Gibizer
7f8dd04108 Rename the nova-grenade-live-migration job to nova-grenade-multinode
We are planning to add non-live-migration multinode tests to this job
so first we rename it to be more generic.

Change-Id: I7571ff508fcccba40cf2307a8788c1850d2d9fd8
2019-08-30 15:35:46 -04:00
Zuul
5450f77d10 Merge "Follow up for specifying az to unshelve" 2019-08-30 16:12:04 +00:00
Sean Mooney
58ffff49ac fix lxml compatibility issues
Some unit tests were performing string matches on xml data
with inputs that were generated using lxml. This is problematic
as while white space between element tags is important in xml
ordering of attributes within a tag is not. In the latest version
of lxml the ordering asserted in the test no longer matches the
order returned by lxml on python 3.6+.

This change updates the failing test to use the XMLMatcher
class to compare xml strings instead.

Closes-Bug: #1838666
Related-Bug: #1841667
Change-Id: I1649a850ccb9ac85d7a962936ffef51d573b6f78
2019-08-30 13:48:58 +01:00
Adam Spiers
4f6fcfbdd0 libvirt/host.py: remove unnecessary temporary variable
This is a simple fixup to address feedback from:

  https://review.opendev.org/#/c/673151/16/nova/virt/libvirt/host.py@764

blueprint: amd-sev-libvirt-support
Change-Id: I2b3d01756defb7be206f487bcb21b7486ff0553c
2019-08-30 12:56:53 +01:00
Zuul
744615559e Merge "Add functional test for AggregateMultiTenancyIsolation + migrate" 2019-08-30 11:27:40 +00:00
Adam Spiers
f5964dfaa2 Provide HW_CPU_X86_AMD_SEV trait when SEV is supported
Add code to init_host() in the libvirt driver which detects support
for AMD SEV in the compute host hardware and hypervisor, by checking
that:

   a) 'sev' parameter of the kvm-amd kernel module is enabled, and

   b) the <features> element returned from libvirt's
      virConnectGetDomainCapabilities API contains an

         <sev supported='yes'>

      element.  This second check is achieved by utilising the domain
      capability retrieval and parsing code which was added in
      I4c35b6a27db05349213429422ffe62adb09bd921.

When both checks pass, provide the standard trait HW_CPU_X86_AMD_SEV
(which was newly available in os-traits 0.13.0) on the resource
provider for the compute host.  This indicates the ability of the
compute host to boot instances with AMD SEV (Secure Encrypted
Virtualization) enabled.

Note that this is not the last of the detection code.  Future commits
will add support for the use of a new MEM_ENCRYPTION_CONTEXT resource
class in the compute host inventory which tracks how many guests with
encrypted memory can be run on a host concurrently.  Until that point,
SEV functionality cannot be used, and this trait is effectively
dormant.

blueprint: amd-sev-libvirt-support
Change-Id: I2b41f1cce0af8b9d36b74a4663aa4ff227e17cc6
2019-08-30 10:36:53 +01:00
Zuul
c43ec6b471 Merge "Move live_migration test hooks under gate/" 2019-08-30 05:37:46 +00:00
Zuul
d94918a1fd Merge "Add FUP unit test for port heal allocations" 2019-08-30 03:50:35 +00:00