The ServiceLauncher and ProcessLauncher in oslo.service will,
by default, log config options at DEBUG level at the start
of a service, which is what would happen when starting nova-api
using eventlet.
Running nova-api under wsgi has been supported since Pike, but
the wsgi app code doesn't log the debug options like oslo.service
would, so this adds that back in.
The placement-api wsgi app code would log the options but based on
whether or not debug logging is enabled, which is different from how
it works in oslo.service, so the config option that is checked is
changed in this patch, and a release note is added for that subtle
behavior change.
Closes-Bug: #1732000
Change-Id: I680fd9761a049cac619b7793fa5c60e6daf4fa47
When storage network for booting an instance from iSCSI volume is
managed by neutron, an IP address for volume connector cannot be
registered by an operator in advance. A MAC address can be registered
as a volume connector for an Ironic node. The IP address may be
required depending on cinder backend drivers.
This patch gets an IP address for a volume connector based on a MAC
address assigned to a volume connector in ironic. To bind VIFs to
ironic ports before connecting a volume, VIFs are attached earlier
with a new virt driver API. Nova can get an IP address assigned to
a VIF attached to an ironic port by retrieving the port with the
MAC address.
Co-Authored-By: Satoru Moriya <satoru.moriya.br@hitachi.com>
Implements: blueprint ironic-volume-connector-ip
Change-Id: I999bbfc0e28ec43390298deb59e2b6f6e10bf8ea
When set reclaim_instance_interval > 0, and then delete an
instance which booted from volume with `delete_on_termination`
set as true. After reclaim_instance_interval time pass,
all volumes boot instance will with state: attached and in-use,
but attached instances was deleted.
This bug case as admin context from
`nova.compute.manager._reclaim_queued_deletes` did not have
any token info, then call cinder api would be failed.
So add user/project CONF with admin role at cinder group,
and when determine context is_admin and without token, do
authenticaion with user/project info to call cinder api.
Change-Id: I3c35bba43fee81baebe8261f546c1424ce3a3383
Closes-Bug: #1733736
Closes-Bug: #1734025
Partial-Bug: #1736773
Including updating their unit tests. We can now remove parted from
the rootwrap configuration.
Change-Id: I8cbfe296238976001e38997842059ec2f137f660
blueprint: hurrah-for-privsep
Since libvirt doesn't allow us to hot-unplug mediated devices, we need to
short-circuit the suspend action if the instance has mediated devices
and set it back to the ACTIVE state.
Change-Id: I01147bb3c66d94fdecb76395e5205767a905d18a
This patch enables the flavor extra spec 'required:[traits]'. The
admin can specifiy a set of traits that a flavor required.
To enable this, the placement 1.17 is required, it added trait
support in the `GET /allocation_candidates` API. So bump the
minimal placement version requirement to 1.17 and update the
check in cmd `nova-status`.
Implement blueprint request-traits-in-nova
Change-Id: Ia51ace951e9a0402f873ce9751a8cd3c0283db08
StorPool is distributed data storage software running on standard x86
servers. StorPool aggregates the performance and capacity of all drives
into a shared pool of storage distributed among the servers. Within
this storage pool the user creates thin-provisioned volumes that are
exposed to the clients as block devices. StorPool consists of two parts
wrapped in one package - a server and a client. The StorPool server
allows a hypervisor to act as a storage node, while the StorPool client
allows a hypervisor node to access the storage pool and act as a compute
node. In OpenStack terms the StorPool solution allows each hypervisor
node to be both a storage and a compute node simultaneously.
This driver allows StorPool volumes defined in Cinder to be attached as
additional disks to a virtual machine.
Change-Id: I3d40009eb17d054f33b3ed641643e285ba094ec2
Implements: blueprint libvirt-storpool-volume-attach
When we reboot a guest (like when stopping/starting it), we recreate the guest
XML. Since we need to know the existing mediated devices before destroying
the guest, let's pass them when recreating the XML.
NOTE: I'm also amending the reno file to exactly mention what has been tested
and fully functional and what is not working for all the instance actions.
I'm also considering that change as the last patch that makes the feature
production-ready as most of the left quirks can be easily worked around by
either rebuilding the instance (for example when you resize) or just shelving
the instance instead of suspending it.
Next changes in the series will address those issues but won't be identified
as part of the blueprint itself.
Change-Id: Idba99f3f9b4abe77c042a6edaf9f5fe5c75ac32c
Implements: blueprint add-support-for-vgpu
Currently there is no indication that the rebuild was refused,
and worse, we may have a wrong imageref for the instance.
This patch set the instance to ERROR status if rebuild failed in the
scheduling stage. The user can rebuild the instance with valid image
to get it out of ERROR state and reset with right instance metadata and
properties.
Closes-Bug: 1744325
Change-Id: Ibb7bee15a3d4ee6f0ef53ba12e8b41f65a1fe999
The performance of filtering instance by IP is poor, due to the fact that
the IP address is part of a JSON-encoded filed and we need to unpack the
field and do a search on the field for each record in instances table, we
have to iterate one by one to find the instance that matches the request.
As discussed in Queens PTG[1], one possible solution is to get filtered
ports from Neutron and retrieve the instance uuid from the port.device_id
and then merge to the other filters.
As Nova provides regex matching manner filtering for IP filter, so this
depend on Neutron changes that add substring matching functionality to
the GET /ports API[2]
The performance test results of the POC are presented in[3].
[1]http://lists.openstack.org/pipermail/openstack-dev/2017-September/122258.html
[2]https://bugs.launchpad.net/neutron/+bug/1718605
[3]http://lists.openstack.org/pipermail/openstack-dev/2018-January/125990.html
Depends-On: I9549b2ba676e1bad0812682c3f3f3c97de15f5f6
Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>
Implements: blueprint improve-filter-instances-by-ip-performance
Change-Id: I06aabfdce4aaefde080c7e122552ce4f36da5804
There is a lacking feature in libvirt (due to the nature of what is mediated
device as just a sysfs file) that mediated devices are not persisted.
When you reboot your host, you loose all of them which can be a big pain for
operators having allocated guests using vGPUs.
This change will iterate over all the instances, see if they have a nested
vGPU, check if the related mediated device exists, and if not, rebalance
between physical GPUs to find a proper one which can fit.
Note that due to the fact we don't persist neither in Nova the mediated device
information, mediated devices can be created on different physical devices that
they were before the reboot. That's not a big deal since we only support one
type at the moment, but that could become a problem later as we would need to
figure out which type the mediated device was before the reboot.
Partially-Implements: blueprint add-support-for-vgpu
Change-Id: Ie6c9108808a461359d717d8a9e9399c8a407bfe9
This change introduces a new microversion which must be used
to create a server from a multiattach volume or attach a multiattach
volume to an existing server instance.
Attaching a multiattach volume to a shelved offloaded instance is not
supported since an instance in that state does not have a compute host
so we can't tell if the compute would support the multiattach volume
or not. This is consistent with the tagged attach validation with 2.49.
When creating a server from a multiattach volume, we'll check to see
if all computes in all cells are upgraded to the point of even supporting
the compute side changes, otherwise the server create request fails with
a 409. We do this because we don't know which compute node the scheduler
will pick and we don't have any compute capability filtering in the
scheduler for multiattach volumes (that may be a future improvement).
Similarly, when attaching a multiattach volume to an existing instance,
if the compute isn't new enough to support multiattach or the virt
driver simply doesn't support the capability, a 409 response is returned.
Presumably, operators will use AZs/aggregates to organize which hosts
support multiattach if they have a mixed hypervisor deployment, or will
simply disable multiattach support via Cinder policy.
The unit tests are covering error conditions with the new flow. A new
functional scenario test is added for happy path testing of the new boot
from multiattach volume flow and attaching a multiattach volume to more
than one instance.
Tempest integration testing for multiattach is added in change
I80c20914c03d7371e798ca3567c37307a0d54aaa.
Devstack support for multiattach is added in change
I46b7eabf6a28f230666f6933a087f73cb4408348.
Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>
Implements: blueprint multi-attach-volume
Change-Id: I02120ef8767c3f9c9497bff67101e57e204ed6f4
The nova noVNC proxy server has gained the ability to use the VeNCrypt
authentication scheme to secure network communications with the compute
node VNC servers. This documents how to configure the QEMU/KVM compute
nodes and the noVNC proxy server nodes.
Change-Id: If3cea87568efff0874cd8851cabc6770812c545b
Blueprint: websocket-proxy-to-host-security
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
This patch add new query parameter `required` to the
`GET /allocation_candidates` API, which is used to filter candidates
with required traits. The candidate attached traits return in the
provider summary also. Those API changes are added by new microversion.
Also using specific exception TraitNotFound instead of the generic
exception ValueError when invalid traits in the request.
Change-Id: Id821b5b2768dcc698695ba6570c6201e1e9a8233
Implement blueprint add-trait-support-in-allocation-candidates
If an allocation is asking for a VGPU, then libvirt will look at the
available mediated devices and call the sysfs to create one of them if
needed.
Please note I commented in the relnote all the caveats we currently have
with mediated devices in libvirt, but I'll provide some workarounds for
those in the next changes.
Change-Id: Ibf210dd27972fed2651d6c9bd73a0bcf352c8bab
Partially-Implements: blueprint add-support-for-vgpu
All image signature properties should not be inherited from the metadata
of the original image when creating a snapshot of an instance. Otherwise
Glance will attempt to verify the signature of the snapshot image and
fail as this has changed from that of the original.
Closes-bug: #1737513
Change-Id: Ia3d80bf2f81c7317fec117aecbc3c560d51a7d4e
Instead of doing straight passthrough of the RFB protocol from the
tenant sock to the compute socket, insert an RFB security proxy. This
will MITM the initial RFB protocol handshake in order to negotiate an
authentication scheme with the compute node that is distinct from that
used by the tenant.
Based on earlier work by Solly Ross <sross@redhat.com>
Change-Id: I9cc9a380500715e60bd05aa5c29ee46bc6f8d6c2
Co-authored-by: Stephen Finucane <sfinucan@redhat.com>
Implements: bp websocket-proxy-to-host-security
Provide an implementation for the VeNCrypt RFB authentication
scheme, which uses TLS and x509 certificates to provide both
encryption and mutual authentication / authorization.
Based on earlier work by Solly Ross <sross@redhat.com>
Change-Id: I6a63d2535e86faf369ed1c0eeba6cb5a52252b80
Co-authored-by: Stephen Finucane <sfinucan@redhat.com>
Implements: bp websocket-proxy-to-host-security
Introduce a framework for providing RFB authentication scheme
implementations. This will be later used by the websocket RFB security
proxy code.
Change-Id: I98403ca922b83a460a4e7baa12bd5f596a79c940
Co-authored-by: Stephen Finucane <sfinucan@redhat.com>
Implements: bp websocket-proxy-to-host-security