1187 Commits

Author SHA1 Message Date
He Jie Xu
284ba35c33 Fix nits in support traits changes
Addresses the comments from earlier patches:

https://review.openstack.org/535642

https://review.openstack.org/536085

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Change-Id: I366b97ef3c141834f48949700edb968a7c7c4167
2018-01-31 11:07:07 -05:00
Matt Riedemann
53dc0917c5 Log options at debug when starting API services under wsgi
The ServiceLauncher and ProcessLauncher in oslo.service will,
by default, log config options at DEBUG level at the start
of a service, which is what would happen when starting nova-api
using eventlet.

Running nova-api under wsgi has been supported since Pike, but
the wsgi app code doesn't log the debug options like oslo.service
would, so this adds that back in.

The placement-api wsgi app code would log the options but based on
whether or not debug logging is enabled, which is different from how
it works in oslo.service, so the config option that is checked is
changed in this patch, and a release note is added for that subtle
behavior change.

Closes-Bug: #1732000

Change-Id: I680fd9761a049cac619b7793fa5c60e6daf4fa47
2018-01-31 15:45:27 +00:00
Stephen Finucane
bd3da5d763 trivial: Fix typos in release notes
Change-Id: I989fa12f115075c27b29b4863cbb5240abfb5978
2018-01-31 10:51:03 +00:00
Zuul
b0a027ef0e Merge "Add release note for QEMU native LUKS decryption" 2018-01-31 08:15:53 +00:00
Zuul
64c452d568 Merge "Avoid suspending guest with attached vGPUs" 2018-01-31 00:29:54 +00:00
Balazs Gibizer
43cafb684c reno for notification-transformation-queens
Depends-On: https://review.openstack.org/#/c/465081/
Change-Id: Id8b2defca083f86047aba445bfe73bfa7eeb6f80
2018-01-30 10:13:37 +01:00
Zuul
55ea961cef Merge "Ironic: Get IP address for volume connector" 2018-01-27 22:05:30 +00:00
Zuul
ec8b9a59c0 Merge "libvirt: pass the mdevs when rebooting the guest" 2018-01-27 15:29:41 +00:00
Zuul
e6d1bf6bdf Merge "Move remaining uses of parted to privsep." 2018-01-27 15:28:43 +00:00
Zuul
fa80eb1cd9 Merge "placement: support traits in allocation candidates API" 2018-01-27 03:34:12 +00:00
Zuul
92754a0f91 Merge "Deprecate allowed_direct_url_schemes and nova.image.download.modules" 2018-01-27 00:15:49 +00:00
Zuul
606689a05d Merge "Add the Nova libvirt StorPool attachment driver." 2018-01-26 12:47:53 +00:00
Zuul
ac60fda9e5 Merge "Use neutron port_list when filtering instance by ip" 2018-01-26 09:53:31 +00:00
Hironori Shiina
23d935b3a6 Ironic: Get IP address for volume connector
When storage network for booting an instance from iSCSI volume is
managed by neutron, an IP address for volume connector cannot be
registered by an operator in advance. A MAC address can be registered
as a volume connector for an Ironic node. The IP address may be
required depending on cinder backend drivers.

This patch gets an IP address for a volume connector based on a MAC
address assigned to a volume connector in ironic. To bind VIFs to
ironic ports before connecting a volume, VIFs are attached earlier
with a new virt driver API. Nova can get an IP address assigned to
a VIF attached to an ironic port by retrieving the port with the
MAC address.

Co-Authored-By: Satoru Moriya <satoru.moriya.br@hitachi.com>
Implements: blueprint ironic-volume-connector-ip
Change-Id: I999bbfc0e28ec43390298deb59e2b6f6e10bf8ea
2018-01-25 14:10:02 -05:00
melanie witt
e04ef32244 Add release note for QEMU native LUKS decryption
Part of blueprint libvirt-qemu-native-luks

Change-Id: Ifad80fbad54e31986af5da265d37b8ce4a01ef10
2018-01-25 19:01:39 +00:00
lixipeng
ca6daf148d Fix bug case by none token context
When set reclaim_instance_interval > 0, and then delete an
instance which booted from volume with `delete_on_termination`
set as true. After reclaim_instance_interval time pass,
all volumes boot instance will with state: attached and in-use,
but attached instances was deleted.

This bug case as admin context from
`nova.compute.manager._reclaim_queued_deletes` did not have
any token info, then call cinder api would be failed.

So add user/project CONF with admin role at cinder group,
and when determine context is_admin and without token, do
authenticaion with user/project info to call cinder api.

Change-Id: I3c35bba43fee81baebe8261f546c1424ce3a3383
Closes-Bug: #1733736
Closes-Bug: #1734025
Partial-Bug: #1736773
2018-01-24 20:53:50 -05:00
Michael Still
635d205268 Move remaining uses of parted to privsep.
Including updating their unit tests. We can now remove parted from
the rootwrap configuration.

Change-Id: I8cbfe296238976001e38997842059ec2f137f660
blueprint: hurrah-for-privsep
2018-01-24 22:26:36 +00:00
Sylvain Bauza
8e037894f1 Avoid suspending guest with attached vGPUs
Since libvirt doesn't allow us to hot-unplug mediated devices, we need to
short-circuit the suspend action if the instance has mediated devices
and set it back to the ACTIVE state.

Change-Id: I01147bb3c66d94fdecb76395e5205767a905d18a
2018-01-24 09:54:05 -08:00
He Jie Xu
6a818ebc19 placement: enable required traits from the flavor extra specs
This patch enables the flavor extra spec 'required:[traits]'. The
admin can specifiy a set of traits that a flavor required.

To enable this, the placement 1.17 is required, it added trait
support in the `GET /allocation_candidates` API. So bump the
minimal placement version requirement to 1.17 and update the
check in cmd `nova-status`.

Implement blueprint request-traits-in-nova

Change-Id: Ia51ace951e9a0402f873ce9751a8cd3c0283db08
2018-01-24 23:35:54 +08:00
Zuul
dc63965ab4 Merge "Set server status to ERROR if rebuild failed" 2018-01-24 05:19:26 +00:00
Peter Penchev
b100eb05ab Add the Nova libvirt StorPool attachment driver.
StorPool is distributed data storage software running on standard x86
servers.  StorPool aggregates the performance and capacity of all drives
into a shared pool of storage distributed among the servers.  Within
this storage pool the user creates thin-provisioned volumes that are
exposed to the clients as block devices.  StorPool consists of two parts
wrapped in one package - a server and a client.  The StorPool server
allows a hypervisor to act as a storage node, while the StorPool client
allows a hypervisor node to access the storage pool and act as a compute
node.  In OpenStack terms the StorPool solution allows each hypervisor
node to be both a storage and a compute node simultaneously.

This driver allows StorPool volumes defined in Cinder to be attached as
additional disks to a virtual machine.

Change-Id: I3d40009eb17d054f33b3ed641643e285ba094ec2
Implements: blueprint libvirt-storpool-volume-attach
2018-01-24 00:01:42 +02:00
Sylvain Bauza
39382587fe libvirt: pass the mdevs when rebooting the guest
When we reboot a guest (like when stopping/starting it), we recreate the guest
XML. Since we need to know the existing mediated devices before destroying
the guest, let's pass them when recreating the XML.

NOTE: I'm also amending the reno file to exactly mention what has been tested
and fully functional and what is not working for all the instance actions.

I'm also considering that change as the last patch that makes the feature
production-ready as most of the left quirks can be easily worked around by
either rebuilding the instance (for example when you resize) or just shelving
the instance instead of suspending it.

Next changes in the series will address those issues but won't be identified
as part of the blueprint itself.

Change-Id: Idba99f3f9b4abe77c042a6edaf9f5fe5c75ac32c
Implements: blueprint add-support-for-vgpu
2018-01-23 18:07:31 +00:00
int32bit
d03a890a34 Set server status to ERROR if rebuild failed
Currently there is no indication that the rebuild was refused,
and worse, we may have a wrong imageref for the instance.

This patch set the instance to ERROR status if rebuild failed in the
scheduling stage. The user can rebuild the instance with valid image
to get it out of ERROR state and reset with right instance metadata and
properties.

Closes-Bug: 1744325

Change-Id: Ibb7bee15a3d4ee6f0ef53ba12e8b41f65a1fe999
2018-01-23 11:39:45 -05:00
Kevin_Zheng
3f35fe6a88 Use neutron port_list when filtering instance by ip
The performance of filtering instance by IP is poor, due to the fact that
the IP address is part of a JSON-encoded filed and we need to unpack the
field and do a search on the field for each record in instances table, we
have to iterate one by one to find the instance that matches the request.

As discussed in Queens PTG[1], one possible solution is to get filtered
ports from Neutron and retrieve the instance uuid from the port.device_id
and then merge to the other filters.

As Nova provides regex matching manner filtering for IP filter, so this
depend on Neutron changes that add substring matching functionality to
the GET /ports API[2]

The performance test results of the POC are presented in[3].

[1]http://lists.openstack.org/pipermail/openstack-dev/2017-September/122258.html
[2]https://bugs.launchpad.net/neutron/+bug/1718605
[3]http://lists.openstack.org/pipermail/openstack-dev/2018-January/125990.html

Depends-On: I9549b2ba676e1bad0812682c3f3f3c97de15f5f6

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Implements: blueprint improve-filter-instances-by-ip-performance

Change-Id: I06aabfdce4aaefde080c7e122552ce4f36da5804
2018-01-23 10:10:53 +08:00
Zuul
66eb27fa7b Merge "Recreate mediated devices on reboot" 2018-01-22 23:42:06 +00:00
Zuul
7371238f13 Merge "[api] Allow multi-attach in compute api" 2018-01-22 22:22:00 +00:00
Zuul
4e3b07430d Merge "doc: Document TLS security setup for noVNC proxy" 2018-01-22 17:35:05 +00:00
Sylvain Bauza
26c8c6a86c Recreate mediated devices on reboot
There is a lacking feature in libvirt (due to the nature of what is mediated
device as just a sysfs file) that mediated devices are not persisted.
When you reboot your host, you loose all of them which can be a big pain for
operators having allocated guests using vGPUs.

This change will iterate over all the instances, see if they have a nested
vGPU, check if the related mediated device exists, and if not, rebalance
between physical GPUs to find a proper one which can fit.

Note that due to the fact we don't persist neither in Nova the mediated device
information, mediated devices can be created on different physical devices that
they were before the reboot. That's not a big deal since we only support one
type at the moment, but that could become a problem later as we would need to
figure out which type the mediated device was before the reboot.

Partially-Implements: blueprint add-support-for-vgpu
Change-Id: Ie6c9108808a461359d717d8a9e9399c8a407bfe9
2018-01-22 18:07:38 +01:00
Ildiko Vancsa
7e6ae9afd9 [api] Allow multi-attach in compute api
This change introduces a new microversion which must be used
to create a server from a multiattach volume or attach a multiattach
volume to an existing server instance.

Attaching a multiattach volume to a shelved offloaded instance is not
supported since an instance in that state does not have a compute host
so we can't tell if the compute would support the multiattach volume
or not. This is consistent with the tagged attach validation with 2.49.

When creating a server from a multiattach volume, we'll check to see
if all computes in all cells are upgraded to the point of even supporting
the compute side changes, otherwise the server create request fails with
a 409. We do this because we don't know which compute node the scheduler
will pick and we don't have any compute capability filtering in the
scheduler for multiattach volumes (that may be a future improvement).

Similarly, when attaching a multiattach volume to an existing instance,
if the compute isn't new enough to support multiattach or the virt
driver simply doesn't support the capability, a 409 response is returned.
Presumably, operators will use AZs/aggregates to organize which hosts
support multiattach if they have a mixed hypervisor deployment, or will
simply disable multiattach support via Cinder policy.

The unit tests are covering error conditions with the new flow. A new
functional scenario test is added for happy path testing of the new boot
from multiattach volume flow and attaching a multiattach volume to more
than one instance.

Tempest integration testing for multiattach is added in change
I80c20914c03d7371e798ca3567c37307a0d54aaa.

Devstack support for multiattach is added in change
I46b7eabf6a28f230666f6933a087f73cb4408348.

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Implements: blueprint multi-attach-volume
Change-Id: I02120ef8767c3f9c9497bff67101e57e204ed6f4
2018-01-22 10:45:13 -05:00
Daniel Berrange
7c593dc505 doc: Document TLS security setup for noVNC proxy
The nova noVNC proxy server has gained the ability to use the VeNCrypt
authentication scheme to secure network communications with the compute
node VNC servers.  This documents how to configure the QEMU/KVM compute
nodes and the noVNC proxy server nodes.

Change-Id: If3cea87568efff0874cd8851cabc6770812c545b
Blueprint: websocket-proxy-to-host-security
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
2018-01-22 14:48:51 +00:00
He Jie Xu
8a307bbdf4 placement: support traits in allocation candidates API
This patch add new query parameter `required` to the
`GET /allocation_candidates` API, which is used to filter candidates
with required traits.  The candidate attached traits return in the
provider summary also. Those API changes are added by new microversion.

Also using specific exception TraitNotFound instead of the generic
exception ValueError when invalid traits in the request.

Change-Id: Id821b5b2768dcc698695ba6570c6201e1e9a8233
Implement blueprint add-trait-support-in-allocation-candidates
2018-01-22 22:10:10 +08:00
Zuul
f1cc291776 Merge "libvirt: create vGPU for instance" 2018-01-19 14:37:41 +00:00
Zuul
0ff7af38ff Merge "conf: Remove 'vendordata_driver' opt" 2018-01-19 11:41:15 +00:00
Sylvain Bauza
8dc0f636f4 libvirt: create vGPU for instance
If an allocation is asking for a VGPU, then libvirt will look at the
available mediated devices and call the sysfs to create one of them if
needed.

Please note I commented in the relnote all the caveats we currently have
with mediated devices in libvirt, but I'll provide some workarounds for
those in the next changes.

Change-Id: Ibf210dd27972fed2651d6c9bd73a0bcf352c8bab
Partially-Implements: blueprint add-support-for-vgpu
2018-01-19 10:40:51 +01:00
Zuul
daeadc6869 Merge "Fix accumulated nits" 2018-01-17 05:15:55 +00:00
Zuul
73ea39dbce Merge "hyper-v: Deprecates support for Windows / Hyper-V Server 2012" 2018-01-17 05:15:20 +00:00
Zuul
f96e89cc51 Merge "conf: Do not inherit image signature props with snapshots" 2018-01-16 17:38:00 +00:00
Stephen Finucane
025d73dbf6 Fix accumulated nits
Fix nits from the following reviews:

- https://review.openstack.org/#/c/345397/35
- https://review.openstack.org/#/c/345398/36
- https://review.openstack.org/#/c/345399/42

Change-Id: Iafa5b4432a85353a216da4d55e46e1eda30842e3
Blueprint: websocket-proxy-to-host-security
2018-01-16 14:54:04 +00:00
Zuul
4258d743eb Merge "Do not attempt volume swap when guest is stopped/suspended" 2018-01-15 16:15:38 +00:00
Zuul
b07080a04c Merge "console: Provide an RFB security proxy implementation" 2018-01-15 14:15:18 +00:00
Zuul
4fe3f9b3d2 Merge "console: introduce the VeNCrypt RFB authentication scheme" 2018-01-15 14:10:02 +00:00
Zuul
459353eec2 Merge "console: introduce framework for RFB authentication" 2018-01-14 07:35:39 +00:00
Lee Yarwood
82c823bcec conf: Do not inherit image signature props with snapshots
All image signature properties should not be inherited from the metadata
of the original image when creating a snapshot of an instance. Otherwise
Glance will attempt to verify the signature of the snapshot image and
fail as this has changed from that of the original.

Closes-bug: #1737513
Change-Id: Ia3d80bf2f81c7317fec117aecbc3c560d51a7d4e
2018-01-13 11:33:36 +00:00
Zuul
a0e23fc273 Merge "Qualify the Placement 1.15 release note" 2018-01-13 00:54:33 +00:00
Zuul
30e71d608a Merge "Fix up formatting for deprecate-api-extensions-policies release note" 2018-01-13 00:54:23 +00:00
Zuul
2315d09965 Merge "Fix typo in release note" 2018-01-12 21:09:10 +00:00
Zuul
135a9d709d Merge "Add a warning in 'nova-manage cell_v2 delete_cell'" 2018-01-12 20:40:05 +00:00
Daniel P. Berrange
30ceaaff5d console: Provide an RFB security proxy implementation
Instead of doing straight passthrough of the RFB protocol from the
tenant sock to the compute socket, insert an RFB security proxy. This
will MITM the initial RFB protocol handshake in order to negotiate an
authentication scheme with the compute node that is distinct from that
used by the tenant.

Based on earlier work by Solly Ross <sross@redhat.com>

Change-Id: I9cc9a380500715e60bd05aa5c29ee46bc6f8d6c2
Co-authored-by: Stephen Finucane <sfinucan@redhat.com>
Implements: bp websocket-proxy-to-host-security
2018-01-12 10:00:42 +00:00
Stephen Finucane
c5a1a9e711 console: introduce the VeNCrypt RFB authentication scheme
Provide an implementation for the VeNCrypt RFB authentication
scheme, which uses TLS and x509 certificates to provide both
encryption and mutual authentication / authorization.

Based on earlier work by Solly Ross <sross@redhat.com>

Change-Id: I6a63d2535e86faf369ed1c0eeba6cb5a52252b80
Co-authored-by: Stephen Finucane <sfinucan@redhat.com>
Implements: bp websocket-proxy-to-host-security
2018-01-12 10:00:42 +00:00
Stephen Finucane
3c7770f1af console: introduce framework for RFB authentication
Introduce a framework for providing RFB authentication scheme
implementations. This will be later used by the websocket RFB security
proxy code.

Change-Id: I98403ca922b83a460a4e7baa12bd5f596a79c940
Co-authored-by: Stephen Finucane <sfinucan@redhat.com>
Implements: bp websocket-proxy-to-host-security
2018-01-12 10:00:42 +00:00