46082 Commits

Author SHA1 Message Date
Lee Yarwood
9c23cdc247 encryptors: Switch to os-brick encryptor classes
This change drops the encryptor classes and supporting code from the
codebase in favor of the classes provided by os-brick. This is made
possible by the following os-brick change that introduced new encryption
provider constants during Ocata :

Ic155bd29d46059832cce970bf60375e7e472eca6

Thanks to the following bugfix also released as part of 1.11.0 for Ocata
the constants present in os-brick also support the use of the deprecated
legacy class paths from Nova, for example
nova.volume.encryptors.luks.LuksEncryptor, while using the os-brick
provided classes :

I3ec6e3fe919bc03d158da04a18fb8b651002ed52

Implements: blueprint switch-to-os-brick-encryptor-classes
Change-Id: I37ffc90c0bd57029fced251b5cfd7cd4318a0292
Depends-On: Iae12605dc7d0607e78020a24b5b8801606c2f169
2017-04-25 13:51:41 +00:00
Jenkins
3d09b67205 Merge "Adding auto_disk_config field to InstancePayload" 2017-04-25 13:05:41 +00:00
Jenkins
23923134c1 Merge "libvirt: drop MIN_LIBVIRT_HUGEPAGE_VERSION" 2017-04-25 13:04:01 +00:00
Jenkins
5822a29b87 Merge "libvirt: drop MIN_LIBVIRT_NUMA_VERSION" 2017-04-25 13:03:34 +00:00
Jenkins
8c9291b458 Merge "libvirt: drop MIN_QEMU_NUMA_HUGEPAGE_VERSION" 2017-04-25 13:02:56 +00:00
Jenkins
96977e1124 Merge "Remove BuildRequest.block_device_mapping clone workaround" 2017-04-25 10:00:39 +00:00
Jenkins
658ee87995 Merge "fix typos" 2017-04-25 01:05:05 +00:00
Dan Smith
32ecd3b50a Sort CellMappingList.get_all() for safety
The result of this method should be stable in database order, but since
we depend on it for some pagination, this patch sorts the results by
the most efficient thing we have to be sure.

Related to blueprint cells-aware-api

Change-Id: I81fb8a2a342ad54adc386c02a5fe4a90800fcbb0
2017-04-24 12:00:22 -07:00
Dan Smith
ef1c539ad1 Add workaround to disable group policy check upcall
Right now, the compute node is responsible for validating that the scheduler
honored the affinity policy of a server group. It does this by listing all
the instances in a server group and validating that affinity or anti-affinity
was kept true. With cellsv2, this check requires an upcall to the API
database as server groups are stored there. This violates our goal of not
allowing any upcalls to be congruent with known restrictions that actual
cells(v1) deployers require.

This adds a workaround flag, disabled by default, which defeats this check
in the compute node. For smaller deployments, isolation of the cell and api
services is not as big of a deal, so by default, this check will continue
to work as it does today. Larger deployments which care more about the
isolation than they do the absolute affinity guarantees can enable this
workaround to avoid the upcall check. A user can still detect a violation of
their affinity request by examining the obscured host identifier.

Longer-term, this problem goes away when we're doing claims in the scheduler
and placement has a notion of affinity that it can enforce early in the
boot process, eliminating the race and the need for this check entirely.

Related to blueprint cells-aware-api

Change-Id: I22e1a0736a269d89b55b71c2413fa763d5f1fd0c
2017-04-24 12:00:22 -07:00
Dan Smith
1226c57884 Make server groups api aware of multiple cells for membership
This teaches the server groups api to query all of the cells that have
an instance claiming to be a member of the group. Instead of polling all
cells, this collects the subset of cells that actually have members,
according to our InstanceMapping data, and then checks for the deleted-ness
in the actual cell database.

This augments the tests to have one instance in each cell, plus an instance
that is not yet scheduled in any cell to give us coverage of all the
possibilities.

This replaces the previously-flawed and now reverted change:

  If571f9e7c9d0ef6265a249ff808d30a24ab462af

Change-Id: Idd2e35bc95ed98ebc0340ff62e109e23c8adcb21
2017-04-24 11:59:54 -07:00
Jenkins
c3597c87d8 Merge "add tags field to instance.update notification" 2017-04-24 16:49:26 +00:00
Matt Riedemann
14960061cc Remove BuildRequest.block_device_mapping clone workaround
This was originally added in change
I60abcd4f27dc877c4e420071be77c9fdb697ad99 because we still
created the BDMs in the API, and we did it based on the
build_request.block_device_mapping set in the _provision_instances
method.

We had to workaround the fact that BuildRequest.create()
would serialize the BDMs to store them in the DB, and then
deserialize them to set them on the BuildRequest object field.
As a result, the BuildRequest.block_device_mapping field was
orphaned (it lost it's internal RequestContext). So later on
when creating the BDMs from BuildRequest.block_device_mapping,
the update_or_create() remotable call would fail since the
object didn't have a context set. To workaround that we made a
clone of the BlockDeviceMappingList when creating the BuildRequest
and then set the original "dirty" list on the BuildRequest object
which was used later to create the BDMs.

In change I8742071b55f018f864f5a382de20075a5b444a79 we stopped
creating the BDMs in the API service and started created them
in the conductor service when calling schedule_and_build_instances.
And the block_device_mapping list passed to that conductor
method is the same one that we hacked into the BuildRequest object
in the earlier change (for the same reason really).

So at this point, we no longer need this confusing workaround and
can remove it.

Change-Id: I88e4d5cf38ee9ab4be9b8d7ae48bf273478fe3ef
2017-04-24 12:46:18 -04:00
Jenkins
0039231719 Merge "Remove archaic reference to QEMU errors during post live migration" 2017-04-24 09:49:05 +00:00
OpenStack Proposal Bot
1ad41c0b0c Updated from global requirements
Change-Id: Iae12605dc7d0607e78020a24b5b8801606c2f169
2017-04-23 20:48:37 +00:00
Matt Riedemann
b6996365d3 Remove archaic reference to QEMU errors during post live migration
The compute manager should be virt-driver agnostic for the
most part. This reference to errors in post-live migration
for QEMU predates gerrit and goes back to a diablo commit
with no further explanation of what's going on with this
change, but it looks like it's what added live migration
way back when:

1518e66d0c4a4a72b6893cb117648f81dd3e3aff

It doesn't even make sense now, so remove it.

Change-Id: I144711f9e000c7e723f3b4e3dce56e3e922cf6e4
2017-04-21 14:55:26 -04:00
Jenkins
893ee87d3c Merge "Tell people that the nova-cells man page is for cells v1" 2017-04-21 17:57:51 +00:00
Jenkins
60ad364963 Merge "Add release note and update cell install guide for multi-cell limitations" 2017-04-21 17:57:06 +00:00
Jenkins
4a25af8237 Merge "PowerVM Driver: power_on/off and reboot" 2017-04-21 17:43:44 +00:00
Matt Riedemann
a2fd38f45a Tell people that the nova-cells man page is for cells v1
We have to continue scrubbing our documentation for anything
that generically refers to 'cells' since older documentation is
referring to cells v1, and we don't want people to confuse that
with cells v2. This simply provides a warning that cells v1 is
not recommended for new deployments and if you were looking for
help with cells v2 commands, we need to redirect you.

Change-Id: I3bd0332607bff169b5f496d17e4fc7d372ea991b
2017-04-21 10:11:39 -07:00
Matt Riedemann
9a5c3cd7da Add release note and update cell install guide for multi-cell limitations
As of change If1e03c9343b8cc9c34bd51c2b4d25acdb21131ff, using
"nova hypervisor-list" before compute hosts are mapped to a cell
will result in an empty list.

Our cells v2 install steps mention using 'nova hypervisor-list' after
creating a cell and starting compute services to tell when to run
the discover_hosts command, but now hypervisor-list won't work until
you've run discover_hosts, so it's a catch-22.

This change adds a release note to let people writing deployment tools
to know about the change in behavior and also updates the install steps
to use service-list instead of hypervisor-list, since service-list does
not require the compute host to be mapped to the cell first.

We are going to need to make a similar change in the OpenStack install
guide since that also mentions using 'nova hypervisor-list' before
discover_hosts.

Change-Id: If2baab40c2e2a3de20e561bba50688d615b002ef
Closes-Bug: #1682060
2017-04-21 10:11:03 -07:00
Jenkins
fe8dbdeae7 Merge "Using max api version in notification sample test" 2017-04-21 15:26:58 +00:00
Jenkins
ddc807f85a Merge "remove log translation tags from nova.cells" 2017-04-21 15:24:59 +00:00
Jenkins
60996ee9e2 Merge "Update detach to use V3 Cinder API" 2017-04-21 15:24:13 +00:00
Jenkins
f817b5feca Merge "Add description to policies in security_groups.py" 2017-04-21 12:44:38 +00:00
Sujitha
bb7d5214a4 Adding auto_disk_config field to InstancePayload
Add auto_disk_config field to InstancePayload based on
instance.auto_disk_config boolean value. If true, set it to
AUTO else MANUAL.

Implements: bp additional-notification-fields-for-searchlight
Change-Id: I18af99479562e2fe5e74e6e1252b804b074fee58
2017-04-21 12:45:39 +02:00
Sujitha
eb1ac77dd7 add tags field to instance.update notification
Tags field is added to the InstanceUpdatePayload based on the
tags field fo the related Instance object. Adding tags to instance.create
notification will be handled in the bp tag-instances-when-boot.

Implements: bp additional-notification-fields-for-searchlight

Change-Id: Ifad6487de200767fc16d035ad830c77bba49154a
2017-04-21 12:44:12 +02:00
Kevin_Zheng
be7f40c92b Add description to policies in hypervisors.py
blueprint policy-docs

Change-Id: Ie27ba40d8d5cfd811dc5a82f01fe9fa1e49e4ed6
Co-authored-by: Stephen Finucane <sfinucan@redhat.com>
2017-04-21 06:58:14 +00:00
Eric Fried
29ef20bf8b PowerVM Driver: power_on/off and reboot
This change set introduces PowerVMDriver implementations for power_on,
power_off, and reboot.

Change-Id: I39233602c3485d2a93dd59fadeacd7795c8ab9bc
Partially-Implements: blueprint powervm-nova-compute-driver
2017-04-21 00:09:16 +00:00
Jenkins
3cc34c2aa3 Merge "PowerVM Driver: spawn/destroy #2: functional" 2017-04-20 23:40:45 +00:00
Béla Vancsics
c2d38a50ba Using max api version in notification sample test
Notification payloads always reflect the data needed for every
supported API microversion so we can safely use the latest
API version in the notification sample tests. This helps the test
to use the new API features too.

Change-Id: I851bd2e7ef13a1efa4fed4176996d71f6e2de94a
2017-04-20 18:50:22 -04:00
Eric Fried
b2ba4c86f4 PowerVM Driver: spawn/destroy #2: functional
Building on Ic45bb064f4315ea9e63698a7c0e541c5b0de5051, this change set
makes the spawn and destroy methods functional in a basic way.  A
subsequent change will introduce TaskFlow framework.

The VMs still have no network or storage - those will be coming in
future change sets.

Change-Id: I85f740999b8d085e803a39c35cc1897c0fb063ad
Partially-Implements: blueprint powervm-nova-compute-driver
2017-04-20 16:04:20 -05:00
Jenkins
1cbe4d4560 Merge "Fix mitaka online migration for PCI devices" 2017-04-20 18:49:48 +00:00
Jenkins
ba9a42e1ac Merge "PowerVM Driver: spawn/delete #1: no-ops" 2017-04-20 16:10:22 +00:00
john.griffith8@gmail.com
39e2760f35 Update detach to use V3 Cinder API
Use the new Cinder V3 attachment delete method
during detach if the BDM has an attachment_id in it.
This will only be present in the BDM if/when the
new attachment_create API is called. Otherwise, we
revert to the old calls.

Edge cases are handled in separate patches.

Partially Implements: blueprint cinder-new-attach-apis

Co-Authored-By: Steve Noyes <steve.noyes@oracle.com>

Change-Id: I91b9a60268354ffbed86b1e7d173906cfd7b97bd
2017-04-20 11:09:44 -04:00
Jenkins
4ab4d44939 Merge "Mock timeout in test__get_node_console_with_reset_wait_timeout" 2017-04-20 14:22:51 +00:00
Jenkins
559eb958f5 Merge "Document and provide useful error message for volume-backed backup" 2017-04-20 12:47:22 +00:00
Jenkins
d807979dcc Merge "api-ref: Fix response code and parameters in evacuate" 2017-04-20 11:51:31 +00:00
OpenStack Proposal Bot
9569bc3552 Updated from global requirements
Change-Id: I1e19585ad7011b8de04a3cd9d6d37158102d604e
2017-04-19 17:54:40 +00:00
melanie witt
5f2b187ae4 Mock timeout in test__get_node_console_with_reset_wait_timeout
This test is failing intermittently because it's waiting for a real
timeout from BackOffLoopingCall which does an exponential backoff
each interval. Sometimes the test times out entirely and fails.

This changes the test to mock BackOffLoopingCall and raise the timeout
exception to simulate a timeout. The only thing not covered by this is
calling of the _wait_state() function, but that is covered by other
unit tests in the file.

Closes-Bug: #1683953

Change-Id: Ie3ff6b0020dca8a5f5292b327aa2491920e338ef
2017-04-19 16:30:51 +00:00
Jenkins
d62245dadd Merge "Fix docstring in _validate_requested_port_ids" 2017-04-19 16:14:12 +00:00
fuzk
aadb8bd664 fix typos
Change-Id: I780269ec20600dd07f03dbb1d7f1289fe68daf3e
2017-04-19 03:36:04 -07:00
Matt Riedemann
75a7e6fc7d Remove unused os-pci API
The os-pci API was never part of the v2.0 API and was added
to the v3 API, but when the v3 API turned into the v2.1 API
which is backward compatible with the v2.0 API, the os-pci
API was removed from v2.1. The original intent was to enable
it in a microversion but that never happened.

We should just delete this API since it has a number of issues
anyway:

1. It's not documented (which makes sense since it's not enabled).
2. The PciHypervisorController just takes the compute_nodes.pci_stats
   dict and dumps it to json out of the REST API with no control over
   the keys in the response. That means if we ever change the fields
   in the PciDevicePool object, we implicitly introduce a backward
   incompatible change in the REST API.
3. We don't want to be reporting host stats out of the API [1].
4. To make the os-hypervisors extension work in a multi-cell environment
   we'd have to add uuids to the PciDevices model and change the API to
   return and take in uuids to identify the devices for GET requests.
5. And last but not least, no one has asked for this in over two years.

As a result of removing this API we can also remove the join on the
pci_devices table when showing details about an instance or listing
instances, which were added years ago because of the PciServerController:

Id3c8a0b187e399ce2acecd4aaa37ac95e731d46c

Id3e60c3c56c2eb4209e8aca8a2c26881ca86b435

[1] https://docs.openstack.org/developer/nova/policies.html?#metrics-gathering

Closes-Bug: #1426241
Closes-Bug: #1673869

Change-Id: I9099744264eeec175672d10d04da69648dec1a9d
2017-04-18 23:47:40 -04:00
Steven Webster
7f3f0ef1fb Fix mitaka online migration for PCI devices
Currently, a validation error is thrown if we find any PCI device
records which have not populated the parent_addr column on a nova
upgrade. However, the only PCI records for which a parent_addr
makes sense for are those with a device type of 'type-VF' (ie. an
SRIOV virtual function).  PCI records with a device type of 'type-PF'
or 'type-PCI' will not have a parent_addr.  If any of those records
are present on upgrade, the validation will fail.

This change checks that the device type of the PCI record is
'type-VF' when making sure the parent_addr has been correctly
populated

Closes-Bug: #1680918
Change-Id: Ia7e773674a4976fc03deee3f08a6ddb45568ec11
2017-04-18 17:44:36 -04:00
Matt Riedemann
4341d140f9 Fix docstring in _validate_requested_port_ids
We return a two-item tuple from this method, not three.

Change-Id: I963f39e8fd7b0a7980d31609350a0b63b9ef3b4a
2017-04-18 14:43:28 -04:00
Jenkins
4b37c38eaa Merge "Fix the evacuate API without json-schema validation in 2.13" 2017-04-18 16:23:02 +00:00
Jenkins
d9cdbf8e06 Merge "Remove json-schema extension variable for resize" 2017-04-18 16:20:33 +00:00
Jenkins
7e6c905d7d Merge "Remove usage of parameter enforce_type" 2017-04-18 12:59:33 +00:00
Jenkins
8ddf5134fc Merge "Remove test_init_nonexist_schedulerdriver" 2017-04-18 12:57:38 +00:00
Jenkins
ef2203f36f Merge "Expand help text for [libvirt]/disk_cachemodes" 2017-04-18 12:57:09 +00:00
Jenkins
a41136dfee Merge "Deprecate CONF.api.allow_instance_snapshots" 2017-04-18 12:54:52 +00:00