55402 Commits

Author SHA1 Message Date
Balazs Gibizer
167e593ed5 Functional reproduction for bug 1845291
Nova does not try to re-schedule a migration if the original boot was
done with --availability-zone <az:host> or <az::node>.

Related-Bug #1845291

Change-Id: Id78627c5c08090de6220249a5f44d26bf32724af
2019-09-26 15:58:39 +00:00
Zuul
38c774a8e0 Merge "Add librsvg2* to bindep" 2019-09-24 02:36:21 +00:00
Zuul
d1ff9bd66f Merge "Functional reproduction for bug 1844993" 2019-09-24 00:22:03 +00:00
Zuul
c246dd9010 Merge "Refactor pre-live-migration work out of _do_live_migration" 2019-09-23 20:22:04 +00:00
Zuul
c174dcabb6 Merge "Add reminder to update corresponding glance docs" 2019-09-23 19:58:54 +00:00
Zuul
f14ce63091 Merge "Tune up db.instance_get_all_uuids_by_hosts" 2019-09-23 19:58:43 +00:00
Zuul
0c8ca68818 Merge "Use multiple attachments in test_list_volume_attachments" 2019-09-23 19:43:40 +00:00
Zuul
f7badfae30 Merge "Remove stubs from VolumeAttachmentsSample API sample test" 2019-09-23 19:43:34 +00:00
Zuul
10e950b3b5 Merge "[Trivial]Removed unused helper _extract_query_params" 2019-09-23 19:31:49 +00:00
Zuul
6fe2373964 Merge "[Trivial]Remove unused helper should_switch_to_postcopy" 2019-09-23 19:31:43 +00:00
Zuul
31e9914ce5 Merge "Get pci_devices from _list_devices" 2019-09-23 19:24:41 +00:00
Zuul
597b34cd87 Merge "Add note about needing noVNC >= v1.1.0 with using ESX" 2019-09-23 17:24:00 +00:00
Balazs Gibizer
a3024b5cea Functional reproduction for bug 1844993
If the compute RPC is pinned to < 5.2 a server with QoS port cannot be
migrated. However nova not just fails the migration but also leaves the
server and the QoS port in an inconsistent state.

Change-Id: Ia8dc51d11b8ce93c372ee77f2c0b43910f992574
Related-Bug: #1844993
2019-09-23 14:33:23 +02:00
Zuul
5a1c2d4ffa Merge "Rename 'nova.common.config' module to 'nova.middleware'" 2019-09-23 12:25:59 +00:00
Matt Riedemann
4f261f98e1 Revert "Temporarily skip TestNovaMigrationsMySQL"
This reverts commit aebebe713a50d1057d5dfdfa4f13be6cc3ce625b.

We said we would since the skip was temporary.

Change-Id: I5f7f5779735e158688992ccc948e7305bae1c9c4
Related-Bug: #1823251
2019-09-21 09:46:34 -04:00
Zuul
09bf71407f Merge "trivial: Use sane indent" 2019-09-21 11:06:29 +00:00
Zuul
2c22a39bb9 Merge "libvirt: Get the CPU model, not 'arch' from get_capabilities()" 2019-09-21 05:15:28 +00:00
Zuul
1ec87c227c Merge "libvirt: Correctly handle non-CPU flag traits" 2019-09-21 03:40:40 +00:00
Zuul
764a60a1fe Merge "libvirt: Enable driver configuring PMEM namespaces" 2019-09-21 03:40:33 +00:00
Zuul
0ff3fd9bf4 Merge "Add reshaper for PCPU" 2019-09-21 02:07:35 +00:00
Zuul
47e7502f08 Merge "libvirt: Mock 'libvirt_utils.file_open' properly" 2019-09-21 02:07:27 +00:00
Zuul
28b80d975d Merge "fakelibvirt: Make 'Connection.getHostname' unique" 2019-09-21 02:07:17 +00:00
Zuul
a5a875f53b Merge "Add support for translating CPU policy extra specs, image meta" 2019-09-21 02:07:09 +00:00
Zuul
4e5889e0c2 Merge "Include both VCPU and PCPU in core quota count" 2019-09-21 01:53:00 +00:00
Zuul
f79c072a3e Merge "tests: Additional functional tests for pinned instances" 2019-09-21 01:41:47 +00:00
Zuul
337a35cef8 Merge "libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait" 2019-09-21 01:41:40 +00:00
ya.wang
af45b6e981 libvirt: Get the CPU model, not 'arch' from get_capabilities()
This fixes a regression introduced with change
Ib37ff6fc2e1f9de0e60adca54e87c74f45f12ffa.

Without this, setting [libvirt]/cpu_model_extra_flags
to some value results in libvirt failing to start with
an error like this:

  Configured extra flag: aes it not correct, or the host CPU
  does not support this flag. Please correct the config and
  try again. Migration pre-check error: CPU doesn't have
  compatibility.

Closes-Bug: 1844211
Change-Id: I9c7e842c4ef29cc78057a7ed9701f28ecb5e1a8b
2019-09-20 16:04:46 +00:00
LuyaoZhong
0e4ca43311 libvirt: Enable driver configuring PMEM namespaces
Add one configuration option CONF.libvirt.pmem_namespaces:
"$LABEL:$NSNAME[|$NSNAME][,$LABEL:$NSNAME[|$NSNAME]]"

e.g. "128G:ns0|ns1|ns2|ns3,262144MB:ns4|ns5,MEDIUM:ns6|ns7"

Change-Id: I98e5ddbd7a9f2211a16221b5049bc36452a49a75
Partially-Implements: blueprint virtual-persistent-memory
Co-Authored-By: He Jie Xu <hejie.xu@intel.com>
2019-09-19 23:15:39 +00:00
Zuul
9b2e00e015 Merge "hardware: Differentiate between shared and dedicated CPUs" 2019-09-19 20:15:34 +00:00
Zuul
a302b22062 Merge "objects: Add 'NUMACell.pcpuset' field" 2019-09-19 20:15:23 +00:00
Zuul
c0f5387621 Merge "Validate CPU config options against running instances" 2019-09-19 20:15:15 +00:00
Zuul
ed6d19e051 Merge "Trigger real BuildAbortException during migrate with bandwidth" 2019-09-19 18:35:24 +00:00
Zuul
fbc0c19677 Merge "make config drives sticky bug 1835822" 2019-09-19 13:12:41 +00:00
Zuul
0012fdfdc3 Merge "objects: Add 'InstanceNUMATopology.cpu_pinning' property" 2019-09-19 11:39:00 +00:00
Zuul
d9f3b3c193 Merge "libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available'" 2019-09-19 11:33:48 +00:00
Zuul
fb3ba44438 Merge "Temporarily skip TestNovaMigrationsMySQL" 2019-09-19 03:21:27 +00:00
Zuul
97dca8db76 Merge "libvirt: Start reporting PCPU inventory to placement" 2019-09-19 00:23:41 +00:00
Matt Riedemann
aebebe713a Temporarily skip TestNovaMigrationsMySQL
There are several changes approved for Train feature freeze
which are still not merged yet nearly a week later because
of gate failures with bug 1823251. This change skips those
tests temporarily so we can land some code in time for Train
RC1 but this change should be reverted prior to RC1 after
all of those other changes are merged. Note that those
changes (vpmems and pcpu series) don't really have anything
to do with data model schema changes at this point so skipping
this test for now won't drop any coverage for what is going in,
and we aren't merging any other data model schema changes at
this point before RC1. Again, this is a temporary measure and
should be reverted within a few days.

Change-Id: I0564c613f30d321de33a1bd71bfe48c37e72ff80
Related-Bug: #1823251
2019-09-18 19:55:14 -04:00
Matt Riedemann
9ae92e3f98 Add librsvg2* to bindep
I3aaea1d15a357f550f529beaa84fb1a1a7748358 added the docs
build requirement on sphinxcontrib-svg2pdfconverter which
needs the native rsvg-convert command. This change adds
the native package that provides that command to bindep.txt.

Change-Id: I064a1f33902405c3db699a46feeb93397fc3b038
Closes-Bug: #1844583
2019-09-18 17:25:49 -04:00
Matt Riedemann
0ff01646ac Refactor pre-live-migration work out of _do_live_migration
The _do_live_migration method has gotten too big so this
helps by refactoring the pre-live-migration portion into
a new private method with documentation. Note that because
of how _do_live_migration was shadowing and overwriting the
migrate_data input argument the new method has to pass that
back as well. Failing to do that causes integration tests
to fail because unit tests were insufficient, so this also
updates one unit test to make sure that works properly.

This is a small part of a bigger long-term effort to try
and cleanup these live migration compute manager methods
so they can be maintained and tested in isolation, and also
reason about error handling for each piece.

Change-Id: I220a4f9337d355a76596208cf7b96113dd34a8ea
2019-09-18 14:32:18 -04:00
Sean Mooney
641feb62a3 make config drives sticky bug 1835822
This change reorders the call in
_update_instance_after_spawn so that we call
configdrive.update_instance before we set
instance.launched_at. This ensures that if the vm
is booted with a config drive because the host had
force_config_drive=true the instance will keep its
config drive across reboots.

This change fixes a regression introduced as part
of fixing bug #1827492 which addressed failing
to boot vms after changing force_config_drive=false
to force_config_drive=true.

Change-Id: I9194423f5f95e9799bd891548e24756131d65e76
Related-Bug: #1827492
Closes-Bug: #1835822
2019-09-18 18:49:31 +01:00
melanie witt
2b23ee7a30 Add note about needing noVNC >= v1.1.0 with using ESX
As discussed on the following review:

  https://review.opendev.org/674916

this adds a note indicating that the version of noVNC needs to be at
least v1.1.0 in order for the nova-novncproxy to work with ESX/ESXi
hypervisors.

Related-Bug: #1822676

Change-Id: Ia4ba37b6d6a1e4b5c75e38f4bcc2bea1d9ba9560
2019-09-18 16:59:00 +00:00
Zuul
c67057dff3 Merge "Remove an unused file and a related description" 2019-09-18 16:00:17 +00:00
Balazs Gibizer
6648e344a9 Trigger real BuildAbortException during migrate with bandwidth
The functional test
test_migrate_server_with_qos_port_pci_update_fail_not_reschedule used
fault injection to simulate that the
_update_pci_request_spec_with_allocated_interface_name call raises
BuildAbortException during migrating a instance with an SRIOV port
having resource request.

We can do a bit better. This patch simulates that neutron names the PF
device RP in placement in an unexpected way. Therefore the test does not
need to mock anything in nova to simulate the same failure.

Change-Id: Ib72f15eb62fd50ea7117df577478f555a8478724
blueprint: support-move-ops-with-qos-ports
2019-09-18 13:46:32 +02:00
Stephen Finucane
c0a6e4be48 trivial: Use sane indent
(it won't break 'git-blame -M')

Change-Id: Ifa7a3d7142ffc7c953d06bb2c7191a4dacb53713
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-09-18 00:21:10 +01:00
Bhagyashri Shewale
242a894348 Add reshaper for PCPU
Added upgrade code using reshape to report PCPU instead of VCPU in case
the host is configured to use pinned CPUs. With this, when an existing
compute node running guests which uses dedicated CPUs is upgraded to
Train release, it will update allocation records of existing guest from
VCPU to PCPU using the reshape functionality.

Part of blueprint cpu-resources

Change-Id: I25d70aa09080b22d1bfa0aa097f0a114de8bf15a
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
2019-09-18 00:21:10 +01:00
Stephen Finucane
35fae09b22 libvirt: Mock 'libvirt_utils.file_open' properly
The current implementation was extremely fragile and fell over if you
attempted to do something like boot multiple servers at once or cold
migrate one when created.

Change-Id: I69a8be3047394fc892c3682d88ebb485b0c3e2c0
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-09-18 00:21:10 +01:00
Stephen Finucane
1e9596df08 fakelibvirt: Make 'Connection.getHostname' unique
This does not suffer from the lambda scoping issues of the previous
approach.

Change-Id: I7bf4d0e14024e608cea6e20d25a2bcc9ab8af7aa
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Co-authored-by: LuyaoZhong <luyao.zhong@intel.com>
2019-09-18 00:21:10 +01:00
Stephen Finucane
278ab01c32 Add support for translating CPU policy extra specs, image meta
Map 'hw:cpu_policy' and 'hw:cpu_thread_policy' as follows:

  hw:cpu_policy
    dedicated -> resources:PCPU=${flavor.vcpus}
    shared    -> resources:VCPU=${flavor.vcpus}

  hw:cpu_thread_policy
    isolate -> trait:HW_CPU_HYPERTHREADING:forbidden
    require -> trait:HW_CPU_HYPERTHREADING:required
    prefer  -> (none, handled later during scheduling)

Ditto for the 'hw_cpu_policy' and 'hw_cpu_thread_policy' image metadata
equivalents.

In addition, increment the requested 'resources:PCPU' by 1 if the
'hw:emulator_threads_policy' extra spec is present and set to 'isolate'.

The scheduler will attempt to get PCPUs allocations and fall back to
VCPUs if that fails. This is okay because the NUMA fitting code from the
'hardware' module used by both the 'NUMATopology' filter and libvirt
driver protects us. That code doesn't know anything about PCPUs or VCPUs
but rather cares about the 'NUMATopology.pcpuset' field, (starting in
change I492803eaacc34c69af073689f9159449557919db), which can be set to
different values depending on whether this is Train with new-style
config, Train with old-style config, or Stein:

- For Train compute nodes with new-style config, 'NUMATopology.pcpuset'
  will be explictly set to the value of '[compute] cpu_dedicated_set'
  or, if only '[compute] cpu_dedicated_set' is configured, 'None' (it's
  nullable) by the virt driver so the calls to
  'hardware.numa_fit_instance_to_host' in the 'NUMATopologyFilter' or
  virt driver will fail if it can't actually fit.

- For Train compute nodes with old-style config, 'NUMATopology.pcpuset'
  will be set to the same value as 'NUMATopology.cpuset' by the virt
  driver.

- For Stein compute nodes, 'NUMATopology.pcpuset' will be unset and
  we'll detect this in 'hardware.numa_fit_instance_to_host' and simply
  set it to the same value as 'NUMATopology.cpuset'.

Part of blueprint cpu-resources

Change-Id: Ie38aa625dff543b5980fd437ad2febeba3b50079
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-09-18 00:21:10 +01:00
He Jie Xu
d6c96436d6 Include both VCPU and PCPU in core quota count
This patch enables counting both VCPU and PCPU when
`CONF.quota.count_usage_from_placement` enabled. Also add functional
test ensure the quota usage as expected.

Change-Id: I8377cbd1fb601d91deb7bb667a1ad5271c667399
Part of blueprint cpu-resources
2019-09-18 00:21:10 +01:00