Commit Graph

53784 Commits (19.0.0)

Author SHA1 Message Date
Zuul d1f37ff804 Merge "Libvirt: gracefully handle non-nic VFs" into stable/stein 2019-04-05 01:01:13 +00:00
Matt Riedemann afe5f0d4de Remove CellMappingPayload database_connection and transport_url fields
Change I019e88fabd1d386c0d6395a7b1969315873485fd in Stein, which
is not yet officially released, exposes the unencrypted
database_connection URL and MQ transport_url to a CellMapping in
the select_destinations versioned notification CellMappingPayload.

While notifications are not meant to be consumed by end users of
the cloud but only internal services of the deployment, it still
seems like a bad idea to give the keys to the nova cell DB and MQ
to an external-to-nova service like ceilometer.

This change removes the fields from the CellMappingPayload and
bumps the major version to 2.0 to signal the change to consumers,
although I don't expect anything is consuming this yet but we should
follow standard versioning procedure anyway.

Note that notification consumers do not request a specific payload
version nor do they get a schema to perform their own backporting,
they just get what they get, so after this there should be no worry
about needing to support the 1.0 format for this payload.

Change-Id: Ib5edea32d15db01000e6730aebceaf119daf8c5c
Closes-Bug: #1823104
(cherry picked from commit  3301449e73)
2019-04-04 21:25:52 +00:00
Sean Mooney 97a806b790 Libvirt: gracefully handle non-nic VFs
As part of adding support for bandwidth based scheduling
I038867c4094d79ae4a20615ab9c9f9e38fcc2e0a introduced
automatic discovery of parent netdev names for PCIe
virtual functions.

Nova's PCI passthrough support was originally developed for
Intel QAT devices and other generic PCI devices. Later support
for Neutron based SR-IOV NIC was added.

The PCI-SIG SR-IOV specification while most often used by NIC
vendors to virtualise a NIC in hardware was designed for devices
of any PCIe class. Support for Intel's QAT device and other
accelerators like AMD's SRIOV based vGPU have therefore been
regressed by the introduction of the new parent_ifname lookup code.

This change simply catches the exception that would be raised
when pci_utils.get_ifname_by_pci_address is called on generic
VFs allowing a graceful fallback to the previous behaviour.

Change-Id: Ib3811f828246311d90b0e3ba71c162c03fb8fe5a
Closes-Bug: #1821938
(cherry picked from commit e7ae6c65cd)
2019-04-03 21:08:37 +00:00
Sylvain Bauza 0fd77f3f65 Add doc on VGPU allocs and inventories for nrp
Now that we reshape inventories and allocations for VGPU resources in Stein,
we think it's good for operators to know how to verify the resources using

Change-Id: Ic58709aac2dd1f20f1b8440a3cea4f29eed9a965
Closes-Bug: #1821015
(cherry picked from commit 44490142be)
2019-04-02 21:12:03 +00:00
Zuul e4857b9857 Merge "Update UPPER_CONSTRAINTS_FILE for stable/stein" into stable/stein 2019-03-26 20:38:17 +00:00
Ian Wienand a40f0c5a00 Replace git:// URLs with https://
This is a mechanically generated change to replace
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (,
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

Change-Id: Ib663b6034647ef3b2e5d622856573946aee8a27b
2019-03-24 20:34:47 +00:00
OpenStack Release Bot 36687ad567 Update UPPER_CONSTRAINTS_FILE for stable/stein
Update the URL to the upper-constraints file to point to the redirect
rule on so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.

Until the requirements repository has as stable/stein branch, tests will
continue to use the upper-constraints list on master.

Change-Id: Ie1c82d871b4dd31be431e5013950a9f929b89c27
2019-03-22 03:50:05 +00:00
OpenStack Release Bot ba01967bcf Update .gitreview for stable/stein
Change-Id: Ib2c6b8e2989983cc0b93797e418e3cf3f1c25449
2019-03-22 03:50:01 +00:00
Zuul c993d4fe2f Merge "Require python-ironicclient>=2.7.0" 2019-03-22 02:55:13 +00:00
Zuul c3386126e0 Merge "Move libvirt calculation of machine type to" 2019-03-21 18:29:02 +00:00
Zuul c9977f2aed Merge "Fix links to neutron QoS minimum bandwidth doc" 2019-03-21 16:41:35 +00:00
Zuul aa1bfb645f Merge "Add a prelude release note for the 19.0.0 Stein GA" 2019-03-21 15:59:33 +00:00
Zuul acbbc3eb8b Merge "Documentation for bandwidth support" 2019-03-21 15:59:23 +00:00
Balazs Gibizer c4295f87ad Fix links to neutron QoS minimum bandwidth doc
The QoS minimum bandwidth feature will have a separate doc from the
generic QoS neutron doc. This patch updates the links in the release
notes and api version history of the 2.72 microversion

blueprint: bandwidth-resource-provider

Change-Id: Ic753112cf73cb10a6e377bc24c6ee51a057c69f8
2019-03-21 11:47:19 +01:00
Chris Dent 0dfbcd7464 Don't register placement opts mutiple times in a test
The test_local_delete_removes_allocations_after_compute_restart test
was trying to register placement config opts 3 times when only once
is necessary, and if there are CLI opts being registered, only once is
allowed. With change I4cd3d637878eb5bb798b78fd73f5be99e141da9d in
placement, those opts gained some CLI opts, causing this test to

The depends-on is to a change in the placement-side PlacementFixture
to make it possible to not register opts when calling the fixture,
allowing the safe reuse of the already registered config.

Depends-On: I360a306b5d05ada75274733038b73ec2f2bdc4d4
Change-Id: I042e41ac8c41c0e5f0389904eb548e0e97d54c60
Closes-Bug: #1821092
2019-03-20 22:33:32 +00:00
Zuul 59f1f187e5 Merge "docs: Misc cleanups" 2019-03-20 18:30:46 +00:00
Zuul 45d66abe6d Merge "Add known issue for minimum bandwidth resource leak" 2019-03-20 17:16:51 +00:00
melanie witt 70989c3eb5 Add known issue for minimum bandwidth resource leak
Nova will leak minimum bandwidth resources in placement if a user
deletes a bound port from Neutron out-of-band. This adds a note about
how users can work around the issue.

Related-Bug: #1820588

Change-Id: I41f42c1a7595d9e6a73d1261bf1ac1d47ddadcdf
2019-03-20 10:59:13 -04:00
melanie witt 028418788b Add a prelude release note for the 19.0.0 Stein GA

Change-Id: I9ef2a5e597345e3919cbd553178ba5c978f52984
2019-03-20 09:51:33 -05:00
Zuul 592658aafc Merge "Move slight bonkers IP management to privsep." 2019-03-20 13:21:29 +00:00
Stephen Finucane ebbd84fbe8 docs: Misc cleanups
Some random cleanups:

- Don't add the root or 'doc/source' directories to PYTHONPATH - it's
  unnecessary since we install nova (ruling out the first) and don't
  import anything from the latter
- Fix weird indentation
- Remove 'sphinx.ext.coverage', which is used to measure API doc
  coverage. This is unnecessary since we don't publish API docs, save
  for the versioned notification docs
- Remove unnecessary settings
  - 'exclude_patterns' referred to directories that haven't existed for
    a long time
  - 'source_suffix', 'add_module_names' and 'show_authors' were set to
    the default value
  - 'release', 'version' and 'html_last_updated_fmt' are all set
    automatically by 'openstackdoctheme' now
  - 'modindex_common_prefix' is useless since we don't expose a module

All rolled into one patch for efficiencies sake.

Change-Id: I0f70c6d71299dedc59884f2bb39c8ea3c2ca8eff
Signed-off-by: Stephen Finucane <>
2019-03-20 09:42:03 +00:00
Zuul 4f5c8d3f83 Merge "Add docs for compute capabilities as traits" 2019-03-20 05:09:12 +00:00
Zuul b2a149d95e Merge "Remove "Fixing the Scheduler DB model" from schedule evolution doc" 2019-03-20 03:56:22 +00:00
Adam Spiers d2f8995103 Move libvirt calculation of machine type to
The libvirt driver contains some code to calculate the
default machine type given an architecture, by looking it up
in CONF.libvirt.hw_machine_type.

This code will need to be reused when introducing calls to
libvirt's getDomainCapabilities() API, which requires the
machine type as one of the parameters.  However those calls
will need to be made from which
has no access to the driver, so move the machine type
calculation code into nova.virt.libvirt.utils so that it can
be reused by both classes.

Also add some unit tests, and warn when an invalid config
value is used.

blueprint: amd-sev-libvirt-support
Change-Id: I055918ff16766c5b106d794a111ad8af8ff9ab23
2019-03-19 22:28:33 +00:00
Zuul 05a01a14b7 Merge "Use assertXmlEqual() helper for all XML comparison tests" 2019-03-19 20:23:04 +00:00
Zuul a295324876 Merge "Clarify policy shortcomings in policy enforcement doc" 2019-03-19 18:14:23 +00:00
Zuul b459c58a5b Merge "Remove additional policy configuration details from policy doc" 2019-03-19 18:14:17 +00:00
Zuul 56947658b0 Merge "Remove unnecessary default provider_tree when getting traits" 2019-03-19 15:59:36 +00:00
Matt Riedemann c16c3062e7 Add docs for compute capabilities as traits
Change I15364d37fb7426f4eec00ca4eaf99bec50e964b6 added the
ability for the compute service to report a subset of driver
capabilities as standard COMPUTE_* traits on the compute node
resource provider.

This adds administrator documentation to the scheduler docs
about the feature and how it could be used with flavors. There
are also some rules and semantic behavior around how these traits
work so that is also documented.

Note that for cases #3 and #4 in the "Rules" section the
update_available_resource periodic task in the compute service
may add the compute-owned traits again automatically but it
depends on the [compute]/resource_provider_association_refresh
configuration option, which if set to 0 will disable that auto
refresh and a restart or SIGHUP is required. To avoid confusion
in these docs, I have opted to omit the mention of that option
and just document the action that will work regardless of
configuration which is to restart or SIGHUP the compute service.

Change-Id: Iaeec92e0b25956b0d95754ce85c68c2d82c4a7f1
2019-03-19 10:09:55 -04:00
Zuul f58f73978e Merge "Remove stale aggregates notes from scheduler evolution doc" 2019-03-19 07:56:25 +00:00
Zuul 0962bb3b2f Merge "qemu: Make disk image conversion dramatically faster" 2019-03-19 00:14:47 +00:00
Lance Bragstad c8b02af65a Clarify policy shortcomings in policy enforcement doc
This commit updates the list of issues with policy enforcement and
describe some of the benefits for operators and developers if we fix
these issues.

Change-Id: Ie5ba2375fd32611aca360765af01c1ba6432b45e
2019-03-18 23:50:01 +00:00
Lance Bragstad 5d38069f66 Remove additional policy configuration details from policy doc
This is removing additional details that were originally reviewed in:


The policy and docs in code work that nova completed in Newton.

Change-Id: I66105fa90036db50249b62fc34442b667a5ee1db
2019-03-18 23:49:33 +00:00
Adam Spiers c29f7026ca Remove unnecessary default provider_tree when getting traits
I15364d37fb7426f4eec00ca4eaf99bec50e964b6 introduced a new
_get_traits() method in ResourceTracker for getting traits from a
provider tree and then merging with capability traits from the driver.
However it included a default of None for the provider_tree parameter
which was a remnant of earlier iterations of that change.  Since this
method is always passed a ProviderTree, remove the superfluous default
of None.

Change-Id: I1868485912d9a8a330bde50836808accf04c728d
2019-03-18 20:16:00 +00:00
Kashyap Chamarthy e7b64eaad8 qemu: Make disk image conversion dramatically faster
tl;dr: Use 'writeback' instead of 'writethrough' as the cache mode of
the target image for `qemu-img convert`.  Two reasons: (a) if the image
conversion completes succesfully, then 'writeback' calls fsync() to
safely write data to the physical disk; and (b) 'writeback' makes the
image conversion a _lot_ faster.

Back-of-the-envelope "benchmark" (on an SSD)

(Ran both the tests thrice each; version: qemu-img-2.11.0)

With 'writethrough':

    $> time (qemu-img convert -t writethrough -f qcow2 -O raw \
            Fedora-Cloud-Base-29.qcow2 Fedora-Cloud-Base-29.raw)
    real    1m43.470s
    user    0m8.310s
    sys     0m3.661s

With 'writeback':

    $> time (qemu-img convert -t writeback  -f qcow2 -O raw \
            Fedora-Cloud-Base-29.qcow2 5-Fedora-Cloud-Base-29.raw)

    real    0m7.390s
    user    0m5.179s
    sys     0m1.780s

I.e. ~103 seconds of elapsed wall-clock time for 'writethrough' vs. ~7
seconds for 'writeback' -- IOW, 'writeback' is nearly _15_ times faster!


Nova commit e6ce9557f8 ("qemu-img do not
use cache=none if no O_DIRECT support") was introduced to make instances
boot on filesystems that don't support 'O_DIRECT' (which bypasses the
host page cache and flushes data directly to the disk), such as 'tmpfs'.
In doing so it introduced the 'writethrough' cache for the target image
for `qemu-img convert`.

This patch proposes to change that to 'writeback'.

Let's addresses the 'safety' concern:

  "What about data integrity in the event of a host crash (especially
   on shared file systems such as NFS)?"

Answer: If the host crashes mid-way during image conversion, then
neither "data integrity" nor the cache mode in use matters.  But if the
image conversion completes _succesfully_, then 'writeback' will safely
write the data to the physical disk, just as 'writethough' does.

So we are as safe as we can, but with the extra benefit of image
conversion being _much_ faster.

        * * *

The `qemu-img convert` command defaults to 'cache=writeback' for the
source image.  And 'cache=unsafe' for the target, because if `qemu-img`
"crashes during the conversion, the user will throw away the broken
output file anyway and start over"[1].  And `qemu-img convert`
supports[2] fsync() for the target image since QEMU 1.1 (2012).

    -- "qemu-img convert: Use cache=unsafe for output image"
    -- "qemu-img: let 'qemu-img convert' flush data"

Closes-Bug: #1818847

Change-Id: I574be2b629aaff23556e25f8db0d740105be6f07
Signed-off-by: Kashyap Chamarthy <>
Looks-good-to-me'd-by: Kevin Wolf <>
2019-03-18 14:53:39 -05:00
Lance Bragstad 0f1b2e3a63 Remove obsolete policy configuration details from docs
The policy-enforcement document was written prior to any of the
policy-in-code or policy documentation efforts took place. This
commit updates the developer reference for policy to remove these
details since they have already been implemented.

Subsequent patches will update details of this document by taking into
account the recent keystone and oslo changes that help fix the
original issues described in this document.

Change-Id: I263b2f72037a588623958baccacf78fb6a6be05d
2019-03-18 13:53:14 +00:00
Zuul 926e584136 Merge "Pass kwargs to exception to get better format of error message" 2019-03-18 10:57:11 +00:00
Balazs Gibizer 47287f6f94 Documentation for bandwidth support
blueprint: bandwidth-resource-provider

Change-Id: I166787e092e16857d20f89fba965be2a5509fb4b
2019-03-18 11:24:56 +01:00
Michael Still 4b46c3ba88 Move slight bonkers IP management to privsep.
Change-Id: Ifdbee5c9f84211314d57e31ab84419987fec4737
2019-03-17 23:19:08 +00:00
Zuul be8af28a4f Merge "Trivial typo fix for REST API in policy enforcement docs" 2019-03-15 17:48:01 +00:00
Zuul ca39bdd17e Merge "Add descriptions of numbered resource classes and traits" 2019-03-15 17:47:53 +00:00
Zuul e272b280f1 Merge "Remove resize caveat from conductor docs" 2019-03-15 17:47:44 +00:00
Zuul d1797d8f58 Merge "docs: cleanup driver parity scope section" 2019-03-15 17:47:36 +00:00
Matt Riedemann 0c72e63948 Remove "Fixing the Scheduler DB model" from schedule evolution doc
Blueprint detach-service-from-computenode in Kilo decoupled the
compute node and services concepts so this section is no longer
relevant and can be removed from the doc - it's no longer evolving.

Change-Id: Ibba2aa83b0afe2be05415b69a1ff8ae86866b860
Related-Bug: #1820283
2019-03-15 11:44:06 -04:00
Matt Riedemann 18c40cacc1 Remove stale aggregates notes from scheduler evolution doc
Since I901184cb1a4b6eb0d6fa6363bc6ffbcaa0c9d21d in Kilo the
aggregates information about a HostState object (which is a
wrapper over a ComputeNode) is cached in the scheduler, so the
comments in the scheduler evolution doc about not accessing the
aggregates table in the DB from filters/weighers and such is
extremely out of date and should just be removed.

Change-Id: Ibcbad227813d3b37b4e314eddbf3bae6e85652ea
Related-Bug: #1820283
2019-03-15 11:39:33 -04:00
Zuul d40125ef7f Merge "add python 3.7 unit test job" 2019-03-15 14:15:41 +00:00
Zuul c8f7246343 Merge "Avoid crashing while getting libvirt capabilities with unknown arch names" 2019-03-15 13:13:56 +00:00
Matt Riedemann 0a44d3ae0a Trivial typo fix for REST API in policy enforcement docs
Change-Id: If17a910f8a891ce93491d931c95f65d9fd9529e5
2019-03-15 08:33:12 -04:00
Matt Riedemann 1308d644bb Remove resize caveat from conductor docs
This document was written back in the liberty release [1]
and says that conductor is not used for orchestrating the
resize/migrate flow, but given the description of how
conductor is used to orchestrate scheduling and reschedules
during a server create, it is unclear why the doc says that
resize is not used the same way since it is used for rescheduling
when prep_resize fails in a selected dest compute. This removes
the caveat to reflect reality.

[1] Ieb9134302d21a11fe9b9ee876bb7b0dd32b437e1

Change-Id: I932a7ac6870a3f9d26556c23c9074115963b3c27
2019-03-15 08:02:52 -04:00
Matt Riedemann 5de08c0966 docs: cleanup driver parity scope section
This fixes some grammar issues, links to the interop
page and fixes a misuse of tenant.

Change-Id: I3ce0e130e3691240a625c67dfb6123bafe7f48b8
2019-03-15 08:01:19 -04:00