openstack/nova
Code Issues Proposed changes

update_provider_tree devref and docstring updates

Browse Source 
Design changes [1] from the Dublin PTG prompted some rewording of the
docstring for ComputeDriver.update_provider_tree.  And to avoid that
docstring becoming too enormous, relevant chunks of the spec [2] are
copied to a new devref document which is linked from that docstring.

[1] https://review.openstack.org/#/c/552122/
[2] http://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/update-provider-tree.html

Change-Id: I06504aa2a3fe6d39ecc1e681de43be8fee9e06f6
blueprint: update-provider-tree
This commit is contained in:
Eric Fried 2018-03-15 11:40:09 -05:00
parent c5fd4c7612
commit f9e13bcfed
4 changed files with 202 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/source/index.rst
View File

@ -266,6 +266,7 @@ looking parts of our architecture. These are collected below.
reference/services
reference/stable-api
reference/threading
reference/update-provider-tree
reference/vm-states
user/index
user/aggregates

2
doc/source/reference/index.rst
View File

@ -22,6 +22,8 @@ The following is a dive into some of the internals in nova.
based on eventlet, and may not be familiar to everyone.
* :doc:`/reference/notifications`: How the notifications subsystem works in
nova, and considerations when adding notifications.
* :doc:`/reference/update-provider-tree`: A detailed explanation of the
``ComputeDriver.update_provider_tree`` method.
Debugging
=========

172
doc/source/reference/update-provider-tree.rst Normal file
View File

@ -0,0 +1,172 @@
..
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
====================================
ComputeDriver.update_provider_tree
====================================
This provides details on the ``ComputeDriver`` abstract method
``update_provider_tree`` for developers implementing this method in their own
virt drivers.
Background
----------
In the movement towards using placement for scheduling and resource management,
the virt driver method ``get_available_resource`` was initially superseded by
``get_inventory``, whereby the driver could specify its inventory in terms
understood by placement. In Queens, a ``get_traits`` driver method was added.
But ``get_inventory`` is limited to expressing only inventory (not traits or
aggregates). And both of these methods are limited to the resource provider
corresponding to the compute node.
Recent developments such as Nested Resource Providers necessitate the ability
for the virt driver to have deeper control over what the resource tracker
configures in placement on behalf of the compute node. This need is filled by
the virt driver method ``update_provider_tree`` and its consumption by the
resource tracker, allowing full control over the placement representation of
the compute node and its associated providers.
The Method
----------
``update_provider_tree`` accepts two parameters:
* A ``nova.compute.provider_tree.ProviderTree`` object representing all the
providers in the tree associated with the compute node, and any sharing
providers (those with the ``MISC_SHARES_VIA_AGGREGATE`` trait) associated via
aggregate with any of those providers (but not *their* tree- or
aggregate-associated providers), as currently known by placement. This
object is fully owned by the ``update_provider_tree`` method, and can
therefore be modified without locking/concurrency considerations. In other
words, the parameter is passed *by reference* with the expectation that the
virt driver will modify the object. Note, however, that it may contain
providers not directly owned/controlled by the compute host. Care must be
taken not to remove or modify such providers inadvertently. In addition,
providers may be associated with traits and/or aggregates maintained by
outside agents. The ``update_provider_tree`` method must therefore also be
careful only to add/remove traits/aggregates it explicitly controls.
* String name of the compute node (i.e. ``ComputeNode.hypervisor_hostname``)
for which the caller is requesting updated provider information. Drivers may
use this to help identify the compute node provider in the ProviderTree.
Drivers managing more than one node (e.g. ironic) may also use it as a cue to
indicate which node is being processed by the caller.
The virt driver is expected to update the ProviderTree object with current
resource provider and inventory information. When the method returns, the
ProviderTree should represent the correct hierarchy of nested resource
providers associated with this compute node, as well as the inventory,
aggregates, and traits associated with those resource providers.
.. note:: Despite the name, a ProviderTree instance may in fact contain more
than one tree. For purposes of this specification, the ProviderTree
passed to ``update_provider_tree`` will contain:
* the entire tree associated with the compute node; and
* any sharing providers (those with the ``MISC_SHARES_VIA_AGGREGATE``
trait) which are associated via aggregate with any of the providers
in the compute node's tree. The sharing providers will be
presented as lone roots in the ProviderTree, even if they happen to
be part of a tree themselves.
Consider the example below. ``SSP`` is a shared storage provider and
``BW1`` and ``BW2`` are shared bandwidth providers; all three have
the ``MISC_SHARES_VIA_AGGREGATE`` trait::
CN1 SHR_ROOT CN2
/ \ agg1 / /\ agg1 / \
NUMA1 NUMA2--------SSP--/--\-----------NUMA1 NUMA2
/ \ / \ / \ / \ / \
PF1 PF2 PF3 PF4--------BW1 BW2------PF1 PF2 PF3 PF4
agg2 agg3
When ``update_provider_tree`` is invoked for ``CN1``, it is passed a
ProviderTree containing::
CN1 (root)
/ \ agg1
NUMA1 NUMA2-------SSP (root)
/ \ / \
PF1 PF2 PF3 PF4------BW1 (root)
agg2
This method supersedes ``get_inventory`` and ``get_traits``: if this method is
implemented, neither ``get_inventory`` nor ``get_traits`` is used.
Driver implementations of ``update_provider_tree`` are expected to use public
``ProviderTree`` methods to effect changes to the provider tree passed in.
Some of the methods which may be useful are as follows:
* ``new_root``: Add a new root provider to the tree.
* ``new_child``: Add a new child under an existing provider.
* ``data``: Access information (name, UUID, parent, inventory, traits,
aggregates) about a provider in the tree.
* ``remove``: Remove a provider **and its descendants** from the tree. Use
caution in multiple-ownership scenarios.
* ``update_inventory``: Set the inventory for a provider.
* ``add_traits``, ``remove_traits``: Set/unset virt-owned traits for a
provider.
* ``add_aggregates``, ``remove_aggregates``: Set/unset virt-owned aggregate
associations for a provider.
.. note:: There is no supported mechanism for ``update_provider_tree`` to
effect changes to allocations. This is intentional: in Nova,
allocations are managed exclusively outside of virt. (Usually by the
scheduler; sometimes - e.g. for migrations - by the conductor.)
Porting from get_inventory
~~~~~~~~~~~~~~~~~~~~~~~~~~
Virt driver developers wishing to move from ``get_inventory`` to
``update_provider_tree`` should use the ``ProviderTree.update_inventory``
method, specifying the compute node as the provider and the same inventory as
returned by ``get_inventory``. For example:
.. code::
def get_inventory(self, nodename):
inv_data = {
'VCPU': { ... },
'MEMORY_MB': { ... },
'DISK_GB': { ... },
}
return inv_data
would become:
.. code::
def update_provider_tree(self, provider_tree, nodename):
inv_data = {
'VCPU': { ... },
'MEMORY_MB': { ... },
'DISK_GB': { ... },
}
provider_tree.update_inventory(nodename, inv_data)
Porting from get_traits
~~~~~~~~~~~~~~~~~~~~~~~
To replace ``get_traits``, developers should use the
``ProviderTree.add_traits`` method, specifying the compute node as the
provider and the same traits as returned by ``get_traits``. For example:
.. code::
def get_traits(self, nodename):
traits = ['HW_CPU_X86_AVX', 'HW_CPU_X86_AVX2', 'CUSTOM_GOLD']
return traits
would become:
.. code::
def update_provider_tree(self, provider_tree, nodename):
provider_tree.add_traits(
nodename, 'HW_CPU_X86_AVX', 'HW_CPU_X86_AVX2', 'CUSTOM_GOLD')

42
nova/virt/driver.py
View File

@ -845,23 +845,35 @@ class ComputeDriver(object):
:note: Renaming a provider (by deleting it from provider_tree and
re-adding it with a different name) is not supported at this time.
See the developer reference documentation for more details:
https://docs.openstack.org/nova/latest/reference/update-provider-tree.html # noqa
:param nova.compute.provider_tree.ProviderTree provider_tree:
A ProviderTree object representing all the providers associated
with the compute node, and any sharing providers (those with the
``MISC_SHARES_VIA_AGGREGATE`` trait) associated via aggregate with
any of those providers (but not *their* tree- or aggregate-
associated providers), as currently known by placement. This
object is fully owned by the ``update_provider_tree`` method, and
can therefore be modified without locking/concurrency
considerations. Note, however, that it may contain providers not
directly owned/controlled by the compute host. Care must be taken
not to remove or modify such providers inadvertently.
A nova.compute.provider_tree.ProviderTree object representing all
the providers in the tree associated with the compute node, and any
sharing providers (those with the ``MISC_SHARES_VIA_AGGREGATE``
trait) associated via aggregate with any of those providers (but
not *their* tree- or aggregate-associated providers), as currently
known by placement. This object is fully owned by the
update_provider_tree method, and can therefore be modified without
locking/concurrency considerations. In other words, the parameter
is passed *by reference* with the expectation that the virt driver
will modify the object. Note, however, that it may contain
providers not directly owned/controlled by the compute host. Care
must be taken not to remove or modify such providers inadvertently.
In addition, providers may be associated with traits and/or
aggregates maintained by outside agents. The
`update_provider_tree`` method must therefore also be careful only
to add/remove traits/aggregates it explicitly controls.
:param nodename:
Name of the compute node for which the caller is updating providers
and inventory. Drivers managing more than one node may use this in
an advisory capacity to restrict changes to only the providers
associated with that one node, but this is not a requirement: the
caller always subsumes all changes regardless.
String name of the compute node (i.e.
ComputeNode.hypervisor_hostname) for which the caller is requesting
updated provider information. Drivers may use this to help identify
the compute node provider in the ProviderTree. Drivers managing
more than one node (e.g. ironic) may also use it as a cue to
indicate which node is being processed by the caller.
:return: True if the provider_tree was changed; False otherwise.
"""
raise NotImplementedError()