Return Selection Objects
In Queens, we are going to be returning a lot more information from the Scheduler to the Conductor. If we return it as a series of nested dictionaries and lists, it will be fragile and confusing. It would be far better to create an object to hold this data and make it accessible in a simpler and documented way. Blueprint: return-selection-objects Change-Id: I7e94525ba9ee22885ea18a752a416024c36b4a40
This commit is contained in:
Ed Leafe
232
specs/queens/approved/return-selection-objects.rst
Normal file
232
specs/queens/approved/return-selection-objects.rst
Normal file
@@ -0,0 +1,232 @@
|
|||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License.
|
||||||
|
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
========================
|
||||||
|
Return Selection Objects
|
||||||
|
========================
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/nova/+spec/return-selection-objects
|
||||||
|
|
||||||
|
In Queens, we will be changing what we return from select_destinations() in
|
||||||
|
order to both provide additional 'alternate' hosts for each requested instance
|
||||||
|
and also the allocation_request for building on each host. Returning this as an
|
||||||
|
unstructured chunk of data will be fragile and potentially confusing. It would
|
||||||
|
be far better to create an object to hold this data and make it accessible in a
|
||||||
|
simpler and documented way.
|
||||||
|
|
||||||
|
Problem description
|
||||||
|
===================
|
||||||
|
|
||||||
|
Before Queens, the scheduler's select_destinations() method returned a list,
|
||||||
|
containing a dictionary representing the selected host for each requested
|
||||||
|
instance. In Queens, we need to return much more information to the caller of
|
||||||
|
select_destinations(). We could attempt to return a list of HostDicts, which
|
||||||
|
represents the selected host as in the past, along with zero or more
|
||||||
|
'alternate' hosts that are in the same cell and also meet the requested
|
||||||
|
resources. Additionally, each of these will also be accompanied by a dictionary
|
||||||
|
for the allocation_request required to claim that host. The end result will be
|
||||||
|
a list, with one item per requested instance. Each item in that list will be a
|
||||||
|
list of 2-tuples of (HostState, allocation_request). The HostState is a simple
|
||||||
|
dict, but the allocation_request is itself a complex nested dict.
|
||||||
|
|
||||||
|
The result of these changes would mean that the data returned would be a
|
||||||
|
complex nested combination of dictionaries, lists, and tuples. This data
|
||||||
|
structure would be both difficult to understand how to use correctly, and
|
||||||
|
confusing to developers looking at the code for the first time (or even after a
|
||||||
|
period of being away from it). It is also unversioned, meaning it is impossible
|
||||||
|
to track and respond to future changes in a reliable manner.
|
||||||
|
|
||||||
|
Use Cases
|
||||||
|
---------
|
||||||
|
|
||||||
|
As an experienced Nova developer, I would like to be able to write code that
|
||||||
|
uses the information returned from select_destinations() without having to
|
||||||
|
decipher a complex data structure.
|
||||||
|
|
||||||
|
As a newcomer to the Nova codebase, I would like to be able to read code that
|
||||||
|
is clear so that I can work with it quicker and with more confidence that my
|
||||||
|
changes won't break something.
|
||||||
|
|
||||||
|
Proposed change
|
||||||
|
===============
|
||||||
|
|
||||||
|
We propose to create a new Selection object that would contain the data that
|
||||||
|
represents a single destination: both the host information as well as the
|
||||||
|
corresponding allocation_request needed for claiming. The host information,
|
||||||
|
which is currently in a dictionary containing hostname, nodename, and limits
|
||||||
|
keys, will now be stored as simple object fields along with the
|
||||||
|
allocation_request. Additionally, the compute_node_uuid field will be added, as
|
||||||
|
it would be useful to have this available in some of our allocation cleanup
|
||||||
|
tasks.
|
||||||
|
|
||||||
|
There is no need for a corresponding SelectionList object, as there is no need
|
||||||
|
for DB creation or retrieval. The select_destinations() method will return
|
||||||
|
simple Python lists of Selection objects. The Scheduler will return one list
|
||||||
|
of Selection objects for each requested instance, representing the selected
|
||||||
|
host as well as any alternates.
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
|
||||||
|
We could cache the allocation_request data in placement, and simply return a
|
||||||
|
key along with the resource providers. When a claim needs to be made, the key
|
||||||
|
would be POSTed instead of the full allocation_request data, and Placement
|
||||||
|
would use the cached data to carry out the claim. This has the advantage that
|
||||||
|
nothing on the Nova side of things ever uses the data in the
|
||||||
|
allocation_request; to Nova, it's an opaque blob. The downside, of course, is
|
||||||
|
that Placement would have to handle the cache.
|
||||||
|
|
||||||
|
We could return the full allocation_request data to the scheduler, and then
|
||||||
|
handle the caching and key management on the Nova side. When a claim/unclaim is
|
||||||
|
needed, the allocation_request would be retrieved from this cache and POSTed to
|
||||||
|
placement. This alternative doesn't require any changes to placement, but
|
||||||
|
requires that both the API-level cell and all local cells have access to some
|
||||||
|
form of 'global ram' cache that is accessible across cells.
|
||||||
|
|
||||||
|
We could just return an unstructured bunch of Python data, and add a ton of
|
||||||
|
comments everywhere it is used in the hope that anyone looking at the code
|
||||||
|
would understand what each bit represents, and that every future change to the
|
||||||
|
data required would be able to be handled without versioning.
|
||||||
|
|
||||||
|
Data model impact
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
There will be no changes to any database schemas, but this will introduce a new
|
||||||
|
versioned object. This object will contain the following fields, along with
|
||||||
|
their types::
|
||||||
|
|
||||||
|
* compute_node_uuid: fields.UUIDField
|
||||||
|
* service_host: fields.StringField
|
||||||
|
* nodename: fields.StringField
|
||||||
|
* cell_uuid: fields.UUIDField
|
||||||
|
* numa_limits: fields.ObjectField("NUMATopologyLimits")
|
||||||
|
* allocation_request: fields.StringField
|
||||||
|
|
||||||
|
There isn't a good field type for the allocation_request value, as it is a
|
||||||
|
complex nested structure, so instead we'll store it as its JSON respresentation
|
||||||
|
in a StringField. The structure of an allocation_request, as described in this
|
||||||
|
spec[2], looks like::
|
||||||
|
|
||||||
|
"allocations": [
|
||||||
|
{
|
||||||
|
"resource_provider": {
|
||||||
|
"uuid": $COMPUTE_NODE1_UUID
|
||||||
|
},
|
||||||
|
"resources": {
|
||||||
|
"VCPU": $AMOUNT_REQUESTED_VCPU,
|
||||||
|
"MEMORY_MB": $AMOUNT_REQUESTED_MEMORY_MB
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"resource_provider": {
|
||||||
|
"uuid": $SHARED_STORAGE_UUID
|
||||||
|
},
|
||||||
|
"resources": {
|
||||||
|
"DISK_GB": $AMOUNT_REQUESTED_DISK_GB
|
||||||
|
}
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
REST API impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Security impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Notifications impact
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Other end user impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Other deployer impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Developer impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
It will make life a little easier for anyone working with the Nova codebase by
|
||||||
|
not making them decipher complex data structures, but other than that, none.
|
||||||
|
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
ed-leafe
|
||||||
|
|
||||||
|
Other contributors:
|
||||||
|
None
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
* Create the Selection object.
|
||||||
|
|
||||||
|
* Modify the scheduler's select_destinations() method to populate these objects
|
||||||
|
with the selected host info and return them.
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
This is one part of the overall sweeping changes being made in Queens, and all
|
||||||
|
of it will have to be tested. The Selection object will need some basic tests,
|
||||||
|
but the bulk of the testing will be in the conductor to verify that it is
|
||||||
|
working with Selection objects for host selection, resource claiming, and
|
||||||
|
retries on build failures.
|
||||||
|
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
The developer reference docs will need to be updated to document this new
|
||||||
|
object. The docs for the scheduler workflow will also need to be updated to
|
||||||
|
reflect these changes.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
The initial problem was documented in a blog post[0], and was then discussed at
|
||||||
|
the Nova Scheduler subteam meeting[1], where this approach was agreed upon.
|
||||||
|
|
||||||
|
[0] https://blog.leafe.com/handling-unstructured-data/
|
||||||
|
[1] http://eavesdrop.openstack.org/meetings/nova_scheduler/2017/nova_scheduler.2017-08-28-14.00.log.html#l-140
|
||||||
|
[2] https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/placement-allocation-requests.html
|
||||||
|
|
||||||
|
History
|
||||||
|
=======
|
||||||
|
|
||||||
|
.. list-table:: Revisions
|
||||||
|
:header-rows: 1
|
||||||
|
|
||||||
|
* - Release Name
|
||||||
|
- Description
|
||||||
|
* - Queens
|
||||||
|
- Introduced
|
||||||
Reference in New Issue
Block a user