This spec aims to extend the placement ``GET /allocation_candidates`` API to include all resource class inventories of entire tree in ``provider_summaries`` field in the response body. Change-Id: I7992e16dd63cc2c5d22c3c3ce58adae9ad741569 Blueprint: placement-return-all-resources
7.3 KiB
Nested Resource Providers - Allocation Candidates
https://blueprints.launchpad.net/nova/+spec/nested-resource-providers-allocation-candidates
Nested resource providers functionality was added to nova and
placement in the Queens and Pike releases, however there remain a numer
of pieces of functionality that need to be completed for the feature to
be wholly useful. One of these pieces is the full integration of nested
resource providers in the allocation requests returned by the
GET /allocation_candidates
HTTP endpoint in the placement
service.
Problem description
With the update_provider_tree work, the resource tracker and virt drivers are now capable of constructing a tree of nested resource providers, each decorated with various traits and containing inventory records of various resource classes.
While this is critical to properly represent the hierarchical relationship of child providers to parent providers, this hierarchical structure is still not used in calculating the set of allocation requests returned to nova scheduler during pre-filtering compute hosts.
The placement service needs to consider a provider's membership within a tree in determining whether a provider can supply the resources in a particular request.
Consider the following arrangement, with a compute node that has some VCPU and MEMORY_MB inventory as well as two SR-IOV physical functions with inventory of SR-IOV virtual functions. Only one of those physical functions is decorated with a trait representing different a type of offload:
compute node -- VCPU: 16, MEMORY_MB: 16384
/ \
/ \
NUMA NODE 0 NUMA NODE 1
| |
| |
SRIOV_NET_VF:4 -- PF 0 PF 1 -- SRIOV_NET_VF:4
HW_NIC_OFFLOAD_GRO
If we request 2 VCPU
and 1 SRIOV_NET_VF
resource, we would expect to get back 2 allocation requests, with each
allocation request containing the compute node resource provider for the
VCPU
resources and one of the physical function resource
providers for the SRIOV_NET_VF
resource.
However, currently, we will get 0 results back from
GET /allocation_candidates
. Similarly, if we request 2
VCPU
and 1 SRIOV_NET_VF
along with
HW_NIC_OFFLOAD_GRO
as a required trait, we would expect to
get back 1 allocation request containing the PF which has the required
trait. However, we currently get 0 results since the calculation of
allocation candidates is not "nested aware".
Use Cases
None
Proposed change
Update the AllocationCandidates.get_by_requests()
method
so that it understands that when provider trees are present, a resource
may be provided by a child provider. When evaluating required traits,
the placement service must ensure that traits are associated with the
providers that are actually supplying the inventory to meet the
allocation request.
Namely, with the host described in Problem description,
GET /allocation_candidates?resources=VCPU:2,SRIOV_NET_VF=1
would provide the following result:
{"allocation_requests": [
{"allocations": {
"35791f28-fb45-4717-9ea9-435b3ef7c3b3": {
"resources": {
"VCPU": 2
,
},
}"7d2590ae-fb85-4080-9306-058b4c915e3f": {
"resources": {
"SRIOV_NET_VF": 1
,
},
},
},
}
{"allocations": {
"35791f28-fb45-4717-9ea9-435b3ef7c3b3": {
"resources": {
"VCPU": 2
,
},
}"99c09379-6e52-4ef8-9a95-b9ce6f68452e": {
"resources": {
"SRIOV_NET_VF": 1
,
},
},
},
},
]"provider_summaries": {
"35791f28-fb45-4717-9ea9-435b3ef7c3b3": {
"resources": {
"VCPU": {
"used": 0,
"capacity": 16
,
}"traits": []
,
},
}"7d2590ae-fb85-4080-9306-058b4c915e3f": {
"resources": {
"SRIOV_NET_VF": {
"used": 0,
"capacity": 4
,
}"traits": []
,
},
}"99c09379-6e52-4ef8-9a95-b9ce6f68452e": {
"resources": {
"SRIOV_NET_VF": {
"used": 0,
"capacity": 4
,
},
}"traits": [
"HW_NIC_OFFLOAD_GRO"
],
},
} }
Note that in provider_summaries
we will show only
resource providers that are in allocation_requests
.
We'd also like to adapt the placement service to return all providers
in a particular provider tree that match the search criteria in
provider_summaries
. This is sequentially enabled by another
spec, Return
resources of entire trees in Placement.
Alternatives
None
Data model impact
None
REST API impact
Introduce a new microversion to signal that provider trees are now
properly handled in the return of
GET /allocation_candidates
.
Security impact
None
Notifications impact
None
Other end user impact
None
Performance Impact
It's going to be slower to evaluate allocation candidates when provider trees are present. There's just no way around this.
Other deployer impact
None
Developer impact
None
Upgrade impact
None
Implementation
Assignee(s)
- Primary assignee:
-
jaypipes
Work Items
- Add a function to retrieve provider trees that meet a set of requested resource amounts and required traits
- Integrate the function into the
AllocationCandidates.get_by_requests()
method and add a microversion to signal the new behavior
Dependencies
None
Testing
Lots of functional tests required for various levels of nesting
Documentation Impact
Would be good to be super-clear about the behaviour of the placement service when evaluating a request for resources and traits when nested resource providers are present in the system.