19c0689fa3
This is generated using: $ tox -e move-implemented-specs -- pike -v Change-Id: I4f3e159430dbe7146e98f93645d8f6be06acb097
7.0 KiB
Allow custom resource classes in flavor extra specs
https://blueprints.launchpad.net/nova/+spec/custom-resource-classes-in-flavors
This spec proposes the ability to create flavors that can select custom resource classes at scheduling time, and override standard resource classes to avoid scheduling them, while maintaining a useful display of the flavor.
Problem description
The background to this problem is well described in the custom resource classes1 spec.
With that spec implemented, we now are able to represent these units of consumable resources, but not schedule to them. The scheduler currently only requests units of VCPU, MEMORY_MB, and DISK_GB. It needs to be able to request custom resource classes as well, to be able to properly filter these resources.
Since the end goal is to be able to schedule only based on the custom resource class, but flavor display and users rely on VCPU, MEMORY_MB, and DISK_GB, we will also need to add overrides for these resource classes. These overrides will allow dashboards, CLIs, etc. to be able to display the resources available for these flavors, without requiring the scheduler to filter on them.
Use Cases
As a deployer of ironic, I wish to be able to create flavors which request a specific class of indivisible baremetal machines. I also want my users to be able to see what sort of CPU/RAM/disk resources are available on my baremetal flavors, without scheduling on that data.
As an NFV deployer, I have hardware that has FPGA devices. I have tagged that hardware in the placement engine with custom resource classes based on the software loaded into the FPGA. I would like to allow my users to select a flavor that targets their instance to a machine with an FPGA that has particular software loaded.
Proposed change
We propose to add handling for a number of flavor extra specs to solve this:
resources:$CUSTOM_RESOURCE_CLASS=$N, where $N is a positive integer value. This will be added to the request made by theFilterSchedulerto the placement engine, to filter scheduling candidates by the custom resource class if present.resources:$STANDARD_RESOURCE_CLASS=0, where $STANDARD_RESOURCE_CLASS is one of "VCPU", "MEMORY_MB", or "DISK_GB". This causes the scheduler to avoid scheduling based on the associated value in the top level Flavor object field, while still using it for flavor display.
When the scheduler is constructing the request to placement, it will start by building a list of resource classes from the base flavor attributes as it does today. Next it will look for any overrides and add classes and/or adjust amounts of existing classes based on the extra_specs it finds.
Existing ironic instances will need to go through a migration (described in the "Data model impact" section) in order to correct allocations if the node has a resource class defined.
Alternatives
We could instead add new field(s) to the Flavor object that are specifically meant for custom resource classes and overrides of standard resource classes. While this does have the benefit of avoiding an off chance that some operator is already using extra specs like this, there's a couple of reasons not to do this. First, it creates an extra cycle before we can require ironic operators to transition their flavors, because the new field wouldn't be available until Pike. Second, as I understand things, we want to avoid adding to flavors right now, as it makes it more work to get rid of them altogether in the long term.
Data model impact
Existing ironic instances won't have the custom resource class
recorded in their instance.flavor attribute. As such,
allocations won't be recorded for this resource class, and the node the
instance exists on will be schedulable with new-style flavors.
For example, let's say an existing ironic instance has a
resource_class of CUSTOM_BAREMETAL_GOLD which is accounted
for in the baremetal service, but is not reported as an allocation
against the resource provider in the placement service. If an operator
configures a flavor's extra_specs and sets
resources:CUSTOM_BAREMETAL_GOLD="1", the scheduler may pick
a node to host the instance that placement thinks is available, as there
are no allocations against it, but the resource is actually already
consumed. This would result in a build failure.
To resolve this, we need to do two things:
- We'll need to check each existing instance at Pike startup, in the
driver layer. If the ironic node that the instance exists on has a
resource class defined, we add the custom resource class to
instance.flavor. - When updating instance allocations in the
update_available_resourceperiodic task in the nova-compute service, the scheduler report client will need to check for custom resource classes on theinstance.flavorand report those into the placement service.
REST API impact
None.
Security impact
None.
Notifications impact
None.
Other end user impact
None.
Performance Impact
Due to the migrations, there will be a small performance penalty to start nova-compute hosts running the ironic driver for the first time in Pike.
Other deployer impact
Deployers will need to adjust their flavors to use custom resource classes, before upgrading to Queens.
Warning
Deployers should not adjust their flavors until the data migration described in the Data model impact section above is complete.
Developer impact
None.
Implementation
Assignee(s)
- Primary assignee:
-
Ed Leafe (edleafe)
- Other contributors:
-
Jay Pipes (jaypipes)
Work Items
- Add code to migrate ironic instance flavor data.
- Add code to report custom resource class allocations.
- Add support for custom resource classes in the scheduler request.
- Add overrides for standard resource classes (the deployer does this).
Dependencies
This change depends on the resource tracker reporting custom resource class inventory, which is tracked in the "Custom Resource Classes (Pike)" blueprint.2
Testing
New style flavors will be added to setup for the job that runs ironic and nova with resource classes on the nodes.
Documentation Impact
These extra specs should be documented in the Install Guide, and also in the Upgrades guide.
References
History
| Release Name | Description |
|---|---|
| Pike | Introduced |