Nova scheduler driver for constraints-based scheduling
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
OpenDev Sysadmins cad03aefda OpenDev Migration Patch 1 month ago
doc/source Fixed some typos 4 years ago
nova_solverscheduler Move out TenantRackConstraint and minor cleanups 3 years ago
.coveragerc Change ignore-errors to ignore_errors 3 years ago
.gitignore Added some additional missing files 4 years ago
.gitreview OpenDev Migration Patch 1 month ago
.mailmap Added some additional missing files 4 years ago
.testr.conf Fix coverage testing settings 3 years ago
CONTRIBUTING.rst Added required files for gate testing,doc,etc 4 years ago
HACKING.rst Fixed some typos 4 years ago
LICENSE Added LICENSE 5 years ago
MANIFEST.in Added required files for gate testing,doc,etc 4 years ago
README.rst Fix formatting problem in README 3 years ago
babel.cfg Added required files for gate testing,doc,etc 4 years ago
openstack-common.conf Added required files for gate testing,doc,etc 4 years ago
requirements.txt Update to latest status of codes 4 years ago
setup.cfg Added required files for gate testing,doc,etc 4 years ago
setup.py Added required files for gate testing,doc,etc 4 years ago
solver_scheduler.rst Move solver-scheduler's top directory from 'nova' to 'nova_solverscheduler' 4 years ago
test-requirements.txt Minimum necessary upgrades for solver scheduler to pass tests in kilo 3 years ago
tox.ini Fix coverage testing settings 3 years ago

README.rst

OpenStack Nova Solver Scheduler

A new OpenStack Nova scheduler driver based on constraints-based optimization solvers.

Overview

Solver Scheduler is an OpenStack Nova scheduler driver that provides smart, efficient, and optimization based compute resource scheduling in Nova. It is a pluggable scheduler driver, that can leverage existing constraint solvers available in open source such as PULP, CVXOPT, Google OR-TOOLS, etc. It can be easily extended to add complex constraint models for various use cases, and to solve complex scheduling problems with pulggable solving frameworks.

From filter-scheduler to solver-scheduler

The nova-solver-scheduler works in a similar (but different) way as Nova's default filter-scheduler. It is designed to have the following 3-layer pluggable architecture, compared with filter-scheduler's 2-layer architecture.

  • Filter-scheduler architecture
    • Scheduler driver: FilterScheduler. This is a driver that realizes the filter based scheduling functionalities. It is plugged into Nova scheduler service, and configured by the second layer plug-in's, which are known as weighers and filters.
    • Configurable plug-ins: Weights/Filters. They are the configuration units that defines the behaviour of the filter-scheduler, where filters decide which hosts are available for user requested instance, and weights are then used to sort filtered hosts to find a best one to place the instance.
  • Solver-scheduler architecture
    • Scheduler driver: SolverScheduler. This is a driver that realizes constraint optimization based scheduling functionalities. It sits in parellel with FilterScheduler and can be used as an (advanced) alternative. It uses pluggable opeimization solver modules to solve scheduling problems.
    • Solver drivers. The solver drivers are pluggavle optimization solvers that solve scheduling problems defined by its lower layer plug-in's, which are costs/constraints. The pluggable solvers provide more flexibility for complex scheduling scenarios as well as the ability to give more optimimal scheduling solutions.
    • Configurable plug-ins: Costs/Constraints. Similar as weights/filters in the filter-scheduler. The constraints define the hard restrictions of hosts that cannot be violated, and the costs define the soft objectives that the scheduler should tend to achieve. Scheduler solver will give an overall optimized solution for the placement of all requested instances in each single instance creation request. The costs/constraints can be plugged and configured in a similar way as weights/filters in Nova's default filter scheduler.

While it appears to the user that the solver scheduler with costs/constraints provides similar functionalities as filter scheduler with weights/filters, solver scheduler can be more efficient in large scale high amount scheduling problems (e.g. placing 500 instances in a 1000 node cluster in a single request), the scheduling solution from the solver scheduler is also more optimal compared with that from filter scheduler due to its more flexible designs.

Basic configurations

In order to enable nova-solver-scheduler, we need to have the following minimal configurations in the "[default]" section of nova.conf. Please overwrite the config options' values if the option keys already exist in the configuration file.

[DEFAULT]
# ... (other options)
scheduler_driver=nova_solverscheduler.scheduler.solver_scheduler.ConstraintSolverScheduler
scheduler_host_manager=nova_solverscheduler.scheduler.solver_scheduler_host_manager.SolverSchedulerHostManager

Solvers

We provide 2 solvers that can plug-in-and-solve the scheduling problems by satisfying all the configured constraints: FastSolver (default), and PulpSolver. The FastSolver runs a fast algorithm that can solve large scale scheduling requests efficiently while giving optimal solutions. The PulpSolver translates the scheduling problems into standard LP (Linear Programming) problems, and invokes a 3rd party LP solver interface (coinor.pulp >= 1.0.4) to solve the scheduling problems. While PulpSolver might be more flexible for complex constraints (which might happen in the future), it works slower than the FastSolver especially in large scale scheduling problems.

We recommend using FastSolver at the current stage, as it covers all (currently) known constraint requirements, and scales better. The following option in the "[solver_scheduler]" section of nova config should be used to specify which solver to use. Please add a new section title called "[solver_scheduler]" if it (probably) doesn't already exist in the nova config file.

[solver_scheduler]
scheduler_host_solver=nova_solverscheduler.scheduler.solvers.fast_solver.FastSolver

Costs and Constraints

Configuring which costs/constraints to use

Solver-scheduler uses "costs" and "constraints" to configure the behaviour of scheduler. They can be set in a similar way as "weights" and "filters" in the filter-scheduler.

Here is an example for setting which "costs" to use, pleas put these options in the "[solver_scheduler]" section of nova config, each "cost" has a multiplier associated with it to specify its weight in the scheduler decision:

[solver_scheduler]
# ... (other options)
scheduler_solver_costs=RamCost,AffinityCost,AntiAffinityCost
ram_cost_multiplier=1.0
affinity_cost_multiplier=2.0
anti_affinity_cost_multiplier=0.5

The followings is an example of how to set which "constraints" to be used by the solver-scheduler.

[solver_scheduler]
# ... (other options)
scheduler_solver_constraints=ActiveHostsConstraint,RamConstraint,NumInstancesConstraint

In the following section we will discuss the detailed cost and constraint classes.

Transition from filter-scheduler to solver-scheduler

The table below lists supported constraints and their counterparts in filter scheduler. Those costs and constraints can be used in the same way as the weights/filters in the filter scheduler except the above option-setting. Please refer to [OpenStack Configuration Reference](http://docs.openstack.org/icehouse/config-reference/content/section_compute-scheduler.html) for detailed explanation of available weights and filters and their usage instructions.

Weight Cost
MetricsWeigher MetricsCost
RAMWeigher RamCost
Filter Constraint
AggregateCoreFilter AggregateVcpuConstraint
AggregateDiskFilter AggregateDiskConstraint
AggregateImagePropertiesIsolation AggregateImagePropertiesIsolationConstraint
AggregateInstanceExtraSpecsFilter AggregateInstanceExtraSpecsConstraint
AggregateMultiTenancyIsolation AggregateMultiTenancyIsolationConstraint
AggregateRamFilter AggregateRamConstraint
AggregateTypeAffinityFilter AggregateTypeAffinityConstraint
AllHostsFilter NoConstraint
AvailabilityZoneFilter AvailabilityZoneConstraint
ComputeCapabilitiesFilter ComputeCapabilitiesConstraint
ComputeFilter ActiveHostsConstraint
CoreFilter VcpuConstraint
DifferentHostFilter DifferentHostConstraint
DiskFilter DiskConstraint
ImagePropertiesFilter ImagePropertiesConstraint
IsolatedHostsFilter IsolatedHostsConstraint
IoOpsFilter IoOpsConstraint
JsonFilter JsonConstraint
MetricsFilter MetricsConstraint
NumInstancesFilter NumInstancesConstraint
PciPassthroughFilter PciPassthroughConstraint
RamFilter RamConstraint
RetryFilter RetryConstraint
SameHostFilter SameHostConstraint
ServerGroupAffinityFilter ServerGroupAffinityConstraint
ServerGroupAntiAffinityFilter ServerGroupAntiAffinityConstraint
SimpleCIDRAffinityFilter SimpleCidrAffinityConstraint
TrustedFilter TrustedHostsConstraint
TypeAffinityFilter TypeAffinityConstraint

Notes

Some of the above constraints directly invoke their filter counterparts to check host availability, others (in the following list) are implemented with improved logic that may result in more optimal placement decisions for multi-instance requests:

  • DiskConstraint
  • AggregateDiskConstraint (inherited from DiskConstraint)
  • RamConstraint
  • AggregateRamConstraint (inherited from RamConstraint)
  • VcpuConstraint
  • AggregateVcpuConstraint (inherited from VcpuConstraint)
  • IoOpsConstraint
  • NumInstancesConstraint
  • PciPassthroughConstraint
  • ServerGroupAffinityConstraint
  • ServerGroupAntiAffinityConstraint