When performing a resize, we'll want to (by default) select
target hosts from the source cell to do a traditional resize
if possible before considering target hosts in another cell
which will be slower and more complicated. If the source cell
is disabled or target flavor is not available in the source cell,
then we'll have no choice but to select a host from another cell.
But all things being equal between hosts, we want to stay within
the source cell (by default). Therefore this change adds a new
CrossCellWeigher and related configuration option to prefer hosts
within the source cell when moving a server. The weigher is
completely noop unless a cross-cell move is permitted by
configuration, which will be provided in a future change.
Part of blueprint cross-cell-resize
Change-Id: Ib18752efa56cfeb860487fe6b26102bb4b1db038
This spec proposes to add ability to allow users to use
``Aggregate``'s ``metadata`` to override the global config options
for weights to achieve more fine-grained control over resource
weights.
blueprint: per-aggregate-scheduling-weight
Change-Id: I6e15c6507d037ffe263a460441858ed454b02504
This resolves the TODO from Ocata change:
I8871b628f0ab892830ceeede68db16948cb293c8
By adding a min=0.0 value to the soft affinity
weight multiplier configuration options.
It also removes the deprecated [DEFAULT] group
alias from Ocata change:
I3f48e52815e80c99612bcd10cb53331a8c995fc3
Change-Id: I79e191010adbc0ec0ed02c9d589106debbf90ea8
The policies is deprecated in
8fa70e5dfc, and is replaced with
policy field.
This patch changes deprecated policies to policy.
Change-Id: Iac3f666d53a48cdef26be4d17754fd0a481306a5
There is concern over the ability for compute nodes to reasonably
determine which events should count against its consecutive build
failures. Since the compute may erronenously disable itself in
response to mundane or otherwise intentional user-triggered events,
this patch adds a scheduler weigher that considers the build failure
counter and can negatively weigh hosts with recent failures. This
avoids taking computes fully out of rotation, rather treating them as
less likely to be picked for a subsequent scheduling
operation.
This introduces a new conf option to control this weight. The default
is set high to maintain the existing behavior of picking nodes that
are not experiencing high failure rates, and resetting the counter as
soon as a single successful build occurs. This is minimal visible
change from the existing behavior with default configuration.
The rationale behind the default value for this weigher comes from the
values likely to be generated by its peer weighers. The RAM and Disk
weighers will increase the score by number of available megabytes of
memory (range in thousands) and disk (range in millions). The default
value of 1000000 for the build failure weigher will cause competing
nodes with similar amounts of available disk and a small (less than ten)
number of failures to become less desirable than those without, even
with many terabytes of available disk.
Change-Id: I71c56fe770f8c3f66db97fa542fdfdf2b9865fb8
Related-Bug: #1742102
nova provides many different types of filter, providing both resource
stacking/spreading and affinity/anti-affinity patterns. Curiously
enough, however, there is no way to configure a stack or spread policy
of CPUs. Add one.
Change-Id: I90ee8a2081c2a0465441a8d81d161f4887b4e1fb
Implements: bp vcpu-weighter
Take three hosts, one with a single PCI device, one with many PCI
devices, and one with no PCI devices. Nova should prioritise these
differently based on the demands of the instance. If the instance
requests a single PCI device, then the first of the hosts should be
preferred. Similarly, if the instance requests multiple PCI devices,
then the second of these hosts would be preferred. Finally, if the
instance does not request a PCI device, then the last of these hosts
should be preferred.
While the first and second of these cases is already somewhat handled by
the NUMA topology filter (which will ensure the correct number and type
of PCI devices are available on the hosts), there is nothing to stop a
instance that *does not* require PCI devices from occupying space on one
of the few hosts that *does* have free PCI devices.
The PCIWeigher is designed to keep hosts with PCI device(s) as free as
possible, to best ensure they can satisfy requests from instances with
PCI requirements when called upon. This change ensures (a) an instance
with PCI requirements has the best possible chance of scheduling
successfully and (b) an instance without these requirements won't clog
up hosts that have these valuable resources.
Change-Id: I04bf7e6b8324dcac6c93b0cb69c38c30fb05be56
Partially Implements: blueprint reserve-numa-with-pci
There are five options marked with TODOs:
* periodic_task_interval - negative values are valid, making the TODO
invalid
* host_subset_size - code was in-place to prevent negative values, so
replace this code with a min parameter
* isolated_images - removing this option requires a little bit of
thought and would likely require a blueprint. Retain this for now.
* max_instances_per_host, attestation_auth_timeout,
soft_affinity_weight_multiplier,
soft_anti_affinity_weight_multiplier - support funky value ranges
which, semantically speaking, shouldn't be allowed. Start logging
any "invalid" values.
Change-Id: I8871b628f0ab892830ceeede68db16948cb293c8
Move all scheduler options into their one of two groups. Many of the
options are simply renamed to remove their 'scheduler_' prefix, with
some exceptions:
* scheduler_default_filters -> enabled_filters
* scheduler_baremetal_default_filters -> baremetal_enabled_filters
* scheduler_driver_task_period -> periodic_task_interval
* scheduler_tracks_instance_changes -> track_instance_changes
Change-Id: I3f48e52815e80c99612bcd10cb53331a8c995fc3
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
Implements: blueprint centralize-config-options-ocata
LOG.warn is deprecated. It is still used in few modules.
Replaced with non-deprecated LOG.warning.
Change-Id: Ia6acc11eca60c652844175a5742f626732e295e3
Closes-Bug: #1508442
By default this is turned on and has equal weight to the
ram weigher.
DocImpact: Added the disk_weight_multiplier config option
Closes-Bug: 1513654
Change-Id: I29ce73122ad1860081b64b75646a297dfbb8d292
ServerGroupSoftAffinityWeigher implements the soft-affinity
policy for server groups as it orders the hosts by the number
of instances running on them from the same group in decreasing
order.
ServerGroupSoftAntiAffinityWeigher implements the
soft-anti-affinity policy for server groups as it orders the
hosts by the number of instances running on them from the same
group in increasing order.
Both weigher assumes the the RequestSpec object refers to a valid
InstanceGroup object which has a populated members field. This
will be provided by the subsequent patch.
New configuration options added:
* soft_affinity_weight_multiplier
* soft_anti_affinity_multiplier
Add docs about the new weigher classes and change the function
name 'weigh_object' to '_weigh_object' in the Weights section
to match the code.
DocImpact
Co-Authored-By: Balazs Gibizer <balazs.gibizer@ericsson.com>
Implements: blueprint soft-affinity-for-server-group
Change-Id: I3f156d5e5df4d9642bb4b0ffac30a6288459ce61
This change moves all of the configuration options previously defined in
nova/scheduler to the new centralized nova/conf/scheduler directory.
A subsequent patch will then improve the help texts.
Blueprint centralize-config-options
Change-Id: I08d50e873f71601a26ce768feac635243d0570f4
Given that dict comprehensions don't allow to have None as a list, we had
a problem with the Ironic gate-ironic-inspector-dsvm job because
host_state.metrics was set to None by default.
Adding a unittest to make sure we don't regress.
Closes-Bug: #1499271
Change-Id: I6e3104bbd9b9865ec2dcc724c2f56da6d0b2b687
The commit I4dfbea27ce6c3eecc1a8658b1f9dc0feb2298705 changes
'HostState.metrics' to 'objects.MonitorMetricList'. But MetricWeigher
still use the old dictionary of 'host_manager.MetricItem'. This patch
corrects the behavior of MetricWeigher with related tests, and delete
unused MetricItem.
Co-Authored-By: Sylvain Bauza <sbauza@redhat.com>
Change-Id: I7c98c2e189318e85a733f0287ef507c7bad36472
Closes-Bug: #1493680
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
Add a new nova scheduler weighter, sort the filter
hosts according to host io ops number, aims to
booting instances on light workload hosts.
DocImpact: Adds io_ops_weight_multiplier to [DEFAULT]
group of nova.conf and an new default schedule weigher.
Change-Id: Ib3c9184f10b2ebe6b1230365a51b5542dffd447c
Implements: blueprint io-ops-weight
WeighedHost's __repr__ was poking under the HostState hood to use
.host in its own __repr__ but this hides some important details.
Specifically it hides the hypervisor_hostname, which is important for
Ironic as one 'node' can have thousands of hostnames. The other
details such as available RAM and so on are also useful for ops trying
to debug scheduling issues, so rather than add hypervisor_hostname, I
am just delegating to the underlying __repr__.
Change-Id: I79e55c32b3d0768430132275ebe050f38c63bc87
This patch added the support of unavailable metrics in the metric
weigher. Based on the configuration, if an unavailable metric is met,
the metric weigher might either throw an exception, or return an
configurable weight value.
This is part of the blueprint utilization-aware-scheduling.
DocImpact: Added support of unavailable metrics. 2 new config options
are added, 1) 'required' is a boolean option indicating whether all
the metrics set by 'weight_setting' are needed to be available when
calculating the weight. 2) 'weight_of_unavailable' is a float option
specifying the value to be returned if the metrics is unavailable. See
etc/nova/nova.conf.sample for details.
Change-Id: Icb951605aae28487b00a12d172a3ac4dafda2cdb
The scheduler filter MetricsFilter filters out those hosts which don't
have the metrics data available as required by metric settings.
This is part of the blueprint utilization-aware-scheduling.
DocImpact: Added a new metrics filter.
Change-Id: Ib4a898774daf683c4496ef3e9953d23027f11ac0
The weight system is being used by the scheduler and the cells code.
Currently this system is using the raw values instead of normalizing them.
This makes difficult to properly use multipliers for establishing the
relative importance between two wheighers (one big magnitude could
shade a smaller one). This change introduces weight normalization so
that:
- From an operator point of view we can prioritize the weighers that
we are applying. The only way to do this is being sure that all the
weighers will give a value in a known range, so that it is
not needed to artificially use a huge multiplier to prioritize a
weigher.
- From a weigher developer point of view, somebody willing to implement
one has to care about 1) returning a list of values, 2) setting the
minimum and maximum values where the weights can range, if they are
needed and they are significant for the weighing. For a weigher
developer there are two use cases:
Case 1: Use of a percentage instead of absolute values (for example, %
of free RAM). If we compare two nodes focusing on the percentage of free
ram, the maximum value for the weigher is 100. If we have two nodes one
with 2048 total/1024 free, and the second one 1024 total/512 free they
will get both the same weight, since they have the same % of free RAM
(that is, the 50%).
Case 2: Use of absolute values. In this case, the maximum of the weigher
will be the maximum of the values in the list (in the case above, 1024)
or the maximum value that the magnitude could take (in the case above,
2048). How this maximum is set, is a decision of the developer. He may
let the operator choose the behaviour of the weigher though.
- From the point of view of the scheduler we ensure that it is using
normalized values, and not leveraging the normalization mechanism to the
weighers.
Changes introduced this commit:
1) it introduces weight normalization so that we can apply multipliers
easily. All the weights for an object will be normalized between 0.0 and
1.0 before being sumed up, so that the final weight for a host will be:
weight = w1_multiplier * norm(w1) + w2_multiplier * norm(w2) + ...
2) weights.BaseWeigher has been changed into an ABC so that we enforce
that all weighers have the expected methods.
3) weights.BaseWeigher.weigh_objects() does no longer sum up the
computer weighs to the object, but it rather returns a list that will be
then normalized and added to the existing weight by BaseWeightHandler
4) Adapt the existing weighers to the above changes. Namely
- New 'offset_weight_multiplier' for the cell weigher
nova.cells.weights.weight_offset.WeightOffsetWeigher
- Changed the name of the existing multiplier methods.
5) unittests for all of the introduced changes.
Implements blueprint normalize-scheduler-weights
DocImpact: Now weights for an object are normalized before suming them
up. This means that each weigher will take a maximum value of 1. This
may have an impact for operators that are using more than one weigher
(currently there is only one weigher: RAMWeiger) and for operators using
cells (where we have several weighers). It is needed to review then the
multipliers used and adjust them properly in case they have been
modified.
Docimpact: There is a new configuration option 'offset_weight_multiplier'
in nova.cells.weights.weight_offset.WeightOffsetWeigher
Change-Id: I81bf90898d3cb81541f4390596823cc00106eb20
The new metrics weigher can compute the weight based on the compute
node host's metrics data. The to-be weighed metrics and their
weighing ratio are specified in the configuration file as the
followings:
metrics_weight_setting = name1=1.0,name2=-1.0
The final weight would be name1.value * 1.0 + name2.value * (-1.0).
This is part of the blueprint utilization-aware-scheduling.
DocImpact
Change-Id: Ib3e68505e6d4d8f6d67b54c5f00de3e1c172738c
Remove a lot of getLogger lines and imports of logging in modules
which never use that functionality.
Change-Id: Icdaee2c540980412b000d02ebf1ec568dcf5b38a
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.
Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.
Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.
Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.
Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.
Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).
Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
Modules import nova.config for two reasons right now - firstly, to
reference nova.config.CONF and, secondly, if they use one of the
options defined in nova.config.
Often modules import nova.openstack.common.cfg and nova.config
which is a bit pointless since they could just use cfg.CONF if
they just want to nova.config in order to reference CONF.
Let's just use cfg.CONF everywhere and we can explicitly state
where we actually require options defined in nova.config.
Change-Id: Ie4184a74e3e78c99658becb18dce1c2087e450bb
This makes scheduling weights more plugin friendly and creates shared
code that can be used by the host scheduler as well as the future cells
scheduler. Weighing classes can now be specified much like you can
specify scheduling host filters.
The new weights code reverses the old behavior where lower weights win.
Higher weights are now the winners.
The least_cost module and configs have been deprecated, but are still
supported for backwards compatibility. The code has moved to
nova.scheduler.weights.least_cost and been modified to work with the new
loadable-class code. If any of the least_cost related config options are
specified, this least_cost weigher will be used.
For those not overriding the default least_cost config values, the new
RamWeigher class will be used. The default behavior of the RamWeigher
class is the same default behavior as the old least_cost module.
The new weights code introduces a new config option
'scheduler_weight_classes' which is used to specify which weigher classes
to use. The default is 'all classes', but modified if least_cost
deprecated config options are used, as mentioned above.
The RamWeigher class introduces a new config option
'ram_weight_multiplier'. The default of 1.0 causes weights equal to the
free memory in MB to be returned, thus hosts with more free memory are
preferred (causes equal spreading). Changing this value to a negative
number such as -1.0 will cause reverse behavior (fill first).
DocImpact
Change-Id: I1e5e5039c299db02f7287f2d33299ebf0b9732ce
Matt Riedemann