openstack/nova
Code Issues Proposed changes

Make scheduler filters more pluggable

Browse Source 
Filters are supposed to be pluggable, but they are not, since you have
to modify __init__.py.

This adds a --scheduler_availabile_filters setting which replaces the
hardcoding in __init__.py.  This setting is a MultiStr (a list, which
you can specify more than once) containing the full paths to filter
classes to make available to the scheduler.  Using a value of
'nova.scheduler.filters.standard_filters' maps to all standard filters
included in nova under nova.scheduler.filters.  This is the default
setting and matches what was in __init__.py before.

Also renamed --default_host_filters to --scheduler_default_filters to
make the flag a bit more clear.

Change-Id: I10eb54e9982b6d42316adfb2cc2600b44a9c3bdf
This commit is contained in:
Chris Behrens 2012-02-24 00:01:57 +00:00
parent f01b9b8dd2
commit 3f42e11ca0
14 changed files with 194 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
doc/source/devref/distributed_scheduler.rst
View File

@ -55,15 +55,28 @@ The filtering (excluding compute nodes incapable of fulfilling the request) and
Host Filter
-----------
As we mentioned earlier, filtering hosts is a very deployment-specific process. Service Providers may have a different set of criteria for filtering Compute nodes than a University. To faciliate this the `nova.scheduler.filters` module supports a variety of filtering strategies as well as an easy means for plugging in your own algorithms.
As we mentioned earlier, filtering hosts is a very deployment-specific process. Service Providers may have a different set of criteria for filtering Compute nodes than a University. To facilitate this, the `DistributedScheduler` supports a variety of filtering strategies as well as an easy means for plugging in your own algorithms. Specifying filters involves 2 settings. One makes filters available for use. The second specifies which filters to use by default (out of the filters available). The reason for this second option is that there may be support to allow end-users to specify specific filters during a build at some point in the future.
The filter used is determined by the `--default_host_filters` flag, which points to a Python Class. By default this flag is set to `[AllHostsFilter]` which simply returns all available hosts. But there are others:
Making filters available:
* `ComputeFilter` provides host filtering based on the memory and disk size specified in the `InstanceType` record passed into `run_instance`.
Filters are made available to the scheduler via the `--scheduler_available_filters` setting. This setting can be specified more than once and should contain lists of filter class names (with full paths) to make available. Specifying 'nova.scheduler.filters.standard_filters' will cause all standard filters under 'nova.scheduler.filters' to be made available. That is the default setting. Additionally, you can specify your own classes to be made available. For example, 'myfilter.MyFilterClass' can be specified. Now that you've configured which filters are available, you should set which ones you actually want to use by default.
Setting the default filtering classes:
The default filters to use are set via the `--scheduler_default_filters` setting. This setting should contain a list of class names. You should not specify the full paths with these class names. By default this flag is set to `['AvailabilityZoneFilter', 'RamFilter', 'ComputeFilter']`. Below is a list of standard filter classes:
* `AllHostsFilter` includes all hosts (essentially is a No-op filter)
* `AvailabilityZoneFilter` provides host filtering based on availability_zone
* `ComputeFilter` provides host filtering based on `InstanceType` extra_specs, comparing against host capability announcements
* `CoreFilter` provides host filtering based on number of cpu cores
* `DifferentHostFilter` provides host filtering based on scheduler_hint's 'different_host' value. With the scheduler_hints extension, this allows one to put a new instance on a different host from another instance
* `IsolatedHostsFilter` provides host filtering based on the 'isolated_hosts' and 'isolated_images' flags/settings.
* `JSONFilter` filters hosts based on simple JSON expression grammar. Using a LISP-like JSON structure the caller can request instances based on criteria well beyond what `ComputeFilter` specifies. See `nova.tests.scheduler.test_host_filters` for examples.
* `RamFilter` provides host filtering based on the memory needed vs memory free
* `SameHostFilter` provides host filtering based on scheduler_hint's 'same_host' value. With the scheduler_hints extension, this allows one to put a new instance on the same host as another instance
* `SimpleCIDRAffinityFilter` provides host filtering based on scheduler_hint's 'build_near_host_ip' value. With the scheduler_hints extension, this allows one to put a new instance on a host within the same IP block.
To create your own `HostFilter` the user simply has to derive from `nova.scheduler.filters.AbstractHostFilter` and implement one method: `host_passes`. This method accepts a `HostState` instance describing a host as well as a `filter_properties` dictionary. Host capabilities can be found in `HostState`.capabilities and other properites can be found in `filter_properties` like `instance_type`, etc. Your method should return True if it passes the filter.
To create your own `HostFilter` the user simply has to derive from `nova.scheduler.filters.BaseHostFilter` and implement one method: `host_passes`. This method accepts a `HostState` instance describing a host as well as a `filter_properties` dictionary. Host capabilities can be found in `HostState`.capabilities and other properites can be found in `filter_properties` like `instance_type`, etc. Your method should return True if it passes the filter.
Flags
-----
@ -73,10 +86,12 @@ Here are some of the main flags you should set in your `nova.conf` file:
::
--scheduler_driver=nova.scheduler.distributed_scheduler.DistributedScheduler
--default_host_filters=nova.scheduler.filters.AllHostsFilter
--scheduler_available_filters=nova.scheduler.filters.standard_filters
# --scheduler_available_filters=myfilter.MyOwnFilter
--scheduler_default_filters=RamFilter,ComputeFilter,MyOwnFilter
`scheduler_driver` is the real workhorse of the operation. For Distributed Scheduler, you need to specify a class derived from `nova.scheduler.distributed_scheduler.DistributedScheduler`.
`default_host_filter` is the host filter to be used for filtering candidate Compute nodes.
`scheduler_default_filters` are the host filters to be used for filtering candidate Compute nodes.
Some optional flags which are handy for debugging are:

96
nova/scheduler/filters/__init__.py
View File

@ -14,30 +14,74 @@
# under the License.
"""
There are three filters included: AllHosts, InstanceType & JSON.
AllHosts just returns the full, unfiltered list of hosts.
InstanceType is a hard coded matching mechanism based on flavor criteria.
JSON is an ad-hoc filter grammar.
Why JSON? The requests for instances may come in through the
REST interface from a user or a parent Zone.
Currently InstanceTypes are used for specifing the type of instance desired.
Specific Nova users have noted a need for a more expressive way of specifying
instance requirements. Since we don't want to get into building full DSL,
this filter is a simple form as an example of how this could be done.
In reality, most consumers will use the more rigid filters such as the
InstanceType filter.
Scheduler host filters
"""
from nova.scheduler.filters.abstract_filter import AbstractHostFilter
from nova.scheduler.filters.affinity_filter import DifferentHostFilter
from nova.scheduler.filters.affinity_filter import SameHostFilter
from nova.scheduler.filters.affinity_filter import SimpleCIDRAffinityFilter
from nova.scheduler.filters.all_hosts_filter import AllHostsFilter
from nova.scheduler.filters.availability_zone_filter \
import AvailabilityZoneFilter
from nova.scheduler.filters.isolated_hosts_filter import IsolatedHostsFilter
from nova.scheduler.filters.compute_filter import ComputeFilter
from nova.scheduler.filters.core_filter import CoreFilter
from nova.scheduler.filters.json_filter import JsonFilter
from nova.scheduler.filters.ram_filter import RamFilter
import os
import types
from nova import exception
from nova import utils
class BaseHostFilter(object):
"""Base class for host filters."""
def host_passes(self, host_state, filter_properties):
raise NotImplemented()
def _full_name(self):
"""module.classname of the filter."""
return "%s.%s" % (self.__module__, self.__class__.__name__)
def _is_filter_class(cls):
"""Return whether a class is a valid Host Filter class."""
return type(cls) is types.TypeType and issubclass(cls, BaseHostFilter)
def _get_filter_classes_from_module(module_name):
"""Get all filter classes from a module."""
classes = []
module = utils.import_object(module_name)
for obj_name in dir(module):
itm = getattr(module, obj_name)
if _is_filter_class(itm):
classes.append(itm)
return classes
def standard_filters():
"""Return a list of filter classes found in this directory."""
classes = []
filters_dir = __path__[0]
for dirpath, dirnames, filenames in os.walk(filters_dir):
relpath = os.path.relpath(dirpath, filters_dir)
if relpath == '.':
relpkg = ''
else:
relpkg = '.%s' % '.'.join(relpath.split(os.sep))
for fname in filenames:
root, ext = os.path.splitext(fname)
if ext != '.py' or root == '__init__':
continue
module_name = "%s%s.%s" % (__package__, relpkg, root)
mod_classes = _get_filter_classes_from_module(module_name)
classes.extend(mod_classes)
return classes
def get_filter_classes(filter_class_names):
"""Get filter classes from class names."""
classes = []
for cls_name in filter_class_names:
obj = utils.import_class(cls_name)
if _is_filter_class(obj):
classes.append(obj)
elif type(obj) is types.FunctionType:
# Get list of classes from a function
classes.extend(obj())
else:
raise exception.ClassNotFound(class_name=cls_name,
exception='Not a valid scheduler filter')
return classes

25
nova/scheduler/filters/abstract_filter.py
View File

@ -1,25 +0,0 @@
# Copyright (c) 2011 Openstack, LLC.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
class AbstractHostFilter(object):
"""Base class for host filters."""
def host_passes(self, host_state, filter_properties):
return True
def _full_name(self):
"""module.classname of the filter."""
return "%s.%s" % (self.__module__, self.__class__.__name__)

4
nova/scheduler/filters/affinity_filter.py
View File

@ -15,14 +15,14 @@
# limitations under the License.
import abstract_filter
import netaddr
from nova.compute import api as compute
from nova import flags
from nova.scheduler import filters
class AffinityFilter(abstract_filter.AbstractHostFilter):
class AffinityFilter(filters.BaseHostFilter):
def __init__(self):
self.compute_api = compute.API()

4
nova/scheduler/filters/all_hosts_filter.py
View File

@ -14,10 +14,10 @@
# under the License.
import abstract_filter
from nova.scheduler import filters
class AllHostsFilter(abstract_filter.AbstractHostFilter):
class AllHostsFilter(filters.BaseHostFilter):
"""NOP host filter. Returns all hosts."""
def host_passes(self, host_state, filter_properties):

4
nova/scheduler/filters/availability_zone_filter.py
View File

@ -14,10 +14,10 @@
# under the License.
import abstract_filter
from nova.scheduler import filters
class AvailabilityZoneFilter(abstract_filter.AbstractHostFilter):
class AvailabilityZoneFilter(filters.BaseHostFilter):
"""Filters Hosts by availabilty zone."""
def host_passes(self, host_state, filter_properties):

4
nova/scheduler/filters/compute_filter.py
View File

@ -14,14 +14,14 @@
# under the License.
from nova import log as logging
from nova.scheduler.filters import abstract_filter
from nova.scheduler import filters
from nova import utils
LOG = logging.getLogger(__name__)
class ComputeFilter(abstract_filter.AbstractHostFilter):
class ComputeFilter(filters.BaseHostFilter):
"""HostFilter hard-coded to work with InstanceType records."""
def _satisfies_extra_specs(self, capabilities, instance_type):

4
nova/scheduler/filters/core_filter.py
View File

@ -18,7 +18,7 @@
from nova import flags
from nova import log as logging
from nova.openstack.common import cfg
from nova.scheduler.filters import abstract_filter
from nova.scheduler import filters
LOG = logging.getLogger(__name__)
@ -31,7 +31,7 @@ FLAGS = flags.FLAGS
FLAGS.register_opt(cpu_allocation_ratio_opt)
class CoreFilter(abstract_filter.AbstractHostFilter):
class CoreFilter(filters.BaseHostFilter):
"""CoreFilter filters based on CPU core utilization."""
def host_passes(self, host_state, filter_properties):

4
nova/scheduler/filters/isolated_hosts_filter.py
View File

@ -14,14 +14,14 @@
# under the License.
import abstract_filter
from nova import flags
from nova.scheduler import filters
FLAGS = flags.FLAGS
class IsolatedHostsFilter(abstract_filter.AbstractHostFilter):
class IsolatedHostsFilter(filters.BaseHostFilter):
"""Returns host."""
def host_passes(self, host_state, filter_properties):

4
nova/scheduler/filters/json_filter.py
View File

@ -17,10 +17,10 @@
import json
import operator
from nova.scheduler.filters import abstract_filter
from nova.scheduler import filters
class JsonFilter(abstract_filter.AbstractHostFilter):
class JsonFilter(filters.BaseHostFilter):
"""Host Filter to allow simple JSON-based grammar for
selecting hosts.
"""

4
nova/scheduler/filters/ram_filter.py
View File

@ -17,7 +17,7 @@
from nova import flags
from nova import log as logging
from nova.openstack.common import cfg
from nova.scheduler.filters import abstract_filter
from nova.scheduler import filters
LOG = logging.getLogger(__name__)
@ -29,7 +29,7 @@ FLAGS = flags.FLAGS
FLAGS.register_opt(ram_allocation_ratio_opt)
class RamFilter(abstract_filter.AbstractHostFilter):
class RamFilter(filters.BaseHostFilter):
"""Ram Filter with over subscription flag"""
def host_passes(self, host_state, filter_properties):

32
nova/scheduler/host_manager.py
View File

@ -18,7 +18,6 @@ Manage hosts in the current zone.
"""
import datetime
import types
import UserDict
from nova import db
@ -26,6 +25,7 @@ from nova import exception
from nova import flags
from nova import log as logging
from nova.openstack.common import cfg
from nova.scheduler import filters
from nova import utils
@ -36,14 +36,20 @@ host_manager_opts = [
cfg.IntOpt('reserved_host_memory_mb',
default=512,
help='Amount of memory in MB to reserve for host/dom0'),
cfg.ListOpt('default_host_filters',
cfg.MultiStrOpt('scheduler_available_filters',
default=['nova.scheduler.filters.standard_filters'],
help='Filter classes available to the scheduler which may '
'be specified more than once. An entry of '
'"nova.scheduler.filters.standard_filters" '
'maps to all filters included with nova.'),
cfg.ListOpt('scheduler_default_filters',
default=[
'AvailabilityZoneFilter',
'RamFilter',
'ComputeFilter'
],
help='Which filters to use for filtering hosts when not '
'specified in the request.'),
help='Which filter class names to use for filtering hosts '
'when not specified in the request.'),
]
FLAGS = flags.FLAGS
@ -157,20 +163,8 @@ class HostManager(object):
def __init__(self):
self.service_states = {} # { <host> : { <service> : { cap k : v }}}
self.filter_classes = self._get_filter_classes()
def _get_filter_classes(self):
"""Get the list of possible filter classes"""
# Imported here to avoid circular imports
from nova.scheduler import filters
def get_itm(nm):
return getattr(filters, nm)
return [get_itm(itm) for itm in dir(filters)
if (type(get_itm(itm)) is types.TypeType)
and issubclass(get_itm(itm), filters.AbstractHostFilter)
and get_itm(itm) is not filters.AbstractHostFilter]
self.filter_classes = filters.get_filter_classes(
FLAGS.scheduler_available_filters)
def _choose_host_filters(self, filters):
"""Since the caller may specify which filters to use we need
@ -179,7 +173,7 @@ class HostManager(object):
of acceptable filters.
"""
if filters is None:
filters = FLAGS.default_host_filters
filters = FLAGS.scheduler_default_filters
if not isinstance(filters, (list, tuple)):
filters = [filters]
good_filters = []

112
nova/tests/scheduler/test_host_filters.py
View File

@ -18,6 +18,7 @@ Tests For Scheduler Host Filters.
import json
from nova import context
from nova import exception
from nova import flags
from nova.scheduler import filters
from nova import test
@ -25,6 +26,15 @@ from nova.tests.scheduler import fakes
from nova import utils
class TestFilter(filters.BaseHostFilter):
pass
class TestBogusFilter(object):
"""Class that doesn't inherit from BaseHostFilter"""
pass
class HostFiltersTestCase(test.TestCase):
"""Test case for host filters."""
@ -34,9 +44,35 @@ class HostFiltersTestCase(test.TestCase):
self.json_query = json.dumps(
['and', ['>=', '$free_ram_mb', 1024],
['>=', '$free_disk_mb', 200 * 1024]])
# This has a side effect of testing 'get_filter_classes'
# when specifing a method (in this case, our standard filters)
classes = filters.get_filter_classes(
['nova.scheduler.filters.standard_filters'])
self.class_map = {}
for cls in classes:
self.class_map[cls.__name__] = cls
def test_get_filter_classes(self):
classes = filters.get_filter_classes(
['nova.tests.scheduler.test_host_filters.TestFilter'])
self.assertEqual(len(classes), 1)
self.assertEqual(classes[0].__name__, 'TestFilter')
# Test a specific class along with our standard filters
classes = filters.get_filter_classes(
['nova.tests.scheduler.test_host_filters.TestFilter',
'nova.scheduler.filters.standard_filters'])
self.assertEqual(len(classes), 1 + len(self.class_map))
def test_get_filter_classes_raises_on_invalid_classes(self):
self.assertRaises(exception.ClassNotFound,
filters.get_filter_classes,
['nova.tests.scheduler.test_host_filters.NoExist'])
self.assertRaises(exception.ClassNotFound,
filters.get_filter_classes,
['nova.tests.scheduler.test_host_filters.TestBogusFilter'])
def test_all_host_filter(self):
filt_cls = filters.AllHostsFilter()
filt_cls = self.class_map['AllHostsFilter']()
host = fakes.FakeHostState('host1', 'compute', {})
self.assertTrue(filt_cls.host_passes(host, {}))
@ -46,7 +82,7 @@ class HostFiltersTestCase(test.TestCase):
self.stubs.Set(utils, 'service_is_up', fake_service_is_up)
def test_affinity_different_filter_passes(self):
filt_cls = filters.DifferentHostFilter()
filt_cls = self.class_map['DifferentHostFilter']()
host = fakes.FakeHostState('host1', 'compute', {})
instance = fakes.FakeInstance(context=self.context,
params={'host': 'host2'})
@ -59,7 +95,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_affinity_different_filter_fails(self):
filt_cls = filters.DifferentHostFilter()
filt_cls = self.class_map['DifferentHostFilter']()
host = fakes.FakeHostState('host1', 'compute', {})
instance = fakes.FakeInstance(context=self.context,
params={'host': 'host1'})
@ -72,7 +108,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertFalse(filt_cls.host_passes(host, filter_properties))
def test_affinity_same_filter_passes(self):
filt_cls = filters.SameHostFilter()
filt_cls = self.class_map['SameHostFilter']()
host = fakes.FakeHostState('host1', 'compute', {})
instance = fakes.FakeInstance(context=self.context,
params={'host': 'host1'})
@ -85,7 +121,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_affinity_same_filter_fails(self):
filt_cls = filters.SameHostFilter()
filt_cls = self.class_map['SameHostFilter']()
host = fakes.FakeHostState('host1', 'compute', {})
instance = fakes.FakeInstance(context=self.context,
params={'host': 'host2'})
@ -98,7 +134,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertFalse(filt_cls.host_passes(host, filter_properties))
def test_affinity_simple_cidr_filter_passes(self):
filt_cls = filters.SimpleCIDRAffinityFilter()
filt_cls = self.class_map['SimpleCIDRAffinityFilter']()
host = fakes.FakeHostState('host1', 'compute', {})
affinity_ip = flags.FLAGS.my_ip.split('.')[0:3]
@ -113,7 +149,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_affinity_simple_cidr_filter_fails(self):
filt_cls = filters.SimpleCIDRAffinityFilter()
filt_cls = self.class_map['SimpleCIDRAffinityFilter']()
host = fakes.FakeHostState('host1', 'compute', {})
affinity_ip = flags.FLAGS.my_ip.split('.')
@ -129,7 +165,7 @@ class HostFiltersTestCase(test.TestCase):
def test_compute_filter_passes(self):
self._stub_service_is_up(True)
filt_cls = filters.ComputeFilter()
filt_cls = self.class_map['ComputeFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024}}
capabilities = {'enabled': True}
service = {'disabled': False}
@ -140,7 +176,7 @@ class HostFiltersTestCase(test.TestCase):
def test_ram_filter_fails_on_memory(self):
self._stub_service_is_up(True)
filt_cls = filters.RamFilter()
filt_cls = self.class_map['RamFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024}}
capabilities = {'enabled': True}
service = {'disabled': False}
@ -151,7 +187,7 @@ class HostFiltersTestCase(test.TestCase):
def test_compute_filter_fails_on_service_disabled(self):
self._stub_service_is_up(True)
filt_cls = filters.ComputeFilter()
filt_cls = self.class_map['ComputeFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024}}
capabilities = {'enabled': True}
service = {'disabled': True}
@ -162,7 +198,7 @@ class HostFiltersTestCase(test.TestCase):
def test_compute_filter_fails_on_service_down(self):
self._stub_service_is_up(False)
filt_cls = filters.ComputeFilter()
filt_cls = self.class_map['ComputeFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024}}
capabilities = {'enabled': True}
service = {'disabled': False}
@ -173,7 +209,7 @@ class HostFiltersTestCase(test.TestCase):
def test_compute_filter_passes_on_volume(self):
self._stub_service_is_up(True)
filt_cls = filters.ComputeFilter()
filt_cls = self.class_map['ComputeFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024}}
capabilities = {'enabled': False}
service = {'disabled': False}
@ -184,7 +220,7 @@ class HostFiltersTestCase(test.TestCase):
def test_compute_filter_passes_on_no_instance_type(self):
self._stub_service_is_up(True)
filt_cls = filters.ComputeFilter()
filt_cls = self.class_map['ComputeFilter']()
filter_properties = {}
capabilities = {'enabled': False}
service = {'disabled': False}
@ -195,7 +231,7 @@ class HostFiltersTestCase(test.TestCase):
def test_compute_filter_passes_extra_specs(self):
self._stub_service_is_up(True)
filt_cls = filters.ComputeFilter()
filt_cls = self.class_map['ComputeFilter']()
extra_specs = {'opt1': 1, 'opt2': 2}
capabilities = {'enabled': True, 'opt1': 1, 'opt2': 2}
service = {'disabled': False}
@ -208,7 +244,7 @@ class HostFiltersTestCase(test.TestCase):
def test_compute_filter_fails_extra_specs(self):
self._stub_service_is_up(True)
filt_cls = filters.ComputeFilter()
filt_cls = self.class_map['ComputeFilter']()
extra_specs = {'opt1': 1, 'opt2': 3}
capabilities = {'enabled': True, 'opt1': 1, 'opt2': 2}
service = {'disabled': False}
@ -222,7 +258,7 @@ class HostFiltersTestCase(test.TestCase):
def test_isolated_hosts_fails_isolated_on_non_isolated(self):
self.flags(isolated_images=['isolated'], isolated_hosts=['isolated'])
filt_cls = filters.IsolatedHostsFilter()
filt_cls = self.class_map['IsolatedHostsFilter']()
filter_properties = {
'request_spec': {
'instance_properties': {'image_ref': 'isolated'}
@ -233,7 +269,7 @@ class HostFiltersTestCase(test.TestCase):
def test_isolated_hosts_fails_non_isolated_on_isolated(self):
self.flags(isolated_images=['isolated'], isolated_hosts=['isolated'])
filt_cls = filters.IsolatedHostsFilter()
filt_cls = self.class_map['IsolatedHostsFilter']()
filter_properties = {
'request_spec': {
'instance_properties': {'image_ref': 'non-isolated'}
@ -244,7 +280,7 @@ class HostFiltersTestCase(test.TestCase):
def test_isolated_hosts_passes_isolated_on_isolated(self):
self.flags(isolated_images=['isolated'], isolated_hosts=['isolated'])
filt_cls = filters.IsolatedHostsFilter()
filt_cls = self.class_map['IsolatedHostsFilter']()
filter_properties = {
'request_spec': {
'instance_properties': {'image_ref': 'isolated'}
@ -255,7 +291,7 @@ class HostFiltersTestCase(test.TestCase):
def test_isolated_hosts_passes_non_isolated_on_non_isolated(self):
self.flags(isolated_images=['isolated'], isolated_hosts=['isolated'])
filt_cls = filters.IsolatedHostsFilter()
filt_cls = self.class_map['IsolatedHostsFilter']()
filter_properties = {
'request_spec': {
'instance_properties': {'image_ref': 'non-isolated'}
@ -265,7 +301,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_json_filter_passes(self):
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024,
'root_gb': 200,
'ephemeral_gb': 0},
@ -278,7 +314,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_json_filter_passes_with_no_query(self):
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024,
'root_gb': 200,
'ephemeral_gb': 0}}
@ -290,7 +326,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_json_filter_fails_on_memory(self):
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024,
'root_gb': 200,
'ephemeral_gb': 0},
@ -303,7 +339,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertFalse(filt_cls.host_passes(host, filter_properties))
def test_json_filter_fails_on_disk(self):
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
filter_properties = {'instance_type': {'memory_mb': 1024,
'root_gb': 200,
'ephemeral_gb': 0},
@ -316,7 +352,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertFalse(filt_cls.host_passes(host, filter_properties))
def test_json_filter_fails_on_caps_disabled(self):
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
json_query = json.dumps(
['and', ['>=', '$free_ram_mb', 1024],
['>=', '$free_disk_mb', 200 * 1024],
@ -333,7 +369,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertFalse(filt_cls.host_passes(host, filter_properties))
def test_json_filter_fails_on_service_disabled(self):
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
json_query = json.dumps(
['and', ['>=', '$free_ram_mb', 1024],
['>=', '$free_disk_mb', 200 * 1024],
@ -351,7 +387,7 @@ class HostFiltersTestCase(test.TestCase):
def test_json_filter_happy_day(self):
"""Test json filter more thoroughly"""
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
raw = ['and',
'$capabilities.enabled',
['=', '$capabilities.opt1', 'match'],
@ -425,7 +461,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertFalse(filt_cls.host_passes(host, filter_properties))
def test_json_filter_basic_operators(self):
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
host = fakes.FakeHostState('host1', 'compute',
{'capabilities': {'enabled': True}})
# (operator, arguments, expected_result)
@ -476,18 +512,16 @@ class HostFiltersTestCase(test.TestCase):
self.assertFalse(filt_cls.host_passes(host, filter_properties))
def test_json_filter_unknown_operator_raises(self):
filt_cls = filters.JsonFilter()
filt_cls = self.class_map['JsonFilter']()
raw = ['!=', 1, 2]
filter_properties = {'query': json.dumps(raw)}
capabilities = {'enabled': True, 'opt1': 'no-match'}
host = fakes.FakeHostState('host1', 'compute',
{'capabilities': {'enabled': True}})
self.assertRaises(KeyError,
filt_cls.host_passes, host, filter_properties)
def test_json_filter_empty_filters_pass(self):
filt_cls = filters.JsonFilter()
capabilities = {'enabled': True, 'opt1': 'no-match'}
filt_cls = self.class_map['JsonFilter']()
host = fakes.FakeHostState('host1', 'compute',
{'capabilities': {'enabled': True}})
@ -499,8 +533,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_json_filter_invalid_num_arguments_fails(self):
filt_cls = filters.JsonFilter()
capabilities = {'enabled': True, 'opt1': 'no-match'}
filt_cls = self.class_map['JsonFilter']()
host = fakes.FakeHostState('host1', 'compute',
{'capabilities': {'enabled': True}})
@ -513,8 +546,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertFalse(filt_cls.host_passes(host, filter_properties))
def test_json_filter_unknown_variable_ignored(self):
filt_cls = filters.JsonFilter()
capabilities = {'enabled': True, 'opt1': 'no-match'}
filt_cls = self.class_map['JsonFilter']()
host = fakes.FakeHostState('host1', 'compute',
{'capabilities': {'enabled': True}})
@ -527,7 +559,7 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_core_filter_passes(self):
filt_cls = filters.CoreFilter()
filt_cls = self.class_map['CoreFilter']()
filter_properties = {'instance_type': {'vcpus': 1}}
self.flags(cpu_allocation_ratio=2)
host = fakes.FakeHostState('host1', 'compute',
@ -535,13 +567,13 @@ class HostFiltersTestCase(test.TestCase):
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_core_filter_fails_safe(self):
filt_cls = filters.CoreFilter()
filt_cls = self.class_map['CoreFilter']()
filter_properties = {'instance_type': {'vcpus': 1}}
host = fakes.FakeHostState('host1', 'compute', {})
self.assertTrue(filt_cls.host_passes(host, filter_properties))
def test_core_filter_fails(self):
filt_cls = filters.CoreFilter()
filt_cls = self.class_map['CoreFilter']()
filter_properties = {'instance_type': {'vcpus': 1}}
self.flags(cpu_allocation_ratio=2)
host = fakes.FakeHostState('host1', 'compute',
@ -561,14 +593,14 @@ class HostFiltersTestCase(test.TestCase):
}
def test_availability_zone_filter_same(self):
filt_cls = filters.AvailabilityZoneFilter()
filt_cls = self.class_map['AvailabilityZoneFilter']()
service = {'availability_zone': 'nova'}
request = self._make_zone_request('nova')
host = fakes.FakeHostState('host1', 'compute', {'service': service})
self.assertTrue(filt_cls.host_passes(host, request))
def test_availability_zone_filter_different(self):
filt_cls = filters.AvailabilityZoneFilter()
filt_cls = self.class_map['AvailabilityZoneFilter']()
service = {'availability_zone': 'nova'}
request = self._make_zone_request('bad')
host = fakes.FakeHostState('host1', 'compute', {'service': service})

5
nova/tests/scheduler/test_host_manager.py
View File

@ -20,7 +20,6 @@ import datetime
from nova import db
from nova import exception
from nova import log as logging
from nova.scheduler import host_manager
from nova import test
from nova.tests.scheduler import fakes
@ -45,14 +44,14 @@ class HostManagerTestCase(test.TestCase):
self.host_manager = host_manager.HostManager()
def test_choose_host_filters_not_found(self):
self.flags(default_host_filters='ComputeFilterClass3')
self.flags(scheduler_default_filters='ComputeFilterClass3')
self.host_manager.filter_classes = [ComputeFilterClass1,
ComputeFilterClass2]
self.assertRaises(exception.SchedulerHostFilterNotFound,
self.host_manager._choose_host_filters, None)
def test_choose_host_filters(self):
self.flags(default_host_filters=['ComputeFilterClass2'])
self.flags(scheduler_default_filters=['ComputeFilterClass2'])
self.host_manager.filter_classes = [ComputeFilterClass1,
ComputeFilterClass2]