e5ba849437
The weight system is being used by the scheduler and the cells code. Currently this system is using the raw values instead of normalizing them. This makes difficult to properly use multipliers for establishing the relative importance between two wheighers (one big magnitude could shade a smaller one). This change introduces weight normalization so that: - From an operator point of view we can prioritize the weighers that we are applying. The only way to do this is being sure that all the weighers will give a value in a known range, so that it is not needed to artificially use a huge multiplier to prioritize a weigher. - From a weigher developer point of view, somebody willing to implement one has to care about 1) returning a list of values, 2) setting the minimum and maximum values where the weights can range, if they are needed and they are significant for the weighing. For a weigher developer there are two use cases: Case 1: Use of a percentage instead of absolute values (for example, % of free RAM). If we compare two nodes focusing on the percentage of free ram, the maximum value for the weigher is 100. If we have two nodes one with 2048 total/1024 free, and the second one 1024 total/512 free they will get both the same weight, since they have the same % of free RAM (that is, the 50%). Case 2: Use of absolute values. In this case, the maximum of the weigher will be the maximum of the values in the list (in the case above, 1024) or the maximum value that the magnitude could take (in the case above, 2048). How this maximum is set, is a decision of the developer. He may let the operator choose the behaviour of the weigher though. - From the point of view of the scheduler we ensure that it is using normalized values, and not leveraging the normalization mechanism to the weighers. Changes introduced this commit: 1) it introduces weight normalization so that we can apply multipliers easily. All the weights for an object will be normalized between 0.0 and 1.0 before being sumed up, so that the final weight for a host will be: weight = w1_multiplier * norm(w1) + w2_multiplier * norm(w2) + ... 2) weights.BaseWeigher has been changed into an ABC so that we enforce that all weighers have the expected methods. 3) weights.BaseWeigher.weigh_objects() does no longer sum up the computer weighs to the object, but it rather returns a list that will be then normalized and added to the existing weight by BaseWeightHandler 4) Adapt the existing weighers to the above changes. Namely - New 'offset_weight_multiplier' for the cell weigher nova.cells.weights.weight_offset.WeightOffsetWeigher - Changed the name of the existing multiplier methods. 5) unittests for all of the introduced changes. Implements blueprint normalize-scheduler-weights DocImpact: Now weights for an object are normalized before suming them up. This means that each weigher will take a maximum value of 1. This may have an impact for operators that are using more than one weigher (currently there is only one weigher: RAMWeiger) and for operators using cells (where we have several weighers). It is needed to review then the multipliers used and adjust them properly in case they have been modified. Docimpact: There is a new configuration option 'offset_weight_multiplier' in nova.cells.weights.weight_offset.WeightOffsetWeigher Change-Id: I81bf90898d3cb81541f4390596823cc00106eb20 |
||
---|---|---|
.. | ||
CA | ||
api | ||
bundle | ||
cells | ||
cert | ||
compute | ||
conductor | ||
console | ||
consoleauth | ||
db | ||
fake_loadables | ||
glance | ||
image | ||
integrated | ||
keymgr | ||
monkey_patch_example | ||
network | ||
objects | ||
pci | ||
scheduler | ||
servicegroup | ||
ssl_cert | ||
virt | ||
volume | ||
README.rst | ||
__init__.py | ||
cast_as_call.py | ||
conf_fixture.py | ||
fake_crypto.py | ||
fake_hosts.py | ||
fake_instance.py | ||
fake_instance_actions.py | ||
fake_ldap.py | ||
fake_network.py | ||
fake_network_cache_model.py | ||
fake_notifier.py | ||
fake_policy.py | ||
fake_processutils.py | ||
fake_utils.py | ||
fake_volume.py | ||
fakeguestfs.py | ||
matchers.py | ||
policy_fixture.py | ||
test_api_validation.py | ||
test_availability_zones.py | ||
test_baserpc.py | ||
test_bdm.py | ||
test_block_device.py | ||
test_cinder.py | ||
test_configdrive2.py | ||
test_context.py | ||
test_crypto.py | ||
test_exception.py | ||
test_flavors.py | ||
test_hooks.py | ||
test_instance_types_extra_specs.py | ||
test_iptables_network.py | ||
test_ipv6.py | ||
test_linuxscsi.py | ||
test_loadables.py | ||
test_manager.py | ||
test_matchers.py | ||
test_metadata.py | ||
test_notifications.py | ||
test_nova_manage.py | ||
test_objectstore.py | ||
test_pipelib.py | ||
test_policy.py | ||
test_quota.py | ||
test_safeutils.py | ||
test_service.py | ||
test_test.py | ||
test_test_utils.py | ||
test_unit.py | ||
test_utils.py | ||
test_versions.py | ||
test_weights.py | ||
test_wsgi.py | ||
utils.py |
README.rst
OpenStack Nova Testing Infrastructure
This README file attempts to provide current and prospective contributors with everything they need to know in order to start creating unit tests for nova.
Note: the content for the rest of this file will be added as the work items in the following blueprint are completed: https://blueprints.launchpad.net/nova/+spec/consolidate-testing-infrastructure
Test Types: Unit vs. Functional vs. Integration
TBD
Writing Unit Tests
TBD
Using Fakes
TBD
test.TestCase
The TestCase class from nova.test (generally imported as test) will automatically manage self.stubs using the stubout module and self.mox using the mox module during the setUp step. They will automatically verify and clean up during the tearDown step.
If using test.TestCase, calling the super class setUp is required and calling the super class tearDown is required to be last if tearDown is overriden.
Writing Functional Tests
TBD
Writing Integration Tests
TBD
Tests and Exceptions
A properly written test asserts that particular behavior occurs. This can be a success condition or a failure condition, including an exception. When asserting that a particular exception is raised, the most specific exception possible should be used.
In particular, testing for Exception being raised is almost always a mistake since it will match (almost) every exception, even those unrelated to the exception intended to be tested.
This applies to catching exceptions manually with a try/except block, or using assertRaises().
Example:
self.assertRaises(exception.InstanceNotFound, db.instance_get_by_uuid,
elevated, instance_uuid)
If a stubbed function/method needs a generic exception for testing purposes, test.TestingException is available.
Example:
def stubbed_method(self):
raise test.TestingException()
self.stubs.Set(cls, 'inner_method', stubbed_method)
obj = cls()
self.assertRaises(test.TestingException, obj.outer_method)
Stubbing and Mocking
Whenever possible, tests SHOULD NOT stub and mock out the same function.
If it's unavoidable, tests SHOULD define stubs before mocks since the TestCase cleanup routine will un-mock before un-stubbing. Doing otherwise results in a test that leaks stubbed functions, causing hard-to-debug interference between tests1.
If a mock must take place before a stub, any stubs after the mock call MUST be manually unset using self.cleanUp calls within the test.