tobiko/tobiko/openstack/heat
Federico Ressi 3c2155cb4d Skip test cases when nova scheduler hints are not respected
THE CHANGE

This also introduce some new features:
 - allows to override a stack verification step to be executed
   at the end of stack creation to check the stack is in healthy
   cond and eventually retry creating it
 - restructure fail safe and concurrent creation loop
   to make it easier to understand and maintain
 - this should fix some different-host of same-host test case
   failures tipically seen in downstream jobs by checking
   nova scheduler hints are respected and eventually skip neutron
   test cases thare pretend to check connectivity between
   vms running on different (or the same) hypervisor host.

THE PROBLEM

When schedulers hints are given to nova to chose where to allocate
new VMs, there are no warranties to make sure it actually follows
them. In special it has saw many times that it fails to do it
expecially when many VMs are being created on the same time, for
example when running tobiko from a machine with many CPU core.
On such case test runner decides to spawn many parallel workers
(typically 2 for every core when executing on hyperthreading CPUs).
It has been seen the more test runner workers executing tests
the highest is the probability these hints are unmet by Nova.

THE WORKAROUND

To workaround this problem with this change we are verifying
scheduler hints has met. If not it retries creating them up
to two times before raising a SkipException to skip test
current case.

Change-Id: I8c0cff5ca69680aba3842bd7738f27651a677633
2019-12-13 17:39:14 +00:00
..
__init__.py Skip test cases when nova scheduler hints are not respected 2019-12-13 17:39:14 +00:00
_client.py Fix code style for passing PEP8 verifications. 2019-09-19 11:54:50 +02:00
_stack.py Skip test cases when nova scheduler hints are not respected 2019-12-13 17:39:14 +00:00
_template.py Pass mypy type verifications 2019-11-04 13:46:52 +01:00