rally-openstack/rally
Boris Pavlovic 82cd95fa7e New task config and verification refactoring
*) Change task config format
   . Split "context" & "runner" stuff

*) Refactor Verification
   . Move validation to context.base, runner.base and scenario.base
   . Validate whole config fully before starting any of tasks
   .  Optimize scenario args validation (create only one time clients)
   . Optimize order of validation:
     1) Validate names of benchmarks
     2) Validate all static parameters, e.g. configuration of runner
        and context
     3) If everything is ok in all benchmarks, then start validation
        of scenario args.
   . Store validation result (exception) in task["verification_log"]
   . Remove verification logic from BenchmarkEngine.__exit__
   . Remove scenario args verification results from task["results"]

*) Fix & Swtich to new format doc/samples/tasks
   . Switch to new fromat
   . Add missing task configratuion
   . Better formatting
   . json & yaml samples

*) Refactored unit tests
   . tests.rally.benchmark.test_engine
   . tests.rally.benchmark.context.base
   . tests.orcestrator.test_api.start_task
       cover validation step as well and new change format

*) Refactor orchestrator api start task
   . Remove benchmark engine context
   . Call verify explicity
   . Do not raise any excpetion in case of validation error
   . Catch in start task any unexcepted Exceptions a set deployment
     in incosistance state

*) Refactor CLI
   . Properly handle new behaviour of verification
   . Replace table on task start to just message
   . Add HINTs to task detailed command

*) Add unit test for checking doc samples

*) Improve benchmark engine LOGing

blueprint benchmark-new-task-config

Change-Id: I23d3f6b3439fdb44946a7c2491d5a9b3559dc671
2014-03-17 17:19:53 +04:00
..
__init__.py Add rally.sample.conf to project 2013-08-14 14:08:09 +04:00
consts.py New task config and verification refactoring 2014-03-17 17:19:53 +04:00