openstack/rally
Code Issues Proposed changes

Introudce benchmark.context() decorator

Browse Source 
Before this patch we have to specify:

__ctx_name__, __ctx_order__ and __ctx_hidden__ by hands as attributes
of context class.

This thing produced few issues:

1) It looks ugly
2) It is absolutelly unclear. So you don't know what they mean, and
   there is no easy way to find information about it. So now it's
   much simpler e.g. you should just take a look at decorator
   implementation
3) This was source of bugs, cause it was not tested by syntax
   of language => so we have to have mad tests to check that
   all __ctx_name__, __ctx_order__ were specified

* Use decorator instead of direct setting attributes

* Rename attributes:
  __ctx_name__ -> _ctx_name
  __ctx_order__ -> _ctx_order
  __ctx_hidden__ -> _ctx_hidden

  This was done cause of unification with other parts of code

* Improve a bit "servers" get uuid of image and flavor only once

* Impove order of context

* Make secgroup and allow_ssh not hidden context. There is no
  need to hide them, and probably somebody would like to use
  them.

* Docs are updated

Change-Id: Ib089b398ba0c7f54d5246eb6ba29f5bbcfd2deee
This commit is contained in:
Boris Pavlovic 2014-11-27 05:02:07 +04:00
parent d4b843d6e5
commit 83c9309ded
18 changed files with 86 additions and 111 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
doc/samples/plugins/context/context_plugin.py
View File

@ -9,6 +9,7 @@ LOG = logging.getLogger(__name__)
CONF = cfg.CONF
@base.context(name="create_flavor", order=1000)
class CreateFlavorContext(base.Context):
"""This sample create flavor with specified options before task starts and
delete it after task completion.
@ -17,10 +18,6 @@ class CreateFlavorContext(base.Context):
rally.benchmark.context.base.Context
"""
__ctx_name__ = "create_flavor"
__ctx_order__ = 1000
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
"$schema": utils.JSON_SCHEMA,

14
doc/source/concepts.rst
View File

@ -264,15 +264,14 @@ From the developer's view, contexts management is implemented via **Context clas
.. parsed-literal::
from rally import utils
from rally.benchmark.context import base
@base.context(name="your_context", *# Corresponds to the context field name in task configuration files*
order=100500, *# a number specifying the priority with which the context should be set up*
hidden=False) *# True if the context cannot be configured through the input task file*
class YourContext(base.Context):
*"""Yet another context class."""*
__ctx_name__ = "your_context" *# Corresponds to the context field name in task configuration files*
__ctx_order__ = xxx *# a 3-digit number specifying the priority with which the context should be set up*
__ctx_hidden__ = False *# True if the context cannot be configured through the task configuration file*
*# The schema of the context configuration format*
CONFIG_SCHEMA = {
"type": "object",
@ -312,9 +311,10 @@ Consequently, the algorithm of initiating the contexts can be roughly seen as fo
context2.cleanup()
context1.cleanup()
- where the order of contexts in which they are set up depends on the value of their *__ctx_order__* attribute. Contexts with lower *__ctx_order__* have higher priority: *1xx* contexts are reserved for users-related stuff (e.g. users/tenants creation, roles assignment etc.), *2xx* - for quotas etc.
- where the order of contexts in which they are set up depends on the value of their *order* attribute. Contexts with lower *order* have higher priority: *1xx* contexts are reserved for users-related stuff (e.g. users/tenants creation, roles assignment etc.), *2xx* - for quotas etc.
The *__ctx_hidden__* attribute defines whether the context should be a *hidden* one. **Hidden contexts** cannot be configured by end-users through the task configuration file as shown above, but should be specified by a benchmark scenario developer through a special *@base.scenario(context={...})* decorator. Hidden contexts are typically needed to satisfy some specific benchmark scenario-specific needs, which don't require the end-user's attention. For example, the hidden **"allow_ssh" context** (:mod:`rally.benchmark.context.secgroup`) is used in the **VMTasks.boot_runcommand_delete benchmark scenario** (:mod:`rally.benchmark.scenarios.vm.vmtasks`) to enable the SSH access to the servers. The fact that end-users do not have to worry about such details about SSH while launching this benchmark scenarios obviously makes their life easier and shows why hiddent contexts are of great importance in Rally.
The *hidden* attribute defines whether the context should be a *hidden* one. **Hidden contexts** cannot be configured by end-users through the task configuration file as shown above, but should be specified by a benchmark scenario developer through a special *@base.scenario(context={...})* decorator. Hidden contexts are typically needed to satisfy some specific benchmark scenario-specific needs, which don't require the end-user's attention. For example, the hidden **"cleanup" context** (:mod:`rally.benchmark.context.cleanup.context`) is used to make generic cleanup after running benchmark. So user can't change
it configuration via task and break his cloud.
If you want to dive deeper, also see the context manager (:mod:`rally.benchmark.context.base`) class that actually implements the algorithm described above.

40
rally/benchmark/context/base.py
View File

@ -25,7 +25,31 @@ from rally import utils
LOG = logging.getLogger(__name__)
def context(name, order, hidden=False):
"""Context class wrapper.
Each context class has to be wrapped by context() wrapper. It
sets essential configuration of context classes. Actually this wrapper just
adds attributes to the class.
:param name: Name of the class, used in the input task
:param order: As far as we can use multiple context classes that sometimes
depend on each other we have to specify order of execution.
Contexts with smaller order are run first
:param hidden: If it is true you won't be able to specify context via
task config
"""
def wrapper(cls):
cls._ctx_name = name
cls._ctx_order = order
cls._ctx_hidden = hidden
return cls
return wrapper
@six.add_metaclass(abc.ABCMeta)
@context(name="base", order=0, hidden=True)
class Context(object):
"""This class is a factory for context classes.
@ -39,14 +63,10 @@ class Context(object):
4) Order of context creation
"""
__ctx_name__ = "base"
__ctx_order__ = 0
__ctx_hidden__ = True
CONFIG_SCHEMA = {}
def __init__(self, context):
self.config = context.get("config", {}).get(self.__ctx_name__, {})
self.config = context.get("config", {}).get(self.get_name(), {})
self.context = context
self.task = context["task"]
@ -61,8 +81,8 @@ class Context(object):
@classmethod
def validate(cls, config, non_hidden=False):
if non_hidden and cls.__ctx_hidden__:
raise exceptions.NoSuchContext(name=cls.__ctx_name__)
if non_hidden and cls._ctx_hidden:
raise exceptions.NoSuchContext(name=cls.get_name())
jsonschema.validate(config, cls.CONFIG_SCHEMA)
@classmethod
@ -71,17 +91,17 @@ class Context(object):
@classmethod
def get_name(cls):
return cls.__ctx_name__
return cls._ctx_name
@classmethod
def get_order(cls):
return cls.__ctx_order__
return cls._ctx_order
@staticmethod
def get_by_name(name):
"""Return Context class by name."""
for context in utils.itersubclasses(Context):
if name == context.__ctx_name__:
if name == context.get_name():
return context
raise exceptions.NoSuchContext(name=name)

10
rally/benchmark/context/cleanup/context.py
View File

@ -43,13 +43,10 @@ class CleanupMixin(object):
pass
@base.context(name="admin_cleanup", order=200, hidden=True)
class AdminCleanup(CleanupMixin, base.Context):
"""Context class for admin resources cleanup."""
__ctx_hidden__ = True
__ctx_name__ = "admin_cleanup"
__ctx_order__ = 200
@classmethod
def validate(cls, config, non_hidden=False):
super(AdminCleanup, cls).validate(config, non_hidden)
@ -70,13 +67,10 @@ class AdminCleanup(CleanupMixin, base.Context):
users=self.context.get("users", []))
@base.context(name="cleanup", order=201, hidden=True)
class UserCleanup(CleanupMixin, base.Context):
"""Context class for user resources cleanup."""
__ctx_hidden__ = True
__ctx_name__ = "cleanup"
__ctx_order__ = 201
@classmethod
def validate(cls, config, non_hidden=False):
super(UserCleanup, cls).validate(config, non_hidden)

5
rally/benchmark/context/images.py
View File

@ -28,13 +28,10 @@ from rally import utils as rutils
LOG = logging.getLogger(__name__)
@base.context(name="images", order=410)
class ImageGenerator(base.Context):
"""Context class for adding images to each user for benchmarks."""
__ctx_name__ = "images"
__ctx_order__ = 411
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
"$schema": rutils.JSON_SCHEMA,

5
rally/benchmark/context/keypair.py
View File

@ -26,11 +26,8 @@ from rally import utils
LOG = logging.getLogger(__name__)
@base.context(name="keypair", order=301)
class Keypair(base.Context):
__ctx_name__ = "keypair"
__ctx_order__ = 300
__ctx_hidden__ = True
KEYPAIR_NAME = "rally_ssh_key"
def _generate_keypair(self, endpoint):

5
rally/benchmark/context/quotas/quotas.py
View File

@ -27,13 +27,10 @@ from rally import utils
LOG = logging.getLogger(__name__)
@base.context(name="quotas", order=300)
class Quotas(base.Context):
"""Context class for updating benchmarks' tenants quotas."""
__ctx_name__ = "quotas"
__ctx_order__ = 210
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
"$schema": utils.JSON_SCHEMA,

5
rally/benchmark/context/roles.py
View File

@ -24,13 +24,10 @@ from rally import utils as rutils
LOG = logging.getLogger(__name__)
@base.context(name="roles", order=303)
class RoleGenerator(base.Context):
"""Context class for adding temporary roles for benchmarks."""
__ctx_name__ = "roles"
__ctx_order__ = 101
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "array",
"$schema": rutils.JSON_SCHEMA,

5
rally/benchmark/context/sahara/sahara_cluster.py
View File

@ -31,13 +31,10 @@ CONF = cfg.CONF
LOG = logging.getLogger(__name__)
@base.context(name="sahara_cluster", order=441)
class SaharaCluster(base.Context):
"""Context class for setting up the Cluster an EDP job."""
__ctx_name__ = "sahara_cluster"
__ctx_order__ = 413
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
"$schema": rutils.JSON_SCHEMA,

5
rally/benchmark/context/sahara/sahara_edp.py
View File

@ -27,13 +27,10 @@ from rally import utils as rutils
LOG = logging.getLogger(__name__)
@base.context(name="sahara_edp", order=442)
class SaharaEDP(base.Context):
"""Context class for setting up the environment for an EDP job."""
__ctx_name__ = "sahara_edp"
__ctx_order__ = 414
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
"$schema": rutils.JSON_SCHEMA,

5
rally/benchmark/context/sahara/sahara_image.py
View File

@ -25,13 +25,10 @@ from rally import utils as rutils
LOG = logging.getLogger(__name__)
@base.context(name="sahara_image", order=440)
class SaharaImage(base.Context):
"""Context class for adding and tagging Sahara images."""
__ctx_name__ = "sahara_image"
__ctx_order__ = 412
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
"$schema": rutils.JSON_SCHEMA,

4
rally/benchmark/context/secgroup.py
View File

@ -77,10 +77,8 @@ def _prepare_open_secgroup(endpoint):
return rally_open
@base.context(name="allow_ssh", order=302)
class AllowSSH(base.Context):
__ctx_name__ = "allow_ssh"
__ctx_order__ = 301
__ctx_hidden__ = True
def __init__(self, context):
super(AllowSSH, self).__init__(context)

29
rally/benchmark/context/servers.py
View File

@ -25,14 +25,12 @@ from rally import utils as rutils
LOG = logging.getLogger(__name__)
@base.context(name="servers", order=430)
class ServerGenerator(base.Context):
"""Context class for adding temporary servers for benchmarks.
Servers are added for each tenant.
"""
__ctx_name__ = "servers"
__ctx_order__ = 412
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
@ -75,6 +73,13 @@ class ServerGenerator(base.Context):
servers_per_tenant = self.config["servers_per_tenant"]
current_tenants = []
clients = osclients.Clients(self.context["users"][0]["endpoint"])
image_id = types.ImageResourceType.transform(clients=clients,
resource_config=image)
flavor_id = types.FlavorResourceType.transform(clients=clients,
resource_config=flavor)
for user in self.context["users"]:
if user["tenant_id"] not in current_tenants:
LOG.debug("Booting servers for user tenant %s "
@ -82,11 +87,6 @@ class ServerGenerator(base.Context):
current_tenants.append(user["tenant_id"])
clients = osclients.Clients(user["endpoint"])
nova_scenario = nova_utils.NovaScenario(clients=clients)
image_id = types.ImageResourceType.transform(
clients=clients, resource_config=image)
flavor_id = types.FlavorResourceType.transform(
clients=clients, resource_config=flavor)
server_name_prefix = nova_scenario._generate_random_name()
LOG.debug("Calling _boot_servers with server_name_prefix=%s "
@ -97,8 +97,7 @@ class ServerGenerator(base.Context):
current_servers = []
servers = nova_scenario._boot_servers(
server_name_prefix, image_id,
flavor_id,
server_name_prefix, image_id, flavor_id,
servers_per_tenant)
for server in servers:
@ -107,11 +106,11 @@ class ServerGenerator(base.Context):
LOG.debug("Adding booted servers %s to context"
% current_servers)
self.context["servers"].append(
{"server_ids": current_servers,
"endpoint": user["endpoint"],
"tenant_id": user["tenant_id"]}
)
self.context["servers"].append({
"server_ids": current_servers,
"endpoint": user["endpoint"],
"tenant_id": user["tenant_id"]
})
@rutils.log_task_wrapper(LOG.info, _("Exit context: `Servers`"))
def cleanup(self):

5
rally/benchmark/context/tempest.py
View File

@ -29,11 +29,8 @@ from rally.verification.verifiers.tempest import tempest
LOG = logging.getLogger(__name__)
@base.context(name="tempest", order=666, hidden=True)
class Tempest(base.Context):
__ctx_name__ = "tempest"
__ctx_order__ = 666
__ctx_hidden__ = True
@utils.log_task_wrapper(LOG.info, _("Enter context: `tempest`"))
def setup(self):
self.verifier = tempest.Tempest(self.task.task.deployment_uuid)

9
rally/benchmark/context/users.py
View File

@ -52,13 +52,10 @@ CONF.register_opts(context_opts,
title='benchmark context options'))
@base.context(name="users", order=100)
class UserGenerator(base.Context):
"""Context class for generating temporary users/tenants for benchmarks."""
__ctx_name__ = "users"
__ctx_order__ = 100
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
"$schema": rutils.JSON_SCHEMA,
@ -239,7 +236,7 @@ class UserGenerator(base.Context):
if len(self.context["tenants"]) < self.config["tenants"]:
raise exceptions.ContextSetupFailure(
ctx_name=self.__ctx_name__,
ctx_name=self.get_name(),
msg=_("Failed to create the requested number of tenants."))
users_num = self.config["users_per_tenant"] * self.config["tenants"]
@ -249,7 +246,7 @@ class UserGenerator(base.Context):
if len(self.context["users"]) < users_num:
raise exceptions.ContextSetupFailure(
ctx_name=self.__ctx_name__,
ctx_name=self.get_name(),
msg=_("Failed to create the requested number of users."))
@rutils.log_task_wrapper(LOG.info, _("Exit context: `users`"))

5
rally/benchmark/context/volumes.py
View File

@ -24,13 +24,10 @@ from rally import utils as rutils
LOG = logging.getLogger(__name__)
@base.context(name="volumes", order=420)
class VolumeGenerator(base.Context):
"""Context class for adding volumes to each user for benchmarks."""
__ctx_name__ = "volumes"
__ctx_order__ = 500
__ctx_hidden__ = False
CONFIG_SCHEMA = {
"type": "object",
"$schema": rutils.JSON_SCHEMA,

37
tests/unit/benchmark/context/test_base.py
View File

@ -19,7 +19,6 @@ import mock
from rally.benchmark.context import base
from rally import exceptions
from rally import utils
from tests.unit import fakes
from tests.unit import test
@ -58,14 +57,19 @@ class BaseContextTestCase(test.TestCase):
@mock.patch("rally.benchmark.context.base.utils.itersubclasses")
def test_get_by_name(self, mock_itersubclasses):
A = mock.MagicMock()
A.__ctx_name__ = "a"
B = mock.MagicMock()
B.__ctx_name__ = "b"
mock_itersubclasses.return_value = [A, B]
self.assertEqual(A, base.Context.get_by_name("a"))
self.assertEqual(B, base.Context.get_by_name("b"))
@base.context(name="some_fake1", order=1)
class SomeFake1(base.Context):
pass
@base.context(name="some_fake2", order=1)
class SomeFake2(base.Context):
pass
mock_itersubclasses.return_value = [SomeFake1, SomeFake2]
self.assertEqual(SomeFake1, base.Context.get_by_name("some_fake1"))
self.assertEqual(SomeFake2, base.Context.get_by_name("some_fake2"))
@mock.patch("rally.benchmark.context.base.utils.itersubclasses")
def test_get_by_name_non_existing(self, mock_itersubclasses):
@ -107,18 +111,11 @@ class BaseContextTestCase(test.TestCase):
ctx.cleanup.assert_called_once_with()
def test_all_context_have_ctxt_order(self):
for cls in utils.itersubclasses(base.Context):
self.assertNotEqual(cls.get_order(), 0, str(cls))
def test_all_context_have_ctxt_name(self):
for cls in utils.itersubclasses(base.Context):
self.assertNotEqual(cls.get_name(), "base", str(cls))
def test_lt(self):
@base.context(name="fake_lt", order=fakes.FakeContext.get_order() - 1)
class FakeLowerContext(fakes.FakeContext):
__ctx_order__ = fakes.FakeContext.get_order() - 1
pass
ctx = mock.MagicMock()
self.assertTrue(FakeLowerContext(ctx) < fakes.FakeContext(ctx))
@ -127,8 +124,9 @@ class BaseContextTestCase(test.TestCase):
def test_gt(self):
@base.context(name="fake_gt", order=fakes.FakeContext.get_order() + 1)
class FakeBiggerContext(fakes.FakeContext):
__ctx_order__ = fakes.FakeContext.get_order() + 1
pass
ctx = mock.MagicMock()
self.assertTrue(FakeBiggerContext(ctx) > fakes.FakeContext(ctx))
@ -137,8 +135,9 @@ class BaseContextTestCase(test.TestCase):
def test_eq(self):
@base.context(name="fake2", order=fakes.FakeContext.get_order() + 1)
class FakeOtherContext(fakes.FakeContext):
__ctx_order__ = fakes.FakeContext.get_order() + 1
pass
ctx = mock.MagicMock()
self.assertFalse(FakeOtherContext(ctx) == fakes.FakeContext(ctx))

4
tests/unit/fakes.py
View File

@ -1249,11 +1249,9 @@ class FakeTimer(rally_utils.Timer):
return 10
@base_ctx.context("fake", order=1)
class FakeContext(base_ctx.Context):
__ctx_name__ = "fake"
__ctx_order__ = 1
CONFIG_SCHEMA = {
"type": "object",
"$schema": rally_utils.JSON_SCHEMA,