257 lines
8.5 KiB
Python
Raw Normal View History

# Copyright (c) 2014 Mirantis Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import copy
import traceback
import uuid
import eventlet.debug
from oslo_config import cfg
from oslo_log import log as logging
import oslo_messaging as messaging
from oslo_messaging import target
from oslo_serialization import jsonutils
Nova Network support Adds a support for Nova Network if Neutron is not present in the current OpenStack deployment. Supporting the Nova Network requires modifications in three different parts of generated Heat Stack: 1) Generated Security Groups and their rules should be of type 'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup' 2) Security Group assignments should go to security_groups property of Instance resource, not the network port (as port concept is not present when using NovaNetwork) 3) FloatingIP should be of type OS::Nova::FloatingIP and should be associated with an Instance by OS::Nova::FloatingIPAssociation resource. To achieve p1 a SecurityGroupManager class of Core Library is made abstract and is inherited by two concrete implementations: NeutronSecurityGroupManager (containing the old MuranoPL code which generated templates based on OS::Neutron::SecurityGroup) and a new AwsSecurityGroupManager, which generates AWS-compliant firewall rules which are consumed by NovaNetwork. The particular concreate instance of this class is generated by the default network of environment: Network class has got a new method called generateSecurityGroupManager which returns an appropriate implementation. For pp 2-3 a new inheritor of Network class has been added to the Core Library: an io.murano.resources.NovaNetwork. It generates FloatingIP association resources if needed and returns a securityGroupName object as one of the outputs of its joinInstance methods. The Instance class has been modified to properly handle these types of outputs. The instance of the NovaNetwork class is generated at the API side when a new Environment is created and a is assigned to the defaultNetworks.environment property of the environment if the neutron is not defined in keystone. Also this change moves the auth_utils module from engine to common, as Keystone Client it contains is now used by the API process as well. This changed is based on some of the code from the outdated changeset I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7 Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com> Co-Authored-By: Stan Lagun <slagun@mirantis.com> DocImpact Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1 Implements-Blueprint: nova-network-support
2015-04-02 20:37:09 +03:00
from murano.common import auth_utils
from murano.common.helpers import token_sanitizer
Initial implementation of Plugable Classes Adds a PluginLoader which loads classes defined as stevedore plugins at io.murano.extension namespace and registers them as MuranoPL classes in class loader. Modifies the ClientManager class to make the _get_client method public, so other code may use it to add custom clients. This is useful for plugins which may define their own clients. Modifies the configuration settings adding 'enabled_plugins' parameter to control which of the installed plugins are active. Adds an example plugin which encapsulates Glance interaction logic to: * List all available glance images * Get Image by ID * Get Image by Name * Output image info with murano-related metadata Adds a demo application which demonstrates the usage of plugin. The app consist of the following components: * An 'ImageValidatorMixin' class which inherits generic instance class (io.murano.resources.Instance) and adds a method capable to validate Instance's image for having appropriate murano metadata type. This class may be used as a mixin when added to inheritance hierarchy of concrete instance classes. * A concrete class called DemoInstance which inherits from io.murano.resources.LinuxMuranoInstance and ImageValidatorMixin to add the image validation logic to standard Murano-enabled Linux-based instance. * An application which deploys a single VM using the DemoInstance class if the tag on user-supplied image matches the user-supplied constant. The ImageValidatorMixin demonstrates the instantiation of plugin-provided class and its usage, as well as handling of exception which may be thrown if the plugin is not installed in the environment. Change-Id: I978339d87033bbe38dad4c2102612d8f3a1eb3c3 Implements-blueprint: plugable-classes
2015-03-17 17:25:19 +03:00
from murano.common import plugin_loader
from murano.common import rpc
from murano.dsl import dsl_exception
from murano.dsl import serializer
from murano.engine import environment
from murano.engine import executor as engine_executor
from murano.engine import package_loader
from murano.engine.system import status_reporter
from murano.common.i18n import _LI, _LE, _LW
from murano.policy import model_policy_enforcer as enforcer
CONF = cfg.CONF
Initial implementation of Plugable Classes Adds a PluginLoader which loads classes defined as stevedore plugins at io.murano.extension namespace and registers them as MuranoPL classes in class loader. Modifies the ClientManager class to make the _get_client method public, so other code may use it to add custom clients. This is useful for plugins which may define their own clients. Modifies the configuration settings adding 'enabled_plugins' parameter to control which of the installed plugins are active. Adds an example plugin which encapsulates Glance interaction logic to: * List all available glance images * Get Image by ID * Get Image by Name * Output image info with murano-related metadata Adds a demo application which demonstrates the usage of plugin. The app consist of the following components: * An 'ImageValidatorMixin' class which inherits generic instance class (io.murano.resources.Instance) and adds a method capable to validate Instance's image for having appropriate murano metadata type. This class may be used as a mixin when added to inheritance hierarchy of concrete instance classes. * A concrete class called DemoInstance which inherits from io.murano.resources.LinuxMuranoInstance and ImageValidatorMixin to add the image validation logic to standard Murano-enabled Linux-based instance. * An application which deploys a single VM using the DemoInstance class if the tag on user-supplied image matches the user-supplied constant. The ImageValidatorMixin demonstrates the instantiation of plugin-provided class and its usage, as well as handling of exception which may be thrown if the plugin is not installed in the environment. Change-Id: I978339d87033bbe38dad4c2102612d8f3a1eb3c3 Implements-blueprint: plugable-classes
2015-03-17 17:25:19 +03:00
RPC_SERVICE = None
Initial implementation of Plugable Classes Adds a PluginLoader which loads classes defined as stevedore plugins at io.murano.extension namespace and registers them as MuranoPL classes in class loader. Modifies the ClientManager class to make the _get_client method public, so other code may use it to add custom clients. This is useful for plugins which may define their own clients. Modifies the configuration settings adding 'enabled_plugins' parameter to control which of the installed plugins are active. Adds an example plugin which encapsulates Glance interaction logic to: * List all available glance images * Get Image by ID * Get Image by Name * Output image info with murano-related metadata Adds a demo application which demonstrates the usage of plugin. The app consist of the following components: * An 'ImageValidatorMixin' class which inherits generic instance class (io.murano.resources.Instance) and adds a method capable to validate Instance's image for having appropriate murano metadata type. This class may be used as a mixin when added to inheritance hierarchy of concrete instance classes. * A concrete class called DemoInstance which inherits from io.murano.resources.LinuxMuranoInstance and ImageValidatorMixin to add the image validation logic to standard Murano-enabled Linux-based instance. * An application which deploys a single VM using the DemoInstance class if the tag on user-supplied image matches the user-supplied constant. The ImageValidatorMixin demonstrates the instantiation of plugin-provided class and its usage, as well as handling of exception which may be thrown if the plugin is not installed in the environment. Change-Id: I978339d87033bbe38dad4c2102612d8f3a1eb3c3 Implements-blueprint: plugable-classes
2015-03-17 17:25:19 +03:00
PLUGIN_LOADER = None
LOG = logging.getLogger(__name__)
eventlet.debug.hub_exceptions(False)
def _prepare_rpc_service(server_id):
endpoints = [TaskProcessingEndpoint()]
transport = messaging.get_transport(CONF)
s_target = target.Target('murano', 'tasks', server=server_id)
return messaging.get_rpc_server(transport, s_target, endpoints, 'eventlet')
def get_rpc_service():
global RPC_SERVICE
if RPC_SERVICE is None:
RPC_SERVICE = _prepare_rpc_service(str(uuid.uuid4()))
return RPC_SERVICE
Initial implementation of Plugable Classes Adds a PluginLoader which loads classes defined as stevedore plugins at io.murano.extension namespace and registers them as MuranoPL classes in class loader. Modifies the ClientManager class to make the _get_client method public, so other code may use it to add custom clients. This is useful for plugins which may define their own clients. Modifies the configuration settings adding 'enabled_plugins' parameter to control which of the installed plugins are active. Adds an example plugin which encapsulates Glance interaction logic to: * List all available glance images * Get Image by ID * Get Image by Name * Output image info with murano-related metadata Adds a demo application which demonstrates the usage of plugin. The app consist of the following components: * An 'ImageValidatorMixin' class which inherits generic instance class (io.murano.resources.Instance) and adds a method capable to validate Instance's image for having appropriate murano metadata type. This class may be used as a mixin when added to inheritance hierarchy of concrete instance classes. * A concrete class called DemoInstance which inherits from io.murano.resources.LinuxMuranoInstance and ImageValidatorMixin to add the image validation logic to standard Murano-enabled Linux-based instance. * An application which deploys a single VM using the DemoInstance class if the tag on user-supplied image matches the user-supplied constant. The ImageValidatorMixin demonstrates the instantiation of plugin-provided class and its usage, as well as handling of exception which may be thrown if the plugin is not installed in the environment. Change-Id: I978339d87033bbe38dad4c2102612d8f3a1eb3c3 Implements-blueprint: plugable-classes
2015-03-17 17:25:19 +03:00
def get_plugin_loader():
global PLUGIN_LOADER
if PLUGIN_LOADER is None:
PLUGIN_LOADER = plugin_loader.PluginLoader()
return PLUGIN_LOADER
class TaskProcessingEndpoint(object):
@classmethod
def handle_task(cls, context, task):
result = cls.execute(task)
rpc.api().process_result(result, task['id'])
@staticmethod
def execute(task):
s_task = token_sanitizer.TokenSanitizer().sanitize(task)
LOG.info(_LI('Starting processing task: {task_desc}').format(
task_desc=jsonutils.dumps(s_task)))
result = None
reporter = status_reporter.StatusReporter(task['id'])
try:
task_executor = TaskExecutor(task, reporter)
result = task_executor.execute()
return result
finally:
LOG.info(_LI('Finished processing task: {task_desc}').format(
task_desc=jsonutils.dumps(result)))
class TaskExecutor(object):
@property
def action(self):
return self._action
@property
def environment(self):
return self._environment
@property
def model(self):
return self._model
def __init__(self, task, reporter=None):
if reporter is None:
reporter = status_reporter.StatusReporter(task['id'])
self._action = task.get('action')
self._model = task['model']
self._environment = environment.Environment()
self._environment.token = task['token']
self._environment.tenant_id = task['tenant_id']
self._environment.system_attributes = self._model.get('SystemData', {})
self._reporter = reporter
self._model_policy_enforcer = enforcer.ModelPolicyEnforcer(
self._environment)
def execute(self):
try:
self._create_trust()
except Exception as e:
return self.exception_result(e, None, '<system>')
murano_client_factory = \
lambda: self._environment.clients.get_murano_client()
with package_loader.CombinedPackageLoader(
murano_client_factory,
self._environment.tenant_id) as pkg_loader:
result = self._execute(pkg_loader)
self._model['SystemData'] = self._environment.system_attributes
result['model'] = self._model
if (not self._model.get('Objects')
and not self._model.get('ObjectsCopy')):
try:
self._delete_trust()
except Exception:
LOG.warn(_LW('Cannot delete trust'), exc_info=True)
return result
def _execute(self, pkg_loader):
get_plugin_loader().register_in_loader(pkg_loader)
executor = engine_executor.Executor(
pkg_loader, self.environment)
try:
obj = executor.load(self.model)
except Exception as e:
return self.exception_result(e, None, '<load>')
if obj is not None:
try:
self._validate_model(obj.object, self.action, pkg_loader)
except Exception as e:
return self.exception_result(e, obj, '<validate>')
try:
LOG.info(_LI('Invoking pre-cleanup hooks'))
self.environment.start()
executor.cleanup(self._model)
except Exception as e:
return self.exception_result(e, obj, '<GC>')
finally:
LOG.info(_LI('Invoking post-cleanup hooks'))
self.environment.finish()
self._model['ObjectsCopy'] = copy.deepcopy(self._model.get('Objects'))
action_result = None
if self.action:
try:
LOG.info(_LI('Invoking pre-execution hooks'))
self.environment.start()
try:
action_result = self._invoke(executor)
finally:
try:
self._model = serializer.serialize_model(obj, executor)
except Exception as e:
return self.exception_result(e, None, '<model>')
except Exception as e:
return self.exception_result(e, obj, self.action['method'])
finally:
LOG.info(_LI('Invoking post-execution hooks'))
self.environment.finish()
try:
action_result = serializer.serialize(action_result)
except Exception as e:
return self.exception_result(e, None, '<result>')
return {
'action': {
'result': action_result,
'isException': False
}
}
def exception_result(self, exception, root, method_name):
if isinstance(exception, dsl_exception.MuranoPlException):
LOG.error('\n' + exception.format(prefix=' '))
exception_traceback = exception.format()
else:
exception_traceback = traceback.format_exc()
LOG.exception(
_LE("Exception %(exc)s occurred"
" during invocation of %(method)s"),
{'exc': exception, 'method': method_name})
self._reporter.report_error(root, str(exception))
return {
'action': {
'isException': True,
'result': {
'message': str(exception),
'details': exception_traceback
}
}
}
def _validate_model(self, obj, action, package_loader):
if CONF.engine.enable_model_policy_enforcer:
if action is not None and action['method'] == 'deploy':
self._model_policy_enforcer.validate(
obj.to_dictionary(), package_loader)
def _invoke(self, mpl_executor):
obj = mpl_executor.object_store.get(self.action['object_id'])
method_name, kwargs = self.action['method'], self.action['args']
if obj is not None:
return obj.type.invoke(method_name, mpl_executor, obj, (), kwargs)
def _create_trust(self):
if not CONF.engine.use_trusts:
return
trust_id = self._environment.system_attributes.get('TrustId')
if not trust_id:
Nova Network support Adds a support for Nova Network if Neutron is not present in the current OpenStack deployment. Supporting the Nova Network requires modifications in three different parts of generated Heat Stack: 1) Generated Security Groups and their rules should be of type 'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup' 2) Security Group assignments should go to security_groups property of Instance resource, not the network port (as port concept is not present when using NovaNetwork) 3) FloatingIP should be of type OS::Nova::FloatingIP and should be associated with an Instance by OS::Nova::FloatingIPAssociation resource. To achieve p1 a SecurityGroupManager class of Core Library is made abstract and is inherited by two concrete implementations: NeutronSecurityGroupManager (containing the old MuranoPL code which generated templates based on OS::Neutron::SecurityGroup) and a new AwsSecurityGroupManager, which generates AWS-compliant firewall rules which are consumed by NovaNetwork. The particular concreate instance of this class is generated by the default network of environment: Network class has got a new method called generateSecurityGroupManager which returns an appropriate implementation. For pp 2-3 a new inheritor of Network class has been added to the Core Library: an io.murano.resources.NovaNetwork. It generates FloatingIP association resources if needed and returns a securityGroupName object as one of the outputs of its joinInstance methods. The Instance class has been modified to properly handle these types of outputs. The instance of the NovaNetwork class is generated at the API side when a new Environment is created and a is assigned to the defaultNetworks.environment property of the environment if the neutron is not defined in keystone. Also this change moves the auth_utils module from engine to common, as Keystone Client it contains is now used by the API process as well. This changed is based on some of the code from the outdated changeset I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7 Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com> Co-Authored-By: Stan Lagun <slagun@mirantis.com> DocImpact Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1 Implements-Blueprint: nova-network-support
2015-04-02 20:37:09 +03:00
trust_id = auth_utils.create_trust(self._environment.token,
self._environment.tenant_id)
self._environment.system_attributes['TrustId'] = trust_id
self._environment.trust_id = trust_id
def _delete_trust(self):
trust_id = self._environment.trust_id
if trust_id:
Nova Network support Adds a support for Nova Network if Neutron is not present in the current OpenStack deployment. Supporting the Nova Network requires modifications in three different parts of generated Heat Stack: 1) Generated Security Groups and their rules should be of type 'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup' 2) Security Group assignments should go to security_groups property of Instance resource, not the network port (as port concept is not present when using NovaNetwork) 3) FloatingIP should be of type OS::Nova::FloatingIP and should be associated with an Instance by OS::Nova::FloatingIPAssociation resource. To achieve p1 a SecurityGroupManager class of Core Library is made abstract and is inherited by two concrete implementations: NeutronSecurityGroupManager (containing the old MuranoPL code which generated templates based on OS::Neutron::SecurityGroup) and a new AwsSecurityGroupManager, which generates AWS-compliant firewall rules which are consumed by NovaNetwork. The particular concreate instance of this class is generated by the default network of environment: Network class has got a new method called generateSecurityGroupManager which returns an appropriate implementation. For pp 2-3 a new inheritor of Network class has been added to the Core Library: an io.murano.resources.NovaNetwork. It generates FloatingIP association resources if needed and returns a securityGroupName object as one of the outputs of its joinInstance methods. The Instance class has been modified to properly handle these types of outputs. The instance of the NovaNetwork class is generated at the API side when a new Environment is created and a is assigned to the defaultNetworks.environment property of the environment if the neutron is not defined in keystone. Also this change moves the auth_utils module from engine to common, as Keystone Client it contains is now used by the API process as well. This changed is based on some of the code from the outdated changeset I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7 Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com> Co-Authored-By: Stan Lagun <slagun@mirantis.com> DocImpact Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1 Implements-Blueprint: nova-network-support
2015-04-02 20:37:09 +03:00
auth_utils.delete_trust(self._environment.trust_id)
self._environment.system_attributes['TrustId'] = None
self._environment.trust_id = None