openstack/murano
Code Issues Proposed changes

Nova Network support

Browse Source 
Adds a support for Nova Network if Neutron is not present in the
current OpenStack deployment.

Supporting the Nova Network requires modifications in three different
parts of generated Heat Stack:
	1) Generated Security Groups and their rules should be of type
	   'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup'
	2) Security Group assignments should go to security_groups property
	   of Instance resource, not the network port (as port concept is
	   not present when using NovaNetwork)
	3) FloatingIP should be of type OS::Nova::FloatingIP and should be
	   associated with an Instance by OS::Nova::FloatingIPAssociation
	   resource.

To achieve p1 a SecurityGroupManager class of Core Library is made
abstract and is inherited by two concrete implementations:
NeutronSecurityGroupManager (containing the old MuranoPL code which
generated templates based on OS::Neutron::SecurityGroup) and a new
AwsSecurityGroupManager, which generates AWS-compliant firewall rules
which are consumed by NovaNetwork.

The particular concreate instance of this class is generated by the
default network of environment: Network class has got a new method called
generateSecurityGroupManager which returns an appropriate implementation.

For pp 2-3 a new inheritor of Network class has been added to the Core
Library: an io.murano.resources.NovaNetwork. It generates FloatingIP
association resources if needed and returns a securityGroupName object
as one of the outputs of its joinInstance methods.
The Instance class has been modified to properly handle these types of
outputs.
The instance of the NovaNetwork class is generated at the API side
when a new Environment is created and a is assigned to the
defaultNetworks.environment property of the environment if the neutron
is not defined in keystone.

Also this change moves the auth_utils module from engine to common, as
Keystone Client it contains is now used by the API process as well.

This changed is based on some of the code from the outdated changeset
I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7

Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Co-Authored-By: Stan Lagun <slagun@mirantis.com>

DocImpact

Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1
Implements-Blueprint: nova-network-support
This commit is contained in:
Alexander Tivelkov 2015-04-02 20:37:09 +03:00
parent 521039d511
commit 796a0b2c9d
17 changed files with 273 additions and 61 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
doc/source/install/configure_network.rst
View File

@ -17,17 +17,38 @@
Network Configuration
---------------------
To work with Murano, tenant network in Openstack installation should be configured in a certain way.
This configuration may be set up automatically with the provision of several parameters in config file or manually.
Murano may work in various networking environments and is capable to detect the
current network configuration and choose the appropriate settings automatically.
However, some additional actions are required to support advanced scenarios.
Murano has advanced networking features that give you ability to not care about configuring networks
for your application. By default it will create an isolated network for each environment and join
Nova network support
^^^^^^^^^^^^^^^^^^^^
Nova Network is simplest networking solution, which has limited capabilities
but is available on any OpenStack deployment without the need to deploy any
additional components.
When a new Murano Environment is created, Murano checks if a dedicated
networking service (i.e. Neutron) exists in the current OpenStack deployment.
It relies on Keystone's service catalog for that.
If such a service is not present, Murano automatically falls back to Nova
Network. No further configuration is needed in this case, all the VMs spawned
by Murano will be joining the same Network.
Neutron support
^^^^^^^^^^^^^^^
If Neutron is installed, Murano enables its advanced networking features that
give you ability to not care about configuring networks for your application.
By default it will create an isolated network for each environment and join
all VMs needed by your application to that network. To install and configure application in
just spawned virtual machine Murano also requires a router connected to the external network.
Automatic network configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Automatic Neutron network configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To create router automatically, provide the following parameters in config file:
@ -49,8 +70,8 @@ To figure out the name of the external network, perform the following command:
During the first deploy, required networks and router with specified name will be created and set up.
Manual network configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Manual neutron network configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Step 1. Create public network

14
meta/io.murano/Classes/Environment.yaml
View File

@ -47,14 +47,22 @@ Methods:
- $generatedEnvironmentName: randomName()
- $.setAttr(generatedEnvironmentName, $generatedEnvironmentName)
- $this.agentListener: new(sys:AgentListener, name => $generatedEnvironmentName)
- $stackDescriptionFormat: 'This stack was generated by Murano for environment {0} (ID: {1})'
- $stackDescriptionFormat: 'This stack was generated by Murano for environment {0} (ID: {1})'
- $this.stack: new(sys:HeatStack,
name => 'murano-' + $generatedEnvironmentName,
description => $stackDescriptionFormat.format($.name, $.id()))
- $this.instanceNotifier: new(sys:InstanceNotifier, environment => $this)
- $this.reporter: new(sys:StatusReporter, environment => $this)
- $this.securityGroupManager: new(sys:SecurityGroupManager, environment => $this)
- $net: null
- If: $.defaultNetworks.environment != null
Then:
- $net: $.defaultNetworks.environment
- If: $.defaultNetworks.flat != null
Then:
- $net: $.defaultNetworks.flat
- If: $net != null
Then:
- $this.securityGroupManager: $net.generateSecurityGroupManager($this)
deploy:
Usage: Action

16
meta/io.murano/Classes/resources/Instance.yaml
View File

@ -180,8 +180,10 @@ Methods:
assignFloatingIp => $assignFip,
sharedIps => $sharedIps
)
- $.instanceTemplate: $.instanceTemplate.mergeWith($joinResult.template)
- If: $joinResult.template != null
Then:
- $.instanceTemplate: $.instanceTemplate.mergeWith($joinResult.template)
- If: $joinResult.portRef != null
Then:
- $template:
@ -192,6 +194,16 @@ Methods:
- port:
$joinResult.portRef
- $.instanceTemplate: $.instanceTemplate.mergeWith($template)
- If: $joinResult.secGroupName != null
Then:
- $template:
resources:
$.name:
properties:
security_groups:
- $joinResult.secGroupName
- $.instanceTemplate: $.instanceTemplate.mergeWith($template)
- $._floatingIpOutputName: coalesce($._floatingIpOutputName, $joinResult.instanceFipOutput)
- If: $assignFip and $joinResult.instanceFipOutput != null

5
meta/io.murano/Classes/resources/Network.yaml
View File

@ -18,3 +18,8 @@ Methods:
- sharedIps:
Contract:
- $.class(std:SharedIp)
generateSecurityGroupManager:
Arguments:
- environment:
Contract: $.class(std:Environment).notNull()

9
meta/io.murano/Classes/resources/NeutronNetworkBase.yaml
View File

@ -1,6 +1,7 @@
Namespaces:
=: io.murano.resources
std: io.murano
sys: io.murano.system
Name: NeutronNetworkBase
@ -80,3 +81,11 @@ Methods:
portRef:
get_resource: $portName
instanceFipOutput: $instanceFipOutput
generateSecurityGroupManager:
Arguments:
- environment:
Contract: $.class(std:Environment).notNull()
Body:
- Return: new(sys:NeutronSecurityGroupManager, environment => $environment)

57
meta/io.murano/Classes/resources/NovaNetwork.yaml Normal file
View File

@ -0,0 +1,57 @@
Namespaces:
=: io.murano.resources
std: io.murano
sys: io.murano.system
Name: NovaNetwork
Extends: Network
Methods:
joinInstance:
Arguments:
- instance:
Contract: $.class(Instance).notNull()
- securityGroupName:
Contract: $.string()
- assignFloatingIp:
Contract: $.bool().notNull()
- sharedIps:
Contract:
- $.class(std:SharedIp)
Body:
- $fipName: null
- $template: null
- $instanceFipOutput: null
- If: $assignFloatingIp
Then:
- $instanceFipOutput: $instance.name + '-floatingIPaddress'
- $fipName: format('fip-nn-{0}', $instance.name)
- $template:
resources:
$fipName:
type: 'OS::Nova::FloatingIP'
$fipName + 'Assignment':
type: 'OS::Nova::FloatingIPAssociation'
properties:
floating_ip:
get_resource: $fipName
server_id:
get_resource: $instance.name
outputs:
$instanceFipOutput:
value:
get_attr: [$fipName, ip]
description: format('Floating IP of {0}', $instance.name)
- Return:
template: $template
secGroupName:
get_resource: $securityGroupName
instanceFipOutput: instanceFipOutput
generateSecurityGroupManager:
Arguments:
- environment:
Contract: $.class(std:Environment).notNull()
Body:
- Return: new(sys:AwsSecurityGroupManager, environment => $environment)

57
meta/io.murano/Classes/system/AwsSecurityGroupManager.yaml Normal file
View File

@ -0,0 +1,57 @@
Namespaces:
=: io.murano.system
std: io.murano
Name: AwsSecurityGroupManager
Extends: SecurityGroupManager
Methods:
addGroupIngress:
Arguments:
- rules:
Contract:
- FromPort: $.int().notNull()
ToPort: $.int().notNull()
IpProtocol: $.string().notNull()
External: $.bool().notNull()
- groupName:
Contract: $.string().notNull()
Default: $this.defaultGroupName
Body:
- $ext_keys:
true:
ext_key: remote_ip_prefix
ext_val: '0.0.0.0/0'
false:
ext_key: remote_mode
ext_val: remote_group_id
- $stack: $.environment.stack
- $template:
resources:
$groupName:
type: 'AWS::EC2::SecurityGroup'
properties:
GroupDescription: format('Composite security group of Murano environment {0}', $.environment.name)
SecurityGroupIngress:
- FromPort: '-1'
ToPort: '-1'
IpProtocol: icmp
CidrIp: '0.0.0.0/0'
- $.environment.stack.updateTemplate($template)
- $ingress: $rules.select(dict(
FromPort => str($.FromPort),
ToPort => str($.ToPort),
IpProtocol => $.IpProtocol,
CidrIp => '0.0.0.0/0'
))
- $template:
resources:
$groupName:
type: 'AWS::EC2::SecurityGroup'
properties:
SecurityGroupIngress: $ingress
- $.environment.stack.updateTemplate($template)

19
meta/io.murano/Classes/SecurityGroupManager.yaml → meta/io.murano/Classes/system/NeutronSecurityGroupManager.yaml
View File

@ -1,16 +1,10 @@
Namespaces:
=: io.murano.system
std: io.murano
=: io.murano.system
std: io.murano
Name: SecurityGroupManager
Name: NeutronSecurityGroupManager
Properties:
environment:
Contract: $.class(std:Environment).notNull()
defaultGroupName:
Contract: $.string()
Default: format('MuranoSecurityGroup-{0}', $.environment.name)
Extends: SecurityGroupManager
Methods:
addGroupIngress:
@ -61,8 +55,3 @@ Methods:
properties:
rules: $ingress
- $.environment.stack.updateTemplate($template)

26
meta/io.murano/Classes/system/SecurityGroupManager.yaml Normal file
View File

@ -0,0 +1,26 @@
Namespaces:
=: io.murano.system
std: io.murano
Name: SecurityGroupManager
Properties:
environment:
Contract: $.class(std:Environment).notNull()
defaultGroupName:
Contract: $.string()
Default: format('MuranoSecurityGroup-{0}', $.environment.name)
Methods:
addGroupIngress:
Arguments:
- rules:
Contract:
- FromPort: $.int().notNull()
ToPort: $.int().notNull()
IpProtocol: $.string().notNull()
External: $.bool().notNull()
- groupName:
Contract: $.string().notNull()
Default: $this.defaultGroupName

5
meta/io.murano/manifest.yaml
View File

@ -22,7 +22,6 @@ Classes:
io.murano.SharedIp: SharedIp.yaml
io.murano.File: File.yaml
io.murano.system.SecurityGroupManager: SecurityGroupManager.yaml
io.murano.resources.Network: resources/Network.yaml
io.murano.resources.Instance: resources/Instance.yaml
@ -35,6 +34,7 @@ Classes:
io.murano.resources.NeutronNetworkBase: resources/NeutronNetworkBase.yaml
io.murano.resources.NeutronNetwork: resources/NeutronNetwork.yaml
io.murano.resources.ExistingNeutronNetwork: resources/ExistingNeutronNetwork.yaml
io.murano.resources.NovaNetwork: resources/NovaNetwork.yaml
io.murano.system.Agent: system/Agent.yaml
io.murano.system.AgentListener: system/AgentListener.yaml
@ -43,3 +43,6 @@ Classes:
io.murano.system.InstanceNotifier: system/InstanceNotifier.yaml
io.murano.system.StatusReporter: system/StatusReporter.yaml
io.murano.system.NetworkExplorer: system/NetworkExplorer.yaml
io.murano.system.SecurityGroupManager: system/SecurityGroupManager.yaml
io.murano.system.NeutronSecurityGroupManager: system/NeutronSecurityGroupManager.yaml
io.murano.system.AwsSecurityGroupManager: system/AwsSecurityGroupManager.yaml

2
murano/api/v1/environments.py
View File

@ -59,7 +59,7 @@ class Controller(object):
try:
environment = envs.EnvironmentServices.create(
body.copy(),
request.context.tenant)
request.context)
except db_exc.DBDuplicateEntry:
msg = _('Environment with specified name already exists')
LOG.exception(msg)

2
murano/api/v1/templates.py
View File

@ -183,7 +183,7 @@ class Controller(object):
try:
environment = envs.EnvironmentServices.create(
body.copy(), request.context.tenant)
body.copy(), request.context)
except db_exc.DBDuplicateEntry:
msg = _('Environment with specified name already exists')
LOG.exception(msg)

22
murano/engine/auth_utils.py → murano/common/auth_utils.py
View File

@ -18,11 +18,11 @@ from oslo.config import cfg
from oslo.utils import importutils
def get_client(environment):
def get_client(token, tenant_id):
settings = _get_keystone_settings()
kwargs = {
'token': environment.token,
'tenant_id': environment.tenant_id,
'token': token,
'tenant_id': tenant_id,
'auth_url': settings['auth_url']
}
kwargs.update(settings['ssl'])
@ -58,12 +58,12 @@ def _admin_client(trust_id=None, project_name=None):
return client
def get_client_for_trusts(environment):
return _admin_client(environment.trust_id)
def get_client_for_trusts(trust_id):
return _admin_client(trust_id)
def create_trust(environment):
client = get_client(environment)
def create_trust(token, tenant_id):
client = get_client(token, tenant_id)
settings = _get_keystone_settings()
trustee_id = get_client_for_admin(
@ -74,14 +74,14 @@ def create_trust(environment):
trustee_user=trustee_id,
impersonation=True,
role_names=roles,
project=environment.tenant_id)
project=tenant_id)
return trust.id
def delete_trust(environment):
keystone_client = get_client_for_trusts(environment)
keystone_client.trusts.delete(environment.trust_id)
def delete_trust(trust_id):
keystone_client = get_client_for_trusts(trust_id)
keystone_client.trusts.delete(trust_id)
def _get_keystone_settings():

7
murano/common/engine.py
View File

@ -21,6 +21,7 @@ from oslo import messaging
from oslo.messaging import target
from oslo.serialization import jsonutils
from murano.common import auth_utils
from murano.common import config
from murano.common.helpers import token_sanitizer
from murano.common import plugin_loader
@ -28,7 +29,6 @@ from murano.common import rpc
from murano.dsl import dsl_exception
from murano.dsl import executor
from murano.dsl import serializer
from murano.engine import auth_utils
from murano.engine import client_manager
from murano.engine import environment
from murano.engine import package_class_loader
@ -233,13 +233,14 @@ class TaskExecutor(object):
return
trust_id = self._environment.system_attributes.get('TrustId')
if not trust_id:
trust_id = auth_utils.create_trust(self._environment)
trust_id = auth_utils.create_trust(self._environment.token,
self._environment.tenant_id)
self._environment.system_attributes['TrustId'] = trust_id
self._environment.trust_id = trust_id
def _delete_trust(self):
trust_id = self._environment.trust_id
if trust_id:
auth_utils.delete_trust(self._environment)
auth_utils.delete_trust(self._environment.trust_id)
self._environment.system_attributes['TrustId'] = None
self._environment.trust_id = None

46
murano/db/services/environments.py
View File

@ -14,15 +14,23 @@
import yaml
from keystoneclient import exceptions as ks_exceptions
from murano.common import auth_utils
from murano.common import config
from murano.common import uuidutils
from murano.db import models
from murano.db.services import sessions
from murano.db import session as db_session
from murano.openstack.common import log as logging
from murano.services import states
DEFAULT_NETWORK_TYPE = 'io.murano.resources.NeutronNetwork'
LOG = logging.getLogger(__name__)
DEFAULT_NETWORK_TYPES = {
"nova": 'io.murano.resources.NovaNetwork',
"neutron": 'io.murano.resources.NeutronNetwork'
}
class EnvironmentServices(object):
@ -78,24 +86,25 @@ class EnvironmentServices(object):
return states.EnvironmentStatus.READY
@staticmethod
def create(environment_params, tenant_id):
def create(environment_params, context):
# tagging environment by tenant_id for later checks
"""Creates environment with specified params, in particular - name
:param environment_params: Dict, e.g. {'name': 'env-name'}
:param tenant_id: Tenant Id
:param context: request context to get the tenant id and the token
:return: Created Environment
"""
objects = {'?': {
'id': uuidutils.generate_uuid(),
}}
network_driver = EnvironmentServices.get_network_driver(context)
objects.update(environment_params)
if not objects.get('defaultNetworks'):
objects['defaultNetworks'] = \
EnvironmentServices.generate_default_networks(objects['name'])
EnvironmentServices.generate_default_networks(objects['name'],
network_driver)
objects['?']['type'] = 'io.murano.Environment'
environment_params['tenant_id'] = tenant_id
environment_params['tenant_id'] = context.tenant
data = {
'Objects': objects,
@ -196,26 +205,25 @@ class EnvironmentServices(object):
session.save(unit)
@staticmethod
def generate_default_networks(env_name):
def generate_default_networks(env_name, network_driver):
net_config = config.CONF.find_file(
config.CONF.networking.network_config_file)
if net_config:
LOG.debug("Loading network configuration from file")
with open(net_config) as f:
data = yaml.safe_load(f)
return EnvironmentServices._objectify(data, {
'ENV': env_name
})
# TODO(ativelkov):
# This is a temporary workaround. Need to find a better way:
# These objects have to be created in runtime when the environment is
# deployed for the first time. Currently there is no way to persist
# such changes, so we have to create the objects on the API side
network_type = DEFAULT_NETWORK_TYPES[network_driver]
LOG.debug("Setting '{0}' as environment's "
"default network".format(network_type))
return {
'environment': {
'?': {
'id': uuidutils.generate_uuid(),
'type': DEFAULT_NETWORK_TYPE
'type': network_type
},
'name': env_name + '-network'
},
@ -250,3 +258,15 @@ class EnvironmentServices(object):
for key, value in replacements.iteritems():
data = data.replace('%' + key + '%', value)
return data
@staticmethod
def get_network_driver(context):
ks = auth_utils.get_client(context.auth_token, context.tenant)
try:
ks.service_catalog.url_for(service_type='network')
except ks_exceptions.EndpointNotFound:
LOG.debug("Will use NovaNetwork as a network driver")
return "nova"
else:
LOG.debug("Will use Neutron as a network driver")
return "neutron"

8
murano/engine/client_manager.py
View File

@ -19,11 +19,12 @@ import muranoclient.v1.client as muranoclient
import neutronclient.v2_0.client as nclient
from oslo.config import cfg
from murano.common import auth_utils
from murano.common import config
from murano.dsl import helpers
from murano.engine import auth_utils
from murano.engine import environment
try:
# integration with congress is optional
import congressclient.v1.client as congress_client
@ -77,8 +78,9 @@ class ClientManager(object):
if not config.CONF.engine.use_trusts:
use_trusts = False
env = self._get_environment(context)
factory = lambda _1, _2: auth_utils.get_client_for_trusts(env) \
if use_trusts else auth_utils.get_client(env)
factory = lambda _1, _2: \
auth_utils.get_client_for_trusts(env.trust_id) \
if use_trusts else auth_utils.get_client(env.token, env.tenant_id)
return self.get_client(context, 'keystone', use_trusts, factory)

2
murano/tests/unit/api/base.py
View File

@ -80,6 +80,8 @@ class MuranoApiTestCase(base.MuranoWithDBTestCase, FakeLogMixin):
return_value=self.mock_engine_rpc).start()
mock.patch(self.RPC_IMPORT + '.api',
return_value=self.mock_api_rpc).start()
mock.patch('murano.db.services.environments.EnvironmentServices.'
'get_network_driver', return_value='neutron').start()
self.addCleanup(mock.patch.stopall)