64 Commits

Author SHA1 Message Date
Alexander Tivelkov
796a0b2c9d Nova Network support
Adds a support for Nova Network if Neutron is not present in the
current OpenStack deployment.

Supporting the Nova Network requires modifications in three different
parts of generated Heat Stack:
	1) Generated Security Groups and their rules should be of type
	   'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup'
	2) Security Group assignments should go to security_groups property
	   of Instance resource, not the network port (as port concept is
	   not present when using NovaNetwork)
	3) FloatingIP should be of type OS::Nova::FloatingIP and should be
	   associated with an Instance by OS::Nova::FloatingIPAssociation
	   resource.

To achieve p1 a SecurityGroupManager class of Core Library is made
abstract and is inherited by two concrete implementations:
NeutronSecurityGroupManager (containing the old MuranoPL code which
generated templates based on OS::Neutron::SecurityGroup) and a new
AwsSecurityGroupManager, which generates AWS-compliant firewall rules
which are consumed by NovaNetwork.

The particular concreate instance of this class is generated by the
default network of environment: Network class has got a new method called
generateSecurityGroupManager which returns an appropriate implementation.

For pp 2-3 a new inheritor of Network class has been added to the Core
Library: an io.murano.resources.NovaNetwork. It generates FloatingIP
association resources if needed and returns a securityGroupName object
as one of the outputs of its joinInstance methods.
The Instance class has been modified to properly handle these types of
outputs.
The instance of the NovaNetwork class is generated at the API side
when a new Environment is created and a is assigned to the
defaultNetworks.environment property of the environment if the neutron
is not defined in keystone.

Also this change moves the auth_utils module from engine to common, as
Keystone Client it contains is now used by the API process as well.

This changed is based on some of the code from the outdated changeset
I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7

Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Co-Authored-By: Stan Lagun <slagun@mirantis.com>

DocImpact

Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1
Implements-Blueprint: nova-network-support
2015-04-08 13:09:21 +03:00
Alexander Tivelkov
62c1f10e7b Initial implementation of Plugable Classes
Adds a PluginLoader which loads classes defined as stevedore plugins at
io.murano.extension namespace and registers them as MuranoPL classes in
class loader.

Modifies the ClientManager class to make the _get_client method public,
so other code may use it to add custom clients. This is useful for
plugins which may define their own clients.

Modifies the configuration settings adding 'enabled_plugins' parameter to
control which of the installed plugins are active.

Adds an example plugin which encapsulates Glance interaction logic to:
	* List all available glance images
	* Get Image by ID
	* Get Image by Name
	* Output image info with murano-related metadata

Adds a demo application which demonstrates the usage of plugin. The app
consist of the following components:
	* An 'ImageValidatorMixin' class which inherits generic instance
	  class (io.murano.resources.Instance) and adds a method capable to
	  validate Instance's image for having appropriate murano metadata
	  type. This class may be used as a mixin when added to inheritance
	  hierarchy of concrete instance classes.
	* A concrete class called DemoInstance which inherits from
	  io.murano.resources.LinuxMuranoInstance and ImageValidatorMixin
	  to add the image validation logic to standard Murano-enabled
	  Linux-based instance.
	* An application which deploys a single VM using the DemoInstance
	  class if the tag on user-supplied image matches the user-supplied
	  constant.

The ImageValidatorMixin demonstrates the instantiation of
plugin-provided class and its usage, as well as handling of exception
which may be thrown if the plugin is not installed in the environment.

Change-Id: I978339d87033bbe38dad4c2102612d8f3a1eb3c3
Implements-blueprint: plugable-classes
2015-03-19 14:06:24 +00:00
Stan Lagun
0d42f01f68 Configurable environment's default network config
Network configuration is extracted into a separate file.
By default this is netconfig.yaml residing near murano.conf
But the name and path can be changed in config file.
Example of net-config can be found in etc/murano/netconfig.yaml.sample
If no file present old behavior is kept

Change-Id: I7b74eea69ee2ffe1c721b751e564b54252dcfbe3
Implements: blueprint configure-environment-network-defaults
2015-03-17 20:26:41 +03:00
Dmytro Dovbii
ad6a1da399 Fix pep8 issues in imports
Fixes imports grouping.

Change-Id: I038089cf2d3fc15fefa7d5bc878dad2f16756923
2015-03-13 10:16:08 +02:00
Jenkins
53601f4dcf Merge "Fixes agent call may hanged upon action call" 2015-03-10 08:55:04 +00:00
Stan Lagun
83ba5fa731 Fixes agent call may hanged upon action call
When action called AgentListener automatically starts listening upon
first EP send to the agent. But Environment.deploy() were the only
place where AgentLister was stopped. So when action other than
Environment.deploy() was called there is no one to stop listener.
Thus on each action call new listener on the same RabbitMQ queue
was started causing listeners to steal messages from each other.
Agent.call() that never received response from agent caused
deployment/action hang.

Change-Id: Ia778c816a0e2f57d1f694fd1f128848f61b21a2d
Closes-Bug: #1425963
2015-03-09 08:44:16 +00:00
Stan Lagun
b3e665c63a Makes exception_traceback optional for exception_result
Change-Id: Icd6d655e615fbe92bbf7ba6caaffe5b74c67ffab
Closes-Bug: #1428058
2015-03-05 15:09:42 +00:00
Jenkins
42c320a085 Merge "Update API policy" 2015-03-02 14:28:05 +00:00
Ekaterina Chernova
d82b74b62f Update API policy
* Set admin rule for a several API calls and remove direct check in code
* Now admin can configure policy.json and enable package management for regular users
* Update common policy module

Closes-Bug: #1412868

Change-Id: I8d0725b613564529d32a5acef289f4822f32915c
2015-03-02 15:28:14 +03:00
Stan Lagun
73f8368024 Adds API to obtain action result
Also adds File type to core library for common convention type for files

Partially implements: blueprint actions-return-result

Change-Id: I5cbfb9ed6f4ae56e931815841f9c042f25a1d0ca
2015-03-02 14:08:36 +03:00
Ekaterina Chernova
5feedfb8ba Update from global requirements
Change-Id: I955c2945de793ee7f3ce4eb9688e43ec63a6b8e2
2015-02-25 16:09:22 +03:00
Kirill Zaitsev
ceba21573d Include missing log string format specifier
Correctly log non-dsl_exception exceptions
during action execution.

Change-Id: I48d5a3bc033f2830e5bc14a899df49fe7ede014a
2015-02-24 16:24:29 +00:00
Kirill Zaitsev
4e036191ff Use oslo.i18n for translation
Remove gettextutils in favor of oslo.i18n suite for
internationalization purposes. Wrap murano.common.i18n around
oslo.i18n. Mark all logs messages of levels higher than
DEBUG for translation with _/_LI/_LW/_LE/_LC to conform with
oslo.i18n guidelines.

Change-Id: I09a2e2fc802e404f5c59fa4edd2a2124ad24101a
Implements: blueprint organize-translation
2015-02-19 18:51:22 +03:00
Jenkins
2bfe4e3e69 Merge "Add timeouts to murano-agent calls" 2015-02-05 16:11:08 +00:00
Dmytro Dovbii
69c4711ce8 Add timeouts to murano-agent calls
Now there is no way to be sure that the agent successfully started execution
on a VM. Also there is no control of the execution time of scripts on agent.
This patch adds timeouts to agent`s calls on murano engine side

Implements blueprint murano-agent-timeouts
Change-Id: I81b1cf526e07be5ee32b51e259b93b6b40b122cf
2015-02-05 16:31:59 +02:00
Ekaterina Chernova
a8518aecce Resolve and enable H702 PEP8 rule
H702  Formatting operation should be outside of localization method call

Change-Id: I83e3a7e3d7f19d3238ee91dcab730d55d479a65e
2015-02-05 12:10:22 +03:00
Ekaterina Chernova
9a26a3f4c3 Resolve and enable H307 pep8 rule
* H307  like imports should be grouped together

Change-Id: I9fa443c9b7380e4c589793caf35c80ba6afbe9f0
2015-02-05 12:10:22 +03:00
Ekaterina Chernova
017d25f49e Resolve and enable E265 pep8 issue
E265 block comment should start with '# '

Change-Id: I1a8b971776deb8b05b84be169571b9f1e573d2ba
2015-02-05 12:10:22 +03:00
Ekaterina Chernova
87a63bf24e Resolve H305 pep8 issue
H305  imports not grouped correctly

Change-Id: Ia990dd628da195097838a13d876ef987363d3f8e
2015-02-05 12:10:22 +03:00
Ekaterina Chernova
8a15466301 Enable and resolve some PEP8 issues
* E128 continuation line under-indented for visual indent
* H501 don't use locals() for formatting strings. to also check
* H402 first line of docstring should end with punctuation
* E122 continuation line missing indentation or outdented
* E713 test for membership should be 'not in'

Change-Id: I4a30350778a4452075e468400effcbc4155d24d8
2015-02-05 12:10:18 +03:00
Jenkins
f18b9d2af4 Merge "Remove 'murano_metadata_url' from config options" 2015-02-02 11:05:07 +00:00
Ekaterina Chernova
66d703d836 Remove 'murano_metadata_url' from config options
This option was used, when there was a separate service for murano repository.
Now it's obsolete and should be removed.

Change-Id: I2e9611ec8ce7968ff2656660eebdafe87ac1811e
2015-01-30 18:16:28 +03:00
ondrej.vojta
a933d6e3b8 Initial implementation of policy enforcement point
Adds ModelPolicyEnforcer that calls congress client (added by commit 2ea56d5b).
Enforcer called only when config property set to true (default false).
Integration test will follow in the next commit (https://review.openstack.org/#/c/147515).

Partially Implements blueprint policy-enforcement-point

Change-Id: Ie53b985ba759c3297e2fe2228bd48fce220ea32f
2015-01-26 16:11:57 +01:00
Natasha Beck
e5026e098c Add Mistarl client to murano
Change-Id: I780a9166177db14197d2cffb28eb1aeaa0e70fa9
Implements: blueprint murano-mistral-integration
2015-01-19 13:53:49 +02:00
Serg Melikyan
ed80169da0 Log details about finished deployment
We are logging whole environment object model when it's returned
from the engine to the API, this log message is not convenient
for getting environment status from reading log.

This change is adding additional logging message that outputs
following information:

 * Environment ID
 * Tenant ID
 * Deployment State
 * List of deployed applications

Example: <EnvId: 29892048920482 TenantId: 2349872384723894
Status: Success Apps: io.murano.Tomcat, io.murano.Postgres>

Change-Id: I0ebfdcbff419fee268f12df7cfed48f6377fe71f
Closes-bug: 1409446
2015-01-14 16:23:36 +00:00
Stan Lagun
b5f0b0f245 Adds per-class configs
Adds ability to have per-class configuration and special properties
with usage "Config". Such properties get their values from config
(if it is present) rather than from object model.

Config files can also modify defaults for other property types.

Config files are stored in special folder that is configured in
[engine] section of Murano config file under class_configs key.

Config files must me named using %FQ class name%.json or
%FQ class name%.yaml pattern and contain dictionary of a form
propertyName -> propertyValue

Change-Id: I0f45fa7064183f5605c5ef393b5b00e8c8ae2bda
Implements: blueprint class-configs
2014-12-17 00:48:51 +03:00
Ruslan Kamaldinov
734d407332 Replace anyjson with oslo.serialization
Now, when we started to use oslo.serialization it is safe
to replace all the usages of anyjson with jsonutils from
oslo library.
oslo.serialization uses anyjson under the hood, so there
shouldn't be any performance changes.

Change-Id: I8d6fbfbf88e657f5586c7361de849683c064d2e2
2014-12-13 18:10:53 +03:00
Ruslan Kamaldinov
18694ec63d Use oslo.serialization
Change-Id: I028409bb4fe05d4d2949f7f028298cd4704345b4
2014-12-11 02:15:39 +03:00
Ruslan Kamaldinov
bf05953c28 Use oslo.utils
Files from oslo.utils will go away once Murano
migrates to oslo.serialization.

Change-Id: Ib3711164562bea4ac30a96384733d77753ffa69c
2014-12-11 01:49:37 +03:00
Ruslan Kamaldinov
aa2c8f0a12 Update from oslo incubator
Details about this change:
* Cleaned up openstack-common.conf, this file should contain
  only direct dependencies of Murano
* Removed unused files from openstack/common
* Moved xmlutils to murano/common; this module was removed from
  oslo-incubator long time ago, but is still used by
  common/wsgi.py, which also was deprecated by Oslo and adopted
  by Murano

Change-Id: I118de30bb0bae577d24d86fa723522580beb13d0
2014-12-10 09:29:14 +00:00
Jenkins
af2a623c2b Merge "Use Keystone trusts to get fresh token" 2014-12-10 00:06:51 +00:00
Dmitrii Dovbii
a55acc1266 Remove #noqa from gettextutils imports
Removed #noqa from gettextutils and added them to import_exceptions.
I think it is better to specify option in one place (tox.ini) than
every time take care that you do not forget to specify this tag.

Also removed a few unused imports that were revealed in the process.

Change-Id: Ic4ca9cf374870075a36b88269ff8aea5a8e24a90
2014-11-14 19:43:14 +02:00
Jenkins
238e3a60e5 Merge "Environment in delete failed state was in progress forever" 2014-11-14 15:17:16 +00:00
Stan Lagun
f40169327b Use Keystone trusts to get fresh token
Instead of using user's auth token (which can expire) for interactions with
other services engine creates Keystone trust that impersonate user and
create new tokens on demand.

Heat stack is created on deployment start using token rather than trust so that
Heat could establish trust of its own (trusts cannot be chained).

New behavior is disabled by default and can be enabled using [engine]/use_trusts = True in murano.conf.
With trusts enabled engine will not work with Heat prior to Juno.
For Heat stacks with deferred actions or long deployment time to work it is also required to turn on trusts in Heat itself.
This can be done via [DEFAULT]/deferred_auth_method=trusts in heat.conf and ensuring that current user
has heat_stack_owner role (or any other that is in [DEFAULT]/trusts_delegated_roles=trusts in heat.conf)

Change-Id: Ic9f3f956ddb6ff2a300a08056ee841cf3c0db870
Implements: blueprint auth-for-long-running-requests
2014-11-14 13:58:17 +03:00
Dmytro Dovbii
eccc6be0c8 Remove unused functional from murano/common/wsgi.py
Removed unused function which was mistakenly added in commit
https://review.openstack.org/#/c/129671/

Change-Id: I6c42aba8f2968bd0bf6c1d10b4d4ae99813a701a
2014-11-14 11:01:31 +02:00
Dmytro Dovbii
76ffd35806 Use oslo.config generator in murano
Similarly with the way it was done in the murano-agent, added
the ability to use the oslo-config-generator for creating a sample
configuration file murano.conf.sample.

Now to create the file, you must run:
tox -egenconfig

Change-Id: I8b06336d75eccfd598ce9033333884f75f393a09
2014-11-12 17:09:35 +02:00
Stan Lagun
c301b5e763 Environment in delete failed state was in progress forever
Deletion sessions were in 'deploying' state instead of 'deleting'.
Because of this such sessions could not be found by RPC result
processing code and remained in 'deploying' status causing UI
to display progress bar forever.

Also there were 2 duplicate SessionStates enums in the code with
and the second copy was outdated and didn't contained DELETING
status as well as other new session statuses. Because buggy code
was using that outdated enum it was necessary to merge both enums
into single declaration

Change-Id: I852f1f3dd1051c7b40afaa2575a4335b0f3c3104
Closes-Bug: #1386068
2014-11-10 18:43:55 +00:00
Ekaterina Chernova
552f04a674 Extract version definition to a separate file
Before, version was stores in the init file.
It's better not to store such kind of public data
as version to init file.
Moreover, whole murano package needed to be imported to identify project version
Now, separate file is used for that

Also, doc builder config was updated to support this change

Change-Id: Iede1dfbe9ae4d8c6fb5ecdeb8f8f25124e4bc34f
2014-11-10 13:02:56 +04:00
Ryan Peters
09fe0567a2 Creates a router if one doesn't exist
If it doesn't find a router by the name specified in router_name
in murano.conf, create a router with that name. Uses
external_network_id or external_network_name as the
external_gateway_info ID. Requires:
https://review.openstack.org/#/c/119800

Change-Id: If8b966a7d43eb2af485113de2a0708e554605725
2014-09-24 17:46:55 +00:00
Serg Melikyan
f1905be336 Use only specific router for created networks
Murano should not use first available router it may create
different networking issues cause router may not have access to
external network.

Closes-bug: #1366124
Change-Id: I223a4862c3b20cfb20281be6bd9376be3644ad92
2014-09-24 16:55:49 +00:00
Ruslan Kamaldinov
caf247b29b Updated from global requirements
Also slightly modified sample config, including default value
for packages_cache.

Change-Id: I43e294a7e2328e916aa8273d808cfcabd60f7024
2014-09-24 16:30:47 +04:00
Jenkins
ff5876ccc8 Merge "Fix deployment failure detection" 2014-09-04 14:19:32 +00:00
Ekaterina Fedorova
08a6713240 Move wsgi module to murano/common
* Resolve all import issues
* Update sample config
* Resolve pep8 doc string issues

Change-Id: Ib95add96b1f3cd04ecf3666d24ef307e3c6bba42
Closes-Bug: #bug/1363892
2014-09-03 19:53:45 +04:00
Serg Melikyan
51cf4d1775 Fix deployment failure detection
Closes-Bug: #1364370
Change-Id: If242d815aca565909a6ed747e41eeab66f2cc2b0
2014-09-03 14:14:18 +04:00
Ekaterina Fedorova
d6a3a2b2e6 Enable H202, H402, H404 rules
* H202 assertRaises Exception too broad
* H402 one line docstring needs punctuation
* H404 multi line docstring should start without a leading new line

Change-Id: I2f662b8b97d14daa501620c8237bf93bd2251243
2014-09-01 09:10:51 +00:00
Stan Lagun
e47ef4e849 Adds REST API endpoint for action execution
Deployment is replaced with a more generic concept called 'action'.
Action can be performed on any object of Object Model.
Actions are marked with 'Usage: Action' in MuranoPL code. They can have arguments.
List of available actions can be obtained from Object Model itself after initial deployment.
This commit adds ability to REST API invoke actions by providing its unique id (from OM) and parameters.
Also refactors API code to use tasks.

Change-Id: If21809340bb799af58a8d1a2d148e52565028970
Partially-Implements: blueprint application-actions
2014-08-27 17:09:29 +04:00
Ruslan Kamaldinov
2bfe3a9482 Migrate to oslo.db
Move from oslo-incubator db module to oslo.db

Change-Id: I01e04387ee400be92345a12de4f0f83a99980842
2014-08-23 16:35:07 +04:00
Angus Salkeld
b909d29058 Fix pep H101(TODO) and H231(py3 exceptions)
Change-Id: I6797c4df1595466a12f84721a383eb541e6401f7
2014-08-20 17:20:26 +10:00
Stan Lagun
56b2d5df27 Fixes silent deletion of environments
Now environment deletion is done as a regular deployment that can fail.
Environments that are deleted, but deletion process has failed remain in database
and shown in dashboard with status 'delete failure'. Environments that are being deleted
has status 'deleting' and do not disappear before they really got deleted on engine side

Also improved status reporting for environments. Now it also reports status of last deployment -
'deploy failure', 'delete failure'

P.S. Functional tests were slightly refactored and fixed to reflect changes
in deletion logic

Change-Id: I05625dd71f7ca9559bb88319b26b122214f15019
Closes-Bug: #1325101
2014-08-19 12:52:52 +04:00
Steve McLellan
e2bea76426 Allow murano-agent to be disabled
In some circumstances murano-agent isn't required (e.g. in
environments where heat SW config is capable alone of performing
configuration). In this case it's not necessary to have the
additional overhead of rabbitMQ connections for the AgentListener
that will never receive a message.

Patch adds a config option 'disable_murano_agent' that no-ops
AgentLister.start() and raises an exception on Agent._send()

Change-Id: I565caaae21925c48f2a0adea18036239cac91c77
Implements: blueprint disable-murano-agent
2014-08-13 09:03:14 -05:00