Base class for modify actions.
Entry points based action registration.
Action manager loads and execute actions.
Out-of-the-box actions implementation.
Partially implements: blueprint policy-based-env-modification
Change-Id: Ib148c5f2b7efb1186d77f5461ef9324cca1287c2
This commit reworks mechanism how context chain
is built at each point. The goal was to make
entire chain be dependent on format version
number. This would allow for example to use
yaql 1.0 legacy mode context for MuranoPL/1.0
and not to use it for 2.0.
Not the entire chain is built on demand up to the
root instead of attaching to existed parent context
as it was before. DSL host (engine) is no more
required to have custom MuranoDslExecutor implementation
but a new ContextManager interface that is used
to control contexts on each layer. Engine uses
this interface to register system classes and
engine level yaql functions. Also a ContextManager
is a foundation for method mocking because now
all MuranoPL methods are yaql functions stored
in context and with ContextManager it is possible
to inject/replace function in each scope (
global, package, class, object)
Partially implements: blueprint murano-versioning
Change-Id: I0a553e8044061fe780a83bc04d70b8f80580988f
During reworking of how MuranoPL works with packages and
removal or class-loader plugin support code was broken and
temporary disabled. This commit restores previous functionality
using new PackageLoader and MuranoPackage classes
Change-Id: Id43807bfbb9093ddf9a1fbd9273d2fc34e8e5f59
With this change MuranoPackage becomes first-class DSL citizen.
Packages have version, runtime_version (that is specified
in Format attribute of the manifest file) and a list of classes.
Previously engine used to have package loader which had most
of "load" functionality and class loader that mostly acted as an
adapter from package loader to interface that DSL used to
get classes. Now class loader is gone and is replaced with
package loader at the DSL level. Package loader is responsible
for loading packages by either package or class name (as it was
before) plus semantic_version spec (for example ">=1.2,<2.0").
Package loader can now keep track of several versions of the same
package.
Also packages now have requirements with version specs.
All class names that are encountered in application code are
looked up within requirements only. As a consequence
packages that use other packages without referencing
them explicitly will become broken. An exception from this rule
is core library which is referenced automatically.
Partially implements: blueprint murano-versioning
Change-Id: I8789ba45b6210e71bf4977a766f82b66d2a2d270
* Code migrated to yaql 1.0.0
* New MuranoPL object initialization
* Lots of refactoring
See referenced specs for more information
Implements: blueprint migrate-to-yaql-vnext
Implements: blueprint object-construction
Depends-on: I7f314634ab5f08a521e51082d5c84dffca4b0b5c
Closes-Bug: #1454264
Change-Id: I740a4f83c76d8b56a1da585a739d770ef823a524
Before if an exception occured during dsl cleanup exception_result would
use empty ObjectsCopy and empty Attributes for result. In case exception
happened during env deletion this would cause env to be deleted by API,
because it ignored isException and treated any result as valid.
Now exception_result also includes original ObjectsCopy/Attributes in case
they're empty in exception_result.
Api not only checks count of 'error' statuses, in session, but also
checks isException attribute of the result, and treats exception results
as Errors, therefore marking deployment/deletion as failed.
Logging of results in API is now aware, that objects can be empty during
app deletion and no longer throws AttributeError because of that.
Change-Id: Idec8191ee25d1cac606741673719bbb8a72709b0
Closes-Bug: #1456724
This package class loader combines two types of
class loaders: loading packages from API and from local directory.
If folders to look packages in are specified,
packages would be loaded from their.
Otherwise, standart loader by API will operate as usual.
Implements blueprint change-murano-class-loader
Change-Id: Ifd8f40a755dc580703a44edc2b32cdd17691669d
To follow latest community practices we need to use oslo_log in
murano instead of handcrafted libs.
Unsupported log level 'audit' was changed to 'info'.
Change-Id: I40c0f3790e34bf80d0b63554e86b3cbc019eefca
In the several places the CONF variable is used from the murano.config.
This patch declares the local CONF variable and uses the oslo_config library.
Change-Id: I1f9a42f7f2cf7fef2cf0d63b185428dfddd9e78e
Currently pre/post deployment are invoked only for
deployment (actions). Those hooks are used to stop
AgentListener. But they were not executed for GC
phase that could also start AgentListener. As a result
we could get 2 AgentListeners running and stealing
messages from each other
Change-Id: If96db92e38f067bfed72cfc4a3d5619de2f0fad3
Closes-Bug: #1449500
(cherry picked from commit a79955709d)
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.
The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.
Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.
Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages
Change-Id: I975592f3694be42d52685ebf606f6d3012caf1a8
The database constraints which were present were enforcing the global
uniqueness of package FQNs and the names of classes defined in them.
This behavior was not correct, as the uniqueness should be enforced per
tenant, so the same package may be uploaded into two isolated tenants
without affecting each other.
This behavior lead to a very serious security issue: any tenant could
upload a package, leave it private and thus block all other tenants of
the cloud from uploading the package with the same name or even other
packages which contain at least one class in common with it. This could
be used to intentionally block all the operations of Murano on any
public environments.
This fix modifies the package name constraint to be unique only in
combination with owner_id, i.e. makes packages unique per tenant. Also
it removes the class name uniquness check from database (as there is no
cross-DB way to check it in a proper way) and adds a check method in
db.api module instead.
As the packages may be made public, this introduces a potential
collision: if the user owns some package, and there is a public package
with the same fully-qualified-name (or defining same class(es)) then the
class loader of the engine will have to choise between these packages
and/or classes defined in them.
To resolve this collision this commit adds a logic to fetch all the
patching packages and then pick the best match. Packages owned by the
current tenant are the most preferred, then the engine will pick public
packages, and non-owned non-public packages are the least preferred
(there may be no such packages now, they may appear when we add other
ways of package sharing).
Closes-bug: #1440094
Change-Id: I5c9b49642dfb6e955cf0c98b42f418da3b82060a
Adds a support for Nova Network if Neutron is not present in the
current OpenStack deployment.
Supporting the Nova Network requires modifications in three different
parts of generated Heat Stack:
1) Generated Security Groups and their rules should be of type
'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup'
2) Security Group assignments should go to security_groups property
of Instance resource, not the network port (as port concept is
not present when using NovaNetwork)
3) FloatingIP should be of type OS::Nova::FloatingIP and should be
associated with an Instance by OS::Nova::FloatingIPAssociation
resource.
To achieve p1 a SecurityGroupManager class of Core Library is made
abstract and is inherited by two concrete implementations:
NeutronSecurityGroupManager (containing the old MuranoPL code which
generated templates based on OS::Neutron::SecurityGroup) and a new
AwsSecurityGroupManager, which generates AWS-compliant firewall rules
which are consumed by NovaNetwork.
The particular concreate instance of this class is generated by the
default network of environment: Network class has got a new method called
generateSecurityGroupManager which returns an appropriate implementation.
For pp 2-3 a new inheritor of Network class has been added to the Core
Library: an io.murano.resources.NovaNetwork. It generates FloatingIP
association resources if needed and returns a securityGroupName object
as one of the outputs of its joinInstance methods.
The Instance class has been modified to properly handle these types of
outputs.
The instance of the NovaNetwork class is generated at the API side
when a new Environment is created and a is assigned to the
defaultNetworks.environment property of the environment if the neutron
is not defined in keystone.
Also this change moves the auth_utils module from engine to common, as
Keystone Client it contains is now used by the API process as well.
This changed is based on some of the code from the outdated changeset
I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7
Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Co-Authored-By: Stan Lagun <slagun@mirantis.com>
DocImpact
Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1
Implements-Blueprint: nova-network-support
Adds a PluginLoader which loads classes defined as stevedore plugins at
io.murano.extension namespace and registers them as MuranoPL classes in
class loader.
Modifies the ClientManager class to make the _get_client method public,
so other code may use it to add custom clients. This is useful for
plugins which may define their own clients.
Modifies the configuration settings adding 'enabled_plugins' parameter to
control which of the installed plugins are active.
Adds an example plugin which encapsulates Glance interaction logic to:
* List all available glance images
* Get Image by ID
* Get Image by Name
* Output image info with murano-related metadata
Adds a demo application which demonstrates the usage of plugin. The app
consist of the following components:
* An 'ImageValidatorMixin' class which inherits generic instance
class (io.murano.resources.Instance) and adds a method capable to
validate Instance's image for having appropriate murano metadata
type. This class may be used as a mixin when added to inheritance
hierarchy of concrete instance classes.
* A concrete class called DemoInstance which inherits from
io.murano.resources.LinuxMuranoInstance and ImageValidatorMixin
to add the image validation logic to standard Murano-enabled
Linux-based instance.
* An application which deploys a single VM using the DemoInstance
class if the tag on user-supplied image matches the user-supplied
constant.
The ImageValidatorMixin demonstrates the instantiation of
plugin-provided class and its usage, as well as handling of exception
which may be thrown if the plugin is not installed in the environment.
Change-Id: I978339d87033bbe38dad4c2102612d8f3a1eb3c3
Implements-blueprint: plugable-classes
When action called AgentListener automatically starts listening upon
first EP send to the agent. But Environment.deploy() were the only
place where AgentLister was stopped. So when action other than
Environment.deploy() was called there is no one to stop listener.
Thus on each action call new listener on the same RabbitMQ queue
was started causing listeners to steal messages from each other.
Agent.call() that never received response from agent caused
deployment/action hang.
Change-Id: Ia778c816a0e2f57d1f694fd1f128848f61b21a2d
Closes-Bug: #1425963
Also adds File type to core library for common convention type for files
Partially implements: blueprint actions-return-result
Change-Id: I5cbfb9ed6f4ae56e931815841f9c042f25a1d0ca
Remove gettextutils in favor of oslo.i18n suite for
internationalization purposes. Wrap murano.common.i18n around
oslo.i18n. Mark all logs messages of levels higher than
DEBUG for translation with _/_LI/_LW/_LE/_LC to conform with
oslo.i18n guidelines.
Change-Id: I09a2e2fc802e404f5c59fa4edd2a2124ad24101a
Implements: blueprint organize-translation
Adds ModelPolicyEnforcer that calls congress client (added by commit 2ea56d5b).
Enforcer called only when config property set to true (default false).
Integration test will follow in the next commit (https://review.openstack.org/#/c/147515).
Partially Implements blueprint policy-enforcement-point
Change-Id: Ie53b985ba759c3297e2fe2228bd48fce220ea32f
Now, when we started to use oslo.serialization it is safe
to replace all the usages of anyjson with jsonutils from
oslo library.
oslo.serialization uses anyjson under the hood, so there
shouldn't be any performance changes.
Change-Id: I8d6fbfbf88e657f5586c7361de849683c064d2e2
Removed #noqa from gettextutils and added them to import_exceptions.
I think it is better to specify option in one place (tox.ini) than
every time take care that you do not forget to specify this tag.
Also removed a few unused imports that were revealed in the process.
Change-Id: Ic4ca9cf374870075a36b88269ff8aea5a8e24a90
Instead of using user's auth token (which can expire) for interactions with
other services engine creates Keystone trust that impersonate user and
create new tokens on demand.
Heat stack is created on deployment start using token rather than trust so that
Heat could establish trust of its own (trusts cannot be chained).
New behavior is disabled by default and can be enabled using [engine]/use_trusts = True in murano.conf.
With trusts enabled engine will not work with Heat prior to Juno.
For Heat stacks with deferred actions or long deployment time to work it is also required to turn on trusts in Heat itself.
This can be done via [DEFAULT]/deferred_auth_method=trusts in heat.conf and ensuring that current user
has heat_stack_owner role (or any other that is in [DEFAULT]/trusts_delegated_roles=trusts in heat.conf)
Change-Id: Ic9f3f956ddb6ff2a300a08056ee841cf3c0db870
Implements: blueprint auth-for-long-running-requests
Now environment deletion is done as a regular deployment that can fail.
Environments that are deleted, but deletion process has failed remain in database
and shown in dashboard with status 'delete failure'. Environments that are being deleted
has status 'deleting' and do not disappear before they really got deleted on engine side
Also improved status reporting for environments. Now it also reports status of last deployment -
'deploy failure', 'delete failure'
P.S. Functional tests were slightly refactored and fixed to reflect changes
in deletion logic
Change-Id: I05625dd71f7ca9559bb88319b26b122214f15019
Closes-Bug: #1325101
Python frames in mixed stack traces were missing file name and pointed to a line
below correct position
Change-Id: I335292f40b3b6ea3dbca80b84f1d8dbed9a6581d
Fixes: bug #1331113
With this change MuranoPL can find correct base class method
where old implementation would throw AmbiguousMethodName.
Also removes possibility to have several methods with the same name
but different signature. This feature didn't worked in most cases,
never used anc could cause unexpected program behavior
Implements: blueprint muranopl-multiple-inheritance-method-resolution
Change-Id: I0a3149b993b1b8a9e9166fce13999e7dd7bf48a5
Methods in MuranoPL now can be marked with Usage attribute
to specify whether particular method available for remote
call or not. By default usage is Runtime (not available for
remote call).
Workflow:
migrateVm:
Usage: Action
Arguments:
- killExisting:
Contract: $.bool()
Default: True
Change-Id: If3da3c6bf67aa79d522d82abbf3b5378f72e87ae
Partially-Implements: blueprint application-actions
Adds support for packages consisting of single HOT template.
Most of HOT features are supported with exception of
environments, attachments and JSON parameter type.
Implements: blueprint hot-packages
Change-Id: I927af0e96f1613e8843ac47844e9c19fa00fdaa6