The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.
The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.
Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.
Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages
Change-Id: I975592f3694be42d52685ebf606f6d3012caf1a8
The database constraints which were present were enforcing the global
uniqueness of package FQNs and the names of classes defined in them.
This behavior was not correct, as the uniqueness should be enforced per
tenant, so the same package may be uploaded into two isolated tenants
without affecting each other.
This behavior lead to a very serious security issue: any tenant could
upload a package, leave it private and thus block all other tenants of
the cloud from uploading the package with the same name or even other
packages which contain at least one class in common with it. This could
be used to intentionally block all the operations of Murano on any
public environments.
This fix modifies the package name constraint to be unique only in
combination with owner_id, i.e. makes packages unique per tenant. Also
it removes the class name uniquness check from database (as there is no
cross-DB way to check it in a proper way) and adds a check method in
db.api module instead.
As the packages may be made public, this introduces a potential
collision: if the user owns some package, and there is a public package
with the same fully-qualified-name (or defining same class(es)) then the
class loader of the engine will have to choise between these packages
and/or classes defined in them.
To resolve this collision this commit adds a logic to fetch all the
patching packages and then pick the best match. Packages owned by the
current tenant are the most preferred, then the engine will pick public
packages, and non-owned non-public packages are the least preferred
(there may be no such packages now, they may appear when we add other
ways of package sharing).
Closes-bug: #1440094
Change-Id: I5c9b49642dfb6e955cf0c98b42f418da3b82060a
Adds a support for Nova Network if Neutron is not present in the
current OpenStack deployment.
Supporting the Nova Network requires modifications in three different
parts of generated Heat Stack:
1) Generated Security Groups and their rules should be of type
'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup'
2) Security Group assignments should go to security_groups property
of Instance resource, not the network port (as port concept is
not present when using NovaNetwork)
3) FloatingIP should be of type OS::Nova::FloatingIP and should be
associated with an Instance by OS::Nova::FloatingIPAssociation
resource.
To achieve p1 a SecurityGroupManager class of Core Library is made
abstract and is inherited by two concrete implementations:
NeutronSecurityGroupManager (containing the old MuranoPL code which
generated templates based on OS::Neutron::SecurityGroup) and a new
AwsSecurityGroupManager, which generates AWS-compliant firewall rules
which are consumed by NovaNetwork.
The particular concreate instance of this class is generated by the
default network of environment: Network class has got a new method called
generateSecurityGroupManager which returns an appropriate implementation.
For pp 2-3 a new inheritor of Network class has been added to the Core
Library: an io.murano.resources.NovaNetwork. It generates FloatingIP
association resources if needed and returns a securityGroupName object
as one of the outputs of its joinInstance methods.
The Instance class has been modified to properly handle these types of
outputs.
The instance of the NovaNetwork class is generated at the API side
when a new Environment is created and a is assigned to the
defaultNetworks.environment property of the environment if the neutron
is not defined in keystone.
Also this change moves the auth_utils module from engine to common, as
Keystone Client it contains is now used by the API process as well.
This changed is based on some of the code from the outdated changeset
I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7
Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Co-Authored-By: Stan Lagun <slagun@mirantis.com>
DocImpact
Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1
Implements-Blueprint: nova-network-support
Adds a PluginLoader which loads classes defined as stevedore plugins at
io.murano.extension namespace and registers them as MuranoPL classes in
class loader.
Modifies the ClientManager class to make the _get_client method public,
so other code may use it to add custom clients. This is useful for
plugins which may define their own clients.
Modifies the configuration settings adding 'enabled_plugins' parameter to
control which of the installed plugins are active.
Adds an example plugin which encapsulates Glance interaction logic to:
* List all available glance images
* Get Image by ID
* Get Image by Name
* Output image info with murano-related metadata
Adds a demo application which demonstrates the usage of plugin. The app
consist of the following components:
* An 'ImageValidatorMixin' class which inherits generic instance
class (io.murano.resources.Instance) and adds a method capable to
validate Instance's image for having appropriate murano metadata
type. This class may be used as a mixin when added to inheritance
hierarchy of concrete instance classes.
* A concrete class called DemoInstance which inherits from
io.murano.resources.LinuxMuranoInstance and ImageValidatorMixin
to add the image validation logic to standard Murano-enabled
Linux-based instance.
* An application which deploys a single VM using the DemoInstance
class if the tag on user-supplied image matches the user-supplied
constant.
The ImageValidatorMixin demonstrates the instantiation of
plugin-provided class and its usage, as well as handling of exception
which may be thrown if the plugin is not installed in the environment.
Change-Id: I978339d87033bbe38dad4c2102612d8f3a1eb3c3
Implements-blueprint: plugable-classes
When action called AgentListener automatically starts listening upon
first EP send to the agent. But Environment.deploy() were the only
place where AgentLister was stopped. So when action other than
Environment.deploy() was called there is no one to stop listener.
Thus on each action call new listener on the same RabbitMQ queue
was started causing listeners to steal messages from each other.
Agent.call() that never received response from agent caused
deployment/action hang.
Change-Id: Ia778c816a0e2f57d1f694fd1f128848f61b21a2d
Closes-Bug: #1425963
Also adds File type to core library for common convention type for files
Partially implements: blueprint actions-return-result
Change-Id: I5cbfb9ed6f4ae56e931815841f9c042f25a1d0ca
Remove gettextutils in favor of oslo.i18n suite for
internationalization purposes. Wrap murano.common.i18n around
oslo.i18n. Mark all logs messages of levels higher than
DEBUG for translation with _/_LI/_LW/_LE/_LC to conform with
oslo.i18n guidelines.
Change-Id: I09a2e2fc802e404f5c59fa4edd2a2124ad24101a
Implements: blueprint organize-translation
Adds ModelPolicyEnforcer that calls congress client (added by commit 2ea56d5b).
Enforcer called only when config property set to true (default false).
Integration test will follow in the next commit (https://review.openstack.org/#/c/147515).
Partially Implements blueprint policy-enforcement-point
Change-Id: Ie53b985ba759c3297e2fe2228bd48fce220ea32f
Now, when we started to use oslo.serialization it is safe
to replace all the usages of anyjson with jsonutils from
oslo library.
oslo.serialization uses anyjson under the hood, so there
shouldn't be any performance changes.
Change-Id: I8d6fbfbf88e657f5586c7361de849683c064d2e2
Removed #noqa from gettextutils and added them to import_exceptions.
I think it is better to specify option in one place (tox.ini) than
every time take care that you do not forget to specify this tag.
Also removed a few unused imports that were revealed in the process.
Change-Id: Ic4ca9cf374870075a36b88269ff8aea5a8e24a90
Instead of using user's auth token (which can expire) for interactions with
other services engine creates Keystone trust that impersonate user and
create new tokens on demand.
Heat stack is created on deployment start using token rather than trust so that
Heat could establish trust of its own (trusts cannot be chained).
New behavior is disabled by default and can be enabled using [engine]/use_trusts = True in murano.conf.
With trusts enabled engine will not work with Heat prior to Juno.
For Heat stacks with deferred actions or long deployment time to work it is also required to turn on trusts in Heat itself.
This can be done via [DEFAULT]/deferred_auth_method=trusts in heat.conf and ensuring that current user
has heat_stack_owner role (or any other that is in [DEFAULT]/trusts_delegated_roles=trusts in heat.conf)
Change-Id: Ic9f3f956ddb6ff2a300a08056ee841cf3c0db870
Implements: blueprint auth-for-long-running-requests
Now environment deletion is done as a regular deployment that can fail.
Environments that are deleted, but deletion process has failed remain in database
and shown in dashboard with status 'delete failure'. Environments that are being deleted
has status 'deleting' and do not disappear before they really got deleted on engine side
Also improved status reporting for environments. Now it also reports status of last deployment -
'deploy failure', 'delete failure'
P.S. Functional tests were slightly refactored and fixed to reflect changes
in deletion logic
Change-Id: I05625dd71f7ca9559bb88319b26b122214f15019
Closes-Bug: #1325101
Python frames in mixed stack traces were missing file name and pointed to a line
below correct position
Change-Id: I335292f40b3b6ea3dbca80b84f1d8dbed9a6581d
Fixes: bug #1331113
With this change MuranoPL can find correct base class method
where old implementation would throw AmbiguousMethodName.
Also removes possibility to have several methods with the same name
but different signature. This feature didn't worked in most cases,
never used anc could cause unexpected program behavior
Implements: blueprint muranopl-multiple-inheritance-method-resolution
Change-Id: I0a3149b993b1b8a9e9166fce13999e7dd7bf48a5
Methods in MuranoPL now can be marked with Usage attribute
to specify whether particular method available for remote
call or not. By default usage is Runtime (not available for
remote call).
Workflow:
migrateVm:
Usage: Action
Arguments:
- killExisting:
Contract: $.bool()
Default: True
Change-Id: If3da3c6bf67aa79d522d82abbf3b5378f72e87ae
Partially-Implements: blueprint application-actions
Adds support for packages consisting of single HOT template.
Most of HOT features are supported with exception of
environments, attachments and JSON parameter type.
Implements: blueprint hot-packages
Change-Id: I927af0e96f1613e8843ac47844e9c19fa00fdaa6