Adds a support for Nova Network if Neutron is not present in the
current OpenStack deployment.
Supporting the Nova Network requires modifications in three different
parts of generated Heat Stack:
1) Generated Security Groups and their rules should be of type
'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup'
2) Security Group assignments should go to security_groups property
of Instance resource, not the network port (as port concept is
not present when using NovaNetwork)
3) FloatingIP should be of type OS::Nova::FloatingIP and should be
associated with an Instance by OS::Nova::FloatingIPAssociation
resource.
To achieve p1 a SecurityGroupManager class of Core Library is made
abstract and is inherited by two concrete implementations:
NeutronSecurityGroupManager (containing the old MuranoPL code which
generated templates based on OS::Neutron::SecurityGroup) and a new
AwsSecurityGroupManager, which generates AWS-compliant firewall rules
which are consumed by NovaNetwork.
The particular concreate instance of this class is generated by the
default network of environment: Network class has got a new method called
generateSecurityGroupManager which returns an appropriate implementation.
For pp 2-3 a new inheritor of Network class has been added to the Core
Library: an io.murano.resources.NovaNetwork. It generates FloatingIP
association resources if needed and returns a securityGroupName object
as one of the outputs of its joinInstance methods.
The Instance class has been modified to properly handle these types of
outputs.
The instance of the NovaNetwork class is generated at the API side
when a new Environment is created and a is assigned to the
defaultNetworks.environment property of the environment if the neutron
is not defined in keystone.
Also this change moves the auth_utils module from engine to common, as
Keystone Client it contains is now used by the API process as well.
This changed is based on some of the code from the outdated changeset
I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7
Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Co-Authored-By: Stan Lagun <slagun@mirantis.com>
DocImpact
Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1
Implements-Blueprint: nova-network-support
YAQL expression for FIP resource were incorrect and left as is (as a string).
Thus there were single resource for FIP with incorrect name instead of resource
per VM
Change-Id: Ieecdfdb0a7fbad82ef4344b4bac1a55656991e9e
Closes-Bug: #1435787
Closes-Bug: #1435961
When action called AgentListener automatically starts listening upon
first EP send to the agent. But Environment.deploy() were the only
place where AgentLister was stopped. So when action other than
Environment.deploy() was called there is no one to stop listener.
Thus on each action call new listener on the same RabbitMQ queue
was started causing listeners to steal messages from each other.
Agent.call() that never received response from agent caused
deployment/action hang.
Change-Id: Ia778c816a0e2f57d1f694fd1f128848f61b21a2d
Closes-Bug: #1425963
Also adds File type to core library for common convention type for files
Partially implements: blueprint actions-return-result
Change-Id: I5cbfb9ed6f4ae56e931815841f9c042f25a1d0ca
Also refactores networking code to move networking code from
Instance so that different types of network can be represented by
single simple interface
Implements: blueprint join-existing-neutron-networks
Change-Id: I90afcea6b1c12d7f6f564d8524f5c0d30e1b4686
New class is created that is responsible to run Mistral workflows.
The workflow that can be run hare is already uploaded workflow.
Partially Implements: blueprint murano-mistral-integration
Change-Id: I6b6251d06aa3ce9bf103a67fe354b8c49f52d193
'io.murano.resources.Instance' class has no availabilityZone property,
while is shoud. Since it's optional value, no error occured.
But for more flexability, it's better to have it.
Change-Id: Ied375db17dc7adb75b4befcf0a083edaca78bfd6
Closes-Bug: #1413938
This fix is supposed to be a workaround for an issue when
instances loose their network upon subsequent stack updates.
Change-Id: I57e7e433099309906d4c85d9dff8227a2a89acf4
Workaround-for: #1393376
Closes-bug: #1414008
Some properties of Neutron resources in Heat were renamed
(e.g. network_id => network in OS::Neutron::RouterInterface),
this change introduces same changes in Core Library.
Change-Id: Ic1f82adb6c325e43c37528a61ac7cebc9b47cfb5
Closes-bug: #1381241
FloatingIPAssociation was designed to attach floating IP created outside
of the Heat stack to resources in the stack. Since we create floating IP by
ourselvs, there is no reason to use FloatingIPAssociation at all.
Change-Id: I7575ab2fa7f166a46b2328479c1691a33c812ec1
Closes-bug: #1412658
Instead of using user's auth token (which can expire) for interactions with
other services engine creates Keystone trust that impersonate user and
create new tokens on demand.
Heat stack is created on deployment start using token rather than trust so that
Heat could establish trust of its own (trusts cannot be chained).
New behavior is disabled by default and can be enabled using [engine]/use_trusts = True in murano.conf.
With trusts enabled engine will not work with Heat prior to Juno.
For Heat stacks with deferred actions or long deployment time to work it is also required to turn on trusts in Heat itself.
This can be done via [DEFAULT]/deferred_auth_method=trusts in heat.conf and ensuring that current user
has heat_stack_owner role (or any other that is in [DEFAULT]/trusts_delegated_roles=trusts in heat.conf)
Change-Id: Ic9f3f956ddb6ff2a300a08056ee841cf3c0db870
Implements: blueprint auth-for-long-running-requests
When deployment of environment fails with exception AgentListener.stop() is not
called and thus there remains a listener for RabbitMQ response queue.
Besides being a resource leak it introduces another problems: when that environment
get redeployed it becomes 2 listeners on the same queue and responses from agents
may be stolen by zombie listener making workflow wait forever or response
(hang deployment)
Change-Id: Ic4cedd323ab7b55690d095ed8addcb0dc3e335a7
Closes-Bug: #1369589
Server.addresses's implementation in Heat is dependent on a nova extension
which is not guaranteed to be present. I'm not sure if this is something
that should be fixed in Heat or not, but this is a workaround that
results in the same information (instance.ipAddresses will be set to a
list of all IP addresses across all networks the instance is assigned
to).
Change-Id: I5154f7e2a69ab8e64f9fbd1d8a26daf07f2482cd
Closes-Bug: 1364446
A common paradigm with userdata is to supply:
userdata:
str_replace:
template: |
<some script>
params:
param1: value1
With the contract on customUserData being a string, passing str_replace
as the argument results in coercion to a string, which makes it invalid.
Change-Id: I8bcf788522fc2428eaa3c8529ee11b5ed385099b
Defaults to null (which in turn will cause heat to use its
default) but allows control over software deployment signalling,
in particular to use 'NO_SIGNAL' if required not to wait for success
or failure.
Change-Id: I7b8bb553724173261d9475e2d574fc0e413d2b3e
Instance class implementation is based on OS::Nova::Server
resource in Heat and related networking resources. When deploy
called on Instance number of resources are allocated in Heat.
Some classes (e.g. descendants of Instance) can allocate other
resources that depends on presence of resources allocated by Instance,
for example HeatSWConfigInstance allocate OS::Heat::SoftwareDeployment
that depends on underlying OS::Nova::Server.
When Instance is destroyed and clean-up stage is finished underlying
resources are removed from stack. But since some resources still depend
on removed ones whole stack becomes invalid and deployment fails.
This change implements quick workaround by removing resource clean-up
in Instance class. Warning, this change introduce resource leak, cause
resoures are allocated during Instance deployment now will be deleted
only when whole Environment (and therefor stack) will be deleted.
Change-Id: I772f0b35918cd554fcbe1e830766ec23d5641176
Closes-Bug: #1359998
Murano uses Heat to allocate OpenStack resources by creating and
updating one Heat stack. All changes to stack are stored in shared
variable exposed via call to environment.stack.updateStack.
Allocation of resources needed by Instance class is done in several calls
to environment.stack.updateStack therefor in case of simultaneous deployment
of two instances of class Instance race condition is possible.
Instance class was rewritten to have only one call to stack.updateStack
that is going to eliminate possible race condition.
P.S. This change also introduce new function mergeWith that can merge
two dicts.
Change-Id: Ib3b0be04e7d0b6107466bd5837278a12f01cf766
Closes-Bug: #1364824
Stack-delete was failing periodically because of heat bug
https://bugs.launchpad.net/heat/+bug/1299259, whereby there's
an implicit dependency from floating IPs to the router interface.
There is a fix in progress for heat, but until then, this patch
adds an explicit depends_on to the router interface.
Change-Id: I846abf1fbd90d5f5c71b1c13941463743ace48ad
Closes-Bug: 1356721
Addition to reviews/113676 (reducing number of api requests during
deploy). InstanceNotifier yaml stub was mistakenly called
InstanceReportNotifier; patch corrects manifest.yaml and the class
name, and renames the yaml file.
Change-Id: I47c143ed92456374936e13bb27ae5988e5676e8e
Closes-Bug: 1358061
For non-structured configs, configSection cannot be restricted to a
Map. Patch removes the contract type, although it is worth revisiting
this to refactor slightly in future (which should be possible without
changing the interface).
Change-Id: I179e350a724965c97e90c3a1927f955c20cc2a72
Closes-Bug: 1358046
For every class name, a request is made to the API server to retrieve
the package to which it belongs. This is unnecessary, since a package
contains a list of the classes it owns. This patch adds a second cache
to PackageClassLoader that maps class names to the package, and reduces
the number of calls for the io.murano classes to 1.
The second improvement is to add blank YAML definitions for system
classes (the function definitions are in python code) which allows
the engine to cache those, too.
Change-Id: Iee3c39b7c4965d5b8015d458afb78c324dfd2826
Adds a method to Instance to allow subclasses to perform additional
updates to heat templates; subclasses should take care to expose
as little or as much as makes sense regarding implementation details.
Additions are added after the Instance Heat fragments are added but
before the stack is pushed.
Also adds a function to HeatSWConfigInstance to add SoftwareConfig
(and StructuredConfig) elements and their Deployments.
Change-Id: I1cd127a32e3115765f8a26886eeeeb5696f686fd
Implements: blueprint software-config-at-deploy-time
This fix moves queue creation code to initialize section in order to
have a queue created before Murano agent on the spawned VM will try to use it.
Change-Id: Ic7352042e9e1b48d227a8b572ae8ae91798c7487
Closes-Bug: #1349472
In https://review.openstack.org/105117 despite what commit message says only queue name was changed.
This change makes Heat stack name be set to that name with 'murano-' prefix and description
in HOT template telling what environment owns the stack
Change-Id: I4a398a265031d0428497c6f91dfed88c7003ad21
Added new action 'destroy' to io.murano.Environment that handles
deletion of Heat stack when environment is deleted.
Note: changeset include changes in io.murano package, during update
this package should be re-uploaded to application catalog
Closes-Bug: #1321487
Change-Id: I92268b90cb762d58669208aef3f5fce85e06d355
To expose IP addresses of Instance in MuranoPL we place new entry to
outputs section in Heat template that reference corresponding
OS::Nova::Server resource. During clean-up stage, if some application
was deleted from environment, we remove corresponding OS::Nova::Server
resource, but reference in outputs section stays.
Stack with reference to resource that is not present in template is
invalid - environment deployment fails.
This change add code that removes references to OS::Nova::Server when
corresponding Instance is deleted.
Attention: this change fix issue that can break murano-ci gate
Closes-bug: #1339630
Co-Authored-By: Stan Lagun <slagun@mirantis.com>
Change-Id: I74d32034969dd7f554d74fac87f407388e52dd7e
In order to pass raw userdata or heat software config resources to an
instance it's necessary to tell Heat what to expect. This patch adds
user_data_format defaulting to HEAT_CFNTOOLS (Heat's default) and a
function allowing it to be overriden (to RAW or SOFTWARE_CONFIG).
Derived classes should override user_data_format as part of
prepareUserData which is now expected to return a dictionary whose
keys are [data, format]. A class HeatSWConfigInstance extends
Instance to add this; a HeatSWConfigLinuxInstance is provided
as a convenience (heat SW + SSH access).
Change-Id: I59f66d42742bf179e7fa1de3cd5de07dbf9fb42b
This change resolve systax error in Environment.yaml
introduced by change I44de2083aa2d50e6cd3b3b79b204facedaf2b7df
Attention: this change fix issue that broke murano-ci gate
Change-Id: Id69dcb30935cde315dc72632724be4684d51b450
Closes-bug: 1339668
