This prevents from the situation, when tenant is no longer exist
and there is no way to delete an application
Change-Id: I1539d0331634e8c665ff056f1bfde4de47880985
Closes-Bug: #1436618
Adds a support for Nova Network if Neutron is not present in the
current OpenStack deployment.
Supporting the Nova Network requires modifications in three different
parts of generated Heat Stack:
1) Generated Security Groups and their rules should be of type
'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup'
2) Security Group assignments should go to security_groups property
of Instance resource, not the network port (as port concept is
not present when using NovaNetwork)
3) FloatingIP should be of type OS::Nova::FloatingIP and should be
associated with an Instance by OS::Nova::FloatingIPAssociation
resource.
To achieve p1 a SecurityGroupManager class of Core Library is made
abstract and is inherited by two concrete implementations:
NeutronSecurityGroupManager (containing the old MuranoPL code which
generated templates based on OS::Neutron::SecurityGroup) and a new
AwsSecurityGroupManager, which generates AWS-compliant firewall rules
which are consumed by NovaNetwork.
The particular concreate instance of this class is generated by the
default network of environment: Network class has got a new method called
generateSecurityGroupManager which returns an appropriate implementation.
For pp 2-3 a new inheritor of Network class has been added to the Core
Library: an io.murano.resources.NovaNetwork. It generates FloatingIP
association resources if needed and returns a securityGroupName object
as one of the outputs of its joinInstance methods.
The Instance class has been modified to properly handle these types of
outputs.
The instance of the NovaNetwork class is generated at the API side
when a new Environment is created and a is assigned to the
defaultNetworks.environment property of the environment if the neutron
is not defined in keystone.
Also this change moves the auth_utils module from engine to common, as
Keystone Client it contains is now used by the API process as well.
This changed is based on some of the code from the outdated changeset
I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7
Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Co-Authored-By: Stan Lagun <slagun@mirantis.com>
DocImpact
Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1
Implements-Blueprint: nova-network-support
Network configuration is extracted into a separate file.
By default this is netconfig.yaml residing near murano.conf
But the name and path can be changed in config file.
Example of net-config can be found in etc/murano/netconfig.yaml.sample
If no file present old behavior is kept
Change-Id: I7b74eea69ee2ffe1c721b751e564b54252dcfbe3
Implements: blueprint configure-environment-network-defaults
Adds new API calls, responsible for add,
browse and delete categories.
Implements blueprint enable-category-management
Change-Id: I9da0680cfa244ef225be0706a54f492644c0dcba
* Set admin rule for a several API calls and remove direct check in code
* Now admin can configure policy.json and enable package management for regular users
* Update common policy module
Closes-Bug: #1412868
Change-Id: I8d0725b613564529d32a5acef289f4822f32915c
Also adds File type to core library for common convention type for files
Partially implements: blueprint actions-return-result
Change-Id: I5cbfb9ed6f4ae56e931815841f9c042f25a1d0ca
DEPLOYED status of sessions was not handled properly - the bug was
caused that if environment was deployed with failure it may remain with
this status even if it was deployed successfully after that.
Change-Id: I00309a7605067727eca4108e3fedac7bf0998847
Closes-Bug: #1413260
Remove gettextutils in favor of oslo.i18n suite for
internationalization purposes. Wrap murano.common.i18n around
oslo.i18n. Mark all logs messages of levels higher than
DEBUG for translation with _/_LI/_LW/_LE/_LC to conform with
oslo.i18n guidelines.
Change-Id: I09a2e2fc802e404f5c59fa4edd2a2124ad24101a
Implements: blueprint organize-translation
* E128 continuation line under-indented for visual indent
* H501 don't use locals() for formatting strings. to also check
* H402 first line of docstring should end with punctuation
* E122 continuation line missing indentation or outdented
* E713 test for membership should be 'not in'
Change-Id: I4a30350778a4452075e468400effcbc4155d24d8
Now, when we started to use oslo.serialization it is safe
to replace all the usages of anyjson with jsonutils from
oslo library.
oslo.serialization uses anyjson under the hood, so there
shouldn't be any performance changes.
Change-Id: I8d6fbfbf88e657f5586c7361de849683c064d2e2
Type of 'description' field must be Sql Text because description
of apps could easily be longer than 512 characters
Change-Id: I3764d6516b629022dff43f905efd532e8725e387
Closes-Bug: 1379064
Current API package search will return applications sorted by date
created in ascending order. Users naturally will want to see these
ordered alphabetically by name. Change default 'order_by' to name
Change-Id: I880fafb85729fea296ebd0330fcd976279d51a9c
Closes-Bug: 1391607
Removed #noqa from gettextutils and added them to import_exceptions.
I think it is better to specify option in one place (tox.ini) than
every time take care that you do not forget to specify this tag.
Also removed a few unused imports that were revealed in the process.
Change-Id: Ic4ca9cf374870075a36b88269ff8aea5a8e24a90
Deletion sessions were in 'deploying' state instead of 'deleting'.
Because of this such sessions could not be found by RPC result
processing code and remained in 'deploying' status causing UI
to display progress bar forever.
Also there were 2 duplicate SessionStates enums in the code with
and the second copy was outdated and didn't contained DELETING
status as well as other new session statuses. Because buggy code
was using that outdated enum it was necessary to merge both enums
into single declaration
Change-Id: I852f1f3dd1051c7b40afaa2575a4335b0f3c3104
Closes-Bug: #1386068
* H202 assertRaises Exception too broad
* H402 one line docstring needs punctuation
* H404 multi line docstring should start without a leading new line
Change-Id: I2f662b8b97d14daa501620c8237bf93bd2251243
Deployment is replaced with a more generic concept called 'action'.
Action can be performed on any object of Object Model.
Actions are marked with 'Usage: Action' in MuranoPL code. They can have arguments.
List of available actions can be obtained from Object Model itself after initial deployment.
This commit adds ability to REST API invoke actions by providing its unique id (from OM) and parameters.
Also refactors API code to use tasks.
Change-Id: If21809340bb799af58a8d1a2d148e52565028970
Partially-Implements: blueprint application-actions
Now environment deletion is done as a regular deployment that can fail.
Environments that are deleted, but deletion process has failed remain in database
and shown in dashboard with status 'delete failure'. Environments that are being deleted
has status 'deleting' and do not disappear before they really got deleted on engine side
Also improved status reporting for environments. Now it also reports status of last deployment -
'deploy failure', 'delete failure'
P.S. Functional tests were slightly refactored and fixed to reflect changes
in deletion logic
Change-Id: I05625dd71f7ca9559bb88319b26b122214f15019
Closes-Bug: #1325101
This patch incorporates two parts:
1. Fixed length of indexed fields in models.py. This was already done in Alembic
migration scripts, but missed the models.py
2. Fixed bug in DB migration testing where we checked current version of database
but due a race condidition queried wrong (SQLite instead of pre-configured MySQL)
database. Also, a small change is introduced in session.py to adopt protection
of session facade from other projects.
Change-Id: Ieb96d53afe0d6e4b7ebc306933ed9f781d7b10b8
Sets is_public=false in the database model, adds is_public querystring
to API and policy check for it. Will need corresponding client change.
Does not include an alembic migration because altering columns is
apparently difficult, and removing/adding does more damage than good.
New objects should have is_public set to False by the db model.
Relies on changes in https://review.openstack.org/#/c/109151/ (mainly
for unit test refactoring).
Change-Id: I390806b7e2d09f18ab3c496178406e1a3fc8d899
Closes-Bug: #1331243
Now we can use "murano-db-manage version" to print current
migration version applied to the database.
Change-Id: I933faaacd98b85cfa0b86bfc8c2c70d2548ac573
Closes-bug: #1342259
The JSON deserializer doesn't handle null values properly; it raises
a TypeError because it expects a string or buffer. This became apparent
after the supplier info update which added columns that would be null
for existing packages. Patch returns None from the JSON deserializer
in that case, and also refactors unittest base classes.
Change-Id: I6f31500d2dd1aaaf6a1fd26c1901666072d79776
Closes-Bug: #1342306
In 'Status' model, 'text' attribute is declared as String. SQLAlchemy
translates String type to VARCHAR type in MySQL dialect. When column
with VARCHAR type is declared in MySQL, user must specify
size of the column.
We can't specify size for text attribute because Reporter class from
core library that gives ability to publish different deployment statuses
from MuranoPL doesn't limit user input. Solution is to change attribute
type to Text.
Change-Id: If72e6db4b3c2039504f77649118ac9e1f68b8289
Closes-bug: #1341132
Problems we had:
1. If the collation of the DB and table is utf8 and the columns we want to
index are 512 chars, it is too big for the index since the max key length
is 767 bytes. Following error message observed:
(OperationalError) (1071, 'Specified key was too long; max key length is 767 bytes')
2. During migration to Alembic we missed the unique field for the
'fully_qualified_name' column of the table 'package'. This change
fixes that issue.
Fixes:
1. Reduce length of indexed columns to fix problem. 128 chars should be probably enough
2. Add uniqe paramater for column fully_qualified_name of table package
Additional changes:
* Test that column is unique in DB migration tests (on real databases)
* Introduced base for DB-related unit-tests. These tests use in-memory
instance of SQLite
* Test that column is unique in DB-related unit-tests
Closes-Bug: #1339201
Closes-Bug: #1339728
Change-Id: I4816790e11f225c5dbb130747535094fdf06733e
This commit adds the ability to include
additional information about a package's supplier
to the manifest.yaml file.
The format of the optional fields are as follows:
Supplier:
Name: <name of vendor>
CompanyUrl:
Link: <url>
Text: <anchor text>
Summary: <plain text summary of vendor>
Description: <html summary for dashboard>
Logo: <png filename of vendor logo in same dir>
Targets: blueprint additional-author-information
Change-Id: I9bb67089ad1bc554524ee828b9bfda38dc8251f6
This commit migrates Murano database migration framework from
sqlalchemy-migrate to Alembic. sqlalchemy-migrate is considered abandoned. All
other OpenStack projects are in process of migration to Alembic or have
already finished migration.
This change doesn't preserve all the migration scripts, it merges them into a
single migration script. That's what we would do even without Alembic. It's a
common practice to squash migration scripts into a single one right after the
release.
NOTICE:
CLI command to run migrations changed. Now it looks like this:
$ murano-db-manage upgrade
Migration path:
1. Simple solution is just to re-create your database from scratch
2. You can stamp your database with revision to make Alembic think that it
already applied the first migration:
$ tox -e venv -- murano-db-manage --config-file etc/murano/murano.conf stamp --revision head
For usage instructions see:
murano/db/migraiton/alembic_migrations/README
This patch also adds opportunistic tests for migrations. These tests will run in
OpenStack Infrastructure jenkins slaves. It'll use real MySQL and Postgres databases.
Tests will run in a "snake walk" manner, which means that upgrade and downgrade path
will be tested.
Base for migration tests is copied from Nova. Please note, that at this moment we
cannot use code from oslo, because it is not documented and is not tested. Once
test_migrations_base appears in project 'oslo.db', we'll be able to remove our own
test_migrations_base and use the one from 'oslo.db'.
implements: blueprint alembic-migrations
Change-Id: I5aa978f7095efc57f2d6fad81b5553e1880ad931