62 Commits

Author SHA1 Message Date
Jenkins
7e5e4b5642 Merge "Allow any admins to perform actions on packages" 2015-04-08 15:00:12 +00:00
Jenkins
5e909c98e3 Merge "Nova Network support" 2015-04-08 14:05:17 +00:00
Jenkins
bbc7d12826 Merge "Restricted text search on packages to text fields" 2015-04-08 11:01:49 +00:00
Ekaterina Chernova
57b303cf53 Allow any admins to perform actions on packages
This prevents from the situation, when tenant is no longer exist
and there is no way to delete an application

Change-Id: I1539d0331634e8c665ff056f1bfde4de47880985
Closes-Bug: #1436618
2015-04-08 13:22:40 +03:00
Alexander Tivelkov
796a0b2c9d Nova Network support
Adds a support for Nova Network if Neutron is not present in the
current OpenStack deployment.

Supporting the Nova Network requires modifications in three different
parts of generated Heat Stack:
	1) Generated Security Groups and their rules should be of type
	   'AWS::EC2::SecurityGroup', not 'OS::Neutron::SecurityGroup'
	2) Security Group assignments should go to security_groups property
	   of Instance resource, not the network port (as port concept is
	   not present when using NovaNetwork)
	3) FloatingIP should be of type OS::Nova::FloatingIP and should be
	   associated with an Instance by OS::Nova::FloatingIPAssociation
	   resource.

To achieve p1 a SecurityGroupManager class of Core Library is made
abstract and is inherited by two concrete implementations:
NeutronSecurityGroupManager (containing the old MuranoPL code which
generated templates based on OS::Neutron::SecurityGroup) and a new
AwsSecurityGroupManager, which generates AWS-compliant firewall rules
which are consumed by NovaNetwork.

The particular concreate instance of this class is generated by the
default network of environment: Network class has got a new method called
generateSecurityGroupManager which returns an appropriate implementation.

For pp 2-3 a new inheritor of Network class has been added to the Core
Library: an io.murano.resources.NovaNetwork. It generates FloatingIP
association resources if needed and returns a securityGroupName object
as one of the outputs of its joinInstance methods.
The Instance class has been modified to properly handle these types of
outputs.
The instance of the NovaNetwork class is generated at the API side
when a new Environment is created and a is assigned to the
defaultNetworks.environment property of the environment if the neutron
is not defined in keystone.

Also this change moves the auth_utils module from engine to common, as
Keystone Client it contains is now used by the API process as well.

This changed is based on some of the code from the outdated changeset
I6f4b7908bd4bbcd375f64705c7dd06e3954f1ec7

Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Co-Authored-By: Stan Lagun <slagun@mirantis.com>

DocImpact

Change-Id: I4c48f33de100a5730ba1d086540d0d99e8fbf9b1
Implements-Blueprint: nova-network-support
2015-04-08 13:09:21 +03:00
Andrew Pashkin
b7280da8d0 Restricted text search on packages to text fields
Change-Id: I707e4382986881dde3eaf84228a8a93466b8ca76
Closes-Bug: #1419743
2015-04-07 23:28:09 +03:00
Kirill Zaitsev
5542f47b61 Add package_count to Category objects
Partially implements blueprint enable-category-management
Change-Id: I8639586d6dad1b398d54ee6f4ac5755cb38ed8b0
2015-04-06 15:29:00 +00:00
Jenkins
7a08d3b81f Merge "Removed redundant code from models module" 2015-04-05 18:57:12 +00:00
Andrew Pashkin
ea654c0c72 Fixed wrong super call in models.TimestampMixin
Change-Id: I003f8add2ab3f479681517e8a707db84fd11d1b2
2015-03-26 20:07:49 +03:00
Andrew Pashkin
67bb73395e Removed redundant code from models module
It's senseless to have a method with only super call.

Change-Id: Ia5b36d86073291c9f51b50e3ffbd4f73a8628158
2015-03-26 16:07:31 +00:00
Stan Lagun
0d42f01f68 Configurable environment's default network config
Network configuration is extracted into a separate file.
By default this is netconfig.yaml residing near murano.conf
But the name and path can be changed in config file.
Example of net-config can be found in etc/murano/netconfig.yaml.sample
If no file present old behavior is kept

Change-Id: I7b74eea69ee2ffe1c721b751e564b54252dcfbe3
Implements: blueprint configure-environment-network-defaults
2015-03-17 20:26:41 +03:00
Ekaterina Chernova
2c23f73e72 Implement category management API
Adds new API calls, responsible for add,
browse and delete categories.

Implements blueprint enable-category-management

Change-Id: I9da0680cfa244ef225be0706a54f492644c0dcba
2015-03-06 12:36:08 +03:00
Jenkins
42c320a085 Merge "Update API policy" 2015-03-02 14:28:05 +00:00
Ekaterina Chernova
d82b74b62f Update API policy
* Set admin rule for a several API calls and remove direct check in code
* Now admin can configure policy.json and enable package management for regular users
* Update common policy module

Closes-Bug: #1412868

Change-Id: I8d0725b613564529d32a5acef289f4822f32915c
2015-03-02 15:28:14 +03:00
Stan Lagun
73f8368024 Adds API to obtain action result
Also adds File type to core library for common convention type for files

Partially implements: blueprint actions-return-result

Change-Id: I5cbfb9ed6f4ae56e931815841f9c042f25a1d0ca
2015-03-02 14:08:36 +03:00
Jenkins
1c139cd1b8 Merge "Environment Template entity and its service" 2015-02-27 13:43:13 +00:00
Henar Muñoz Frutos
47351c5dac Environment Template entity and its service
Change-Id: Ic46463e9e4900e2b9e3cf7aab4b9c5893da24344
2015-02-26 11:13:17 +01:00
Jenkins
41c4b99045 Merge "Fixed logic of determining environment status" 2015-02-25 12:47:23 +00:00
Andrew Pashkin
4ae29f2672 Fixed logic of determining environment status
DEPLOYED status of sessions was not handled properly - the bug was
caused that if environment was deployed with failure it may remain with
this status even if it was deployed successfully after that.

Change-Id: I00309a7605067727eca4108e3fedac7bf0998847
Closes-Bug: #1413260
2015-02-25 14:19:12 +03:00
Kirill Zaitsev
4e036191ff Use oslo.i18n for translation
Remove gettextutils in favor of oslo.i18n suite for
internationalization purposes. Wrap murano.common.i18n around
oslo.i18n. Mark all logs messages of levels higher than
DEBUG for translation with _/_LI/_LW/_LE/_LC to conform with
oslo.i18n guidelines.

Change-Id: I09a2e2fc802e404f5c59fa4edd2a2124ad24101a
Implements: blueprint organize-translation
2015-02-19 18:51:22 +03:00
Andrew Pashkin
2b4457ada9 Changed models to match migrations
Some fields in models and in migrations was
different.

Closes-Bug: #1415436
Change-Id: I4b8cbbea567d2a3a79f08ce9e72b4b3427719509
2015-02-06 16:04:20 +03:00
Ekaterina Chernova
017d25f49e Resolve and enable E265 pep8 issue
E265 block comment should start with '# '

Change-Id: I1a8b971776deb8b05b84be169571b9f1e573d2ba
2015-02-05 12:10:22 +03:00
Ekaterina Chernova
87a63bf24e Resolve H305 pep8 issue
H305  imports not grouped correctly

Change-Id: Ia990dd628da195097838a13d876ef987363d3f8e
2015-02-05 12:10:22 +03:00
Ekaterina Chernova
8a15466301 Enable and resolve some PEP8 issues
* E128 continuation line under-indented for visual indent
* H501 don't use locals() for formatting strings. to also check
* H402 first line of docstring should end with punctuation
* E122 continuation line missing indentation or outdented
* E713 test for membership should be 'not in'

Change-Id: I4a30350778a4452075e468400effcbc4155d24d8
2015-02-05 12:10:18 +03:00
Ruslan Kamaldinov
734d407332 Replace anyjson with oslo.serialization
Now, when we started to use oslo.serialization it is safe
to replace all the usages of anyjson with jsonutils from
oslo library.
oslo.serialization uses anyjson under the hood, so there
shouldn't be any performance changes.

Change-Id: I8d6fbfbf88e657f5586c7361de849683c064d2e2
2014-12-13 18:10:53 +03:00
Jenkins
afbabf4703 Merge "Use oslo.utils" 2014-12-12 18:54:32 +00:00
Ruslan Kamaldinov
bf05953c28 Use oslo.utils
Files from oslo.utils will go away once Murano
migrates to oslo.serialization.

Change-Id: Ib3711164562bea4ac30a96384733d77753ffa69c
2014-12-11 01:49:37 +03:00
Dmytro Dovbii
dd3fd29ee0 Change the package 'description' field type
Type of 'description' field must be Sql Text because description
of apps could easily be longer than 512 characters

Change-Id: I3764d6516b629022dff43f905efd532e8725e387
Closes-Bug: 1379064
2014-12-10 11:35:39 +02:00
Jenkins
cebe6629cb Merge "Change application index default sort from 'created' to 'name'" 2014-11-29 20:11:41 +00:00
Brian Tully
de3f842c2c Change application index default sort from 'created' to 'name'
Current API package search will return applications sorted by date
created in ascending order. Users naturally will want to see these
ordered alphabetically by name. Change default 'order_by' to name

Change-Id: I880fafb85729fea296ebd0330fcd976279d51a9c
Closes-Bug: 1391607
2014-11-14 16:05:38 -05:00
Dmitrii Dovbii
a55acc1266 Remove #noqa from gettextutils imports
Removed #noqa from gettextutils and added them to import_exceptions.
I think it is better to specify option in one place (tox.ini) than
every time take care that you do not forget to specify this tag.

Also removed a few unused imports that were revealed in the process.

Change-Id: Ic4ca9cf374870075a36b88269ff8aea5a8e24a90
2014-11-14 19:43:14 +02:00
Stan Lagun
c301b5e763 Environment in delete failed state was in progress forever
Deletion sessions were in 'deploying' state instead of 'deleting'.
Because of this such sessions could not be found by RPC result
processing code and remained in 'deploying' status causing UI
to display progress bar forever.

Also there were 2 duplicate SessionStates enums in the code with
and the second copy was outdated and didn't contained DELETING
status as well as other new session statuses. Because buggy code
was using that outdated enum it was necessary to merge both enums
into single declaration

Change-Id: I852f1f3dd1051c7b40afaa2575a4335b0f3c3104
Closes-Bug: #1386068
2014-11-10 18:43:55 +00:00
Ekaterina Fedorova
d6a3a2b2e6 Enable H202, H402, H404 rules
* H202 assertRaises Exception too broad
* H402 one line docstring needs punctuation
* H404 multi line docstring should start without a leading new line

Change-Id: I2f662b8b97d14daa501620c8237bf93bd2251243
2014-09-01 09:10:51 +00:00
Stan Lagun
e47ef4e849 Adds REST API endpoint for action execution
Deployment is replaced with a more generic concept called 'action'.
Action can be performed on any object of Object Model.
Actions are marked with 'Usage: Action' in MuranoPL code. They can have arguments.
List of available actions can be obtained from Object Model itself after initial deployment.
This commit adds ability to REST API invoke actions by providing its unique id (from OM) and parameters.
Also refactors API code to use tasks.

Change-Id: If21809340bb799af58a8d1a2d148e52565028970
Partially-Implements: blueprint application-actions
2014-08-27 17:09:29 +04:00
Ruslan Kamaldinov
2bfe3a9482 Migrate to oslo.db
Move from oslo-incubator db module to oslo.db

Change-Id: I01e04387ee400be92345a12de4f0f83a99980842
2014-08-23 16:35:07 +04:00
Angus Salkeld
b909d29058 Fix pep H101(TODO) and H231(py3 exceptions)
Change-Id: I6797c4df1595466a12f84721a383eb541e6401f7
2014-08-20 17:20:26 +10:00
Stan Lagun
56b2d5df27 Fixes silent deletion of environments
Now environment deletion is done as a regular deployment that can fail.
Environments that are deleted, but deletion process has failed remain in database
and shown in dashboard with status 'delete failure'. Environments that are being deleted
has status 'deleting' and do not disappear before they really got deleted on engine side

Also improved status reporting for environments. Now it also reports status of last deployment -
'deploy failure', 'delete failure'

P.S. Functional tests were slightly refactored and fixed to reflect changes
in deletion logic

Change-Id: I05625dd71f7ca9559bb88319b26b122214f15019
Closes-Bug: #1325101
2014-08-19 12:52:52 +04:00
Jenkins
c00284c365 Merge "Default is_public to false" 2014-08-14 08:56:32 +00:00
Ruslan Kamaldinov
59f8b854a7 Fixed race in DB migration testing
This patch incorporates two parts:
1. Fixed length of indexed fields in models.py. This was already done in Alembic
   migration scripts, but missed the models.py
2. Fixed bug in DB migration testing where we checked current version of database
   but due a race condidition queried wrong (SQLite instead of pre-configured MySQL)
   database. Also, a small change is introduced in session.py to adopt protection
   of session facade from other projects.

Change-Id: Ieb96d53afe0d6e4b7ebc306933ed9f781d7b10b8
2014-08-01 12:41:50 -07:00
Steve McLellan
f5daa12347 Default is_public to false
Sets is_public=false in the database model, adds is_public querystring
to API and policy check for it. Will need corresponding client change.

Does not include an alembic migration because altering columns is
apparently difficult, and removing/adding does more damage than good.
New objects should have is_public set to False by the db model.

Relies on changes in https://review.openstack.org/#/c/109151/ (mainly
for unit test refactoring).

Change-Id: I390806b7e2d09f18ab3c496178406e1a3fc8d899
Closes-Bug: #1331243
2014-07-31 20:01:28 +04:00
Sergey Reshetnyak
b50b209a9d Use with_variant method for dialects db types
* Fix type in models

Change-Id: I8075044c98946445f6c7e2ad259ef5a8d46bd73e
2014-07-29 18:30:05 +04:00
Ruslan Kamaldinov
a25e041242 Add version command to murano-db-manage
Now we can use "murano-db-manage version" to print current
migration version applied to the database.

Change-Id: I933faaacd98b85cfa0b86bfc8c2c70d2548ac573
Closes-bug: #1342259
2014-07-20 19:09:55 +04:00
Steve McLellan
3509b530d0 Handle null JSON blobs
The JSON deserializer doesn't handle null values properly; it raises
a TypeError because it expects a string or buffer. This became apparent
after the supplier info update which added columns that would be null
for existing packages. Patch returns None from the JSON deserializer
in that case, and also refactors unittest base classes.

Change-Id: I6f31500d2dd1aaaf6a1fd26c1901666072d79776
Closes-Bug: #1342306
2014-07-16 10:52:43 -05:00
Jenkins
fe6bc63390 Merge "Fix DB migration script" 2014-07-16 15:08:48 +00:00
Serg Melikyan
2611893c20 Fix issue with incorrect model on MySQL
In 'Status' model, 'text' attribute is declared as String. SQLAlchemy
translates String type to VARCHAR type in MySQL dialect. When column
with VARCHAR type is declared in MySQL, user must specify
size of the column.

We can't specify size for text attribute because Reporter class from
core library that gives ability to publish different deployment statuses
from MuranoPL doesn't limit user input. Solution is to change attribute
type to Text.

Change-Id: If72e6db4b3c2039504f77649118ac9e1f68b8289
Closes-bug: #1341132
2014-07-12 18:55:18 +04:00
Ruslan Kamaldinov
2174fc8a84 Fix DB migration script
Problems we had:
1. If the collation of the DB and table is utf8 and the columns we want to
   index are 512 chars, it is too big for the index since the max key length
   is 767 bytes. Following error message observed:
   (OperationalError) (1071, 'Specified key was too long; max key length is 767 bytes')

2. During migration to Alembic we missed the unique field for the
   'fully_qualified_name' column of the table 'package'. This change
   fixes that issue.

Fixes:
1. Reduce length of indexed columns to fix problem. 128 chars should be probably enough
2. Add uniqe paramater for column fully_qualified_name of table package

Additional changes:
* Test that column is unique in DB migration tests (on real databases)
* Introduced base for DB-related unit-tests. These tests use in-memory
  instance of SQLite
* Test that column is unique in DB-related unit-tests

Closes-Bug: #1339201
Closes-Bug: #1339728
Change-Id: I4816790e11f225c5dbb130747535094fdf06733e
2014-07-12 13:16:04 +00:00
Ankur Rishi
2b4be05007 Add optional fields to packages for supplier info
This commit adds the ability to include
additional information about a package's supplier
to the manifest.yaml file.

The format of the optional fields are as follows:

Supplier:
  Name: <name of vendor>
  CompanyUrl:
    Link: <url>
    Text: <anchor text>
  Summary: <plain text summary of vendor>
  Description: <html summary for dashboard>
  Logo: <png filename of vendor logo in same dir>

Targets: blueprint additional-author-information
Change-Id: I9bb67089ad1bc554524ee828b9bfda38dc8251f6
2014-07-10 11:44:07 -07:00
Jenkins
fc0135390a Merge "Added DB migrations on Alembic" 2014-06-23 11:39:25 +00:00
Ruslan Kamaldinov
9030bc7285 Added DB migrations on Alembic
This commit migrates Murano database migration framework from
sqlalchemy-migrate to Alembic. sqlalchemy-migrate is considered abandoned. All
other OpenStack projects are in  process of migration to Alembic or have
already finished migration.

This change doesn't preserve all the migration scripts, it merges them into a
single migration script. That's what we would do even without Alembic. It's a
common practice to squash migration scripts into a single one right after the
release.

NOTICE:
CLI command to run migrations changed. Now it looks like this:
$ murano-db-manage upgrade

Migration path:
1. Simple solution is just to re-create your database from scratch
2. You can stamp your database with revision to make Alembic think that it
   already applied the first migration:
   $ tox -e venv -- murano-db-manage --config-file etc/murano/murano.conf stamp --revision head

For usage instructions see:
murano/db/migraiton/alembic_migrations/README

This patch also adds opportunistic tests for migrations. These tests will run in
OpenStack Infrastructure jenkins slaves. It'll use real MySQL and Postgres databases.
Tests will run in a "snake walk" manner, which means that upgrade and downgrade path
will be tested.
Base for migration tests is copied from Nova. Please note, that at this moment we
cannot use code from oslo, because it is not documented and is not tested. Once
test_migrations_base appears in project 'oslo.db', we'll be able to remove our own
test_migrations_base and use the one from 'oslo.db'.

implements: blueprint alembic-migrations
Change-Id: I5aa978f7095efc57f2d6fad81b5553e1880ad931
2014-06-20 18:01:34 +04:00
Timur Sufiev
75f778ac30 Support specifying `sort_dir` key in packages.search call
Required to implement 'Previous Page' button at App Catalog page
in dashboard.

Change-Id: I9dc110be02e3aaf47c4c876569d01029da48359e
Targets: blueprint normalize-dashboard-pagination
2014-06-16 19:54:18 +04:00