Commit Graph

52 Commits

Author SHA1 Message Date
shihanzhang
06746baa32 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: I24ca32d9e92cdd2b3a91916337839d116e335084
2017-04-03 12:11:20 +08:00
Huangsm
82bd5858e3 Order the packages for parameter search
Previously, list packages with parameter 'search',
search content in any attributes, but packages are unordered
With this change, packages are  ordered.
Such as: list package with 'search=mysql'
the packages whose name like 'mysql' are ahead of others
whose afn like 'mysql',and so on

Change-Id: Iecc3038ac40e731788975cff0cdf7d42e7f97112
Closes-Bug:#1467487
2016-11-04 17:55:40 -04:00
Duong Ha-Quang
8d09fd5820 Fix typo and make docstring more clear
Rewrite docstrings in murano/common/wsgi.py to make it
more clear. Fix typo in other files.

TrivialFix

Change-Id: Iaa6bde5f3b0356b6ac9e67f39ebf0f7feff4ee45
2016-10-04 15:29:16 +07:00
Cao Xuan Hoang
688dcd0f88 TrivialFix: Remove cfg import unused
This patch removes cfg import unused in
murano/api/middleware/version_negotiation.py
murano/api/versions.py
murano/db/catalog/api.py
murano/engine/system/test_fixture.py
murano/policy/model_policy_enforcer.py
murano/tests/unit/cmd/test_api_workers.py
murano/tests/unit/cmd/test_engine_workers.py

Change-Id: I0f12f15e4def84550ef5bde0840006211cac7809
2016-08-29 09:35:14 +07:00
OpenStack Proposal Bot
bc8943c8cc Modified docstrings to comply with pep8 H405 style check
Closes-Bug: #1563270

Change-Id: I987d846f1a86ecee1fcfb0f99877563a10481478
2016-05-10 20:18:23 +00:00
Nikolay Starodubtsev
8865876b1c Update _get_tags function to prevent race condition
Add `session.begin(nested=True` inside get tags to prevent duplicate
tags creation if murano executes in amount of threads.

Change-Id: I10968e2f2f758a5904943d273a9302a5cfb1a8c9
Closes-Bug: #1498097
2016-02-01 14:55:35 +03:00
Ravi Shekhar Jethani
859888d62a Python3: Replace dict.iteritems with six.iteritems
This also adds a check to murano/hacking/checks.py that
should catch this error in the future.

Blueprint murano-python-3-support
Change-Id: Ib8b2b91a40577d18739f0c1bf2c37b7cf4060560
2016-01-15 03:15:23 -08:00
Aaron-DH
a757b8f842 Modify filter by 'Name' in Package Definition
Search with name only matches package name and
search with keyword can matches names, tags ..etc

Change-Id: Ia86befcf0a126f4e532c847df44170a898b38824
Closes-Bug: #1500865
2016-01-09 17:23:43 +08:00
Jenkins
b0f6e230e4 Merge "Add an option to filter packages by 'id' in API" 2015-12-09 15:25:27 +00:00
Andrew Pashkin
ac9b85e1f5 Add an option to filter packages by 'id' in API
This patch enhances the GET /catalog/packages Murano API call with
option to filter packages by id.

Partially implements bp assign-category-button
Change-Id: I86bbdbde74a7fa086197adef7a91199704811f72
2015-12-09 16:07:42 +03:00
Jenkins
8bbc876278 Merge "Improve message for publish package conflict" 2015-12-02 11:02:11 +00:00
Jenkins
53a9673f64 Merge "Fix the delete package give the error with """" 2015-10-19 06:57:59 +00:00
Lin Yang
06f6d46008 Improve message for publish package conflict
Previously, end user will receive a conflict message "Class with the same full
name is already registered in the visibility scope" when publish package
conflict with one public package with same fqn. The "class" concept might be
not familiar to end user who doesn't know murano package details.

And in most cases, user will modify the class name in package. So it would be
better to check the duplicate fqn at first, which it give back a more straight
message "Package with the same Name is already made public" in this case.

Change-Id: I721ce60cb213fc446f62755b3b59523aca5d5fcb
Closes-Bug: #1506703
2015-10-16 11:48:56 +08:00
zhu.rong
2aa9f52d38 Fix a typo in db/catalog/api.py
a typo frn --> fqn

Change-Id: I8c16a7fd5b784e19dd296be339358415d1fb0710
2015-10-15 21:14:14 +08:00
zhu.rong
fe6fe4b538 Fix the delete package give the error with """
CLI delete package give the error with """

murano package-delete pakage_id

403 Forbidden: Package is not owned by the tenant
"0c5581f43593479d9c44d9ca1ed1ca69" (HTTP 403)

Change-Id: I4634eaab9eb5a29e70e33214b3a3803e411ec134
Closes-Bug: #1506466
2015-10-15 20:52:33 +08:00
Alexander Tivelkov
5c7e20a742 Retry-on-deadlock added to modify package db api
On some HA database deployments such as MySQL Galera the transaction
locks are optimistic, and if the same data gets modifyed concurrently
DB server may throw a DBDeadlock error which is intended to be handled
by the caller app.

This patch adds appropriate retrying logic into the catalog db api
methods, so the operation to upload, modify or delete the package gets
retryed if the deadlock occurs due to this optimistic locking behavior.
It uses oslo_db_api.wrap_db_retry helper decorator from oslo library.

Closes-Bug: #1502589
Change-Id: I8d7b3dfb02a31eba6d55ba0bf89ac0cf05f1313d
2015-10-08 15:49:51 +03:00
Jenkins
4505cf1bbe Merge "Show public packages for non-admin users" 2015-09-18 14:21:25 +00:00
Ekaterina Chernova
d193bf7f4e Show public packages for non-admin users
But only if modification of public packages are allowed in policy file.
Turn off managing public packagef for non-admin users by default.

Currently non-admin users are able to see only their own packages
despite of the policy setting.

Corresponding tests were updated.

Change-Id: I5510f6b8b82d7d28358e1b7dcbffe275079512ee
Closes-Bug: #1496418
2015-09-18 12:10:35 +03:00
xiangxinyong
f433cbce21 Support unicode characters in Applications or Packages's filter
1.Locate to these two pages:
Horizon->Murano->Application Catalog->Applications
Horizon->Murano->Manage->Package Definitions
2.Input the unicode characters in the Filter TextBox.
3.Click Filter Button.
4.Throw error informations.

Change-Id: Ib65457adbbce3f904a2570c35a638318bde33c9d
Closes-Bug: #1496267
2015-09-16 16:10:55 +08:00
Jenkins
72866cf808 Merge "Log refactoring close to new logging spec" 2015-09-09 23:14:05 +00:00
Nikolay Starodubtsev
975bcc4a98 Log refactoring close to new logging spec
* correct log levels
* correct log messages

partial-bug: 1419872
partial-implements-bp: unified-style-logging

Change-Id: I80c2b4c809dc52125f9f4a3beec0cd3b6c4c1213
2015-09-09 20:54:26 +03:00
Ekaterina Chernova
ea50bd3643 Add id to sort keys parameters while pagination
Currently default sort is a 'name'.
But in case there are more packages in catalog
with same name more then 'limit' parameter, all of them will be ignored.

Adding additional sort key is fixing that problem.

Change-Id: Idd86275dc48cbdb2862987f6e3151e1d9fb4e00f
Closes-Bug: #1448782
2015-09-09 16:24:27 +03:00
Ekaterina Chernova
0abdd93870 Add category list pagination support
Also sorting is added.
Default sort is done by category name.

Change-Id: I50254cb58f3924a5a1400859ea30d788e572729f
Closes-Bug: #1474932
2015-08-12 19:42:02 +03:00
Dmytro Dovbii
bef27a609b Delete ability to manage packages by name
Since package name is not unique anymore, abilty to get packages
from CLI by fqn was deleted

Change-Id: I3a205730adf8934460619b58d9d55c640c0bbf5e
Closes-Bug: #1447244
2015-07-28 15:16:56 +03:00
Dmytro Dovbii
7f3c642c5a Fix typo in error message
Change-Id: I8587fd50d17c61f5ccaceea1f70b4814ae279618
Closes-Bug: #1476281
2015-07-23 16:26:14 +03:00
Nikolay Starodubtsev
de5ed2115a Switch to oslo_log
To follow latest community practices we need to use oslo_log in
murano instead of handcrafted libs.
Unsupported log level 'audit' was changed to 'info'.

Change-Id: I40c0f3790e34bf80d0b63554e86b3cbc019eefca
2015-07-15 11:20:45 +03:00
Doug Hellmann
21a14acba4 Drop use of 'oslo' namespace package
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.

The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.

Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.

Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages

Change-Id: I975592f3694be42d52685ebf606f6d3012caf1a8
2015-04-28 20:21:22 +00:00
Kirill Zaitsev
58819afbec Streamline and simplify package filtering api
Introduce catalogue parameter to allow distinguishing between catalogue
of applications (packages that one can deploy) and managable-apps
(packages that one can edit).
Streamline include_disabled filtering.
Allow filtering out your own packages for admin requests.

Change-Id: I19a90b26abebe0833878c560169d6569a9d7421e
Partial-Bug: #1448226
Relates-Bug: #1448135
2015-04-28 20:56:49 +03:00
Alexander Tivelkov
f18661b586 Fix for cross-tenant package and class isolation
The database constraints which were present were enforcing the global
uniqueness of package FQNs and the names of classes defined in them.
This behavior was not correct, as the uniqueness should be enforced per
tenant, so the same package may be uploaded into two isolated tenants
without affecting each other.

This behavior lead to a very serious security issue: any tenant could
upload a package, leave it private and thus block all other tenants of
the cloud from uploading the package with the same name or even other
packages which contain at least one class in common with it. This could
be used to intentionally block all the operations of Murano on any
public environments.

This fix modifies the package name constraint to be unique only in
combination with owner_id, i.e. makes packages unique per tenant. Also
it removes the class name uniquness check from database (as there is no
cross-DB way to check it in a proper way) and adds a check method in
db.api module instead.

As the packages may be made public, this introduces a potential
collision: if the user owns some package, and there is a public package
with the same fully-qualified-name (or defining same class(es)) then the
class loader of the engine will have to choise between these packages
and/or classes defined in them.
To resolve this collision this commit adds a logic to fetch all the
patching packages and then pick the best match. Packages owned by the
current tenant are the most preferred, then the engine will pick public
packages, and non-owned non-public packages are the least preferred
(there may be no such packages now, they may appear when we add other
ways of package sharing).

Closes-bug: #1440094
Change-Id: I5c9b49642dfb6e955cf0c98b42f418da3b82060a
2015-04-14 20:58:49 +03:00
Jenkins
7e5e4b5642 Merge "Allow any admins to perform actions on packages" 2015-04-08 15:00:12 +00:00
Jenkins
bbc7d12826 Merge "Restricted text search on packages to text fields" 2015-04-08 11:01:49 +00:00
Ekaterina Chernova
57b303cf53 Allow any admins to perform actions on packages
This prevents from the situation, when tenant is no longer exist
and there is no way to delete an application

Change-Id: I1539d0331634e8c665ff056f1bfde4de47880985
Closes-Bug: #1436618
2015-04-08 13:22:40 +03:00
Andrew Pashkin
b7280da8d0 Restricted text search on packages to text fields
Change-Id: I707e4382986881dde3eaf84228a8a93466b8ca76
Closes-Bug: #1419743
2015-04-07 23:28:09 +03:00
Kirill Zaitsev
5542f47b61 Add package_count to Category objects
Partially implements blueprint enable-category-management
Change-Id: I8639586d6dad1b398d54ee6f4ac5755cb38ed8b0
2015-04-06 15:29:00 +00:00
Ekaterina Chernova
2c23f73e72 Implement category management API
Adds new API calls, responsible for add,
browse and delete categories.

Implements blueprint enable-category-management

Change-Id: I9da0680cfa244ef225be0706a54f492644c0dcba
2015-03-06 12:36:08 +03:00
Ekaterina Chernova
d82b74b62f Update API policy
* Set admin rule for a several API calls and remove direct check in code
* Now admin can configure policy.json and enable package management for regular users
* Update common policy module

Closes-Bug: #1412868

Change-Id: I8d0725b613564529d32a5acef289f4822f32915c
2015-03-02 15:28:14 +03:00
Kirill Zaitsev
4e036191ff Use oslo.i18n for translation
Remove gettextutils in favor of oslo.i18n suite for
internationalization purposes. Wrap murano.common.i18n around
oslo.i18n. Mark all logs messages of levels higher than
DEBUG for translation with _/_LI/_LW/_LE/_LC to conform with
oslo.i18n guidelines.

Change-Id: I09a2e2fc802e404f5c59fa4edd2a2124ad24101a
Implements: blueprint organize-translation
2015-02-19 18:51:22 +03:00
Ekaterina Chernova
017d25f49e Resolve and enable E265 pep8 issue
E265 block comment should start with '# '

Change-Id: I1a8b971776deb8b05b84be169571b9f1e573d2ba
2015-02-05 12:10:22 +03:00
Jenkins
cebe6629cb Merge "Change application index default sort from 'created' to 'name'" 2014-11-29 20:11:41 +00:00
Brian Tully
de3f842c2c Change application index default sort from 'created' to 'name'
Current API package search will return applications sorted by date
created in ascending order. Users naturally will want to see these
ordered alphabetically by name. Change default 'order_by' to name

Change-Id: I880fafb85729fea296ebd0330fcd976279d51a9c
Closes-Bug: 1391607
2014-11-14 16:05:38 -05:00
Dmitrii Dovbii
a55acc1266 Remove #noqa from gettextutils imports
Removed #noqa from gettextutils and added them to import_exceptions.
I think it is better to specify option in one place (tox.ini) than
every time take care that you do not forget to specify this tag.

Also removed a few unused imports that were revealed in the process.

Change-Id: Ic4ca9cf374870075a36b88269ff8aea5a8e24a90
2014-11-14 19:43:14 +02:00
Ekaterina Fedorova
d6a3a2b2e6 Enable H202, H402, H404 rules
* H202 assertRaises Exception too broad
* H402 one line docstring needs punctuation
* H404 multi line docstring should start without a leading new line

Change-Id: I2f662b8b97d14daa501620c8237bf93bd2251243
2014-09-01 09:10:51 +00:00
Ruslan Kamaldinov
2bfe3a9482 Migrate to oslo.db
Move from oslo-incubator db module to oslo.db

Change-Id: I01e04387ee400be92345a12de4f0f83a99980842
2014-08-23 16:35:07 +04:00
Angus Salkeld
b909d29058 Fix pep H101(TODO) and H231(py3 exceptions)
Change-Id: I6797c4df1595466a12f84721a383eb541e6401f7
2014-08-20 17:20:26 +10:00
Ruslan Kamaldinov
2174fc8a84 Fix DB migration script
Problems we had:
1. If the collation of the DB and table is utf8 and the columns we want to
   index are 512 chars, it is too big for the index since the max key length
   is 767 bytes. Following error message observed:
   (OperationalError) (1071, 'Specified key was too long; max key length is 767 bytes')

2. During migration to Alembic we missed the unique field for the
   'fully_qualified_name' column of the table 'package'. This change
   fixes that issue.

Fixes:
1. Reduce length of indexed columns to fix problem. 128 chars should be probably enough
2. Add uniqe paramater for column fully_qualified_name of table package

Additional changes:
* Test that column is unique in DB migration tests (on real databases)
* Introduced base for DB-related unit-tests. These tests use in-memory
  instance of SQLite
* Test that column is unique in DB-related unit-tests

Closes-Bug: #1339201
Closes-Bug: #1339728
Change-Id: I4816790e11f225c5dbb130747535094fdf06733e
2014-07-12 13:16:04 +00:00
Timur Sufiev
75f778ac30 Support specifying `sort_dir` key in packages.search call
Required to implement 'Previous Page' button at App Catalog page
in dashboard.

Change-Id: I9dc110be02e3aaf47c4c876569d01029da48359e
Targets: blueprint normalize-dashboard-pagination
2014-06-16 19:54:18 +04:00
Ekaterina Fedorova
da8ee01b65 Make categories optional parameter
Change-Id: I5960de1b7e01db491046b89f08fd95324d24551d
Closes-Bug: #1311033
2014-06-10 12:04:38 +04:00
Steve McLellan
da7fda4832 Allow package_get by name
If package lookup by id fails, filter by fully_qualified_name
and choose the first (and what will be only) result. Also allows
delete and update to use FQN as well as ID.

This may need revisiting to allow non-unique FQNs.

Change-Id: I84fb8e155b5aa4bc036f8b598101b6b969962325
Implements: blueprint id-fqn-equivalence
2014-05-29 09:32:43 -05:00
Jenkins
84bff3be68 Merge "Make DELETE request return 404 for nonexistant package" 2014-05-29 14:28:33 +00:00
Timur Sufiev
f0fad150ee Refactor api.catalog.search() method to provide 'next_marker' value
That way it works like Glance (see http://tinyurl.com/jwqtj23) and
allows for complex query pagination (to cycle through pages filtered
by some criteria, muranoclient uses 'next_marker' pointer).

Implements: blueprint app-catalog-pagination
Change-Id: If71da70fa835a1ee7a271ebf7238c0a4c64d4123
2014-05-29 13:44:40 +04:00