Commit Graph

582 Commits (cb80df4f4ab0e8de16d53c7fddff7c44dedd3f94)

Author SHA1 Message Date
Zuul 037cfb6e98 Merge "Don't return the sensitive information to user" 6 years ago
huangtianhua 8cdfc3b293 Don't return the sensitive information to user
We return back the sensitive information to user
when some exceptions happened, for example,
when DBError happened, we return the whole sql
statement to user, it's not safe.
This patch changes to return the message if the
exception is the HeatException, otherwise the message
won't be revealed to user.

Change-Id: I6e01b1003a39106274e79c3b413917a30b5651b6
Closes-Bug: #1708122
6 years ago
rabi 01b5878129 Add REST api support for cancel without rollback
We already have REST api support for cancelling a
UPDATE_IN_PROGRESS stack with rollback. This adds a
new action 'cancel_without_rollback' to allow for
canceling a create/update in_progress stack without


Change-Id: I6c6ffa0502ab8745cfb2f9c5ef263f1e02dfc4ca
Closes-Bug: #1709041
6 years ago
rabi 46adb6649d Fix error for invalid auth_encryption_key
cryptography module throws an InvalidToken exception that does
not have any error message.

Change-Id: Ied520cd9ff60032fad1ae13e4a81dad6847d82ab
Closes-Bug: #1711047
6 years ago
ricolin 552f94b928 Add converge flag in stack update for observing on reality
Add converge parameter for stack update API and RPC call,
that allow triggering observe on reality. This will be
triggered by API call with converge argument (with True
or False value) within. This flag also works for resources
within nested stack.
Implements bp get-reality-for-resources

Change-Id: I151b575b714dcc9a5971a1573c126152ecd7ea93
6 years ago
liyi 8f10215ffd Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.


Change-Id: Ieec8028305099422e1b0f8fc84bc90c9ca6c694f
6 years ago
Zane Bitter 5392273af8 Fix missing spaces in strings
Where strings are continued across multiple lines, it's easy to miss a
space. This fixes several incorrect strings, cleans up some formatting
(don't line break inside a format specifier) and fixes typos.

Change-Id: Id781b16cad03c32973a67c02ad22ce4459c6d126
6 years ago
Lu lei 1c714d9e5b Use i18n to translate msg in
1.Help messages and output info should support oslo.i18n. So we
should use oslo.i18n in LOG.warning()/LOG.error()/

2.Not all files need to translate msg, tests directory and dev directory
are faced with developers, so they need to nothing.

The more information as follow:

Co-Authored-By: shizhihui <>
Change-Id: Iced712f8833e90f3570497cd033713bdc34c27da
7 years ago
Zane Bitter f310a1f6bc Handle ambiguous physical resource IDs
It's possible that we could end up with multiple resources with the same
physical resource ID, but that would be undetectable since we return only
one from the database layer. This change allows us to detect the problem an
return an error where the result is rendered ambiguous.

Change-Id: I2c5ddbe6731c33a09ec7c4a7b91dcfe414da4385
7 years ago
huangtianhua 6c26eb8df6 Fix a small nit
To fix a small nit of bug/1641812.

Change-Id: I9c2c6e605541a6fabe3707d63bcbdd27282c74e6
7 years ago
huangtianhua 6afc7a68d0 Includes project info while admin getting stacks
To return the information of project when super admin
getting stacks across tenants.

Change-Id: I40c0d61325752428b3ccf57706a825b47ae231c9
Closes-Bug: #1641812
7 years ago
rabi 2407c86436 Ignore merged env if environment files are specified
Ignore merged environment from the client, if there are
'environment_files' in the request for merging in the server.
This would avoid the duplicates when 'parameter_merge_strategies'
are specified in the first env file.

Change-Id: I4ee274d69e42a47c4b8fd5a25567fb2eb144a8cc
Related-Bug: #1635409
7 years ago
huangtianhua 62f6045f2a Return condition functions based on the filter param
Add 'with_condition_func' filter param for API
template-function-list, if the param set to true,
the response will include the condition functions.

Change-Id: Icdfbafbb98698373648ff2d78db3c45fe2b924ee
Closes-Bug: #1625505
7 years ago
rabi 1f88c675c4 Remove tenant_safe flag from rpc and db api
We can use admin_context to have access to stacks
and software configs across projects. This removes
the tenant_safe flag from rpc and db api. This is
backward compatible with older rpc clients.

We still support use of global_tenant flag for listing
stacks and software configs. However, by default
an admin(user with admin role in admin_project)
would not need that.

Change-Id: I12303dbf30fb80290f95baba0c67cdf684f5f409
7 years ago
Jenkins bb4fb01cec Merge "Remove unnecessary checks" 7 years ago
Thomas Herve 9a248ad6c5 Check for empty sections in environment
Passing empty section like parameters in the environment can leave the
user with a strange error message. This fixes the issue by providing a
correct error.

Change-Id: I766cd38e8a70c9f76cf10f540e22a929eaed3768
Closes-Bug: #1606148
7 years ago
Dmitriy Uvarenkov 38db9f440e Remove unnecessary checks
Service's _get_stack method already checks
whether the stack is None and if it is, throws
an exception. So methods that use _get_stack
do not need to check the same thing.
Also, looks like there is no way that stack's
self.raw_template.template or self.raw_template.template
can be None or blank, so removed such checks as well
with corresponding tests.

Change-Id: I83d6e9251e0f7b6cacb0ddcfe1a3bc5dcf57d1c3
7 years ago
Steven Hardy 316b5b6381 Add stack files retrieval API
Similar to the recent addition that enables retrieval of the current
environment for a stack, this enables reading the current files map
for a running stack, which is useful if you want to introspect the
current state, and/or deploy another similar stack without necessarily
having the exact command/repo used initially.

Implements: blueprint files-show

Change-Id: I3198b6a7dc06648af24c198d39470f3b0d5d6f11
7 years ago
ChangBo Guo(gcb) ffda52200e Config: no need to set default=None
By default oslo.cfg sets the default value as None. There is no
need to explicitly do this.


Change-Id: I8d47b8815947c1c1e22e49822ffec0b8874d4487
7 years ago
Jenkins 257e0a9f67 Merge "Implement event list nested-depth" 7 years ago
rabi ac86702172 Authorize super admin actions on all projects
This allows admin super user (user with admin role in admin_project)
to do stack operations across all projects.

Change-Id: Ifbf56fde02b89248ee788e6a212ef9d11e665dc0
Partial-Bug: #1466694
7 years ago
Steve Baker 32ade7a243 Implement event list nested-depth
The GET call to list events will support a nested_depth parameter. The
response will have an additional links url with the ref 'root_stack'
to indicate that this API supports nested_depth queries.

This has the following consequences for old/new combinations of
- new heatclient, new server - nested_depth param is set, server
  returns nested events
- new heatclient, old server - nested_depth param is set, server
  returns events with no root_stack, heatclient falls back to
  recursive event fetch
- old heatclient, new server - nested_depth param is never set,
  recursive event fetch works as before

Here are some timings for a TripleO overcloud stack with ~700 events.

Current heat and python-heatclient master:
time openstack stack event list --nested-depth 4 overcloud |wc -l
real    0m17.500s

This change, with heatclient 31278ff5f77b152b5ef7a4197e15c441c72ff163:
time openstack stack event list --nested-depth 4 overcloud  |wc -l
real    0m1.725s

The difference in event count (744 vs 608) is due to the stack events
being filtered out for stacks with zero resources - these are a source
of unnecessary noise so their removal should be considered an improvement.

Closes-Bug: #1588561

Change-Id: I27e1ffb770e00a7f929c081b2a505e2007f5d584
7 years ago
Zane Bitter 5566e6f2c8 Get rid of gratuitous use of six.iterkeys()
In Python3, dict.keys() returns a view object rather than a list. This
behaves differently in that changes to the dict also modify the view, and
in that the view type interacts with various operators in different ways to

One universally correct transformation to preserve Python2 behaviour in
Python3 would be to replace all instances of d.keys() with
list(six.iterkeys(d)), and indeed we did. However, like many automatic
transformations the results are usually unsightly, invariably inefficient,
and frequently absurd. Not least because list(d.keys()) and indeed list(d)
are also equivalent.

This patch changes to using the simplest correct method of accessing the
data we want in each case.

This reverts or rewrites most of commit

Change-Id: Iba3cf48246d8cbc958d8fb577cd700a218b0bebf
7 years ago
Zane Bitter bfa7f8554b Get rid of Parameters._validate_template_parameters()
Apart from the fact that this method relied on knowledge of the template
format, it also checked nothing useful since we now won't even be able to
create the Parameters object unless all of the parameter definitions are in
fact dicts.

Change-Id: I950420b9016d350a427754d6a482b6ddf456bd6f
7 years ago
Peter Razumovsky f5b64b3e4d Add to resource_schema returning description
Add to resource_schema command ability to return
resource description.


Change-Id: I148bb7764af4db45b75ce63d7fa58daa210b3ed4
Closes-bug: #1558049
7 years ago
Peter Razumovsky 42bfc8a282 Add to resource-type-list returning description
Add to resource-type-list command ability to return
resource description.


Change-Id: Ia92699e5ca60a891e9704c468e447e1053be7609
Closes-bug: #1558049
7 years ago
Jay Dobies b0ca694dd7 Add environment retrieval API
Adds a call to the REST API to retrieve the environment for a running

Implements: blueprint environment-show

Change-Id: I7e3577dfc854018245d79afdfee45a9d250d73a7
7 years ago
Jenkins 92db7d590f Merge "Fix for resource search with invalid filter keys" 7 years ago
Kanagaraj Manickam 971723f83d Fix for resource search with invalid filter keys
When invalid fileters are provided as part of resource
filtering, it will raise 400 http status code (bad request)

Closes-bug: #1554333
Change-Id: I27709b1b73e4ed43367a25894d4ca7c9575b0679
7 years ago
Jenkins 9345e1648f Merge "Use oslo.utils.reflection to extract class name" 7 years ago
Jenkins 3e0e5bdc3c Merge "Return HTTPBadRequest error from API for immutable parameters" 7 years ago
Jenkins 31c39e74b0 Merge "Map RequestLimitExceeded exception in heat-cfn-api" 7 years ago
Jason Dunsmore 8627aa2b9a Return HTTPBadRequest error from API for immutable parameters
Return HTTPBadRequest error from API instead of
HTTPInternalServerError when immutable parameters are modified.

Change-Id: I37ff6dd7295292b8cd08a7a5085590b654a7e4f2
7 years ago
Ahmed Elkhouly f52d77f0d2 Resource mark unhealthy RPC and ReST API
There may exist resources that the user (or application) knows are
unhealthy where Heat has no way of determining that.

Add a PATCH handler to the Resource endpoint::


The PATCH method will accept a JSON body of the form::

    'mark_unhealthy': <bool>,
    'resource_status_reason': <string>

This patch Implements:
- RPC API to mark resources as CHECK_FAILED in both the legacy and
convergence architectures in heat-engine
- ReST front end to the RPC API call in heat-api

Change-Id: Ifa48b179723a2100fff548467db9e162bc669d13
Partially-implements: blueprint mark-unhealthy
7 years ago
Jenkins 714627b39b Merge "JSON size violation gives a bad error with nested templates" 7 years ago
Kamal Hussain d32a0a517e Map RequestLimitExceeded exception in heat-cfn-api
The heat-cfn-api wasn't handling the RequestLimitExceeded error.
Instead it returned generic "Internal Server Error". The fix is to
correctly map RequestLimitedExceeded exception in heat-cfn-api.
Added unit test to verify mapping of RequestLimitExceeded error.

Change-Id: I6e13b2f6b1458fb7c194632db67eb2cebdb173d2
Closes-Bug: #1542559
7 years ago
Jenkins 471a34b2ba Merge "Stack resource search" 7 years ago
Kanagaraj Manickam 6c4709696b Stack resource search
heat resource-list -f <filters>

imlements blueprint heat-stack-resource-search


Change-Id: Iaae88f3b32bc2ba7b41a4078ef3aa8ffc07079b7
7 years ago
Bo Wang 4136b47666 Use oslo.utils.reflection to extract class name
The oslo.utils.reflection.get_class_name() handles more variations
of where a class name may come from (on) python 2 and python 3.
Its usage allows getting more accurate class names so we'd better use it.

Change-Id: Iff0e109a558d0185f126781369bac216da930bca
7 years ago
Jenkins 507952e94f Merge "Fix update preview to handle nested stacks" 7 years ago
Kanagaraj Manickam 692059eeb0 Adds ignore_errors for template_validate REST API
Adds additional parameter ignore_errors to ignore set
of errors during the template valation in API.


implements blueprint heat-template-validate-improvements

Change-Id: Ib8a6d6be4509857bbc90c8f604a882b7a706da10
7 years ago
Steven Hardy 4cfd9a10ac Fix update preview to handle nested stacks
Currently the update preview code has no support for previewing
the effect of an update on nested stacks, which I assume was an
oversight in the original implementation.  So this adds a show_nested
flag to the API which allows enabling recursive preview of the whole
update including nested stacks.

Closes-Bug: #1521971
Depends-On: I06f3b52d5d48dd5e6e266321e58ca8e6116d6017
Change-Id: I96af4d2f07056846aac7ae9ad9b6eb160e8bd51a
7 years ago
Jenkins 35b90e3fc5 Merge "Add environment_files to REST and RPC APIs" 7 years ago
Jenkins f97eeede5e Merge "Optimize the fault map for EntityNotFound" 7 years ago
Jay Dobies c2d8a15b24 Add environment_files to REST and RPC APIs
Adds an optional list of environment files that can be specified instead
of resolving multiple environments client-side. This change covers both
the REST API changes and the internal RPC API changes.

Implements: blueprint multi-environments
Change-Id: I45fb9d4480f6ab4e5fe30cb2e3b62a03777484c7
7 years ago
Jenkins 3fdf463136 Merge "Add REST API for stack export" 7 years ago
Zane Bitter c701226cde ReST API: Use constants in get_allowed_params whitelists
Change-Id: I97663c1fb146f9e8350fead5db5f47f753cb2ef5
7 years ago
Zane Bitter f0fa7312a0 Don't calculate attributes for metadata request
The RPC API call describe_stack_resource is used by two separate ReST API
calls and one CFN API call: and Resource.metadata in heat-api
and StackController.describe_stack_resource. The metadata call in
particular throws away all of the results of the describe_stack_resource
output except for the metadata itself. Either this call or the cfn one
(depending on configuration) is also used very, very often: it's the one
that every instance running os-collect-config polls for information about
software deployments (so, to a first approximation, every server in every
stack will be calling this once every 30s).

Since blueprint detailed-resource-show merged in
6d8a5cb35c, every call to the RPC
describe_stack_resource API has resulted in Heat fetching the attribute
values for every single attribute in the resource from its corresponding
OpenStack API (including those calls originating from a request for just
the metadata, or from the cfn API which similarly discards the attribute

This is insanely inefficient (we're making OpenStack API calls to fetch
data that we don't even return to the user). It's quite possibly a
substantial part of the reason why we've had so much difficulty scaling up
Heat to deal with large numbers of servers polling it for config. And it
tends to leave a lot of annoying warnings in the log messages, not just of
Heat but also the other services it is polling, since polling the metadata
generally happens using the heat_stack_user's credentials, not those of the
actual owner of the resource - it can't actually find the resource and
returns a 404.

The fix is to change the default "with_attr" value in the RPC client to
False, and only pass None (the previous default) or a list of extra
attributes to include in the specific case where the ReST API will actually
return the attribute values to the user and not just discard them. Since
the server previously treated any falsey value for the parameter (e.g.
None, False or []) as equivalent, and since nothing was previously passing
False, this change is safe to backport to stable branches. The api and
engine can be updated in any order without change in behaviour until a new
api is talking to a new server (at which point the bug is fixed).

This change also updates the ReST API unit tests to accurately reflect the
data returned from the engine.

Change-Id: Ifffeaa552d3205ca06a79adda09d35a77099a2bf
Closes-Bug: #1507568
7 years ago
Jason Dunsmore c63411eef6 Add REST API for stack export
Closes-Bug: #1353670
Change-Id: I94d5abf5bfe148b4f25a2a8891e3cf4d1774e373
7 years ago
Javeme ed33ec56ab deprecate module heat.api.middleware.ssl
Use oslo_middleware.http_proxy_to_wsgi instead of oslo_middleware.ssl
due to the 'oslo_middleware.ssl' module is deprecated.

Change-Id: Ibb137049ca4005dd9a886de1ecc6b00dbae79789
Closes-Bug: #1526656
7 years ago