Stack and its nested stacks should be in *_COMPLETE state for
it to be migrated. Also, we should not allow migration of a
This also changes stack_get_all_by_owner_id() to not select the
backup stacks. This seems to be used only in convergence code
path and migrate_convergence_1().
This refactors the building of schema from parameter validation to use a
new method (which doesn't keep stacks in memory), and use that new
method for providing proper schema for resource group when the size is
Allow use policy in code to resource type's rule.
Also add test for override the in-code resource type rule in json
Partially-Implements: bp policy-in-code
When handling the command "openstack stack event list --nested-depth=n", we
obtain the list of nested stacks by querying the database for all resources
that share the root stack ID. However, since all we're getting is the stack
IDs, there's no need to query all of the fields and construct a versioned
object for each resource. Just get the set of stack IDs.
When doing "openstack stack resource list --nested-depth=n", we were
lazy-loading the resources' properties data. This is expensive, especially
when there are a large number of resources. Eager-load the data, as we
always use it to show the resources.
For consistency, always eager-load the resource 'data' (even in
resource_get), because the Resource versioned object accesses it
We already have REST api support for cancelling a
UPDATE_IN_PROGRESS stack with rollback. This adds a
new action 'cancel_without_rollback' to allow for
canceling a create/update in_progress stack without
In change I84d2b34d65b3ce7d8d858de106dac531aff509b7, we changed to
call self._converge_create_or_update() in a sub-thread. However,
thread_group_mgr is not set for cancel_update (with rollback),
which in turn calls converge_stack.
This also enables test_cancel_update_server_with_port, as
bug #1607714 seems to be fixed now.
In order to keep the engine service alive, we add a timer that periodically
does nothing. Calls to add_timer() require a stack_id, and currently we
pass cfg.CONF.periodic_interval. This is highly misleading, because the
value you pass for the stack_id has no effect on the interval. The cause
was a copy-paste error in 07884448fe, when
the code changed from calling ThreadGroup.add_timer() to
ThreadGroupManager.add_timer(). Use None as the stack ID instead.
This RPC call only generates a list of the outputs defined in the template,
not their values, so don't load the resource data needed to calculate the
output values (which can be very slow).
Also, explicitly pass resolve_value=False instead of relying on the default
argument (which is different for format_stack_output() and
format_stack_outputs()), to reduce future confusion.
When we show a stack including the outputs, we calculate all of the
resource attributes that are referenced anywhere in the stack. In
convergence, these are either already cached (and therefore fast) or need
to be cached (and therefore the initial slowness will pay off in future).
This isn't the case in the legacy path though, since we are not doing
caching of attributes in the database in that path. So this is
unnecessarily calculating all of the referenced attribute values, which are
potentially very slow to get.
For legacy stacks, only calculate the attribute values needed to show the
Add converge parameter for stack update API and RPC call,
that allow triggering observe on reality. This will be
triggered by API call with converge argument (with True
or False value) within. This flag also works for resources
within nested stack.
Implements bp get-reality-for-resources
For no reason at all, we were calling Stack.resource_get() twice in
handle_signal(). In the worst case, this function does a database query
that loads every resource.
Before returning the stack outputs, update the StackDefinition with the
resource data needed to resolve them. This will enable us to switch to
resolving outputs via the StackDefinition in a subsequent patch.
Partially-Implements: blueprint stack-definition
After refreshing the metadata of a resource its attributes may change, so
we need to update the StackDefinition with the new data.
Partially-Implements: blueprint stack-definition
We ought to iterate over the dependency list in the validate_template call,
to check for any circular dependencies. This check was removed by
d12cbe7959 in order to avoid trying to
calculate implicit dependencies added by resource plugins (as e.g. many
Neutron resources do) at a time when we don't have sufficient data to do a
good job of it anyway.
However, the previous patch d12cbe7959 split
apart the calculation of explicit and implicit dependencies, so we can
easily do this while still checking for circular references in the
explicit dependencies. This patch splits the dependency calculation into
two phases - the first using only data available from the template, and the
second querying the resources for implicit dependencies.
Use strict_validate = False as the criterion for when to avoid calculating
implicit dependencies, since this corresponds exactly to when we don't have
enough data to do so.
Ensure that attributes that are referenced in outputs get cached (and
therefore stored in the database) even when they are not referenced by
In 37ef669705 (in Kilo) we made signalling
a resource asynchronous because things like e.g. scaling an autoscaling
group were too slow to do synchronously (bug 1394095).
However, for resources that don't have a handle_signal() method at all,
this means that if a signal other than one handled by Heat (e.g. to
clear a hook) is received then the user won't be notified of the error.
There will instead be an error message in the logs warning of an
unhandled exception in a thread.
(Previously, similar problems occurred if the user was attempting to
clear a hook and some error occurred. This has since been fixed as bug
1472515, in Liberty.)
If there's no handle_signal() method defined, then calling
Resource.signal() is quick (i.e. it's only going to clear a hook, or
raise an error). So handle this case synchronously. This ensures that
any error is reported to the user and not in the log.
When updating a stack, we overwrite the existing tags.
So if no tags are specified during update, all existing
tags are removed. It would atleast be good to keep the
existing tags when doing a patch update (with `--existing`).
If stack-show is called at an inopportune moment in convergence, we can
attempt to store the attributes of a resource that isn't actually in the
database. This causes occasional gate failures due to an assert.
The patch at I1cb321a3878a0abce9b41832f76bf77c25bf7cb4 properly deleted
the snapshots from the database, but as delete in convergence sets the
template to an empty template, stack.resources is empty. This works
around the problem by deleting the snapshots beforehand.
Store resource attributes that may be cached in the DB, saving the
cost of re-resolving them later. This works for most resources,
specifically those that do not override the get_attribute() method.
To be able to handle parameter defaults properly, we need to load all
possible templates when merging the environments, to generate an
ParameterGroups is added only for the parent stack and all
the nested stack are missing this information. Added
ParameterGroups for all the nested stack also, if present.
Oslo.config deprecated parameter enforce_type and change its default
value to True in Ifa552de0a994e40388cbc9f7dbaa55700ca276b0. Remove
the usage of it to avoid DeprecationWarning: "Using the 'enforce_type'
argument is deprecated in version '4.0' and will be removed in version
'5.0': The argument enforce_type has changed its default value to True
and then will be removed completely."
Why pass a parameter to a method to influence what it does when you can set
some state on the object that the programmer always has to keep in their
head to figure out what any given call will do?
In ced6f78aa0 we stopped doing validations of
nested stacks at stack creation time, on the assumption that they had been
validated when the parent stack was created. This assumption was incorrect;
for children of the stack being created, the strict_validate global is
always set during validation, so property values of resources will not be
validated until it comes time to create the resource.
Instead, prevent only redundant non-strict validations of stacks at nested
depth 2 and greater. This means that every stack other than the root will
be validated exactly twice - once without validating property values when
the root is created, and again including property validation when the
nested stack itself is created.
Most of the performance benefits should remain; in the case of a large
ResourceGroup using index substitution, we will now have to validate a lot
of nearly-identical resource properties, however we still will not load
into memory and validate a nested stack for each one as we originally did.
Since that happens synchronously, it was likely the main contributor to RPC
timeouts when dealing with large scaling groups. (During the validation at
the creation of the root stack, only a single member of a ResourceGroup is
validated even when index substitution is used. For scaling groups with
identical members, only one member is validated since
This change reverts commit ced6f78aa0.
pydoc is part of the standard library and is much more robust at formatting
docstings than any trivial hand-rolled munger could be. For example, it
correctly preserves indentation within the docstring. Use it when
generating descriptions from docstrings for the API.
The list_events rpc api no longer includes resource properties data
when returning multiple events. In the "show event" case where an event
uuid is included in the list_events request, the resource properties data
for that single event continues to be included in the output.
Note that there is no change in behaviour on the heat-api side (see
heat/api/openstack/v1/events.py). Previously when listing multiple
events, it had just been ignoring the resource properties data it
fetched from the list_events rpc api, not including it in the output
it returned (e.g., to python-heatclient).