Stack and its nested stacks should be in *_COMPLETE state for
it to be migrated. Also, we should not allow migration of a
nested stack.
This also changes stack_get_all_by_owner_id() to not select the
backup stacks. This seems to be used only in convergence code
path and migrate_convergence_1().
Change-Id: Icd54465d0c593557a12d853ddee4ee8ce6483499
Closes-Bug: #1767962
Story: #1767962
Task: #17363
This refactors the building of schema from parameter validation to use a
new method (which doesn't keep stacks in memory), and use that new
method for providing proper schema for resource group when the size is
0.
Change-Id: Id3020e8f3fd94e2cef413d5eb9de9d1cd16ddeaa
Closes-Bug: #1751074
Closes-Bug: #1626025
Fix reset_stack_status on legacy, by using a different session for each
stack reset, and handling lock duplicate errors.
Closes-Bug: #1735755
Change-Id: I6bcd7448052e86ec3e4eb4c49ef3139c20d4f919
Allow use policy in code to resource type's rule.
Also add test for override the in-code resource type rule in json
file.
Partially-Implements: bp policy-in-code
Change-Id: Id6c21732e66de6c421427ded98de52f5da0a4db2
When handling the command "openstack stack event list --nested-depth=n", we
obtain the list of nested stacks by querying the database for all resources
that share the root stack ID. However, since all we're getting is the stack
IDs, there's no need to query all of the fields and construct a versioned
object for each resource. Just get the set of stack IDs.
Change-Id: I12155433e2ac1af919aa4b5e780fb965cd5885d8
Related-Bug: #1588561
When doing "openstack stack resource list --nested-depth=n", we were
lazy-loading the resources' properties data. This is expensive, especially
when there are a large number of resources. Eager-load the data, as we
always use it to show the resources.
For consistency, always eager-load the resource 'data' (even in
resource_get), because the Resource versioned object accesses it
unconditionally.
Change-Id: Idb871fddf77bf24828878c315e19e200c28841be
Related-Bug: #1665503
This is to enable preview of the merged environment
without merging the environment on the client side.
Related-Bug: #1635409
Change-Id: I7ec3af729a65164230153021f438bf226cc5e858
We already have REST api support for cancelling a
UPDATE_IN_PROGRESS stack with rollback. This adds a
new action 'cancel_without_rollback' to allow for
canceling a create/update in_progress stack without
rollback.
APIImpact
Change-Id: I6c6ffa0502ab8745cfb2f9c5ef263f1e02dfc4ca
Closes-Bug: #1709041
In change I84d2b34d65b3ce7d8d858de106dac531aff509b7, we changed to
call self._converge_create_or_update() in a sub-thread. However,
thread_group_mgr is not set for cancel_update (with rollback),
which in turn calls converge_stack.
This also enables test_cancel_update_server_with_port, as
bug #1607714 seems to be fixed now.
Change-Id: Ie674fd556418f6aa8e79654458cbe43648851db2
Closes-Bug: #1713952
In order to keep the engine service alive, we add a timer that periodically
does nothing. Calls to add_timer() require a stack_id, and currently we
pass cfg.CONF.periodic_interval. This is highly misleading, because the
value you pass for the stack_id has no effect on the interval. The cause
was a copy-paste error in 07884448fe, when
the code changed from calling ThreadGroup.add_timer() to
ThreadGroupManager.add_timer(). Use None as the stack ID instead.
Change-Id: Ia24a0d3ae9a0295fc811eb5300656399f426408b
This RPC call only generates a list of the outputs defined in the template,
not their values, so don't load the resource data needed to calculate the
output values (which can be very slow).
Also, explicitly pass resolve_value=False instead of relying on the default
argument (which is different for format_stack_output() and
format_stack_outputs()), to reduce future confusion.
Change-Id: I79aae94b6552d465db6707cd4a40cd53ff18455b
Closes-Bug: #1719340
When we show a stack including the outputs, we calculate all of the
resource attributes that are referenced anywhere in the stack. In
convergence, these are either already cached (and therefore fast) or need
to be cached (and therefore the initial slowness will pay off in future).
This isn't the case in the legacy path though, since we are not doing
caching of attributes in the database in that path. So this is
unnecessarily calculating all of the referenced attribute values, which are
potentially very slow to get.
For legacy stacks, only calculate the attribute values needed to show the
outputs.
Change-Id: I35800c7f87b58daf05cbabd05bcbcd75d0c0fadb
Partial-Bug: #1719333
Add converge parameter for stack update API and RPC call,
that allow triggering observe on reality. This will be
triggered by API call with converge argument (with True
or False value) within. This flag also works for resources
within nested stack.
Implements bp get-reality-for-resources
Change-Id: I151b575b714dcc9a5971a1573c126152ecd7ea93
For no reason at all, we were calling Stack.resource_get() twice in
handle_signal(). In the worst case, this function does a database query
that loads every resource.
Change-Id: I9e3f1e77146c71ae35a0896f915f6f9c9fac3fd3
Related-Bug: #1635610
Before returning the stack outputs, update the StackDefinition with the
resource data needed to resolve them. This will enable us to switch to
resolving outputs via the StackDefinition in a subsequent patch.
Change-Id: Ide3920385d9c4841c18024e1eb1456eed5c8eef5
Partially-Implements: blueprint stack-definition
After refreshing the metadata of a resource its attributes may change, so
we need to update the StackDefinition with the new data.
Change-Id: Iec521317bffc0b381a0d965daef7939b1146ce50
Partially-Implements: blueprint stack-definition
We ought to iterate over the dependency list in the validate_template call,
to check for any circular dependencies. This check was removed by
d12cbe7959 in order to avoid trying to
calculate implicit dependencies added by resource plugins (as e.g. many
Neutron resources do) at a time when we don't have sufficient data to do a
good job of it anyway.
However, the previous patch d12cbe7959 split
apart the calculation of explicit and implicit dependencies, so we can
easily do this while still checking for circular references in the
explicit dependencies. This patch splits the dependency calculation into
two phases - the first using only data available from the template, and the
second querying the resources for implicit dependencies.
Use strict_validate = False as the criterion for when to avoid calculating
implicit dependencies, since this corresponds exactly to when we don't have
enough data to do so.
Change-Id: I21c63dcc8d1cad20dbc237b472670753779b0ff8
Closes-bug: #1691575
Ensure that attributes that are referenced in outputs get cached (and
therefore stored in the database) even when they are not referenced by
other resources.
Change-Id: I667ab04f91edddef5c5dbec0a89d465110c312b4
Closes-Bug: #1660831
In 37ef669705 (in Kilo) we made signalling
a resource asynchronous because things like e.g. scaling an autoscaling
group were too slow to do synchronously (bug 1394095).
However, for resources that don't have a handle_signal() method at all,
this means that if a signal other than one handled by Heat (e.g. to
clear a hook) is received then the user won't be notified of the error.
There will instead be an error message in the logs warning of an
unhandled exception in a thread.
(Previously, similar problems occurred if the user was attempting to
clear a hook and some error occurred. This has since been fixed as bug
1472515, in Liberty.)
If there's no handle_signal() method defined, then calling
Resource.signal() is quick (i.e. it's only going to clear a hook, or
raise an error). So handle this case synchronously. This ensures that
any error is reported to the user and not in the log.
Change-Id: I917b4c7d4ab2cde46148197a3a232d5dc70f63d7
Related-Bug: #1472515
When updating a stack, we overwrite the existing tags.
So if no tags are specified during update, all existing
tags are removed. It would atleast be good to keep the
existing tags when doing a patch update (with `--existing`).
Change-Id: I910fe5237605405791eb6daa81c422bf7fd9eaa1
Closes-Bug: #1701098
If stack-show is called at an inopportune moment in convergence, we can
attempt to store the attributes of a resource that isn't actually in the
database. This causes occasional gate failures due to an assert.
Closes-Bug: #1701569
Change-Id: I50de08ae6b40958781ef4223c82a5c195b0757bf
The patch at I1cb321a3878a0abce9b41832f76bf77c25bf7cb4 properly deleted
the snapshots from the database, but as delete in convergence sets the
template to an empty template, stack.resources is empty. This works
around the problem by deleting the snapshots beforehand.
Related-Bug: #1508299
Change-Id: Id1c2c1a293fdcda07c527f29fedc00b716b303bc
Store resource attributes that may be cached in the DB, saving the
cost of re-resolving them later. This works for most resources,
specifically those that do not override the get_attribute() method.
Change-Id: I71f8aa431a60457326167b8c82adc03ca750eda6
Partial-Bug: #1660831
To be able to handle parameter defaults properly, we need to load all
possible templates when merging the environments, to generate an
approximate schema.
Closes-Bug: #1669571
Change-Id: Idab781d3e6af92990c357a727aa60ec0aa8f5f05
ParameterGroups is added only for the parent stack and all
the nested stack are missing this information. Added
ParameterGroups for all the nested stack also, if present.
Change-Id: I032144733bde916f8de8644121b9fb1ef29baef2
Handle the restore operation as a normal convergence update instead of a
legacy one.
Change-Id: I6ee46cdf7a8fdf89c58c9812d08af21c97fb0f9e
Related-Bug: #1687006
Oslo.config deprecated parameter enforce_type and change its default
value to True in Ifa552de0a994e40388cbc9f7dbaa55700ca276b0. Remove
the usage of it to avoid DeprecationWarning: "Using the 'enforce_type'
argument is deprecated in version '4.0' and will be removed in version
'5.0': The argument enforce_type has changed its default value to True
and then will be removed completely."
Change-Id: I91b0f0a52b5ce8654702510eed76d5dea8cc8fe4
Related--Bug: #1517839
Why pass a parameter to a method to influence what it does when you can set
some state on the object that the programmer always has to keep in their
head to figure out what any given call will do?
Change-Id: I020238de0a351dd2eaf51c6f970ae36ad0f931be
In ced6f78aa0 we stopped doing validations of
nested stacks at stack creation time, on the assumption that they had been
validated when the parent stack was created. This assumption was incorrect;
for children of the stack being created, the strict_validate global is
always set during validation, so property values of resources will not be
validated until it comes time to create the resource.
Instead, prevent only redundant non-strict validations of stacks at nested
depth 2 and greater. This means that every stack other than the root will
be validated exactly twice - once without validating property values when
the root is created, and again including property validation when the
nested stack itself is created.
Most of the performance benefits should remain; in the case of a large
ResourceGroup using index substitution, we will now have to validate a lot
of nearly-identical resource properties, however we still will not load
into memory and validate a nested stack for each one as we originally did.
Since that happens synchronously, it was likely the main contributor to RPC
timeouts when dealing with large scaling groups. (During the validation at
the creation of the root stack, only a single member of a ResourceGroup is
validated even when index substitution is used. For scaling groups with
identical members, only one member is validated since
3aebdabf2e78ac9e920b9dd8c748c4fad0d723c3.)
This change reverts commit ced6f78aa0.
Change-Id: I97cf789cee75931edef58b78c88f02da204d2a08
Closes-Bug: #1675589
Related-Bug: #1645336
pydoc is part of the standard library and is much more robust at formatting
docstings than any trivial hand-rolled munger could be. For example, it
correctly preserves indentation within the docstring. Use it when
generating descriptions from docstrings for the API.
Change-Id: Ib565a64d990a45c652a9475255c37805f86070b4
The list_events rpc api no longer includes resource properties data
when returning multiple events. In the "show event" case where an event
uuid is included in the list_events request, the resource properties data
for that single event continues to be included in the output.
Note that there is no change in behaviour on the heat-api side (see
heat/api/openstack/v1/events.py). Previously when listing multiple
events, it had just been ignoring the resource properties data it
fetched from the list_events rpc api, not including it in the output
it returned (e.g., to python-heatclient).
Change-Id: I7ac83d848cdd0e6c313870c0a4d59a5d9b2301f5
Partial-Bug: #1665506
rabi