When generating the node_data() for a resource, catch and store any
exceptions (other than InvalidTemplateAttribute) encountered while
getting attributes. Re-raise the exception at the point where we try to
read the attribute value, including where we try to serialise the
NodeData object to store in the database.
In convergence, we generate and immediately serialise the NodeData, so
this should result in no substantial change in behaviour there.
In other situations (e.g. when we're just loading the data to show the
stack), this prevents an error in attribute calculation from aborting
the whole operation. The exception will still be raised if (and only if)
the erroneous attribute is accessed, but may be handled more
appropriately. For example, errors in calculating output values are
handled by reporting an error only for that particular output.
Change-Id: Idc97aee87405cc13e83be3373078b52e725850ea
Co-Authored-By: Zane Bitter <zbitter@redhat.com>
Closes-Bug: #1712280
Add converge parameter for stack update API and RPC call,
that allow triggering observe on reality. This will be
triggered by API call with converge argument (with True
or False value) within. This flag also works for resources
within nested stack.
Implements bp get-reality-for-resources
Change-Id: I151b575b714dcc9a5971a1573c126152ecd7ea93
If a resource is in a FAILED state, but the plugin overrides
_needs_update() to not raise UpdateReplace, and also no actual update is
required, we simply change the resource's state to UPDATE_COMPLETE.
However, if there were restricted actions set in the environment, this code
path was not executed. The behaviour should be the same regardless of
whether there are restrictions that have been checked.
Change-Id: Ib3ed081f9f26f4f9172c681bf9ebccb7cdc3ca9c
Related-Bug: #1663522
If a resource is in a FAILED state, but the plugin overrides
_needs_update() to not raise UpdateReplace, and also no actual update is
required, we simply change the resource's state to UPDATE_COMPLETE.
However, since the locking/unlocking steps were merged with the resource
state updates in a7376f7494, in doing so we
were ignoring the lock (since at this stage the resource is still unlocked,
and we don't need to take a lock). If another update traversal had acquired
the lock in the meantime, we would still write to the locked resource.
This change adds a new LOCK_RESPECT lock mode to Resource.store(). In this
mode the lock is not acquired or released, but an error will be raised if
the resource is found to be locked.
Change-Id: I8b5cd1e05b88dd13abc13899a73f23810f7e6135
Closes-Bug: #1705794
Related-Bug: #1662585
During update, delete and signal operations, we set the properties of a
resource to the values stored in the database when they were last created
or updated, so that we don't resolve intrinsic functions to new values
caused by possible changes in live attribute values. In fact, since we
started requiring the resources' node data to be pre-populated into the
StackDefinition, we will get only placeholder values for attributes and
resource IDs if we tried to resolve live functions.
This change does the same for the suspend, resume, snapshot, and check
actions. Should any plugin's handler function for any of those actions read
the resource's properties (I'm not sure that any do) then they will now get
the stored values.
Change-Id: If6b6f66877e23b8538a97aa339357ca1a2a29276
When load()ing a Resource in order to check it, we must load its definition
from whatever version of the template it was created or last updated with.
Previously we created a second Stack object with that template in order to
obtain the resource definition. Since all we really need in order to obtain
this is the StackDefinition, create just that instead.
Change-Id: Ia05983c3d1b838d2e28bb5eca38d13e83ccaf368
Implements: blueprint stack-definition
When the dep_attrs function was written, it was used only in convergence
after checking a single resource. However, now we also use it to generate
data for the ResourceProxy objects, which is often done for all resources
simultaneously. That means doing O(n^2) dep_attrs() calls, which can really
slow things down when there is a large number of resources with complex
properties (or metadata).
This change adds an all_dep_attrs() call, which runs as fast as dep_attrs()
on everything except get_attr functions and their arguments, but only needs
to be called once instead of once per resource. (The get_attr function can
in future override the default implementation of all_dep_attrs() to be as
efficient as dep_attrs() as well.) The resulting data is cached in the
ResourceDefinition or OutputDefinition so that subsequent calls to their
get_attr() methods with different (or the same) resource names will use the
existing data.
Change-Id: If95f4c04b841519ce3d7492211f2696588c0ed48
Partially-Implements: blueprint stack-definition
Closes-Bug: #1684272
This is no longer required as the convergence 'lightweight stack' used for
checking individual resources has now merged with the other code paths: in
every case intrinsic functions are resolved using the data about resources
stored in StackDefinition and accessed via ResourceProxy.
Change-Id: I41cf21c8e1babe819b4b6c668749ed5915ae3b54
This unifies the 'lightweight stack' used in convergence with the how
things work the rest of the time: we now always obtain resource data from
the StackDefinition's ResourceProxy objects. This means that when we are
checking an individual resource, we will never create all of the other
Resource objects for the stack in memory (although we already avoided
loading data for them from the database) - now we will only create
ResourceProxy objects as needed.
Change-Id: Id7472557e26d172df88841ff7f20afdd7f5bfada
Implements: blueprint stack-definition
Since function.dep_attrs() returns logical resource names (rather
than actual objects), we can just as easily use the StackDefinition to
calculate it instead of the Stack and Resource objects.
In the legacy path, we must ensure we use the StackDefinition from the
*new* stack to determine which attributes to include in the NodeData, since
that's what we're going to be using it for. In the convergence path the
current stack definition already contains the new template.
Also, update the *new* stack's definition with the NodeData obtained from
completed resources (in addition to the existing stack's), so that that
data may be used in calculating the dep_attrs for future resources. This is
required when get_attr functions are nested in the template.
Change-Id: I23efcc091eae53470f7f9cb3ca21e09f00f43808
Partially-Implements: blueprint stack-definition
While create a resource, we might retry to create or delete that resource.
Within every action properties validate was act as pre-function before
actually call any action. While the first creat run, if anything goes wrong
during property validation, a validation error will raise right away. Which
will definitely riase in the very first action run. But if it successed,
it still required to revalidate each time an action triggered (whatever is
creat or delete). This patch will ignore properties validate if it's not
the first create run.
Closes-Bug: #1691672
Change-Id: Ibf592a254a862613eddb77ea5933ec6ba0cd2d1a
When requesting the NodeData for a Resource, generate placeholder values if
the resource has not been created yet. Currently, all attributes are given
a value of None (consistent with how the get_attr intrinsic functions
substitute them when the resource state indicates that the attributes are
not valid now). Reference IDs are generated as usual (since the logic for
what to return when the state indicates the data are not valid is contained
in the resource plugins).
In future, this will allow us to add placeholder NodeData for all resources
to the StackDefinition prior to validation. In the long term, that should
help us to remove the state-checking logic from the intrinsic functions. In
the short term, it will allow us to validate template data that is parsed
against the StackDefinition (rather than the Stack), since it gives us a
list of valid attribute names. In the medium term, this should give us a
place to substitute in more sophisticated placeholder values to do better
validation.
Change-Id: Ife69f563a3748c6cb611ca02f826229e76bee4d0
Partially-Implements: blueprint stack-definition
When we signal a resource, we want the signal to be interpreted in the
context of the properties of the resource as they were set at the last
stack update, not based on any attributes of other resources that may have
changed.
We already store the properties for comparison during updates. This patch
freezes the properties while we call handle_signal(), in a similar way to
how we freeze the properties during handle_update() (see bug 1584623).
Also make sure that when a ScalingPolicy adjusts an autoscaling group, it
also uses the group's stored properties so that e.g. the definitions of
existing members cannot change on a scale up.
Change-Id: I7e248ad82f2334b1a580655efa3a302e7d84fbd8
Partially-Implements: blueprint stack-definition
The Stack._find_filtered_resources() method returns Resource objects for
all resources associated with the Stack, regardless of whether they are
current (present in the template; latest version in the case of
convergence). To do this, it previously created a new Resource object
for every resource found in the database.
However, for those resources which *are* current this is unnecessary. We
can access the Resource object simply through self.resources. It turns
out this is necessary for obtaining the required_by list for legacy
stacks, because only the Resources obtained from self.resources also
appear in the Dependencies graph obtained from self.dependencies. The
required_by list is read when listing or showing resources, which would
either return an empty list or fail for legacy stacks.
This patch also makes the Resource.required_by() method more robust in
its error handling.
Change-Id: Id438336e5c88dc7c2d168ba01ee703faa17e8b8e
Closes-Bug: #1703703
Related-Bug: #1523748
Ensure that attributes that are referenced in outputs get cached (and
therefore stored in the database) even when they are not referenced by
other resources.
Change-Id: I667ab04f91edddef5c5dbec0a89d465110c312b4
Closes-Bug: #1660831
Calculating all of the attributes of a resource that are referenced in the
template was previously done inside the node_data() method, but this is
useful for the resource to be able to access for other reasons so refactor
it into a separate method.
Change-Id: I8ec943652e6f93a2352f2ea5e6ac46e0088e5458
Related-Bug: #1660831
Partially-Implements: blueprint stack-definition
This changes the logic of getting cancellation grace
period of task runner before closing it: to move the
liveness check into the cancel_all() method in the
scheduler rather than ask the resource if it's IN_PROGRESS.
Change-Id: Ia2a03de227ff15cdce1b3dbb6cd6bff6c5a50a15
Partial-Bug: 1693495
This moves signal validation after hook handling, to be able to properly
get signals during DELETE actions.
Change-Id: I18e2284ab344f6c2e46e40744b771524b7bad9d9
Closes-Bug: #1690806
Previously, all caching of attribute values was done via the Attributes
object. However, some resource types override Resource.get_attribute() to
do custom handling of the trailing attribute path or dynamic attribute
names, and in these cases the resulting values were not cached (since they
don't go through the Attributes object).
This patch adds a caching step for these resources:
* OS::Senlin::Cluster
* OS::Heat::ResourceChain
* OS::Heat::ResourceGroup
* OS::Heat::AutoscalingGroup
* OS::Heat::SoftwareDeployment
* OS::Heat::SoftwareDeploymentGroup
* TemplateResource
* AWS::CloudFormation::Stack
Change-Id: I07ac22cc4370a79bd8712e2431fa3272115bc0eb
Co-Authored-By: Crag Wolfe <cwolfe@redhat.com>
Partial-Bug: #1660831
This code was never used for anything, as the rest of the patch series
never landed and was abandoned.
This reverts commit 15e52ff5e9.
Change-Id: I7d1a22753e8de1d3adf127c14516ebd667513bfa
Store resource attributes that may be cached in the DB, saving the
cost of re-resolving them later. This works for most resources,
specifically those that do not override the get_attribute() method.
Change-Id: I71f8aa431a60457326167b8c82adc03ca750eda6
Partial-Bug: #1660831
The 'show' attribute is used to get all data about a resource. There are
folks using this in lieu of having access to the underlying OpenStack APIs
themselves. By not caching it, we ensure that they will always see a live
value, even once we start storing attribute values in the database when
modifying a resource.
Change-Id: Id81a56f7d774bc18a2b5d62c80b369f1c12bcc5c
Co-Authored-By: Crag Wolfe <cwolfe@redhat.com>
Related-Bug: #1660831
If a traversal is interrupted by a fresh update before a particular
resource is created, then the resource is left stored in the DB with the
old template ID. While an update always uses the new template, a create
assumes that the template ID in the DB is correct. Since the resource has
never been created, the new traversal will create it using the old
template.
To resolve this, detect the case where the resource has not been created
yet and we are about to create it and the traversal ID is still current,
and always use the new resource definition in that case.
Change-Id: Ifa0ce9e1e08f86b30df00d92488301ea05b45b14
Closes-Bug: #1663745
I405888f46451d2657aa28f610f8ca555215ff5cf changed to load only
a few fields when loading the resource from the db. We should
load 'name' field for it to be used when rasing ConcurrentTransaction
exception.
Change-Id: I699961b75c7c417d03e21b51a2464c39e737ca89
Related-Bug: #1583679
Related-Bug: #1680658
While metadata_set() currently can update the metadata and db
atomic_key as intended, it can leave a heat-engine-resource instance's
._atomic_key with an incorrect value (especially when the db
resource's atomic_key was incremented between retries).
Change-Id: I15a1289af70576ee735d81b6b8ac2130bdcc657e
Partial-Bug: #1675286
Previously, if an update or delete on a resource is initiated
concurrently with another action (e.g. a metadata update as the result
of a resource signal) it may end up failing with
exception.UpdateInProgress because its view of the resource's
atomic_key is stale.
Now, we retry, rereading the resource's atomic_key from the db if
needed, up to cfg.CONF.action_retry_limit times.
Change-Id: I4cfa6f691fe916c0d605a712028b88f61ebab4d9
Partial-Bug: #1675286
Thomas Herve