- pass the adopt data in next to the input_data
- since it is not stored in the db, assign the adopt data
from the cache data in Resource.load()
- in service.py make it easier to enable convergence
(just uncomment the raise NotSupported)
Change-Id: I3b3032602f18f86119c41a1bda29432e459c9966
On Python 3, the decorator does nothing. On Python 2, it aliases the
__str__ method to __unicode__ and creates a new __str__ method that
returns the result of __unicode__() encoded with UTF-8.
All instances of __unicode__ have been removed with the
aforementioned decorator in place instead.
Note: There are some instances of __str__ which aren't handled but
will be once we have actual python34 tests running in the gate.
partial blueprint heat-python34-support
Change-Id: I7271a2581e1c2bbc282933c7da73db810c7e09db
1. There is no back-up stack which can take care of deleting the old
resources after the stack update is done. The DB objects, in convergence
model, needs to be deleted up front after they have been deleted from
real world.
2. When the stack fails, do not delete the previous template. There
might be resource from previous template in DB that are not yet deleted.
3. Resource unlock should use the boolean return value instead of
integer.
Change-Id: I49cc319638c81833c6c544c53e8c321e358e9744
Co-Authored-By: Sirushti Murugesan <sirushti.murugesan@hp.com>
Co-Authored-By: Anant Patil <anant.patil@hp.com>
Support to wait up to few minutes(config option
'error_wait_time') for resources already in progress
to complete before forcibly cancelling them when
suspending, resuming, checking and snapshotting.
Change-Id: I05c2f242beb9d6dcd1b78583e5f5084b6f3add0d
Closes-Bug: #1470817
Parameters and attributes of stack need to be updated for stack update
request. This will fix the update rollback in convergence.
Change-Id: Ia45c71ce809ade70ba8f1dcd7fee576cdf9bae98
If we find a resource with template as current template given for
update, then that is the best resource and we should stop continuing.
This happens in case of rollback, when the previous template is used for
updating the stack.
If we encounter a resource with same template as previous template,
we still need to continue to find a possible resource with current
template, to cover the above mentioned case.
Renaming get_existing_db_resource to get_best_existing_db_resource for
better reflection of what it does and moving it out of enclosed method
so that it can be tested.
Change-Id: I5c753e1c0737803ee3de8d36f25c0e620a65f0cf
When a resource fails:
1. Mark the stack as FAILED.
2. If rollback is enabled, invoke rollback for the stack. Otherwise,
clean-up database of stale entries.
After the stack is completed (or failed), the database needs to be
cleaned up.
Change-Id: Ic5713e8c0c78c38ee1aaee8865346cb55228043f
Current template need not be backed up while creating a stack. With
current logic, a circular reference is created with current stack
pointing to current template as previous template.
This happens when the template is stored before invoking stack create.
The template is stored along with the stack just before invoking the
converge_stack for creating the stack. The test cases are modified to
include instrutions to store the stack before running convergence.
Change-Id: I95b629d7d42201bc8273a3f70f3e3819211b32f8
This is a merge of 4 reviews:
I52f1611d34def3474acba0e5eee054e11c5fc5ad
Ic374a38c9d76763be341d3a80f53fa396c9c2256
Iecd21ccb4392369f66fa1b3a0cf55aad754aeac4
I77b81097d2dcf01efa540237ed5ae14896ed1670
- make sure sender is a tuple (otherwise the serialization
function in sync_point breaks.)
- Update updated_time on any lifecycle operation(CREATE/UPDATE/DELETE)
over a stack.
- adjust sync_point logic to account for deletes
Done by having only a single stack sync point
for both updates and deletes.
- Serialize/deserialize input_data for RPC
- Make GraphKey's the norm in convergence worker
- move temp_update_requires functionality to tests
During intial stages of convergence to simulate the entire cycle
some part of worker code was written in stack.py.
Now that the convergence worker is implemented, this code needs to
be executed only in tests.
- Fix dictionary structure that's passed to resoure.(create/update)
- Temporarily disable loading cache_data for stack to help fix other
issues.
Change-Id: Iecd21ccb4392369f66fa1b3a0cf55aad754aeac4
Co-Authored-by: Sirushti Murugesan <sirushti.murugesan@hp.com>
Co-Authored-by: Rakesh H S <rh-s@hp.com>
This patch adds call of _add_event method at the end of the update_task.
So now in event-list output user can see, that updating stack is
completed or failed.
Also status reason for update_task were changed to be inline with status
reason in the main stack_task method.
Method set_state is not used due to using stack.store method in the end
of update to avoid duplicate update.
Change-Id: Ic94c95f1558fd67f351696645426c00de8db1bf8
Closes-Bug: #1465321
In an convergence enabled environment, when an stack-delete
action is completed, the stack was not soft deleted from DB.
This patch soft deletes the stack after all the required
stack-delete operations are completed.
Change-Id: I36d2b76223d3fc5f313b953b3976282d0923e92e
We use stack.prepare_abandon() to make the
snapshot data, but the data have no 'files' then will
result in ValueError raised(No content found in the "files"
section for get_file path) when restore.
This patch will store 'files' for snapshot data(
also for prepare abandon data), then we can get 'files'
from the snapshot data when stack restore.
Change-Id: I0fddf92e950b93dce70eb8e5a562cf8d0046a9ab
Closes-Bug: #1461375
This change uses database queries to count the total number of
resources in a nested stack tree instead of loading every stack
and every resource in that stack.
This should result in a significant improvement in memory use
for stacks with many nested stacks and resources.
Closes-Bug: #1455589
Change-Id: I8c2f1fa9114dbf68ad4a5f99dd3929b6958b5d7a
Currently, the patch does the following:
Kicks off workflow from stack.update_or_create:
Once the dependency graph is calculated, the leaves
of the graph are all casted into the RPC worker bus.
Worker RPC check_resource worfklow:
Workers will then start working on each resource
individually. Once a resource operation is finished,
sync points are used to check if the parent resource
can be worked on. Resources that finish early will
wait for their siblings to finish. The sibling that
finishes last will trigger the creation/updation/deletion
of it's parent. This process then goes on for all nodes
until the roots of the graph are processed.
Marks stack as complete when roots have finished.
Once the roots of the graph are successfully processed,
the previous raw template which was needed for rollback
in case something went wrong will now be deleted from the
database. The stack is then marked as complete.
Largely follows the convergence prototype code in
github.com/zaneb/heat-convergence-prototype/blob/resumable/converge/converger.py
Implements blueprint convergence-check-workflow
Change-Id: I67cfdc452ba406198c96afba57aa4e756408105d
When executing stack snapshot heat do not release stack lock
some undefined amount of time because it needs to prepare
some data for stack snapshot. It leads to situation when stack
snapshot is complete(from user perspective) but nobody can do any
operations with stack. So the fix executes snapshot saving to DB
right before stack becomes complete.
Change-Id: Id011142498fee49fee9ec1437fc0816b25780e48
Closes-bug: #1456672
This change adds a (currently unused) database function
stack_get_root_id to find the root stack ID for any stack.
Scheduler hints are moved to using Stack.root_stack_id() in this change.
Remaining uses of Stack.root_stack() will switch to using
Stack.root_stack_id() later in the series.
Change-Id: I9914b0df5af119edea6346db0d7c62124fbb1313
Partial-Bug: #1455589
Most validation errors are in fact the user's problem, but in the case of
an AssertionError that's a real problem with Heat. We should return 500
status codes and log the tracebacks, not wrap the in validation exceptions.
Change-Id: I71d91a5413d7f026b332014487583b7f2762b322
The event and notifications do not look the stack up in
the DB so we can rely on the in memory status.
Change-Id: Iceb9bf6fa86f7f5ada7d3a8672788f09a48b1f43
Partial-Bug: #1450314
In order to eliminate circular references, we need to define the Stack as
the top of any reference hierarchy. This means we can't hold a reference to
a Resource directly without also holding a reference to its Stack.
Change-Id: I7430b109bbe1c5d6d64be9b8c778b394e9cff269
Related-Bug: #1454873
1. Add a configuration option to enable/disable template parameters
encryption.
2. Encrypt hidden parameters before storing them in the database and
decrypt on stack load.
Change-Id: Ie46c6a149f414f655600616da8deee463e55671c
Implements: blueprint encrypt-hidden-parameters
Co-Authored-By: Jason Dunsmore <jasondunsmore@gmail.com>
Currently, we load all the resources from the database to resolve
template functions of dependent resources. In convergence, because
every worker will load it's own stack object, loading all the
resources for every resource lifecycle operation will be inefficient.
This patch allows creating lightweight stacks using a cache provided
which will never query the database and only depend on the template
and the cache provided. Function resolutions will now happen by querying
the values in the cache provided if it exists else None is returned.
The resultant lightweight stack will effectively be the stack attributes
loaded from the database and it's raw template with the dependent
resource's functions resolved which will serve as the input for
individual resource objects that will be worked upon.
blueprint convergence-lightweight-stack
Change-Id: I6dbaa7ee4e9d534c31823b4812efcb387c695a22
When generating the dependencies, the deletion order is already taken
care, hence we need not reverse the graph.
Change-Id: Iff158443f2281a07e3a136a4cf6f77f7be592d3f
Generates the graph for traversal in convergence.
* Updates current traversal for the stack
* Deletes any sync_point entries of previous traversal
* Generates the graph for traversal based on
- resources loaded from db for the stack
- resources that exist in present template
* Stores resource.current_template_id and resource.requires
* Stores the edges of graph in stack.current_deps
* Creates sync_points for each node in graph
* Creates sync_point for stack.
blueprint convergence-prepare-traversal
Change-Id: I507e67b39c820ed46d3b269fc76d6cf18d0ef2d7
Currently, the event-list output is very resource-centric, despite
being scoped to the stack from an API path perspective.
This, combined with the fact that the stack updated_at timestamp
is only updated after a succesful update (ref bug #1193269) makes
it impossible to derive the time when an update started via any API.
This is a problem when trying to use the new hook/breakpoint API,
because it's necessary to poll for all events since the most recent
update started, disregarding any stale hook events from previous
updates (which may have failed or timed out without the hooks getting
cleared).
To work around this, add an event for each stack state transition,
such that you can detect the transition to UPDATE_IN_PROGRESS,
then use that event as a marker to get post-update-started events.
Without this (or some other way to determine when the stack update
started), the hooks pre-update functionality landed for kilo is
not really usable (particularly mechanically via scripts).
Change-Id: Idff342b3aecc2d145dfbc7c0f610ad0ca8e52c8b
Partial-Bug: #1448155
Using snapshot's enviroment instead of current stack's
enviroment when stack restore.
Change-Id: I7becfbe4b970820bd374560d00fa6f661a85bdab
Closes-Bug: #1430673
We are persisting for a number of reasons:
- so we don't have to pass this through ever rpc call
- the API exposes parent_resource (currently always None as
it is not persisted)
Closes-bug: #1438978
Change-Id: Id2db36c0234a085ec4f0ce2ab114ec483ea29d81
(cherry picked from commit edf86aeac2)
Check the snapshot belongs to stack when deleting and showing
stack's snapshot, and restoring from snapshot.
Change-Id: I8ce170b40b05ae17669524d75f80e06e39986673
Closes-Bug: #1437602
We are persisting for a number of reasons:
- so we don't have to pass this through ever rpc call
- the API exposes parent_resource (currently always None as
it is not persisted)
Closes-bug: #1438978
Change-Id: Id2db36c0234a085ec4f0ce2ab114ec483ea29d81
Add a "hidden_stack_tags" option that contains a list of tag names.
Stacks that have one or more of these tags will be hidden.
Add an option to show hidden stacks in the stack listing.
blueprint stack-tags
Change-Id: I45a5ac6d73a9a61629a56f88270e3a97fafb378a
Pass stack_user_project_id to updated_stack, backup_stack and
oldstack to make sure the success when deleting stack domain user.
Create a common method to get the kwargs to create a stack from
an existing stack.
Co-Authored-By: Angus Salkeld <asalkeld@mirantis.com>
Change-Id: Ieb7726ed738d5ae8046184f312379b9132b6c4a9
Closes-Bug: #1356084
If there is no snapshot data, no need to start a task runner
of rsrc.delete_snapshot.
Change-Id: I1b868eda8a7a0a2ed5f30aa01a41f4c9820193a5
Closes-Bug: #1437588
The scheduler only raises ExceptionGroup if aggregate_exceptions is
True. This change adds a catch-all exception handler that sets the stack
to failed on any exceptions.
Co-Authored-By: Miguel Grinberg <miguel.grinberg@gmail.com>
Change-Id: I76a6e87924416335921115ec3147a991659cfb2e
Closes-Bug: #1436128
This is a mechanism whereby when heat processes
a stack VM resource, the stack id, root stack id,
stack resource id, stack resource name and the
path in the stack (as a list of tuples,
(stackresourcename, stackname)) can be passed to
nova by heat as scheduler hints, to the
configured schedulers for nova.
Implements: blueprint stack-lifecycle-scheduler-hint
Co-Authored-By: Karolyn Chambers <chamberk@us.ibm.com>
Change-Id: I3e006339a41c469451bc3ee740018b285d3e0a65
Store all Resources in an initial INIT_COMPLETE state prior to creating a
stack, rather than waiting until each Resource is create()d. When updating
a stack, store each new Resource just prior to create()ing it.
Change-Id: Ifeee8213c1962cbf45fd0dd59f1832b0eeff1a47
Closes-Bug: #1319813
When we moved parameters from the stack table to the template table
we did it in a "flexible" way, but this led to some difficult to
diagnose problems. So this patch totally removes "env" as an argument
to the Stack class to make absolutely sure that we are doing this
correctly. I came across these bugs when rebasing decouple-nested.
- env was not getting passed into instance group make_templates()
- template deepcopy was not working correctly (causing some problems
when deleting backup stacks)
Change-Id: I85cfb01c68ff5a44e2ff4f6e446870efb7205933
implementation for versioned objects. This contains UserCreds
Co-Authored-By: Michal Jastrzebski (inc0) <michal.jastrzebski@intel.com>
Co-Authored-By: Grzegorz Grasza <grzegorz.grasza@intel.com>
Change-Id: Ibae5504060d2c714225cd8b735b430c7da52e0fa
Implementation of oslo.versionedobjects.
This commit consists basic mechanism and first objects.
This should be base of implementation versoning to other objects
Implements: blueprint versioned-objects
Co-Authored-By: ShaoHe Feng <shaohe.feng@intel.com>
Co-Authored-By: Grzegorz Grasza <grzegorz.grasza@intel.com>
Change-Id: I554162cf3681fe559c75f54c61c6f32c91f5c2f8
Sirushti Murugesan
Angus Salkeld
huangtianhua
Anant Patil
Alexandr Nedopekin
Sergey Kraynev
Rakesh H S
Steve Baker
Tetiana Lashchova
kairat_kushaev
Zane Bitter
Vijendar Komalla
Steven Hardy
Limor Stotland
Jason Dunsmore
Peter Razumovsky
Deliang Fan
Bill Arnold
He Jie Xu
ShaoHe Feng
Michal Jastrzebski