- pass the adopt data in next to the input_data
- since it is not stored in the db, assign the adopt data
from the cache data in Resource.load()
- in service.py make it easier to enable convergence
(just uncomment the raise NotSupported)
Change-Id: I3b3032602f18f86119c41a1bda29432e459c9966
The functional gate is also failing due to a recent
change introduced in devstack -> I613f1bdc6673f0c4bfe29aaab7b514348a617a8c
This fixes both the gates, functional and unit tests.
Co-Authored-By: Thomas Herve <therve@redhat.com>
Closes-Bug: #1473398
Change-Id: Ia77a1fd884c6346edfb90777e91a51d279ccd92a
This patch splits the stack events related test cases out. There are
revisions regarding wrt replacing mox with mock calls.
Change-Id: I596590fd8e0ce3b54a4a29259fbaf7c4de6016f4
Updates engine resource_type listing to consider the
availability of resource based on service deployed
and configured in keystone
implements blueprint keystone-based-resource-availability
Change-Id: Ide88e503a1c5501a6f7685c5f85d3c340c73824f
1. There is no back-up stack which can take care of deleting the old
resources after the stack update is done. The DB objects, in convergence
model, needs to be deleted up front after they have been deleted from
real world.
2. When the stack fails, do not delete the previous template. There
might be resource from previous template in DB that are not yet deleted.
3. Resource unlock should use the boolean return value instead of
integer.
Change-Id: I49cc319638c81833c6c544c53e8c321e358e9744
Co-Authored-By: Sirushti Murugesan <sirushti.murugesan@hp.com>
Co-Authored-By: Anant Patil <anant.patil@hp.com>
Parameters and attributes of stack need to be updated for stack update
request. This will fix the update rollback in convergence.
Change-Id: Ia45c71ce809ade70ba8f1dcd7fee576cdf9bae98
If we find a resource with template as current template given for
update, then that is the best resource and we should stop continuing.
This happens in case of rollback, when the previous template is used for
updating the stack.
If we encounter a resource with same template as previous template,
we still need to continue to find a possible resource with current
template, to cover the above mentioned case.
Renaming get_existing_db_resource to get_best_existing_db_resource for
better reflection of what it does and moving it out of enclosed method
so that it can be tested.
Change-Id: I5c753e1c0737803ee3de8d36f25c0e620a65f0cf
When a resource fails:
1. Mark the stack as FAILED.
2. If rollback is enabled, invoke rollback for the stack. Otherwise,
clean-up database of stale entries.
After the stack is completed (or failed), the database needs to be
cleaned up.
Change-Id: Ic5713e8c0c78c38ee1aaee8865346cb55228043f
Current template need not be backed up while creating a stack. With
current logic, a circular reference is created with current stack
pointing to current template as previous template.
This happens when the template is stored before invoking stack create.
The template is stored along with the stack just before invoking the
converge_stack for creating the stack. The test cases are modified to
include instrutions to store the stack before running convergence.
Change-Id: I95b629d7d42201bc8273a3f70f3e3819211b32f8
This produces a nested exception like:
'ValueError: resources.nested.resources.my_server: it is broken, sorry'
This re-uses the path mechanism that StackValidationFailed exception
uses.
Change-Id: Id5204c15ee96784e04522ab3c5a8e66900f9a1d3
Closes-bug: 1459837
This is a merge of 4 reviews:
I52f1611d34def3474acba0e5eee054e11c5fc5ad
Ic374a38c9d76763be341d3a80f53fa396c9c2256
Iecd21ccb4392369f66fa1b3a0cf55aad754aeac4
I77b81097d2dcf01efa540237ed5ae14896ed1670
- make sure sender is a tuple (otherwise the serialization
function in sync_point breaks.)
- Update updated_time on any lifecycle operation(CREATE/UPDATE/DELETE)
over a stack.
- adjust sync_point logic to account for deletes
Done by having only a single stack sync point
for both updates and deletes.
- Serialize/deserialize input_data for RPC
- Make GraphKey's the norm in convergence worker
- move temp_update_requires functionality to tests
During intial stages of convergence to simulate the entire cycle
some part of worker code was written in stack.py.
Now that the convergence worker is implemented, this code needs to
be executed only in tests.
- Fix dictionary structure that's passed to resoure.(create/update)
- Temporarily disable loading cache_data for stack to help fix other
issues.
Change-Id: Iecd21ccb4392369f66fa1b3a0cf55aad754aeac4
Co-Authored-by: Sirushti Murugesan <sirushti.murugesan@hp.com>
Co-Authored-by: Rakesh H S <rh-s@hp.com>
Refactored the empty template and added it to template module so that it
can be re-used else where. In convergence it is needed for rolling back
when create fails. Also needed for deleting stack, that is, update the
stack with empty template.
The empty_template method can be passed a template header (version) in
order to get an empty template of that version, otherwise by default, an
empty HOT template is returned.
Change-Id: Ic59bd2d4b9cfed1efd1973af0c4ec9274bcc204a
The following tests have been added:
*test_engine_service.test_validate_new_stack_handle_assertion_error
*test_stack.test_validate_assertion_exception_rethrow
*test_stack_resource.test_validate_assertion_exception_rethrow
These tests are focused on checking the cases when we should
return 500 status codes and log the tracebacks for
AssertionException, but not wrap in the validation exception.
Related commit: I71d91a5413d7f026b332014487583b7f2762b322
Closes-Bug: #1456839
Change-Id: I30c2a91043465aa14c39cae5fb0b0d2c17e059e9
Add previous_status parameter to SupportStatus for
better understanding of objects life cycles.
bp deprecating-improvements
Change-Id: I447d001ada9b0a0297de79704d91e263934a185f
This patch adds call of _add_event method at the end of the update_task.
So now in event-list output user can see, that updating stack is
completed or failed.
Also status reason for update_task were changed to be inline with status
reason in the main stack_task method.
Method set_state is not used due to using stack.store method in the end
of update to avoid duplicate update.
Change-Id: Ic94c95f1558fd67f351696645426c00de8db1bf8
Closes-Bug: #1465321
This patch moves registration of GenericResource to the main test class.
There are several places, where original registration was saved:
- heat/tests/autoscaling/test_lbutils.py - need to be consist with type
names of other resources which are used in test.
- heat/tests/test_environment.py - need special type name for testing
environment mapping.
- heat/tests/test_engine_service.py - need two different types fro mapping.
Change-Id: I33bdaf3d59e573526e5ffa24086ad55be6bbe6ae
This patch separates out the test cases related to thread group manager
and the service engine itself.
Change-Id: I8b2f28f3796d611d72c42ae57e6f19f8bec71d42
We use stack.prepare_abandon() to make the
snapshot data, but the data have no 'files' then will
result in ValueError raised(No content found in the "files"
section for get_file path) when restore.
This patch will store 'files' for snapshot data(
also for prepare abandon data), then we can get 'files'
from the snapshot data when stack restore.
Change-Id: I0fddf92e950b93dce70eb8e5a562cf8d0046a9ab
Closes-Bug: #1461375
This change uses database queries to count the total number of
resources in a nested stack tree instead of loading every stack
and every resource in that stack.
This should result in a significant improvement in memory use
for stacks with many nested stacks and resources.
Closes-Bug: #1455589
Change-Id: I8c2f1fa9114dbf68ad4a5f99dd3929b6958b5d7a
Currently, the patch does the following:
Kicks off workflow from stack.update_or_create:
Once the dependency graph is calculated, the leaves
of the graph are all casted into the RPC worker bus.
Worker RPC check_resource worfklow:
Workers will then start working on each resource
individually. Once a resource operation is finished,
sync points are used to check if the parent resource
can be worked on. Resources that finish early will
wait for their siblings to finish. The sibling that
finishes last will trigger the creation/updation/deletion
of it's parent. This process then goes on for all nodes
until the roots of the graph are processed.
Marks stack as complete when roots have finished.
Once the roots of the graph are successfully processed,
the previous raw template which was needed for rollback
in case something went wrong will now be deleted from the
database. The stack is then marked as complete.
Largely follows the convergence prototype code in
github.com/zaneb/heat-convergence-prototype/blob/resumable/converge/converger.py
Implements blueprint convergence-check-workflow
Change-Id: I67cfdc452ba406198c96afba57aa4e756408105d
After heat engine starts, it won't report its status, so
updated_at is None by then. If heat engine die before its first
status report, its record will not remove from service-list.
Change-Id: I55801ceb12ec8e4a3746799d3d43c6e9c01c6cd8
Closes-Bug: #1459997
This has been deprecated for years and is incompatible with deploying
multiple heat-engine processes. Let's at least not turn it on by default.
Change-Id: Iabddbd65be9000a30a5714b26658ea6acea0e103
This patch separates out test cases related to stack create and
validation. The patch also replaced the existing mox calls with mock
calls.
Change-Id: I167edf7b063f66c835a07d5143f02bd07f62e0a0
This change adds support status information to the resource schema
returned by the /resource_types/{resource_name} endpoint.
Change-Id: I1d030544630bcaed06aa6fa60ea2554861033384
Closes-Bug: #1459486
Rename the StructuredDeployments resource to StructuredDeploymentGroup
to avoid awkward situations when talking about the plural form of the
StructuredDeployment resource. This has been agreed at the Vancouver
summit.
The existing 'StructuredDeployments' resource is kept for backwards
compatibility, but with a deprecated status and corresponding
deprecation message.
Closes-Bug: #1458008
Change-Id: I5d274acf17f222c493d10ded50f87e05a075b424
Rename the SoftwareDeployments resource to SoftwareDeploymentGroup to
avoid awkward situations when talking about the plural form of the
SoftwareDeployment resource. This has been agreed at the Vancouver
summit.
The existing 'SoftwareDeployments' resource is kept for backwards
compatibility, but with a deprecated status and corresponding
deprecation message.
Change-Id: Iae38b4afcb924ba626eccadfd68712e708be2bff
Partial-Bug: #1458008
This is the only thread that gets started without an explicit reference to
the stack. This prevents us from using a weakref in the resource, as it
will cause the Stack's reference count to hit 0 before the thread is run.
Now we explicitly pass a reference to the stack.
Change-Id: Ie51be7b54d97ef184e401e395a7e7e3a26ce003b
Related-Bug: #1454873
Use unittest2.assertWarnsRegex() for making sure we generate the
required warnings.
Use WarningsCapture to not print the other repeated warns to the
console.
Change-Id: I7223f8956394208eaf2eb8a1d651ba1425128bc9
This patch separates out test cases related to stack create and
validation. The patch also replaced the existing mox calls with mock
calls.
Change-Id: I346e11d05c6e28fd178f16556e5528b47b39f47c
Add data type for the attributes. This will help
1) validating the attibute against its type
2) resource-schema api will show the attribute type
3) template authors will know what type of value to
expect making indexing and mapping easier
4) generating docs for the attribute type
Implements: blueprint add-type-in-attributes-schema
Change-Id: Ifc92c57ec1ddd2ab5f810587a1d33e762308dd8a
This patch separates out test cases related to software_config and
software_deployments into a standalone module. The patch also replaces a
few mox calls by mock calls.
Change-Id: Ie2ed122dda16e1cbf8036044d3476e2b328f8796
When generating the dependencies, the deletion order is already taken
care, hence we need not reverse the graph.
Change-Id: Iff158443f2281a07e3a136a4cf6f77f7be592d3f
Generates the graph for traversal in convergence.
* Updates current traversal for the stack
* Deletes any sync_point entries of previous traversal
* Generates the graph for traversal based on
- resources loaded from db for the stack
- resources that exist in present template
* Stores resource.current_template_id and resource.requires
* Stores the edges of graph in stack.current_deps
* Creates sync_points for each node in graph
* Creates sync_point for stack.
blueprint convergence-prepare-traversal
Change-Id: I507e67b39c820ed46d3b269fc76d6cf18d0ef2d7
Currently, the event-list output is very resource-centric, despite
being scoped to the stack from an API path perspective.
This, combined with the fact that the stack updated_at timestamp
is only updated after a succesful update (ref bug #1193269) makes
it impossible to derive the time when an update started via any API.
This is a problem when trying to use the new hook/breakpoint API,
because it's necessary to poll for all events since the most recent
update started, disregarding any stale hook events from previous
updates (which may have failed or timed out without the hooks getting
cleared).
To work around this, add an event for each stack state transition,
such that you can detect the transition to UPDATE_IN_PROGRESS,
then use that event as a marker to get post-update-started events.
Without this (or some other way to determine when the stack update
started), the hooks pre-update functionality landed for kilo is
not really usable (particularly mechanically via scripts).
Change-Id: Idff342b3aecc2d145dfbc7c0f610ad0ca8e52c8b
Partial-Bug: #1448155
Currently heat supports to create CFN template based on
the given resource type. And this patch adds an option
to allow user to specify template type to generate(support
HOT template at the same time).
blueprint support-to-generate-hot-templates
Change-Id: I55cfb9b0f87e638350f2f6367fb399d772fff7e1
six.iterkeys() returns iterator. Using this iterator while
removing entries from dict may raise RuntimeError. So we
shouldn't use iterator in this place. Use list of `stack_id`
values instead.
For py2 we can use .keys(), but .keys() method returns `view`
in py3 and this will lead to the same error as with using
six.iterkeys(). Apply list() to the result given by .keys() to
produce the same behaviour for py2 and py3.
Also refactor tests to make sure that there are no problems
with iterators.
Co-Authored-By: Oleksii Chuprykov <ochuprykov@mirantis.com>
Change-Id: Ice43e18173b4b523d98703bca60819d0e62cfc91
Closes-Bug: #1448186
TypeError: call() takes exactly 3 arguments(5 given) is raised
during cancel stack update, this patch will fix the problem.
Change-Id: If85389150a15575d9e01466d950f8c07b8fa2717
Closes-Bug: #1448384
This patch separates out the stack snapshot test cases. It also replaces
mox tests with mock calls.
Change-Id: I437746a3b7c82ed83c6ec0b3ff7d93a79cfa79b3
This patch extracts the common functions used in engine service test
into a separate module that will be shared among sub-unit-tests.
Following patches will split the engine service tests into sub test
units.
Change-Id: I008856aa43e49bd1bd62f5e03291fa6f1f86fc2b
generate_template and resource_schema methods of service.py
raises wrong type of exceptions after catching them.
Also resource validation failure should not raise stack
validation exception.
Change-Id: I44b8d37db7d2687cee2a7b675dd062b3af3ee167
Closes-Bug: #1447194
We are persisting for a number of reasons:
- so we don't have to pass this through ever rpc call
- the API exposes parent_resource (currently always None as
it is not persisted)
Closes-bug: #1438978
Change-Id: Id2db36c0234a085ec4f0ce2ab114ec483ea29d81
(cherry picked from commit edf86aeac2)
Nova API v1.1 is now deprecated and causes warnings.
Change-Id: Ib4b57a308b7637f4015a07b4e888ccd6347cb947
Closes-Bug: #1437158
(cherry picked from commit d9d68cf52b)
Angus Salkeld
Sirushti Murugesan
Oleksii Chuprykov
tengqm
Kanagaraj Manickam
Anant Patil
Sergey Vilgelm
Andrew Orlov
Peter Razumovsky
Sergey Kraynev
Pavlo Shchelokovskyy
huangtianhua
Tetiana Lashchova
Steve Baker
Ethan Lynn
Zane Bitter
Miguel Grinberg
Thomas Spatzier
tyagi
Rakesh H S
Steven Hardy
Michal Rostecki