Non-destructive upgrade for deprecated resources

If you attempt to update a stack containing
OS::Heat::SoftwareDeployments resources, so it uses the new
non-deprecated OS::Heat::SoftwareDeploymentGroup type instead, it
deletes the group, and all of the deployments.

This means that any deployment "actions" property will be
misinterpreted, e.g if you have actions: CREATE, all the deployments
will re-run on the update, even though it's an update, not a create.

This issue exists on all deprecated resoruces, when we trying to upgrade
to new version of it by update.

This patch fix above update issue by check if resoruce was deprecated
and been update by replacing resource (which is the parent class of
existing resource).

Change-Id: Ib7880120a90c4497a7ceea53eee55c220a28d14e
Closes-Bug: #1528958

doc/source/developing_guides/supportstatus.rst

@@ -45,6 +45,11 @@ which has follow options:
     - UNSUPPORTED. Resources with UNSUPPORTED status are not supported by Heat
       team, i.e. user can use it, but it may be broken.
 
+*substitute_class*:
+  Assign substitute class for object. If replacing the object with new object
+  which inherited (or extended) from the substitute class will transfer the
+  object to new class type gracefully (without calling update replace).
+
 *version*:
   Release name, since which current status is active. Parameter is optional,
   but should be defined or changed any time SupportStatus is specified or
@@ -78,9 +83,9 @@ Creating process of object
 ++++++++++++++++++++++++++
 During creating object there is a reason to add support status. So new
 object should contains *support_status* parameter equals to ``SupportStatus``
-class with defined version of object and, maybe, some message. This parameter
+class with defined version of object and, maybe, *substitute_class* or some
-allows user to understand, from which this object OpenStack release this object
+message. This parameter allows user to understand, from which OpenStack
-is available and can be used.
+release this object is available and can be used.
 
 Deprecating process of object
 +++++++++++++++++++++++++++++
@@ -91,7 +96,8 @@ parameter, need to add one with current release otherwise move current status
 to *previous_status* and add to *version* current release as value. If some new
 object replaces old object, it will be good decision to add some information
 about new object to *support_status* message of old object, e.g. 'Use property
-new_property instead.'.
+new_property instead.'. If old object is directly replaceable by new object,
+we should add *substitute_class* to *support_status* in old object.
 
 Removing process of object
 ++++++++++++++++++++++++++
@@ -149,7 +155,7 @@ next steps:
 
 1. If there is some support_status in object, add `previous_status` parameter
    with current ``SupportStatus`` value and change all other parameters for
-   current `status`, `version` and, maybe, `message`.
+   current `status`, `version` and, maybe, `substitute_class` or `message`.
 
 2. If there is no support_status option, add new one with parameters status
    equals to current status, `version` equals to current release note and,
@@ -164,6 +170,7 @@ Using Support Status during resource deprecating looks like:
        support_status=support.SupportStatus(
            status=support.DEPRECATED,
            version='5.0.0',
+           substitute_class=SubstituteResourceWithType,
            message=_('Optional message'),
            previous_status=support.SupportStatus(version='2014.2')
        )
@@ -199,6 +206,7 @@ status should be moved to *previous_status*, e.g.:
         previous_status=support.SupportStatus(
             status=support.DEPRECATED,
             version='2015.1',
+            substitute_class=SubstituteResourceWithType,
             previous_status=support.SupportStatus(version='2014.2')
         )
     )

heat/engine/resource.py

@@ -1113,8 +1113,8 @@ class Resource(object):
                 new_res_def.resource_type, resource_name=self.name)
             restricted_actions = registry.get_rsrc_restricted_actions(
                 self.name)
-
+            is_substituted = self.check_is_substituted(new_res_type)
-            if type(self) is not new_res_type:
+            if type(self) is not new_res_type and not is_substituted:
                 self._check_for_convergence_replace(restricted_actions)
 
             action_rollback = self.stack.action == self.stack.ROLLBACK
@@ -1128,7 +1128,15 @@ class Resource(object):
                                    six.text_type(failure))
                     raise failure
 
-            runner = scheduler.TaskRunner(self.update, new_res_def)
+            # Use new resource as update method if existing resource
+            # need to be substituted.
+            if is_substituted:
+                substitute = new_res_type(self.name, self.t, self.stack)
+                self.stack.resources[self.name] = substitute
+                updater = substitute.update
+            else:
+                updater = self.update
+            runner = scheduler.TaskRunner(updater, new_res_def)
             try:
                 runner(timeout=timeout)
                 update_tmpl_id_and_requires()
@@ -1220,6 +1228,14 @@ class Resource(object):
             self.state_set(action, self.FAILED, six.text_type(failure))
             raise failure
 
+    @classmethod
+    def check_is_substituted(cls, new_res_type):
+            support_status = getattr(cls, 'support_status', None)
+            if support_status:
+                is_substituted = support_status.is_substituted(new_res_type)
+                return is_substituted
+            return False
+
     @scheduler.wrappertask
     def update(self, after, before=None, prev_resource=None):
         """Return a task to update the resource.

heat/engine/resources/openstack/heat/software_deployment.py

@@ -700,7 +700,8 @@ class SoftwareDeployments(SoftwareDeploymentGroup):
         version='7.0.0',
         previous_status=support.SupportStatus(
             status=support.DEPRECATED,
-            version='2014.2'))
+            version='2014.2'),
+        substitute_class=SoftwareDeploymentGroup)
 
 
 def resource_mapping():

heat/engine/support.py

@@ -20,8 +20,8 @@ SUPPORT_STATUSES = (UNKNOWN, SUPPORTED, DEPRECATED, UNSUPPORTED, HIDDEN
 
 class SupportStatus(object):
 
-    def __init__(self, status=SUPPORTED, message=None, version=None,
+    def __init__(self, status=SUPPORTED, message=None,
-                 previous_status=None):
+                 version=None, previous_status=None, substitute_class=None):
         """Use SupportStatus for current status of object.
 
         :param status: current status of object.
@@ -29,8 +29,10 @@ class SupportStatus(object):
                     valid. It may be None, but need to be defined for correct
                     doc generating.
         :param message: specific status message for object.
+        :param substitute_class: assign substitute class.
         """
         self.status = status
+        self.substitute_class = substitute_class
         self.message = message
         self.version = version
         self.previous_status = previous_status
@@ -58,6 +60,12 @@ class SupportStatus(object):
                     'previous_status': self.previous_status.to_dict()
                     if self.previous_status is not None else None}
 
+    def is_substituted(self, substitute_class):
+        if self.substitute_class is None:
+            return False
+
+        return substitute_class is self.substitute_class
+
 
 def is_valid_status(status):
     return status in SUPPORT_STATUSES

heat/engine/update.py

@@ -154,11 +154,13 @@ class StackUpdate(object):
         res_name = new_res.name
 
         if res_name in self.existing_stack:
-            if type(self.existing_stack[res_name]) is type(new_res):
+            existing_res = self.existing_stack[res_name]
-                existing_res = self.existing_stack[res_name]
+            is_substituted = existing_res.check_is_substituted(type(new_res))
+            if type(existing_res) is type(new_res) or is_substituted:
                 try:
                     yield self._update_in_place(existing_res,
-                                                new_res)
+                                                new_res,
+                                                is_substituted)
                 except resource.UpdateReplace:
                     pass
                 else:
@@ -181,7 +183,7 @@ class StackUpdate(object):
 
         yield self._create_resource(new_res)
 
-    def _update_in_place(self, existing_res, new_res):
+    def _update_in_place(self, existing_res, new_res, is_substituted=False):
         existing_snippet = self.existing_snippets[existing_res.name]
         prev_res = self.previous_stack.get(new_res.name)
 
@@ -191,7 +193,12 @@ class StackUpdate(object):
         # is switching template implementations)
         new_snippet = new_res.t.reparse(self.existing_stack,
                                         self.new_stack.t)
-
+        if is_substituted:
+            substitute = type(new_res)(existing_res.name,
+                                       existing_res.t,
+                                       existing_res.stack)
+            existing_res.stack.resources[existing_res.name] = substitute
+            existing_res = substitute
         return existing_res.update(new_snippet, existing_snippet,
                                    prev_resource=prev_res)
 

heat/tests/test_resource.py

@@ -2032,6 +2032,27 @@ class ResourceTest(common.HeatTestCase):
                           new_temp.id, res_data, 'engine-007',
                           -1, new_stack)
 
+    def test_update_convergence_with_substitute_class(self):
+        tmpl = rsrc_defn.ResourceDefinition('test_res',
+                                            'GenericResourceType')
+        res = generic_rsrc.GenericResource('test_res', tmpl, self.stack)
+        res._store()
+
+        new_temp = template.Template({
+            'HeatTemplateFormatVersion': '2012-12-12',
+            'Resources': {
+                'test_res': {'Type': 'ResourceWithPropsType',
+                             'Properties': {'Foo': 'abc'}}
+            }}, env=self.env)
+        new_temp.store(self.stack.context)
+        new_stack = parser.Stack(utils.dummy_context(), 'test_stack',
+                                 new_temp, stack_id=self.stack.id)
+
+        res_data = {}
+        self.assertRaises(exception.UpdateReplace, res.update_convergence,
+                          new_temp.id, res_data, 'engine-007',
+                          -1, new_stack)
+
     def test_update_convergence_checks_resource_class(self):
         tmpl = rsrc_defn.ResourceDefinition('test_res',
                                             'GenericResourceType')

heat/tests/test_stack_update.py

@@ -24,6 +24,7 @@ from heat.engine import rsrc_defn
 from heat.engine import scheduler
 from heat.engine import service
 from heat.engine import stack
+from heat.engine import support
 from heat.engine import template
 from heat.objects import stack as stack_object
 from heat.rpc import api as rpc_api
@@ -2035,3 +2036,67 @@ class StackUpdateTest(common.HeatTestCase):
                          test_stack['Bres'].state)
         self.assertIn('create_b', test_stack.t.t['conditions'])
         self.assertIn('create_b_res', test_stack.t.t['parameters'])
+
+    def test_stack_update_with_deprecated_resource(self):
+        """Test with update deprecated resource to substitute.
+
+        Test checks the following scenario:
+        1. Create stack with deprecated resource.
+        2. Update stack with substitute resource.
+        The test checks that deprecated resource can be update to it's
+        substitute resource during update Stack.
+        """
+
+        class ResourceTypeB(generic_rsrc.GenericResource):
+            count_b = 0
+
+            def update(self, after, before=None, prev_resource=None):
+                ResourceTypeB.count_b += 1
+
+        resource._register_class('ResourceTypeB', ResourceTypeB)
+
+        class ResourceTypeA(ResourceTypeB):
+            support_status = support.SupportStatus(
+                status=support.DEPRECATED,
+                message='deprecation_msg',
+                version='2014.2',
+                substitute_class=ResourceTypeB)
+
+            count_a = 0
+
+            def update(self, after, before=None, prev_resource=None):
+                ResourceTypeA.count_a += 1
+
+        resource._register_class('ResourceTypeA', ResourceTypeA)
+
+        TMPL_WITH_DEPRECATED_RES = """
+        heat_template_version: 2015-10-15
+        resources:
+          AResource:
+            type: ResourceTypeA
+        """
+        TMPL_WITH_PEPLACE_RES = """
+        heat_template_version: 2015-10-15
+        resources:
+          AResource:
+            type: ResourceTypeB
+        """
+        t = template_format.parse(TMPL_WITH_DEPRECATED_RES)
+        templ = template.Template(t)
+        self.stack = stack.Stack(self.ctx, 'update_test_stack',
+                                 templ)
+
+        self.stack.store()
+        self.stack.create()
+        self.assertEqual((stack.Stack.CREATE, stack.Stack.COMPLETE),
+                         self.stack.state)
+        t = template_format.parse(TMPL_WITH_PEPLACE_RES)
+        tmpl2 = template.Template(t)
+        updated_stack = stack.Stack(self.ctx, 'updated_stack',
+                                    tmpl2)
+        self.stack.update(updated_stack)
+        self.assertEqual((stack.Stack.UPDATE, stack.Stack.COMPLETE),
+                         self.stack.state)
+        self.assertIn('AResource', self.stack)
+        self.assertEqual(1, ResourceTypeB.count_b)
+        self.assertEqual(0, ResourceTypeA.count_a)

heat_integrationtests/functional/test_replace_deprecated.py

@@ -0,0 +1,92 @@
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+import yaml
+
+from heat_integrationtests.functional import functional_base
+
+
+class ReplaceDeprecatedResourceTest(functional_base.FunctionalTestsBase):
+    template = '''
+heat_template_version: "2013-05-23"
+parameters:
+  flavor:
+    type: string
+  image:
+    type: string
+  network:
+    type: string
+
+resources:
+  config:
+    type: OS::Heat::SoftwareConfig
+    properties:
+      config: xxxx
+
+  server:
+    type: OS::Nova::Server
+    properties:
+      image: {get_param: image}
+      flavor: {get_param: flavor}
+      networks: [{network: {get_param: network} }]
+      user_data_format: SOFTWARE_CONFIG
+  dep:
+    type: OS::Heat::SoftwareDeployments
+    properties:
+        config: {get_resource: config}
+        servers: {'0': {get_resource: server}}
+        signal_transport: NO_SIGNAL
+outputs:
+  server:
+    value: {get_resource: server}
+'''
+
+    deployment_group_snippet = '''
+type: OS::Heat::SoftwareDeploymentGroup
+properties:
+  config: {get_resource: config}
+  servers: {'0': {get_resource: server}}
+  signal_transport: NO_SIGNAL
+'''
+    enable_cleanup = True
+
+    def test_replace_software_deployments(self):
+        parms = {'flavor': self.conf.minimal_instance_type,
+                 'network': self.conf.fixed_network_name,
+                 'image': self.conf.minimal_image_ref
+                 }
+        deployments_template = yaml.safe_load(self.template)
+        stack_identifier = self.stack_create(
+            parameters=parms,
+            template=deployments_template,
+            enable_cleanup=self.enable_cleanup)
+        expected_resources = {'config': 'OS::Heat::SoftwareConfig',
+                              'dep': 'OS::Heat::SoftwareDeployments',
+                              'server': 'OS::Nova::Server'}
+        resource = self.client.resources.get(stack_identifier, 'server')
+        self.assertEqual(expected_resources,
+                         self.list_resources(stack_identifier))
+        initial_phy_id = resource.physical_resource_id
+        resources = deployments_template['resources']
+        resources['dep'] = yaml.safe_load(self.deployment_group_snippet)
+        self.update_stack(
+            stack_identifier,
+            deployments_template,
+            parameters=parms)
+        resource = self.client.resources.get(stack_identifier, 'server')
+        self.assertEqual(initial_phy_id,
+                         resource.physical_resource_id)
+        expected_new_resources = {'config': 'OS::Heat::SoftwareConfig',
+                                  'dep': 'OS::Heat::SoftwareDeploymentGroup',
+                                  'server': 'OS::Nova::Server'}
+        self.assertEqual(expected_new_resources,
+                         self.list_resources(stack_identifier))