Fixes JSON to YAML conversion bug

Previously running cfn-json2yaml would generate valid YAML, but the
HOT file would not work. The key mappings it generates had 64 instead
of '64'. Since 64 in YAML is a number and not a string, the template
cannot find the key '64' as it is not a string value.

Turns out this issue was caused by simply forgetting to explicitly
re-add quotation marks after converting from a python object to YAML.

In the conversion code (heat/common/template_format.py), the json
string is first converted into a python object via yaml.load. Next
the python object gets converted to yaml via yaml.dump.

For example:
u'__00015__order__32': u'F17-i386-cfntools' in the python object gets
converted to __00015__order__32: F17-i386-cfntools in yaml.

Crucially in YAML, all strings are not explicity quoted except for
strings that contain only digits. For instance, numerical strings like
'32' will automatically get quoted when using yaml.dump

Normally this would be fine, but a subtle problem arises in the
next line of code:

yml = re.sub('__\d*__order__', '', yml)

By removing all instances of the order substring in the yaml, previous
alphanumeric strings become numeric-only strings. However, since this
step comes after the call to yaml.dump, quotes are not explicity set
anymore!

so for example:
__00015__order__32: F17-i386-cfntool becomes 32: F17-i386-cfntool

Then in YAML, the key 32 is now interpreted as number 32 and not
what we wanted, which is a string '32'

So to help fix this issue, replace all numeric-only keys with quoted
keys. For example:
32: F17-i386-cfntool becomes '32': F17-i386-cfntool
This helps to fix the issue of not finding the numerical keys in
the HOT yaml file.

Change-Id: I37208679f0699d088a7ca632a409d8675cad72c4
Closes-Bug: #1286380
Closes-Bug: #1467029
Closes-Bug: #1467026

heat/common/template_format.py

@@ -108,7 +108,7 @@ def convert_json_to_yaml(json_str):
             next(key_order),
             matchobj.group(2))
         return key
-    key_re = re.compile('^(\s*)"([^"]+)"\s*:', re.M)
+    key_re = re.compile('(\s*)"([^"]+)"\s*:')
     json_str = key_re.sub(order_key, json_str)
 
     # parse the string as json to a python structure
@@ -120,4 +120,7 @@ def convert_json_to_yaml(json_str):
 
     # remove ordering from key names
     yml = re.sub('__\d*__order__', '', yml)
+
+    # convert integer keys back to string
+    yml = re.sub('([\s,{])(\d+)(\s*):', r"\1'\2'\3:", yml)
     return yml

heat/tests/templates/WordPress_Single_Instance.template

@@ -82,7 +82,10 @@
       "cc1.4xlarge" : { "Arch" : "64" }
     },
     "DistroArch2AMI": {
-      "F18"      : { "32" : "F18-i386-cfntools", "64" : "F18-x86_64-cfntools" },
+      "F18"      : {
+         "32" : "F18-i386-cfntools",
+         "64" : "F18-x86_64-cfntools"
+      },
       "F17"      : { "32" : "F17-i386-cfntools", "64" : "F17-x86_64-cfntools" },
       "U10"      : { "32" : "U10-i386-cfntools", "64" : "U10-x86_64-cfntools" },
       "RHEL-6.1" : { "32" : "rhel61-i386-cfntools", "64" : "rhel61-x86_64-cfntools" },

heat/tests/test_template_format.py

@@ -14,6 +14,7 @@
 import os
 
 import mock
+import re
 import six
 import yaml
 
@@ -75,6 +76,17 @@ class JsonToYamlTest(common.HeatTestCase):
             yml_str = template_format.convert_json_to_yaml(json_str)
             yield (json_str, yml_str, f.name)
 
+    def test_integer_only_keys_get_translated_correctly(self):
+        path = os.path.join(os.path.dirname(os.path.realpath(__file__)),
+                            'templates/WordPress_Single_Instance.template')
+        with open(path, 'r') as f:
+            json_str = f.read()
+            yml_str = template_format.convert_json_to_yaml(json_str)
+            match = re.search('[\s,{]\d+\s*:', yml_str)
+            # Check that there are no matches of integer-only keys
+            # lacking explicit quotes
+            self.assertEqual(match, None)
+
 
 class YamlMinimalTest(common.HeatTestCase):