The abstract base classes previously defined in 'collections' were moved
to 'collections.abc' in 3.3. The aliases will be removed in 3.10.
Preempt this change now with a simple find-replace:
$ ag -l 'collections.($TYPES)' | \
xargs sed -i 's/\(collections\)\.\($TYPES\)/\1.abc.\2/g'
Where $TYPES is the list of moved ABCs from [1].
[1] https://docs.python.org/3/library/collections.abc.html
Change-Id: Ia282479bb1d466bd2189ebb21b51d91e89b9581e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.
Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.
To review this patch you can use the six documentation [1] to obtain help and
understand choices.
Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.
Latin-1 is equal to iso-8859-1 [3].
This encoding is the default encoding [4] of certain descriptive HTTP
headers.
I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).
HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].
Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'
[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp
Patch 13 of a serie of 28 patches
Change-Id: I09aa3b7ddd93087c3f92c76c893c609cb9473842
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.
Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.
To review this patch you can use the six documentation [1] to obtain help and
understand choices.
Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.
Latin-1 is equal to iso-8859-1 [3].
This encoding is the default encoding [4] of certain descriptive HTTP
headers.
I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).
HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].
Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'
[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp
Patch 3 of a serie of 28 patches
Change-Id: If6e66839c128dde5fb80d90155dedb598da8d53b
We allow users to provide a description for a resource that shows up in
the Heat API. Many resource types also allow users to add descriptions,
and these have typically been exposed as properties rather than by
trying to read the resource's own description.
To prevent the need to duplicate information, use the resource's
description as the default for any top-level properties named
'description'.
One downside of this is that any changes to the resource description
could cause updates to the resource. (However, there are no issues
specific to updgrades, because the subsitution is also done on the
previous properties.) To guard against the worst of this, only enable
this behaviour if the description can be updated without resource
replacement.
Change-Id: I56560d014a02b5f2ddbc08689d39147fbe4ffca4
In the legacy path we reparse a resource definition to refer to a different
StackDefinition when copying a resource definition into the existing
template (as the resource is updated) or into the backup stack's template.
However, the resources 'condition' may reference conditionals that are not
defined in the template it is being copied into (e.g. during a stack update
that creates a new condition), and in this case the destination template
will become unusable.
Since the fact that we care about the resource definition at all indicates
that the condition was enabled, just ignore the condition after reparsing.
This is consistent with what we do for 'if' macros, which is to resolve the
condition part during reparsing.
Change-Id: I59a374435f6275badc8124efbd7b7db2e36e2de5
Story: #2003558
Task: 24847
We used to validate intrinsic functions in the ResourceDefinition by just
passing it to function.validate(). That worked when it also acted as a
CloudFormation template snippet but, since that behaviour was removed by
2c6fc7bcd6, that call now does nothing. This
replaces it with a validate() method on the ResourceDefinition itself.
This also improves path reporting for validation errors in deletion or
update policies.
Change-Id: I58c788002136ddf9bded309366e2072823b3ac77
Closes-Bug: #1732798
It was unclear what the valid arguments to Template.get_section_name() were
(especially since the function is mis-named for what it actually does in
HOT). Define the arguments as constants and don't pass string literals any
more. Be consistent in how we define paths, standardising on the method in
Resource.validate_template().
Change-Id: Ifd073d9889ff60502f78aaa54532cec2b7814d93
This abstracts some internal knowledge of the various Template
implementations that hasn't become part of the public interface yet out of
ResourceDefinition and into the StackDefinition class where the current
implementation is a little more robust for third-party Template plugins,
and which can be further improved in future.
Change-Id: I01b4e6a93dc5b9f07ee0d1cad7af934ff9c9b4f0
When the dep_attrs function was written, it was used only in convergence
after checking a single resource. However, now we also use it to generate
data for the ResourceProxy objects, which is often done for all resources
simultaneously. That means doing O(n^2) dep_attrs() calls, which can really
slow things down when there is a large number of resources with complex
properties (or metadata).
This change adds an all_dep_attrs() call, which runs as fast as dep_attrs()
on everything except get_attr functions and their arguments, but only needs
to be called once instead of once per resource. (The get_attr function can
in future override the default implementation of all_dep_attrs() to be as
efficient as dep_attrs() as well.) The resulting data is cached in the
ResourceDefinition or OutputDefinition so that subsequent calls to their
get_attr() methods with different (or the same) resource names will use the
existing data.
Change-Id: If95f4c04b841519ce3d7492211f2696588c0ed48
Partially-Implements: blueprint stack-definition
Closes-Bug: #1684272
In order to calculate the dependencies of a ResourceDefinition, we have to
pass it a Stack object. This means we can't infer anything about the
resource's dependencies without instantiating the whole stack, and it means
we can't easily cache the result so we have to walk the entire
properties/metadata every time we need the dependencies.
This change splits a required_resource_names() method (the same as added to
OutputDefinition in the previous patch) out of the dependencies() method.
This new method calculates as much as we can know from the resource
definition alone and caches the result, while the dependencies() method
uses the (cached) result to calculate the graph dependencies for a given
Stack.
This also opens the way in future to changing the Function API to return
dependencies by name instead of as ResourceProxy objects.
Change-Id: Icdd0b2dd41116adae5e57ff3ef0e662ba75e6e2c
Don't assume that function.dependencies() always returns the Resource
object from the provided Stack. (In the future it will return a
ResourceProxy object.) Instead, look up the Resource object we want from
the Stack, as we do with e.g. explicit dependencies (depends_on).
Change-Id: I46d74f760c7222ba3c54ed10beb829bd7ea7746f
Partially-Implements: blueprint stack-definition
Disable old translation mechanism and enable new
one. Integrate it with Properties class and correct
tests.
Change-Id: I953a52e9b165d3ea4fb2fc57ceea8083c7f8f30c
Closes-bug: #1620859
Currently Properties has arg "parent_name", which
is used for detailed path in error and allows to
build path for nested properties schemas. But on
top parent_name takes resource name as initial,
which is few incorrect - Properties should raises
with error about Properties, other info (about in
what resource this error raised) should be built out of
Properties module.
Change-Id: I36e6453a1589c02a3f8cf2c080b38693b23b0f1b
Related-bug: #1620859
If a resource is conditionally defined, depending on it raises an error
in the case it's not defined. This patch fixes that by checking if the
resource is present in the template regardless of the conditions.
Change-Id: Iefae1fcea720bee4ed69ad1a5fe403d52d54433c
Closes-Bug: #1649900
Heat no longer accesses resource definitions as if they were snippets of a
CloudFormation template, but rather uses the APIs of the ResourceDefinition
class. The ability to treat this class as a snippet of CloudFormation
template has been deprecated for several release cycles to allow
third-party plugins to move away from it; this change removes that ability.
Change-Id: Ia9c63662438d4c979b897118b3a7a736161bcbca
This change supports:
1. Allow boolean value or condition function as
condition name of 'if' function:
resources:
r1:
...
properties:
a: {if: [true, 'value_true', 'value_false']}
r2:
...
properties:
b: {if: [{equals: [...]}, 'value_true', 'value_false']}
2. Allow boolean value or condition function as
condtiion name in resource definition:
resources:
r1:
...
condition: false
r2:
...
condition: {and: [cd1, cd2]}
3. Allow boolean value or condition function as
condition name in outputs:
outputs:
output1:
value: ...
condition: true
output2:
value: ...
condition: {not: cd3}
Change-Id: I2bf9bb0b608788c928d12425cbfdaf658df9e880
Co-Authored-By: Zane Bitter <zbitter@redhat.com>
Blueprint: support-conditions-function
This allows us to parse the conditions only once instead of reparsing them
all every time we encountered a resource with a condition. It also allows
us to get rid of some of the API surface area and reduces our reliance on
some of the rest.
Partial-Bug: #1618713
Change-Id: I3c2273722171b9c4cb13ef6588f7b522b6689b1c
This adds support for the following to the template:
heat_template_version: 2016-10-14
resources:
...
res_a:
type: OS::Nova::Server
external_id: the-new-server-id
properties:
...
Co-Authored-By: Rico Lin <rico.l@inwinstack.com>
blueprint external-resources
Change-Id: I8fda1380504d1d8b1e96649bf20b86d6309fdeca
The last remaining code using the deprecated methods in the
ResourceDefinition class to treat the definition like a CloudFormation
template snippet is the tmpl_diff in the Resource class. Replace the dict
diff with a Diff object that has an API which can be queried instead. For
backward compatibility with existing resource plugins, the current diff
objects are for now able to be treated like dicts. However, this is marked
deprecated and can be removed along with the equivalent support for
ResourceDefinitions in the future.
Change-Id: I9ef50f0d9b142d464a1b02da7a46536c532d2ebd
The commit 4188c40beb cleaned up a lot of
places where the warnings module was just being used as a non-standard way
to log a warning. However, the way the module was used in the rsrc_defn
module is correct and should not have been changed. By passing a
DeprecationWarning to warnings.warn() we not only make it easy to find
deprecated functions in the source, we also make it possible for developers
to test whether deprecated code is unused by forcing the exception to be
raised. There's also no point in translating these warnings, since they are
targeted at developers (mostly developers of third-party plugins) and not
operators per se. This patch reverts the change to that one file and its
tests.
Change-Id: I9cb1e739e9f173c8f70cb944e886cc3b49abc5f3
If we use unicode as resource name or parameter,
we are likely to encounter unicode problems.
This patch aims to fix unicode problems, and
there are several things to be clear:
1. For the class wrap with six.python_2_unicode_compatible, in
python2 their original __str__ function will map to __unicode__,
and new __str__ function comes from __unicode__.encode.
So we should always return unicode in __str__() if wrap
with python_2_unicode_compatible.
2. python_2_unicode_compatible will not handle __repr__,
__repr__ should return str in all versions of python.
Co-Authored-By: Ethan Lynn <xjunlin@cn.ibm.com>
Closes-Bug: #1524194
Change-Id: Ib4af43d153e0edf9d66308bf8e7b93a3be501d2e
Remove the incorrect __hash__ function added to the Function class in
46233d2c2a and make the class unhashable. Fix
the one place we were trying to hash it, which is when it appeared in the
deletion_policy of a resource.
Change-Id: I3d47c99470a2404183bccb80269a18623aa79041
There's nothing about hashing a ResourceDefinition that is deprecated or is
ever going away - in fact, the actual implementation of __hash__() is in
ResourceDefinitionCore, and it's only explicitly included in
ResourceDefinition because there is also an __eq__() method which would
otherwise cause Python 3 to override __hash__ with None (per
46233d2c2a). Therefore it's
completely wrong to log a warning, since everything will continue to work
fine when ResourceDefinition is replaced by ResourceDefinitionCore.
Change-Id: I33fc8727c9ec92580a2e2e7f4475d0f4fff2234b
Closes-Bug: #1546362
The process of bringing us into compliance with the H405 pep8 rule has left
us with a lot of docstrings that either don't follow the recommendations of
PEP0257, are misleading, wrong, or nonsensical. This patch attempts to fix
them up.
Change-Id: Icfe90b5ba3102a7e13ab659038a8b2af9019e9e6
The part of a ResourceDefinition that lists explicit dependencies was not
round-trip stable. As a result, when we copied a new resource definition
into the existing template during a stack update, we would end up rewriting
the template unnecesarily (i.e. even though we check for changes) every
time if depends_on was not specified in the resource originally. At the end
of each update, we write the new template to the DB in its entirety, which
removes these extra lines again, ensuring that we will experience the same
problem on every update. This was causing a *lot* of unnecessary writes.
This change ensures that the definition remains stable across a round-trip,
so that no unnecessary changes appear in the template.
Change-Id: If7292e49755db0153d7d0db9f7d3875fa9c1d408
Closes-Bug: #1494108
Use ResoruceDefinitionCore methods metadata and deletion_policy
instead of using ResourceDefinition as a dict.
Also, fix wrong ResourceDefinitionCore __repr__ method.
Change-Id: If7d4e77a72115aefc4878257508c1d6270db444f
According to [1], A class that overrides __eq__() and does not define
__hash__() will have its __hash__() implicitly set to None. When the
__hash__() method of a class is None, instances of the class will raise
an appropriate TypeError when a program attempts to retrieve their hash
value, and will also be correctly identified as unhashable when checking
isinstance(obj, collections.Hashable).
A few other classes also implement __eq__ but we don't seem to use them
anywhere as keys in dictionaries, so they don't need to implement the
__hash__ function yet.
[1] https://docs.python.org/3.4/reference/datamodel.html?highlight=hash#object.__hash__
partial blueprint heat-python34-support
Change-Id: Idf24d462f386927496950e65a24e41b9aef69281
Docstring of class ResourceDefinition tells, that backward compatible
methods should be deprecated in favor of using methods in
ResourceDefinitionCore, but there is no any warning message about it.
This patch adds deprecation warnings.
Aslo related tests were changed to check, that deprecation warning is
really raised.
Change-Id: Ie881a1bdb88f4bcb1c03c037bcedf2a328d5d41c
* Use six.moves.reduce
* Update numliteral from 0L to 0
* Use open() instead of file()
* Use rich comparison methods instead of cmp()
partial blueprint heat-python34-support
Change-Id: I9ffd85645563192b5d6124f8dbb71c24245eefc4
In some cases, there is no information about resource and
section, where Property error raised. This patch improves
StackValidationFailed msg, so this msg look like "Property
error : resource_name.section_name.key_name: error_msg", where
section_name is section, where Property error raised, e.g.
'update_policy'.
Change-Id: Iab2a6acdec254b39983de420ab03f994cff48d89
Closes-bug: #1358512
Use parentheses instead of backslash for line continuation.
Plus, usage of escaped quotes in long strings is avoided where it makes
sense.
Change-Id: If2e78012b85a4430c6f03f65784cac2d032cf116
Don't use positional args for kwargs as the ordering can create bugs.
part of blueprint external-resources
Change-Id: I2a494d34d7eab3df6bf5daa9da15a65ea45bf56c
For python3 compatible, using six.string_types instead of
basestring(). six.string_types represent basestring in
python2 and str in python3.
Change-Id: Ib4ba3d164f782601944023b99d6cc1ee2763ee85
Mutables in the method params defaults might cause errors and
that's why it's anti-pattern in most of the cases and should be
removed.
Change-Id: I8bc284f12ce72082a0482410ec2c20c2fc087a4b
Closes-Bug: #1327473
This class provides a comparable, hashable representation of a resource's
definition in a template that is independent of the template format.
Implements: blueprint resource-template-api
Change-Id: I183114596863fbe0c0ff7cd22134b37493ed0dd9
Stephen Finucane