Previously when calling function.validate() we passed the path to the
function in the template (used for debugging purposes) as a string. This
change allows us to pass it as a list of path components, as we do
elsewhere, so that higher-level code that catches StackValidationFailed
can deal with its components separately.
Change-Id: I017aa6f7511b8478ef8273522ab8087684ae71c6
When we add a resource to a template using add_resource(), strip any
references to resources that are not present in the template from
'depends_on'. This means that resources that are present and depended on
but conditionally disabled in the source template will not be depended
on in the destination template, where they're not present at all (and
thus would cause errors when the template is used).
Change-Id: I58a64cd523cc0d1da8cd39793f0c839189458d99
Closes-Bug: #1701677
Related-Bug: #1649900
This unifies the 'lightweight stack' used in convergence with the how
things work the rest of the time: we now always obtain resource data from
the StackDefinition's ResourceProxy objects. This means that when we are
checking an individual resource, we will never create all of the other
Resource objects for the stack in memory (although we already avoided
loading data for them from the database) - now we will only create
ResourceProxy objects as needed.
Change-Id: Id7472557e26d172df88841ff7f20afdd7f5bfada
Implements: blueprint stack-definition
Since function.dep_attrs() returns logical resource names (rather
than actual objects), we can just as easily use the StackDefinition to
calculate it instead of the Stack and Resource objects.
In the legacy path, we must ensure we use the StackDefinition from the
*new* stack to determine which attributes to include in the NodeData, since
that's what we're going to be using it for. In the convergence path the
current stack definition already contains the new template.
Also, update the *new* stack's definition with the NodeData obtained from
completed resources (in addition to the existing stack's), so that that
data may be used in calculating the dep_attrs for future resources. This is
required when get_attr functions are nested in the template.
Change-Id: I23efcc091eae53470f7f9cb3ca21e09f00f43808
Partially-Implements: blueprint stack-definition
Before returning the stack outputs, update the StackDefinition with the
resource data needed to resolve them. This will enable us to switch to
resolving outputs via the StackDefinition in a subsequent patch.
Change-Id: Ide3920385d9c4841c18024e1eb1456eed5c8eef5
Partially-Implements: blueprint stack-definition
Adds a new section 'permutations' for 'repeat'
function to decide whether to do nested loops
to iterate over all the permutations of
the elements in the given lists, the default value
is True if no specify and keep the same behavior as
before.
For example:
repeat:
template:
network: %net%
port: %port%
ip: %ip%
for_each:
%net%: [n1, n2]
%port%: [p1, p2]
%ip%: [ip1, ip2]
permutations: False
Will be resolve to:
[{network: n1, port: p1, ip: ip1},
{network: n2, port: p2, ip: ip2}]
Change-Id: I2a008400fb71453f6a78656f2e041ae2efa098a2
Blueprint: improve-repeat-function
Ensure that attributes that are referenced in outputs get cached (and
therefore stored in the database) even when they are not referenced by
other resources.
Change-Id: I667ab04f91edddef5c5dbec0a89d465110c312b4
Closes-Bug: #1660831
This helps with dynamically building templates, and is similar to the
add_resource method.
Change-Id: I51ce5c913438083d6d43eb6d8a8bf55cfa8ea3ee
Related-Bug: #1660831
Add new function 'contains' to check whether
the specific value is in a sequence. The resolve
result of the new function is a boolean value, so
we put it as a condition function too.
Change-Id: I90074070dd917df13345fb7c8d4ed09e4a76e864
Blueprint: support-contains-function
This replaces the deprecated (in python 3.2) unittest.TestCase
method assertRaisesRegexp() with assertRaisesRegex()
Change-Id: I405875738337433a36cba78023534615dea8b7b6
Adds function 'list_concat_unique' to concat
several lists using python's extend function
and make sure without repeating items.
Blueprint: support-list-concat-unique-function
Change-Id: Ia0e35eb578ebe91543c644155a86030b3258118b
Now the function 'list_concat' allows string type
args, like {'list_concat': 'I am string'} or
{'list_concat': ['v1', 'v2']}, this patch fixes the
validation to avoid string type args.
Change-Id: If6d73a90e25f38a37c0503fe0b1aaa93a6fbc380
Closes-Bug: #1685434
The already existing str_replace_strict function raises an error if a
param is not present in the template. str_replace_vstrict, newly added
in this patch, is identical but also raises an error if any of the
params have an empty value.
Change-Id: I5407135cc0435cfbad2d18964fe2119c999f67a3
This function uses python's extend function for lists in order to
concatenate lists. So, given the input: [['v1', 'v2'], ['v3', 'v4']]
the result would be ['v1', 'v2', 'v3', 'v4'].
This comes as an alternative to using yaql's concat.
Change-Id: I082833e73388540b29f71b78293fd36fe04b7b92
Query strings are not _required_ to have slashes escaped, and in fact RFC
3986 suggests that "it is sometimes better for usability to avoid
percent-escaping those characters" (Section 3.4).
TripleO is currently unable to use the make_url function because its
database URLs contain slashes in the query, and end up in config files that
don't allow percent characters. This is probably not going to be an
isolated occurrence, so don't escape slashes in the query string.
Change-Id: I72f84e737b042ecfaabf5639c6164d46a072b423
A large proportion of uses of the str_replace function is to build URLs out
of various components from various sources. This is invariably brittle,
with a failure to escape special characters, deal with IPv6 addresses, and
so on. The make_url function provides a both a tidier interface and correct
handling of these edge cases.
Change-Id: I61b6dff01cd509b3d1c54bca118632c276569f4e
The facade_resource intrinsic function needs to access the parent resource
of a stack, which it does via the stack.parent_resource property. Accessing
this for the first time loads the parent stack and all of its resources if
they were not already. This made sense when all nested stacks in a tree
were handled in-memory at the same time, but now that they are processed by
separate engines it is inefficient.
This change moves responsibility for lazy-loading the parent stack to a
separate ParentResourceProxy class, and makes access more efficient again
by avoiding the loading of resources if the stack was not already in
memory. It also resolves a circular reference between nested stacks and
their parent stacks.
We now have a well-defined API behind which we can potentially make further
efficiency improvements while giving third-party Function plugin developers
confidence that we won't break them without notice.
Change-Id: Ibfd80544889778f3499bcbe421b83f0a5aa6a7f7
Partially-Implements: blueprint stack-definition
This will allow the snapshotting of attribute/refid values to occur on both
the legacy and convergence paths (currently it is used only for
convergence).
Change-Id: I9a8fce9c6d22d84ec967087b62bff77f5a6de3db
Partially-Implements: blueprint stack-definition
Formalise the format for the output data from a node in the convergence
graph (i.e. resource reference ID, attributes, &c.) by creating an object
with an API rather than ad-hoc dicts.
Change-Id: I7a705b41046bfbf81777e233e56aba24f3166510
Partially-Implements: blueprint stack-definition
If a resource is conditionally defined, depending on it raises an error
in the case it's not defined. This patch fixes that by checking if the
resource is present in the template regardless of the conditions.
Change-Id: Iefae1fcea720bee4ed69ad1a5fe403d52d54433c
Closes-Bug: #1649900
We should allow the case where the keys collide, but the
replacement is the same as the current key (e.g a noop).
This is useful when you want to do a map_replace based on a
user parameter, then the default can be to do a noop replace
but the user parameter may also be any other (non colliding)
value.
Change-Id: I8cc5761c219616b4f8c18b3f44c4bf864c5457f1
Closes-Bug: #1652034
A modulo constraint will be used to restrict numeric values to
leave a certain remainder when divided with certain divisor.
E.g. we can use this to constrain values to even/odd numbers.
Change-Id: I9d7db4307be2a2b93cc928cf5912af7b49c72076
In many cases, a user would rather see an error result if a
str_replace param is not substituted in the template, rather than the
function silently doing nothing.
Since str_replace is set in its ways, introduce a new function
str_replace_strict which behaves identically to str_replace except
that a ValueError is raised if any of the param's are not found
in the template.
Change-Id: I8b8c69bb49dfeb74e05af4871602c20493b081eb
Atrributes would return `None` before the resource is
created, hence resource validation would fail when
using repeat function. Use empty list instead to avoid
it.
Change-Id: I93f65d6604944ccae40f65ef52754dcec69e9886
Closes-Bug: #1629899
Error with path to the function is returned now in the
case of failed validation.
Closes-Bug: #1625723
Change-Id: Ib96a944c751543f3eb7cb5f8ba09911fc5cae586
Previously we implemented the str_replace intrinsic function by repeatedly
calling str.replace(), with the result that some text could be subject to
multiple substitutions (i.e. some new text that was inserted by a
replacement could itself be partly replaced if some part of it matched
another key), although users could not rely on this happening since the
replacement order is explicitly undefined.
We anticipated that users would guard against problems here using some sort
of guard characters like %replace_me% (although we avoided requiring any
particular choice of characters). Over time it has become clear that for a
variety of reasons (of varying legitimacy), this is often not the case.
Sometimes the format of the source data is not under the direct control of
the user.
This patch revises the algorithm to perform the replacements in a single
pass and makes the ordering of keys deterministic. It also refactors the
various versions of the function so that we no longer need two
implementations of the core algorithm.
Change-Id: I43461a0d78226f921018efb32aae6e2255928bd0
This change supports:
1. Allow boolean value or condition function as
condition name of 'if' function:
resources:
r1:
...
properties:
a: {if: [true, 'value_true', 'value_false']}
r2:
...
properties:
b: {if: [{equals: [...]}, 'value_true', 'value_false']}
2. Allow boolean value or condition function as
condtiion name in resource definition:
resources:
r1:
...
condition: false
r2:
...
condition: {and: [cd1, cd2]}
3. Allow boolean value or condition function as
condition name in outputs:
outputs:
output1:
value: ...
condition: true
output2:
value: ...
condition: {not: cd3}
Change-Id: I2bf9bb0b608788c928d12425cbfdaf658df9e880
Co-Authored-By: Zane Bitter <zbitter@redhat.com>
Blueprint: support-conditions-function
Change the definitions of the Template.conditions() method to return a new
heat.engine.conditions.Conditions object. This class encapsulates the error
handling involved in looking up a condition, and lazily caches resolved
values so that conditions we never reference are not evaluated.
By raising ValueError instead of StackValidationFailed inside
Conditions.is_enabled(), we ensure that the user sees the full path to the
erroneous condition name if it appears inside the 'if' intrinsic function.
Change-Id: I266eec7e87d185182c4c0fea864e7ebdafa11bd5
Now that we raise them in only a single place each, there's not a lot of
value in having a custom exception type for condition-related errors in the
template any more, and in fact there was no code to handle them
appropriately in heat-api so it would have resulted in 500 errors to the
client if they ever escaped heat-engine anyway.
Just use the existing StackValidationFailed exception type. This provides
more flexible control over the path than InvalidConditionReference did.
Also, raising ValueError in the Invalid function means that
StackValidationFailed will be raised with the path, so it will be much
easier for users to find where they have used an invalid function.
Change-Id: Iaa6ca030fc144efc75e707910f84bfa7385fa9e2
This reverts commit 36bf170f34.
The construction and validation phases of intrinsic functions are
deliberately separated, to:
- Reduce the amount of unnecessary work done
- Reduce the risk of changes to validation breaking the loading of existing
stacks from the database
- Ensure that the most critical errors are reported to the user first
This patch reverts a commit that sought to combine the two phases. There
was no linked bug and no explanation other than that this 'improved'
things.
It does not, however, restore the check for the Removed function during
properties translation (and the associated test), because I believe its
original existence was based on a misconception. There is no way in
practice for a hot.functions.Removed object to appear in the properties a
translation rule is operating on.
Change-Id: I59d02e540771baf4fe078081cd0993a3251eb9a6
This is the last remaining area of the code where we were slinging around
CloudFormation-formatted snippets of template. Replace it with a proper API
along the lines of ResourceDefinition.
The default implementation of the new template.Template.outputs() method is
equivalent to what we have previously done spread throughout the code, so
that third-party Template plugins will not be affected until they have had
time to write their own custom implementations.
Change-Id: Ib65dad6db55ae5dafab473bebba67e841ca9a984
Zane Bitter