Adds a yaml application tag '!join:' to support defining data as a list
while having the resulting contents automatically joined together with
the specified delimiter and passed as a string into the resulting object
returned by yaml for JJB to process.
This allows users to store long lists of data that is combined together
for use with Jenkins plugins that expect a delimited string of arguments
instead of needing each module in JJB to provide individual support.
Change-Id: I745181ade3926d5c29708963189ae499a0378ece
Refactor base test class inheritance to allow for BaseTest class to be
reused without needing to also import the TestCase class everywhere in
order to avoid having the common test function executed on base classes.
This makes it easier to build base testing classes and then simplifies
the subclassing of these for actual tests.
Change-Id: I89809e8082469f814f245db4a9ab7658aac8a405
This is the first step in removing jenkins_modules.parser.YamlParser
references from jenkins_jobs.builder.Builder entirely, necessary
because load_files is actually a YamlParser behavior, not a Builder
behavior.
Change-Id: I1cad99b4cdb44af25ba398837f7f328cfcbd5bbb
This commit sees JJBConfig start to take the form it ought to have,
namely using attributes to represent different logical sections of
configuration that target specific subsystems of JJB.
It also moves ConfigParser data retrieval from
jenkins_jobs.modules.helpers.get_value_from_yaml_or_config_file() to
jenkins_jobs.config.JJBConfig.get_module_config()
TODO: Add JJBConfig tests to validate the behavior of JJBConfig in
specific circumstances.
Change-Id: I053d165559f5325a2f40b239117a86e6d0f3ef37
Capture logs to attach them on exceptions to simplify the debugging of
errors in tests when they occur.
Change-Id: I5c1b5c306aba37789a8d41eabf65e0ac628af838
Implement custom tags for including files by subclassing YAMLObject to
allow PyYAML to easily convert from the Yaml nodes.
Change to support both lists and strings under the same tag name and
deprecate the old tag naming. Rename tests using the deprecated tags to
ensure they are not used for examples. Remove old tests that simply
duplicate others.
Change-Id: I1a8d3376ea6e4918a2adb05fb856b939a3124d74
Remove the test options suppressing ignoring differences in whitespace
outputted in the XML to allow capturing of issues where the code
incorrectly adds or removes significant whitespace.
Add a helper function to deal with the edge case of parsing XML directly
that contains non significant whitespace to avoid accidentally adding
excess whitespace when minidom outputs the resulting document.
Change-Id: I9936042cd82c204ba2b3c19f575703e33564f7fd
Anchors and aliases were expanded internally within JJB's yaml loading
calls so they were limited to individual documents. Now, included files
will have access to aliases of anchors already defined at previously
processed files.
Example:
- default:
name: default-timeout-wrapper
timeout: &timeout
fail: true
elastic-percentage: 150
elastic-default-timeout: 90
type: elastic
- wrapper: !include include002_1.yaml.inc
Previously was not possible to use '*timeout' alias inside
include002_1.yaml.inc file
Closes-Story: 2000173
Change-Id: Ic031ddbb0310bd11748183fbde9502735c3b7169
Add variants of the include-raw* tags, which accept a list of files.
Those files are concatenated and included as a string data into the
calling yaml construct.
Change-Id: I6af87a298268acc8a73d7a2b50f9f99733d8723a
Replace requiring empty files for tests that produce no output and treat
any non-existant output file candidate as though the output is blank.
This simplifies tests that are intended to trigger exceptions and will
not generate an output.
Change-Id: I978fdabb488bc3e4bc6f22db294d21ca5d68913b
Adds two tests to ensure correct behaviour with referencing yaml
anchors and a third test to verify the expansion internally of yaml
anchors and aliases.
* test that anchors are not carried across subsequent top level
invocations of yaml.load(). This will be used subsequently to ensure
that where anchors are allowed across included files they are
correctly reset on each top level call.
* test that where anchors are defined in a top level file and
subsequently included files, duplicate anchors raise exceptions as
though they were defined within the same file.
* test that data returned from yaml loading contains the additional data
specified by the alias. Uses json to force conversion so that the
outputted yaml contains the results of the anchors and aliases instead
of them.
Update documentation to contain details of the use of anchors and
aliases including a refernce to a simple generic example from the
specification as well as a JJB specific example.
Change-Id: I0f2b55e1e2f2bad09f65b1b981baa0159372ee10
Python 3 enables hash randomization by default, additionally tox 1.7
turns on the same randomization for earlier versions of python by
default. Need to ensure that order of iteration over the yaml data and
resulting XML has deterministic order for testing.
Adapts https://gist.github.com/enaeseth/844388 which ensures data read
by yaml will have its order retained in a predictable manner across
multiple python versions.
Additionally it seems more sensible to ensure that the order of
generated XML snippets corresponding to the input yaml files are
consistently in the same order as the entries in the source files.
Closes-Bug: #1333349
Change-Id: I6bf6d298a2609cc6ddbbc6b02b7f1a04413a5c89
Add support for local tags which are application specific to allow
including of other yaml files or code from scripts. Allows for code to
be maintained and tested as seperate files, as well as reduces
duplication of yaml code that cannot be macro'ed or easily templated by
including it from a common file.
Adds support for the following tags:
'include' - load file as yaml code
'include-raw' - load file as data (for scripts)
'include-raw-escaped' - load file as data with escaping braces '{}'
as default for use with job-templates
Use configuration file options to provide a search path for the files.
- Test behaviour of yaml tags independent of any XML generation
by comparing json result of yaml parsing to verify that certain
tags do/don't recall the yaml.load() method.
- Add examples for the include tags via addition tests for YamlParser
class
Inspired by
http://stackoverflow.com/questions/528281/how-can-i-include-an-yaml-file-inside-another
Change-Id: Ib90a07043112d4739d6529ceddbc9817668bcec0