19 Commits

Author SHA1 Message Date
Darragh Bailey
13df88774a Add helper tag to join arbitrary lists into strings
Adds a yaml application tag '!join:' to support defining data as a list
while having the resulting contents automatically joined together with
the specified delimiter and passed as a string into the resulting object
returned by yaml for JJB to process.

This allows users to store long lists of data that is combined together
for use with Jenkins plugins that expect a delimited string of arguments
instead of needing each module in JJB to provide individual support.

Change-Id: I745181ade3926d5c29708963189ae499a0378ece
2018-04-12 20:36:44 +01:00
Darragh Bailey
4573b3a25d Refactor base test classes inheritance for reuse
Refactor base test class inheritance to allow for BaseTest class to be
reused without needing to also import the TestCase class everywhere in
order to avoid having the common test function executed on base classes.

This makes it easier to build base testing classes and then simplifies
the subclassing of these for actual tests.

Change-Id: I89809e8082469f814f245db4a9ab7658aac8a405
2016-09-14 14:58:57 +01:00
Wayne Warren
c569dc02d3 Move load_files method to YamlParser from Builder.
This is the first step in removing jenkins_modules.parser.YamlParser
references from jenkins_jobs.builder.Builder entirely, necessary
because load_files is actually a YamlParser behavior, not a Builder
behavior.

Change-Id: I1cad99b4cdb44af25ba398837f7f328cfcbd5bbb
2016-07-22 17:33:53 +01:00
Wayne Warren
4f04de1f9a Use JJBConfig in YamlParser.
This commit sees JJBConfig start to take the form it ought to have,
namely using attributes to represent different logical sections of
configuration that target specific subsystems of JJB.

It also moves ConfigParser data retrieval from
jenkins_jobs.modules.helpers.get_value_from_yaml_or_config_file() to
jenkins_jobs.config.JJBConfig.get_module_config()

TODO: Add JJBConfig tests to validate the behavior of JJBConfig in
specific circumstances.

Change-Id: I053d165559f5325a2f40b239117a86e6d0f3ef37
2016-07-22 17:33:25 +01:00
Darragh Bailey
8805a34c0f Reorder imports to match hacking guidelines
Ensure that the imports follow the standard OpenStack hacking
guidelines.

Change-Id: Iaa4326aef118ddfd807dd006934f1d9ca80a1cfa
2015-12-23 15:23:42 -08:00
Michael Jeanson
2f410fe225 Add 'do-not-fingerprint' to 'copyartifact' builder
Change-Id: I5fa1fa8ac1e1e932f91f2428f41f5675f72f27a6
Story: 2000340
2015-11-19 17:16:20 -05:00
Jenkins
3f0ff2dff6 Merge "Capture logs with FakeLogging Fixture" 2015-10-07 16:52:54 +00:00
Darragh Bailey
9bfbffa559 Capture logs with FakeLogging Fixture
Capture logs to attach them on exceptions to simplify the debugging of
errors in tests when they occur.

Change-Id: I5c1b5c306aba37789a8d41eabf65e0ac628af838
2015-09-30 16:25:22 +01:00
Darragh Bailey
3d728c71a8 Separate out custom yaml tags into separate classes
Implement custom tags for including files by subclassing YAMLObject to
allow PyYAML to easily convert from the Yaml nodes.

Change to support both lists and strings under the same tag name and
deprecate the old tag naming. Rename tests using the deprecated tags to
ensure they are not used for examples. Remove old tests that simply
duplicate others.

Change-Id: I1a8d3376ea6e4918a2adb05fb856b939a3124d74
2015-09-18 00:37:00 +01:00
Darragh Bailey
e5738d0852 Allow whitespace to be significant
Remove the test options suppressing ignoring differences in whitespace
outputted in the XML to allow capturing of issues where the code
incorrectly adds or removes significant whitespace.

Add a helper function to deal with the edge case of parsing XML directly
that contains non significant whitespace to avoid accidentally adding
excess whitespace when minidom outputs the resulting document.

Change-Id: I9936042cd82c204ba2b3c19f575703e33564f7fd
2015-08-11 18:31:48 +01:00
Jenkins
a3aef64855 Merge "Allow inclusion of multiple files" 2015-05-29 17:05:32 +00:00
Victor Seva
1191dcfccf Allow use of aliases defined previously inside included files
Anchors and aliases were expanded internally within JJB's yaml loading
calls so they were limited to individual documents. Now, included files
will have access to aliases of anchors already defined at previously
processed files.

Example:

- default:
    name: default-timeout-wrapper
    timeout: &timeout
      fail: true
      elastic-percentage: 150
      elastic-default-timeout: 90
      type: elastic

- wrapper: !include include002_1.yaml.inc

Previously was not possible to use '*timeout' alias inside
include002_1.yaml.inc file

Closes-Story: 2000173

Change-Id: Ic031ddbb0310bd11748183fbde9502735c3b7169
2015-05-14 14:01:11 +02:00
Jenkins
2a8b57aa40 Merge "fix test in include-raw-escape001.yaml" 2015-04-23 15:22:59 +00:00
Jan Hruban
e7690d9481 Allow inclusion of multiple files
Add variants of the include-raw* tags, which accept a list of files.
Those files are concatenated and included as a string data into the
calling yaml construct.

Change-Id: I6af87a298268acc8a73d7a2b50f9f99733d8723a
2015-04-20 17:15:40 +02:00
Darragh Bailey
2338167e4e Treat non-existant output files as empty files
Replace requiring empty files for tests that produce no output and treat
any non-existant output file candidate as though the output is blank.

This simplifies tests that are intended to trigger exceptions and will
not generate an output.

Change-Id: I978fdabb488bc3e4bc6f22db294d21ca5d68913b
2015-03-23 16:33:41 +00:00
Khai Do
4fca8ba2d5 fix test in include-raw-escape001.yaml
The yaml list for job templates is 'job-template' not
'template-job'.

Change-Id: I693cdfd2ecb2dcdb10db93bbcfe158a910d11222
2015-03-12 15:06:06 -07:00
Darragh Bailey
1e70128d30 Add tests for yaml anchor behaviour
Adds two tests to ensure correct behaviour with referencing yaml
anchors and a third test to verify the expansion internally of yaml
anchors and aliases.

* test that anchors are not carried across subsequent top level
  invocations of yaml.load(). This will be used subsequently to ensure
  that where anchors are allowed across included files they are
  correctly reset on each top level call.
* test that where anchors are defined in a top level file and
  subsequently included files, duplicate anchors raise exceptions as
  though they were defined within the same file.
* test that data returned from yaml loading contains the additional data
  specified by the alias. Uses json to force conversion so that the
  outputted yaml contains the results of the anchors and aliases instead
  of them.

Update documentation to contain details of the use of anchors and
aliases including a refernce to a simple generic example from the
specification as well as a JJB specific example.

Change-Id: I0f2b55e1e2f2bad09f65b1b981baa0159372ee10
2015-03-10 14:27:30 +00:00
Darragh Bailey
c99cbccb8e Ensure dict orders are deterministic
Python 3 enables hash randomization by default, additionally tox 1.7
turns on the same randomization for earlier versions of python by
default. Need to ensure that order of iteration over the yaml data and
resulting XML has deterministic order for testing.

Adapts https://gist.github.com/enaeseth/844388 which ensures data read
by yaml will have its order retained in a predictable manner across
multiple python versions.

Additionally it seems more sensible to ensure that the order of
generated XML snippets corresponding to the input yaml files are
consistently in the same order as the entries in the source files.

Closes-Bug: #1333349
Change-Id: I6bf6d298a2609cc6ddbbc6b02b7f1a04413a5c89
2014-09-01 14:15:10 +01:00
Darragh Bailey
3e3996d32a Use yaml local tags to support including files
Add support for local tags which are application specific to allow
including of other yaml files or code from scripts. Allows for code to
be maintained and tested as seperate files, as well as reduces
duplication of yaml code that cannot be macro'ed or easily templated by
including it from a common file.

Adds support for the following tags:
  'include' - load file as yaml code
  'include-raw' - load file as data (for scripts)
  'include-raw-escaped' - load file as data with escaping braces '{}'
      as default for use with job-templates

Use configuration file options to provide a search path for the files.

- Test behaviour of yaml tags independent of any XML generation
  by comparing json result of yaml parsing to verify that certain
  tags do/don't recall the yaml.load() method.
- Add examples for the include tags via addition tests for YamlParser
  class

Inspired by
http://stackoverflow.com/questions/528281/how-can-i-include-an-yaml-file-inside-another

Change-Id: Ib90a07043112d4739d6529ceddbc9817668bcec0
2014-06-20 23:16:23 +01:00