24 Commits

Author SHA1 Message Date
Sorin Sbarnea
8a7d77c180 Removed some deprecation warnings and enabled py35 on tox.
Change-Id: Icc5b28c4636b542a10502b7b1d2d2dc2028cc166
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2017-03-12 19:43:33 +00:00
Jenkins
e47f9629bc Merge "Add view management functionality" 2016-11-10 16:35:23 +00:00
Thanh Ha
1deb3aff4c
Add view management functionality
- Adds the ability for JJB to work with views
- Views can be created, updated, and deleted.
- New modules for List view and Build Pipeline view are added
- New tests for testing the deletion of views

Example View configuration:

    - view:
        name: MyView
        view-type: list

Change-Id: Idb29a4407bcc14593e10a4d951036cb04e8e6c27
Co-Authored-By: Brandon Leonard <brandon.leonard@rackspace.com>
Co-Authored-By: Joao Vale <jpvale@gmail.com>
Co-Authored-By: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
2016-10-14 09:14:21 -04:00
Darragh Bailey
64537f5125 Improve logger output for expanding templates
Output the variable inputs used that trigger an error when expanding
template names to the error logger channel in a sensible format.

Ensures that when indented variable inputs for templates result in
exceptions when expanding a template name, that the project, template
name and variables that failed to be iterated over are outputted in a
log error message along with the original set of inputs from the
project definition to make it easier for end users to find where the
error has been made in a JJB definition.

Add code to allow dumping of variables stored in OrderedDict
transparently to match the input format used in JJB definitions and
hide the implementation detail of using OrderedDict to be within the
localyaml library.

Change-Id: I660bb0ca3b109e1a861948d6a867f185047b90ae
2016-09-14 15:15:04 +01:00
Wayne Warren
8ffb11a4bf Be explicit about objects exported by modules.
Change-Id: Iaea087c6c3a024922034423a8d56571184f33b7c
2016-08-20 00:34:35 -07:00
Wayne Warren
c80e3bc5e9
Cleanup various deprecation warnings.
Change-Id: Iec16a5965d62bebb50d3e7307ab93c59304a9ab6
2016-08-20 01:24:49 -04:00
Wayne Warren
aaae83c623
Factor XmlJobGenerator out of YamlParser.
Separate XML generation from Yaml parsing/interpreting. The goal here
is to allow different sources to provide data for XML generation,
including external API users writing job definitions in pure Python or
JJB developers who would like to work on alternative Yaml parsing code
since the current YamlParser has very likely reached the limits of
what can be reasonably done with one giant expandYaml procedure.

Change-Id: I9da848acac7e944c0e07286b7399b2e1956a58a5
2016-08-18 22:42:24 -04:00
Wayne Warren
ae1fb60f16
Disentangle YamlParser and ModuleRegistry classes
Create the ModuleRegistry anywhere other than inside the YamlParser
class. This will make it slightly easier to factor a XmlGenerator out
of YamlParser, but I also want to work toward eliminating the circular
references between YamlParser and ModuleRegistry which have been
making it difficult to understand overall program flow.

This commit also replaces all YamlParser instances being passed to
Jenkins job config generating functions with a ModuleRegistry. Mostly
it seems like the parser was only needed to call the ModuleRegistry's
'dispatch' method which to be honest I don't fully understand. This is
where the circular references mentioned in previously come in...it
seems like the "dispatch" function needs access to the (mostly) raw
data contained by the parser, so it took that as a parameter.

The need for the YamlParser's job data can be satisfied by assigning
it to a property on the ModuleRegistry object before Yaml expansion or
XML generation begins; by doing this, we allow the ModuleRegistry to
avoid referencing the parser.

Change-Id: I4b571299b81e708540392ad963163fe092acf1d9
2016-08-18 22:42:24 -04:00
Wayne Warren
f9b3c4e6a7 Remove YamlParser references from JJB modules.
Change-Id: Ia9ca686c4040fd27890dbce99fda3db3e99a1102
2016-08-05 09:01:30 -07:00
Wayne Warren
f85cf31826 Don't expose YamlParser methods unnecessarily.
Change-Id: I81ef41c2b411850b3535387115e0e379e2efc038
2016-08-05 08:56:35 -07:00
Wayne Warren
c569dc02d3 Move load_files method to YamlParser from Builder.
This is the first step in removing jenkins_modules.parser.YamlParser
references from jenkins_jobs.builder.Builder entirely, necessary
because load_files is actually a YamlParser behavior, not a Builder
behavior.

Change-Id: I1cad99b4cdb44af25ba398837f7f328cfcbd5bbb
2016-07-22 17:33:53 +01:00
Wayne Warren
0da11b51c5 Use JJBConfig in ModuleRegistry.
Remove reference to ConfigParser object in ModuleRegistry. Instead:
* make use of JJBConfig.get_module_config to grab settings for Hipchat
  Notifier Plugin
* make use of JJBConfig.yamlparser['allow_empty_variables'] rather
  than repeating ConfigParser logic moved out of the YamlParser into
  JJBConfig in an earlier commit.

Change-Id: Icb7ef514826005545e48af993335ce120f973b0d
2016-07-22 17:33:30 +01:00
Wayne Warren
4f04de1f9a Use JJBConfig in YamlParser.
This commit sees JJBConfig start to take the form it ought to have,
namely using attributes to represent different logical sections of
configuration that target specific subsystems of JJB.

It also moves ConfigParser data retrieval from
jenkins_jobs.modules.helpers.get_value_from_yaml_or_config_file() to
jenkins_jobs.config.JJBConfig.get_module_config()

TODO: Add JJBConfig tests to validate the behavior of JJBConfig in
specific circumstances.

Change-Id: I053d165559f5325a2f40b239117a86e6d0f3ef37
2016-07-22 17:33:25 +01:00
Jenkins
5d14a621a0 Merge "Allow id's for different templates to follow same naming" 2016-04-21 23:13:59 +00:00
David Caro
d4fa9d7b23 Added possibility to exclude project combinations
Change-Id: I40603f041d33a27d08bf6912b7d9d885a95278fd
Signed-off-by: David Caro <dcaroest@redhat.com>
2016-03-12 17:51:01 +00:00
Darragh Bailey
398e28b3c3 Allow id's for different templates to follow same naming
It is useful to allow templates that can have different settings for
slightly different jobs while still following the same naming scheme.
Additionally this allows for shorter and more descriptive names to be
used without the confusion of the variable placeholders, and for the
resulting job names from the templates to be altered without needing to
update all references.

Change-Id: Idc3517b44873210a33f988ebff449ea2ed567054
2016-02-20 18:20:51 +00:00
Darragh Bailey
8805a34c0f Reorder imports to match hacking guidelines
Ensure that the imports follow the standard OpenStack hacking
guidelines.

Change-Id: Iaa4326aef118ddfd807dd006934f1d9ca80a1cfa
2015-12-23 15:23:42 -08:00
Jenkins
daf8be4480 Merge "Debug message leads to (n*m) log growth." 2015-12-03 04:59:49 +00:00
James E. Blair
7ef05f7373 Add template-name as a variable that can be used in a template
In complex configurations it may be useful to have a template
print out its name so that a given job can be easily traced to
its definition.

Change-Id: I1cfed9c27a58b45ba48aeed04839bfc8f54a831e
2015-11-24 07:47:26 -08:00
Wayne Warren
43cfc27360 Debug message leads to (n*m) log growth.
Where n = number of variables in scope for a particular job template instance.
and   m = number of job template instance.

Let's imagine for a large set of jobs with n = 100 and m = 50

This debug message makes it difficult to scan through JJB output looking for
signficant debug messages and does not provide a clear value.

It is trivially easy to know which variables are and which are not included in a
job-template's name by simply reading it.

Change-Id: I745e26ac6062277fc477cb1ca70cf2809f5806e3
2015-11-17 16:03:28 -08:00
Darragh Bailey
b023d7e23f Wrap file objects with codecs to handle unicode
Ensure file objects including stdin/stdout objects are wrapped using
codecs to handle unicode translation to the selected encoding for
input/output.

Add tests to simulate different encodings for input/output and
consequently fix the reading of input from stdin. Include test to
trigger failure to encode a unicode valid character using 'ascii'
encoding.

Change-Id: I9a5c4c96d131873db0000377729b8b0a45aa470d
2015-10-09 10:48:01 -07:00
Darragh Bailey
0e74bddb56 Replace open() with io.open() and force 'utf-8'
Use io.open() to allow reading and writing of files in 'utf-8' format
irrespective of the terminal encoding selected.

Change-Id: Ie952617a34c0719efc59a7729d698beafaa477b0
2015-07-25 14:16:12 -07:00
Richard Pijnenburg
b56e8812c7 Adding tests surrounding handling of defaults values
waynr fixed a bug but explained in concat_defaults002
This bug happens when some values are specified in an other defaults
namespace then global.
This causes JJB to fail that its unable to find the required keys
becasue these were not merged correctly

Change-Id: Id0df24ba7cf8281389c2ed7af7ee716047d0e4a5
2015-05-27 18:24:54 +02:00
Wayne
4a8b93b8c2 Remove YamlParser from jenkins_jobs.builder
The goal of this patch is simply to move some classes out of
jenkins_jobs.builder into more appropriately-named modules. This started with
simply moving YamlParser into jenkins_jobs.parser but led to other moves in
order to avoid cyclic imports since YamlParser uses other classes previously
defined in jenkins_jobs.builder.

That said, this patch doesn't intend to address all of the clutter in
jenkins_jobs.builder, mostly just what is necessary to get started working on
YamlParser independent of other classes in that module.

Change-Id: Ie88bf683e495033eb0b670fe29c256a70282735f
2015-04-22 11:59:07 -07:00