Add a list subcommand that allows listing of all the jobs defined in
yaml configuration, or on the Jenkins master, with the option of taking
glob parameters to allow matching some job names.
Also supports the recursive and exclude path control options.
jenkins-jobs list 'wikimedia-fundraising*'
jenkins-jobs list -r -p configs/ 'wikimedia-fundraising*'
Co-Authored-By: Darragh Bailey <dbailey@hpe.com>
Co-Authored-By: Sorin Sbarnea <ssbarnea@redhat.com>
Change-Id: I897b9ed35561e455dc6b89c3bacec74b54777903
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Running a test with input yaml will result in output XML being sent
to the stdout. Use mock on the console to capture this so that running
the tests via testtools directly doesn't output unnecessarily
Change-Id: I4ccbfcb464d40a77c04913cb8281a0167504e1a4
Use a simple object to ensure matching against any default value is
possible to distinguish been not being set by a user versus use of
a negative default value passed.
This prevents triggering an incorrect notification to the user when
the code is querying for plugin options that could override the
default behaviour and the compatibility code attempts to look up an non
existing section. It would fail to distinguish between no section
present and current value to return matching the default value
suggested.
Change-Id: I5597c2628ccb5a4282a97a4ce5d3bbe41bd9eebb
Allows specifying a folder attribute for each job generated, which in
turn is used when creating or uploading to place the job under the
requested folder.
The job name is expanded after defaults are applied, to support the
attribute being defined within a set of defaults applied to a number of
jobs.
This in turn allows for multiple jobs with the same basename to exist,
provided they are targeted at different folders.
Does not support creating the folders if they do not exist.
Change-Id: I8c2157c4c81087cc972a048d1b88d5f08ac65361
Before, `jenkins-job-builder test -o outdir` would write
outdir/job_name
With the new option `--config-xml`, it writes
outdir/job_name/config.xml
to match the layout that Jenkins uses internally. This is more useful
for diffing against existing Jenkins installations or uploading job
definitions by other means.
Also, it seems better to give the output files an .xml extension.
Change-Id: I78fc7aff531925ee071b8e19f40f766b4af0b5ad
- Added failOnError argument for pre-scm-buildstep
- Introduced buildsteps argument and moved list of build steps to match xml configuration of the preSCMbuildstep plugin.
- Updated existing test cases to match the new argument structure
- Added test case for pre-scm-buildstep to verify creation of failOnError tag
- Added verification on version to maintain retro-compatibility
- Updated documentation to fetch example from testcase definition
Change-Id: I25c19cd0552cf2de97b9c391492d95779c1360be
Task: 3163
Story: 2000690
using python's "input" rather than "raw_input" raise exception
since the user's input is interpreted.
In python 3.x raw_input = input, so the interpertation bug
only exists in python 2.x
Change-Id: I28781a8742d6e03bc850fd0178f317474603a9d2
- Adds the ability for JJB to work with views
- Views can be created, updated, and deleted.
- New modules for List view and Build Pipeline view are added
- New tests for testing the deletion of views
Example View configuration:
- view:
name: MyView
view-type: list
Change-Id: Idb29a4407bcc14593e10a4d951036cb04e8e6c27
Co-Authored-By: Brandon Leonard <brandon.leonard@rackspace.com>
Co-Authored-By: Joao Vale <jpvale@gmail.com>
Co-Authored-By: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
The confirm() util incorrectly returns true only when the answer is
false. This patch reverses the logic so that it properly returns true
when the user inputs a true statement.
Include test case for delete-all command usage to accept or abort.
Mocking the input builtin ensures that the behaviour resulting from the
user inputting 'y' or 'n' will remain tested for delete-all.
Change-Id: I742804a8efe7da8ca6aa70f20059d02758910deb
Co-Authored-By: Darragh Bailey <dbailey@hpe.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Ensure that we explicitly check that there were no extra calls in
addition to the expected set of calls to the delete jobs API, when
populating keep_jobs with the list of jobs updated and checking that
the unmanaged job is ignored.
Correct the list of jobs returned by get_jobs call.
Prevent issue introduced by: https://review.openstack.org/352060
Change-Id: I5b7452d593707db7085e7d305893f05a77be82a1
Refactor base test class inheritance to allow for BaseTest class to be
reused without needing to also import the TestCase class everywhere in
order to avoid having the common test function executed on base classes.
This makes it easier to build base testing classes and then simplifies
the subclassing of these for actual tests.
Change-Id: I89809e8082469f814f245db4a9ab7658aac8a405
And rename it to JobCache. The word "Cache" already implies "Storage"
so it makes more sense to name it after the things that it
stores...Jobs!
Change-Id: If7a7a56e8f6a3637f62e285eaa7ba989f67002dc
Since both of these classes essentially build on basic functionality
provided by jenkins.Jenkins, merge them into a single class. Name this
class "JenkinsManager" to avoid confusion with jenkins.Jenkins.
Also moves some tests for update timeout configuration into the
tests.cmd.test_config module from tests.cmd.subcommand.test_update
module, replacing those tests with skipped stubs as reminders to
figure out a better way later.
Change-Id: I13c17bc90e29e702e5e02992e93cf3cdc689731d
Tests that if no encoding is defined on the output that JJB does not
accidentally trigger a recursion issue when iterating over a large
number of jobs and attempting to wrap the output stream.
Inspired by commit message from: https://review.openstack.org/283441
Change-Id: Ie69c3af0beb9fd7c589a436e8c6d2862210f8a4c
Co-Authored-By: Hao Liu <hliu@redhat.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
The deprecation warning message in this method was confusing and
seemed to be intended for builder.Jenkins.update_job but it's hard to
tell for sure. Seems likely to be an artifact of an outdated patchset
in the parallelize-the-things work done last year because it's not
clear how it could apply to builder.Builder.update_jobs which has a
significantly different use case than parallel_update_job.
Change-Id: Ifb3c1a40d81b0d43ac8e85151f0a99d1502e21db
Separate XML generation from Yaml parsing/interpreting. The goal here
is to allow different sources to provide data for XML generation,
including external API users writing job definitions in pure Python or
JJB developers who would like to work on alternative Yaml parsing code
since the current YamlParser has very likely reached the limits of
what can be reasonably done with one giant expandYaml procedure.
Change-Id: I9da848acac7e944c0e07286b7399b2e1956a58a5
Create the ModuleRegistry anywhere other than inside the YamlParser
class. This will make it slightly easier to factor a XmlGenerator out
of YamlParser, but I also want to work toward eliminating the circular
references between YamlParser and ModuleRegistry which have been
making it difficult to understand overall program flow.
This commit also replaces all YamlParser instances being passed to
Jenkins job config generating functions with a ModuleRegistry. Mostly
it seems like the parser was only needed to call the ModuleRegistry's
'dispatch' method which to be honest I don't fully understand. This is
where the circular references mentioned in previously come in...it
seems like the "dispatch" function needs access to the (mostly) raw
data contained by the parser, so it took that as a parameter.
The need for the YamlParser's job data can be satisfied by assigning
it to a property on the ModuleRegistry object before Yaml expansion or
XML generation begins; by doing this, we allow the ModuleRegistry to
avoid referencing the parser.
Change-Id: I4b571299b81e708540392ad963163fe092acf1d9
Sharing code between these two classes will make it easier to work on
extracting an XmlGenerator class from the YamlParser.
Change-Id: If93227d0b4fca4d7bb80de2341d1e83f1e3ddc79
This is the final piece of YamlParser in jenkins_jobs.builder, so also
remove the imports and make trivial fixes to test methods to prevent
borked test cases.
Change-Id: Ida543fa14c70447ae506092ec8afd5cb6caaa629
In a similer vein to the previous commit, when testing subcommands we
shouldn't do so in a way that unnecessarily entangles the JJB
API. (there ought to be separate tests for the API) By mocking
jenkins.Jenkins instead of jenkins.Builder this test wil be easier to
fix when rearranging the API.
Change-Id: Ibf48dd3749f33c47e5ad510b72eb3b6ed5332132
Make sure that flush cache is correct set first by config file, and
additionally that the CLI overrides any defaults.
In the V2 API work, when moving code around the flush cache option was
accidentally removed from being set by the command line or config
files. This ensures it follows the same behaviour as other similar
options.
Change-Id: I9fb1e234e5ed081ada64855389a87d2f7555469e
Previously this test was bound too tightly to implementation details
of jenkins_jobs.builder.Builder and jenkins_jobs.parser.YamlParser;
specifically, it was enforcing the relationship between these two
objects by accessing the Builder's instance of the YamlParser. In
order to proceed with the JJB 2.0 API refactoring, one of the goals of
which is to disentangle YamlParser from Builder, this had to be
rewritten.
The rewrite aims to instead test at the boundaries of the JJB
application --specifically by mocking out various methods in the
jenkins.Jenkins class upon which JJB is built then making a command
line call to jenkins-jobs using the jenkins_jobs.cli.entry.JenkinsJobs
tool.
The one remaining piece of questionable implementation detail binding
in this test is the mocking of jenkins_jobs.builder.Jenkins.is_managed
which I think is unavoidable without mocking return values on
jenkins.Jenkins.get_job_config, which would be tedius and probably
more effort than it's worth since deconstructing the relationship
between jenkins_jobs.builder.Builder and jenkins_jobs.builder.Jenkins
is not necessary for JJB 2.0 (actually, these two classes should
probably just be combined into one)
Change-Id: Ia0e07097e6596be2ac13140668480cefcec8334b
This commit sees JJBConfig start to take the form it ought to have,
namely using attributes to represent different logical sections of
configuration that target specific subsystems of JJB.
It also moves ConfigParser data retrieval from
jenkins_jobs.modules.helpers.get_value_from_yaml_or_config_file() to
jenkins_jobs.config.JJBConfig.get_module_config()
TODO: Add JJBConfig tests to validate the behavior of JJBConfig in
specific circumstances.
Change-Id: I053d165559f5325a2f40b239117a86e6d0f3ef37
* Move behavior of jenkins_jobs.cmd.execute() into
jenkins_jobs.cli.entry.JJBConfig.execute()
* Delete jenkins_jobs/cmd.py
* Fix up unit tests to deal with rearranged code.
Change-Id: Ia0d3a062de16c4be10863372c753f4ba8480e620
Ensure that argparser object only makes use of the JJBConfig object
instead of leaking a command line parser into a config object for API
usage.
Take care to use 'None' as default for store_true/store_false options
in order to ensure that the config file defaults are only overridden
when CLI options are explicitly set.
Change-Id: I4066ad750f9893759c2e9bdfde425fafacc7e672
* Create jenkins_jobs.config module with JJBConfig class.
* Move DEFAULT_CONF from jenkins_jobs.cmd into jenkins_jobs.config
* Move configuration initialization into JJBConfig
* Create method, "do_magical_things" to handle arbitration between
config file and arguments as well as setting default values for
config file settings if it doesn't contain the expected keys.
* Move JenkinsJobs.create_parser into its own module,
jenkins_jobs.cli.parser, it can be used to provide default settings
in the JJBConfig class when an argparse namespace object is not
provided; this is primarily necessary because most of the original
configuration initialization code relies on this being a namespace
object (simple descendant of the object class).
At this point JJBConfig isn't much more than an object-oriented
version of the way configuration handling happened previously. Its
current form, however, is more amenable to the ultimate goal of the
2.0 refactorings--namely, being able to pass a single config object
around rather than breaking it up into apparently arbitrary settings
necessary to instantiate the Builder class and its delegate objects.
Change-Id: Ic0147e1dccbe620aaaba039a434e7cea6c670054
As far back as its introduction (2cefa400), the `--delete-old` option
has only ever deleted jobs that were marked with the special
"<!-- Managed by Jenkins Job Builder -->" comment that JJB adds to
descriptions.
This change fixes the documentation of the `--delete-old` option,
which erroneously used to state that even jobs that were never
managed by JJB would be subject to deletion.
The test for the feature was slightly enriched as well: the fact
that unmanaged jobs were not deleted, was not actually tested.
Change-Id: I438a7b555b6e122869988b3a2b9ea40896004122
This commit intentionally introduces a number of important API
breakages. Specifically, the jenkins_jobs.cmd module has been pared
down to some of its most difficult-to-refactor elements.
* Create jenkins_jobs.cli.entry.JenkinsJobs class to organize command
line parsing and execution.
* Remove references to ConfigParser object in test code, hidden as an
implementation detail of JenkinsJobs command line parsing. This will
be necessary in the next stage of JJB 2.0 code which will be to
create a JJBConfig object that handles logic and presentation of
configuration from various sources--defaults, command line
arguments, configuration file, and maybe environment variables in
the future.
* Remove references to Namespace object produced by argparse module.
Required rewrite of multipath & recursive path tests with a new
MatchesDir testtools Matcher class that validates the expected
output for a run of JJB against a given set of yamldirs with the
specified command line arguments.
* Use stevedore to dynamically load subcommand parsers.
* Move configuration loading/testing to its own test file. Also fix
the global vs home directory JJB config file test.
Change-Id: If62280418ba7319c313033ab387af4284237747e
The system-global path of
/etc/jenkins_jobs/jenkins_jobs.ini
is not applicable on Windows. To solve that, and as Linux users would
benefit from a per-user configuration file, too, introduce a fallback at
~/.config/jenkins_jobs/jenkins_jobs.ini
following the XDG Base Directory Specification. Try to use that per-user
file if no configuration file is given on the command line and none is
present in the script directory for development purposes.
Change-Id: I1a22454999ee8ac60d6f22545850c33aa03bb29b
Make sure to perform wrap external to loop, as otherwise this will
recursively wrap the output object and potentially result in getattr
lookups exceeding the recursion limit in python.
Change-Id: Ie9318a3d23a721e62c16e7b05f52d5b9bec1bb82
* Only update_jobs uses the parallel features right now
* --workers N
- If set to 0, it will use parallel execution and
use the number of cores in the machine as thread count
- Any other value enables the parallel extensions and sets the
number of concurrent threads to that value
- Will use 1 by default
Add some tests to make sure the parallel execution helpers work as
expected.
Change-Id: Ib0abd34ea7525f75fff4ff480287a6e589deba90
Signed-off-by: David Caro <dcaroest@redhat.com>
Change Ie952617a34c0719e forced utf-8 format but forgot to decode the
xml when passing to update_job. This broke jjb for python 3:
File "/Users/khaido/PycharmProjects/jenkins-job-builder
/jenkins_jobs/builder.py", line
134, in update_job
self.jenkins.create_job(job_name, xml)
File "/Users/khaido/PycharmProjects/jenkins-job-builder/.tox
/py34/lib/python3.4/site-packages/jenkins/__init__.py",
line 852, in create_job
config_xml.encode('utf-8'), DEFAULT_HEADERS))
AttributeError: 'function' object has no attribute 'encode'
Change-Id: Iceda46214bf4709ccd8141ef654cf3ec81e8af06