Rewrite YAML parser, YAML objects and parameters expansion logic to
enable better control over expansion logic.
Broken backward compatilibity:
* More agressive parameter expansion. This may lead to parameters
expanded in places where they were not expanded before.
* Top-level elements, which is not known to parser (such as 'job',
'view', 'project' etc), are now lead to parse failures.
Prepend them with underscore to be ignored by parser.
* Files included using '!include-raw:' elements and having formatting in
it's path ('lazy-loaded' in previous implementation) are now expanded
too.
Use '!include-raw-escape:' for them instead.
See changes in these tests for examples:
tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml
tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml
tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml
* Parameters with template value using itself were substituted as is.
For example: "timer: '{timer}'" was expanded to "{timer}". Now it
leads to recursive parameter error.
See changes in this test for example:
tests/yamlparser/job_fixtures/parameter_name_reuse_default.*
->
tests/yamlparser/error_fixtures/parameter_name_reuse_default.*
* When job group includes a job which was never declared, it was just
ignored. Now it fails: job is missing.
See changes in this test for example:
tests/yamlparser/job_fixtures/job_group_includes_missing_job.*
->
tests/yamlparser/error_fixtures/job_group_includes_missing_job.*
Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
Rename fixtures directory to job_fixtures to prepare for adding view_fixtures
directory in the following commits.
Change-Id: Ic20997cae020b542ddc22bf444fa6b92fbcae064
The heading level of 'Recursive Searching of Paths' and 'Excluding Paths' should be below 'Running'.
Change-Id: I9069b2c205e0dfc945855ea5d7bff02149e1cd51
Sphinx was pinned to >1.5.0 <=1.7.0 4 years ago. It is now on v4.4.0,
and we're seeing an incompatibility with the latest version of jinja2.
Along with updating the Sphinx version, this also includes fixes for
several warnings generated in the latest version:
* Outdated docs due to the removal of the "parser" param from
Base.get_xml
* Class lookup issues
* External links for which we have extlinks defined
* Deprated PyModulelevel is replaced by PyFunction
* sphinx.ext.autodoc.Documenter.get_doc signature changed
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Change-Id: Iba65fb724f098f3e4a0b20f2a183cb2ea3c450ea
Setting desired version of black to 22.3.0 prevents issues with
the click dependency described in the #1966879.
Introduction of black 22.3.0 forced style changes in following files:
- doc/source/conf.py
- jenkins_jobs/local_yaml.py
- tests/base.py
These changes do not modify behavior of the programs concerned.
Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I42a46f9ed31ea0f404d698eec73d30975e2e6f4c
The links pointing to full example and anchor & aliases on
yaml website are no longer valid, causing failures for
docs-linkcheck. This change updates the links in the doc.
curl https://yaml.org/spec/1.2/#id2761803
<meta http-equiv="refresh" content="0; url=/spec/1.2.2/" />
curl https://yaml.org/spec/1.2/#id2765878
<meta http-equiv="refresh" content="0; url=/spec/1.2.2/" />
Change-Id: I1d57e9b414694e2dd8e0dca24ef2e6ea96cb35a0
Jenkins Wiki has been fully taken off line, so update the remaining
links to reference either the relevant plugin page or the github repo.
Add extlink target for repo in jenkinsci github org.
The sonatype-clm plugin seems to be more-or-less completely deprecated,
so update the description to indicate this and link to the
nexus-artifact-uploader plugin.
Update the jjb sphinx plugin so that it generates references for the
yamlfunctions.
Change-Id: If2241e751d01a60a8cb4cbaea5b3176aeb92eab4
Signed-off-by: Pat Long <pllong@arista.com>
It looks like the YAML specifications are no longer available in:
yaml.org/spec/1.2/spec.html
but in:
yaml.org/spec/1.2/
See:
$ curl https://yaml.org/spec/1.2/spec.html
<meta http-equiv="refresh" content="0; url=https://yaml.org/spec/1.2/" />
Update these links to avoid errors reported by jjb-tox-docs-linkcheck.
Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
Change-Id: Ie928adf2e4c321e900bfbf36c19ae34a3dc7311d
This adds a job type "githuborg", which creates multibranch pipelines
for all repos in the org that contain the specified Jenkinsfile.
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Change-Id: I148e2300f3b1ee148b51d25a2a805ed0b5a97862
Commit 30d61a21e8 changed the default but
missed updating the documentation to mention that it is False by
default. This update resolves the documentation mismatch.
Change-Id: Ibe2c31d7e9ca0380efe0e6ef5014c05f1c42eae8
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Some links have been permanently redirected to a new URL so
adjust the links to go directly to the new URL.
Change-Id: I900091f8bdc39aa5fd06eae04a9b18ef23add8f2
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
The Jenkins Wiki page is deprecated and Jenkins community is expected
to migrate their documentation to plugins.jenkins.io URL. This patch
updates all publishers that have a relevant plugins.jenkins.io URL
documentation link.
Change-Id: I94448d04f2b28fbf3befde2e49c76d196e352d7e
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
This toxenv will allow us to flag URLs that no longer work. We
also switch from using old style setup.py build_sphinx to the
newer sphinx-build command.
Change-Id: I62f42918814fb6eea4876e0d22c6bccddf86a826
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Please review the following patch containing the code changes in
the repo. This patch is a transition patch and is the auto-generated
output of the python-black tool.
Change-Id: I2d2de71da8a105fb62b561899ae78441ddab4032
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
My previous submission added support for property strategy: all branches
get the same properties. This patch adds support for the second of 2
options, "named branches get different properties", which allows for
a default definition of properties + exceptions for individual refspecs
based on their literal branch name. Also includes some re-org of the
existing prop. strat. support in order to DRY up the code for both.
Also adds sphinx.ext.doctest to docs/src/conf.py extensions to allow for
running `make doctest` locally.
Change-Id: Icd143fe25b1e2d5c8d1e7e8b0650d91f40838043
Signed-off-by: sbussetti <steve.bussetti@gmail.com>
Jenkins2 GitHub scms default to automatically sending updates on build
status to github for pull requests. The plugin this enables allows
control of this by enabling or disabling the feature via an SCM trait,
allowing you to either no notify at all or manually notify via other
means.
Change-Id: I4f5629c3ee8031d41054e3be5e853201d8f0fed6
Signed-off-by: sbussetti <steve.bussetti@gmail.com>
Add options --jobs-only and --views-only to job_builder
section in config file.
By default JJB updates both jobs and views. Some cases
non-admin users are not allowed to update views on Jenkins
which requires explicity using -j flag.
Allow users to set a config option 'update=jobs|views|all'
in the 'job_builder' section implicitly use --jobs-only or
--views-only and control these flags from the config file.
Note: CLI options takes precedence over the config file.
Change-Id: I2a94e5a2d671ccbfc505de2f19b578ecfef9e9d7
Co-Authored-By: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Anil Belur <askb23@gmail.com>
Signed-off-by: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Anil Belur <askb23@gmail.com>
Took inspiration from my StackOverflow reply here
https://stackoverflow.com/questions/50177308 to document a practical
use of setting variable defaults and inheritenence.
Change-Id: I45934341d3ff3fbf4ff711a435924197d2f12712
Signed-off-by: Thanh Ha <zxiiro@linux.com>
Preparatory move for adopting pre-commit hooks.
Shoudl only remove newlines and spaces at end of files.
Change-Id: Ia33679467c533722b9308f7e7b74be395c926428
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
If set to True, YAML anchors can be referenced across files, allowing jobs to be
composed from bits of YAML defined in separate files. False by default.
Story: 2000338
Task: 2547
Change-Id: I034ce3bce0030093cb8d4266dabbdb06d96306d6
Allow views to also be configured via templates similar to
job-templates.
This adds a new project key called "views" and parser type called
"view-template" allowing the user to add custom views.
Example:
- view-template:
name: '{name}-template-{seq}'
description: 'testing view templates feature'
view-type: list
regex: 'test-view-.*'
- project:
name: 'test-view'
views:
- '{name}-template-{seq}'
seq:
- a
- b
- c
Change-Id: I6eb225f24bc3c7d790c7dcab6391735c579ac71a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
This work was based on original work done by
Joshua Harlow on top of older code from:
https://github.com/abnamrocoesd/jenkins-job-builder/
Credit is due to `Joost van der Griendt` for doing this
work. This builds on his work in that repo, and adjusts it
so that it can get merged into upstream and released
as a fully supported job type.
SCM implementation is different than than normal SCM module,
supporting: BitBucket, git, and GitHub in this initial
patch.
Change-Id: If50a54d282dd7d901c16edb9fe04874bdd83c9ef
Co-Authored-By: Joshua Harlow <jxharlow@godaddy.com>
Co-Authored-By: Sorin Sbarnea <ssbarnea@redhat.com>
Co-Authored-By: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Signed-off-by: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Add print_job_urls = True to [job_builder] to enable output as urls
instead of simple job names when changing them.
Change-Id: I3454606a50ca385c87d8c13d3eab5d30d94cf02f
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
JJB 2.0 update does not allow updating only views
or jobs which would be useful, which is implemented with
delete and delete-all. This patch add's thoses flags
for the update option.
Change-Id: I3712c8552ea83a4c29c05461894c4464aa41230f
Signed-off-by: Anil Belur <askb23@gmail.com>
View documentation was not properly being generated and linked
in the index. Let's add that in.
Change-Id: I6197dbecc89ddcf8c6f6b0e6a3d4c517224dc77b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Inject the template-name into parameters earlier, so it's possible to
use it in the global defaults. Mark the template name as 'verbatim' so
the interpolation is skipped by the formatter.
Change-Id: I5d98e32ed2bedaa6b536e61fbdec589f358f861f
Vsevolod Fedorov