Merge "Add a JJB quick start guide"

This commit is contained in:
Jenkins 2015-08-12 21:45:17 +00:00 committed by Gerrit Code Review
commit 19aed6592f
4 changed files with 353 additions and 258 deletions

265
doc/source/execution.rst Normal file
View File

@ -0,0 +1,265 @@
Configuration File
------------------
After installation, you will need to create a configuration file. By
default, ``jenkins-jobs`` looks in
``/etc/jenkins_jobs/jenkins_jobs.ini`` but you may specify an
alternative location when running ``jenkins-jobs``. The file should have
the following format:
.. literalinclude:: ../../etc/jenkins_jobs.ini-sample
:language: ini
job_builder section
^^^^^^^^^^^^^^^^^^^
**ignore_cache**
(Optional) If set to True, Jenkins Job Builder won't use any cache.
**keep_descriptions**
By default `jenkins-jobs` will overwrite the jobs descriptions even if no
description has been defined explicitly.
When this option is set to True, that behavior changes and it will only
overwrite the description if you specified it in the yaml. False by default.
**include_path**
(Optional) Can be set to a ':' delimited list of paths, which jenkins
job builder will search for any files specified by the custom application
yaml tags 'include', 'include-raw' and 'include-raw-escaped'.
**recursive**
(Optional) If set to True, jenkins job builder will search for job
definition files recursively.
**exclude**
(Optional) If set to a list of values separated by ':', these paths will be
excluded from the list of paths to be processed when searching recursively.
Values containing no ``/`` will be matched against directory names at all
levels, those starting with ``/`` will be considered absolute, while others
containing a ``/`` somewhere other than the start of the value will be
considered relative to the starting path.
**allow_duplicates**
(Optional) By default `jenkins-jobs` will abort when a duplicate macro,
template, job-group or job name is encountered as it cannot establish the
correct one to use. When this option is set to True, only a warning is
emitted.
**allow_empty_variables**
(Optional) When expanding strings, by default `jenkins-jobs` will raise an
exception if there's a key in the string, that has not been declared in the
input YAML files. Setting this option to True will replace it with the empty
string, allowing you to use those strings without having to define all the
keys it might be using.
jenkins section
^^^^^^^^^^^^^^^
**user**
This should be the name of a user previously defined in Jenkins.
Appropriate user permissions must be set under the Jenkins security
matrix: under the ``Global`` group of permissions, check ``Read``,
then under the ``Job`` group of permissions, check ``Create``,
``Delete``, ``Configure`` and finally ``Read``.
**password**
The API token for the user specified. You can get this through the
Jenkins management interface under ``People`` -> username ->
``Configure`` and then click the ``Show API Token`` button.
**url**
The base URL for your Jenkins installation.
**query_plugins_info**
Whether to query the Jenkins instance for plugin info when a configuration
file is provided. If a configuration file is not provided `jenkins-jobs` will
ignore this setting and skip querying for plugin information. True by
default.
hipchat section
^^^^^^^^^^^^^^^
**send-as**
This is the hipchat user name that will be used when sending notifications.
**authtoken**
The API token necessary to send messages to hipchat. This can be generated in
the hipchat web interface by a user with administrative access for your
organization. This authtoken is set for each job individually; the
JJB Hipchat Plugin does not currently support setting different tokens for
different projects, so the token you use will have to be scoped such that it
can be used for any room your jobs might be configured to notify. For more
information on this topic, please see the `Hipchat API Documentation`__
__ https://www.hipchat.com/docs/apiv2/auth
Running
-------
After it's installed and configured, you can invoke Jenkins Job
Builder by running ``jenkins-jobs``. You won't be able to do
anything useful just yet without a configuration; that is
discussed in the next section.
Test Mode
^^^^^^^^^
Once you have a configuration defined, you can run the job builder in test mode.
If you want to run a simple test with just a single YAML job definition file
and see the XML output on stdout::
jenkins-jobs test /path/to/foo.yaml
You can also pass JJB a directory containing multiple job definition files::
jenkins-jobs test /path/to/defs -o /path/to/output
which will write XML files to the output directory for all of the jobs
defined in the defs directory.
.. _updating-jobs:
Updating Jobs
^^^^^^^^^^^^^
When you're satisfied with the generated XML from the test, you can run::
jenkins-jobs update /path/to/defs
which will upload the job definitions to Jenkins if needed. Jenkins Job
Builder maintains, for each host, a cache [#f1]_ of previously configured jobs,
so that you can run that command as often as you like, and it will only
update the jobs configurations in Jenkins if the defined definitions has
changed since the last time it was run. Note: if you modify a job
directly in Jenkins, jenkins-jobs will not know about it and will not
update it.
To update a specific list of jobs, simply pass the job names as additional
arguments after the job definition path. To update Foo1 and Foo2 run::
jenkins-jobs update /path/to/defs Foo1 Foo2
Passing Multiple Paths
^^^^^^^^^^^^^^^^^^^^^^
It is possible to pass multiple paths to JJB using colons as a path separator on
\*nix systems and semi-colons on Windows systems. For example::
jenkins-jobs test /path/to/global:/path/to/instance:/path/to/instance/project
This helps when structuring directory layouts as you may selectively include
directories in different ways to suit different needs. If you maintain multiple
Jenkins instances suited to various needs you may want to share configuration
between those instances (global). Furthermore, there may be various ways you
would like to structure jobs within a given instance.
Recursive Searching of Paths
----------------------------
In addition to passing multiple paths to JJB it is also possible to enable
recursive searching to process all yaml files in the tree beneath each path.
For example::
For a tree:
/path/
to/
defs/
ci_jobs/
release_jobs/
globals/
macros/
templates/
jenkins-jobs update -r /path/to/defs:/path/to/globals
JJB will search defs/ci_jobs, defs/release_jobs, globals/macros and
globals/templates in addition to the defs and globals trees.
Excluding Paths
---------------
To allow a complex tree of jobs where some jobs are managed differently without
needing to explicitly provide each path, the recursive path processing supports
excluding paths based on absolute paths, relative paths and patterns. For
example::
For a tree:
/path/
to/
defs/
ci_jobs/
manual/
release_jobs/
manual/
qa_jobs/
globals/
macros/
templates/
special/
jenkins-jobs update -r -x man*:./qa_jobs -x /path/to/defs/globals/special \
/path/to/defs:/path/to/globals
JJB will search the given paths, ignoring the directories qa_jobs,
ci_jobs/manual, release_jobs/manual, and globals/special when
building the list of yaml files to be processed. Absolute paths
are denoted by starting from the root, relative by containing
the path separator, and patterns by having neither.
Patterns use simple shell globing to match directories.
Deleting Jobs
^^^^^^^^^^^^^
Jenkins Job Builder supports deleting jobs from Jenkins.
To delete a specific job::
jenkins-jobs delete Foo1
To delete a list of jobs, simply pass them as additional
arguments after the command::
jenkins-jobs delete Foo1 Foo2
The ``update`` command includes a ``delete-old`` option to remove obsolete
jobs::
jenkins-jobs update --delete-old /path/to/defs
Obsolete jobs are *all* jobs not managed by JJB, even jobs which
were *never* managed by JJB.
There is also a command to delete **all** jobs.
**WARNING**: Use with caution::
jenkins-jobs delete-all
Globbed Parameters
^^^^^^^^^^^^^^^^^^
Jenkins job builder supports globbed parameters to identify jobs from a set of
definition files. This feature only supports JJB managed jobs.
To update jobs that only have 'foo' in their name::
jenkins-jobs update ./myjobs \*foo\*
To delete jobs that only have 'foo' in their name::
jenkins-jobs delete --path ./myjobs \*foo\*
.. _command-reference:
Command Reference
^^^^^^^^^^^^^^^^^
.. program-output:: jenkins-jobs --help
.. program-output:: jenkins-jobs test --help
.. program-output:: jenkins-jobs update --help
.. program-output:: jenkins-jobs delete-all --help
.. program-output:: jenkins-jobs delete --help
.. rubric:: Footnotes
.. [#f1] The cache default location is at ``~/.cache/jenkins_jobs``, which
can be overridden by setting the ``XDG_CACHE_HOME`` environment
variable.

View File

@ -8,7 +8,9 @@ Contents
.. toctree:: .. toctree::
:maxdepth: 3 :maxdepth: 3
quick-start
installation installation
execution
definition definition
extending extending

View File

@ -40,261 +40,3 @@ Test Coverage
To measure test coverage, execute the command:: To measure test coverage, execute the command::
tox -e cover tox -e cover
Configuration File
------------------
After installation, you will need to create a configuration file. By
default, ``jenkins-jobs`` looks in
``/etc/jenkins_jobs/jenkins_jobs.ini`` but you may specify an
alternative location when running ``jenkins-jobs``. The file should have
the following format:
.. literalinclude:: ../../etc/jenkins_jobs.ini-sample
:language: ini
job_builder section
^^^^^^^^^^^^^^^^^^^
**ignore_cache**
(Optional) If set to True, Jenkins Job Builder won't use any cache.
**keep_descriptions**
By default `jenkins-jobs` will overwrite the jobs descriptions even if no
description has been defined explicitly.
When this option is set to True, that behavior changes and it will only
overwrite the description if you specified it in the yaml. False by default.
**include_path**
(Optional) Can be set to a ':' delimited list of paths, which jenkins
job builder will search for any files specified by the custom application
yaml tags 'include', 'include-raw' and 'include-raw-escaped'.
**recursive**
(Optional) If set to True, jenkins job builder will search for job
definition files recursively
**exclude**
(Optional) If set to a list of values separated by ':', these paths will be
excluded from the list of paths to be processed when searching recursively.
Values containing no ``/`` will be matched against directory names at all
levels, those starting with ``/`` will be considered absolute, while others
containing a ``/`` somewhere other than the start of the value will be
considered relative to the starting path.
**allow_duplicates**
(Optional) By default `jenkins-jobs` will abort any time a duplicate macro,
template, job-group or job name is encountered as it cannot establish the
correct one to use. When this option is set to True, only a warning is
emitted.
**allow_empty_variables**
(Optional) When expanding strings, by default `jenkins-jobs` will raise an
exception if there's a key in the string, that has not been declared on the
yamls. Setting this options to True, will replace it with the empty string,
allowing you to use those strings without having to define all the keys it
might be using.
jenkins section
^^^^^^^^^^^^^^^
**user**
This should be the name of a user previously defined in Jenkins.
Appropriate user permissions must be set under the Jenkins security
matrix: under the ``Global`` group of permissions, check ``Read``,
then under the ``Job`` group of permissions, check ``Create``,
``Delete``, ``Configure`` and finally ``Read``.
**password**
The API token for the user specified. You can get this through the
Jenkins management interface under ``People`` -> username ->
``Configure`` and then click the ``Show API Token`` button.
**url**
The base URL for your Jenkins installation.
**query_plugins_info**
Whether to query the Jenkins instance for plugin info when a configuration
file is provided. If a configuration file is not provided `jenkins-jobs` will
ignore this setting and skip querying for plugin information. True by
default.
hipchat section
^^^^^^^^^^^^^^^
**send-as**
This is the hipchat user name that will be used when sending notifications.
**authtoken**
The API token necessary to send messages to hipchat. This can be generated in
the hipchat web interface by a user with administrative access for your
organization. This is the authtoken is set for each job individually; the
JJB Hipchat Plugin does not currently support setting different tokens for
different projects, so the token you use will have to be scoped such that it
can be used for any room your jobs might be configured to notify. For more
information on this topic, please see the `Hipchat API Documentation`__
__ https://www.hipchat.com/docs/apiv2/auth
Running
-------
After it's installed and configured, you can invoke Jenkins Job
Builder by running ``jenkins-jobs``. You won't be able to do anything
useful just yet without a configuration which is discussed in the next
section.
Test Mode
^^^^^^^^^
Once you have a configuration defined, you can run the job builder in test mode.
If you want to run a simple test with just a single YAML job definition file
and see the XML output on stdout::
jenkins-jobs test /path/to/foo.yaml
You can also pass JJB a directory containing multiple job definition files::
jenkins-jobs test /path/to/defs -o /path/to/output
which will write XML files to the output directory for all of the jobs
defined in the defs directory.
Updating Jobs
^^^^^^^^^^^^^
When you're satisfied with the generated XML from the test, you can run::
jenkins-jobs update /path/to/defs
which will upload the job definitions to Jenkins if needed. Jenkins Job
Builder maintains, for each host, a cache [#f1]_ of previously configured jobs,
so that you can run that command as often as you like, and it will only
update the jobs configurations in Jenkins if the defined definitions has
changed since the last time it was run. Note: if you modify a job
directly in Jenkins, jenkins-jobs will not know about it and will not
update it.
To update a specific list of jobs, simply pass them as additional
arguments after the job definition path. To update Foo1 and Foo2 run::
jenkins-jobs update /path/to/defs Foo1 Foo2
Passing Multiple Paths
^^^^^^^^^^^^^^^^^^^^^^
It is possible to pass multiple paths to JJB using colons as a path separator on
\*nix systems and semi-colons on Windows systems. For example::
jenkins-jobs test /path/to/global:/path/to/instance:/path/to/instance/project
This helps when structuring directory layouts as you may selectively include
directories in different ways to suit different needs. If you maintain multiple
Jenkins instances suited to various needs you may want to share configuration
between those instances (global). Furthermore, there may be various ways you
would like to structure jobs within a given instance.
Recursive Searching of Paths
----------------------------
In addition to passing multiple paths to JJB it is also possible to enable
recursive searching to process all yaml files in the tree beneath each path.
For example::
For a tree:
/path/
to/
defs/
ci_jobs/
release_jobs/
globals/
macros/
templates/
jenkins-jobs update -r /path/to/defs:/path/to/globals
JJB will search defs/ci_jobs, defs/release_jobs, globals/macros and
globals/templates in addition to the defs and globals trees.
Excluding Paths
---------------
To allow a complex tree of jobs where some jobs are managed differently without
needing to explicitly provide each path, the recursive path processing supports
excluding paths based on absolute paths, relative paths and patterns. For
example::
For a tree:
/path/
to/
defs/
ci_jobs/
manual/
release_jobs/
manual/
qa_jobs/
globals/
macros/
templates/
special/
jenkins-jobs update -r -x man*:./qa_jobs -x /path/to/defs/globals/special \
/path/to/defs:/path/to/globals
JJB search the given paths, ignoring the directories qa_jobs, ci_jobs/manual,
release_jobs/manual, and globals/special when building the list of yaml files
to be processed. Absolute paths are denoted by starting from the root,
relative by containing the path separator, and patterns by having neither.
Patterns use simple shell globing to match directories.
Deleting Jobs
^^^^^^^^^^^^^
Jenkins Job Builder supports deleting jobs from Jenkins.
To delete a specific job::
jenkins-jobs delete Foo1
To delete a list of jobs, simply pass them as additional
arguments after the command::
jenkins-jobs delete Foo1 Foo2
The ``update`` command includes a ``delete-old`` option to remove obsolete
jobs. Obsolete jobs are *all* jobs not managed by JJB, even jobs which
were *never* managed by JJB::
jenkins-jobs update --delete-old /path/to/defs
There is also a command to delete **all** jobs.
**WARNING**: Use with caution::
jenkins-jobs delete-all
Globbed Parameters
^^^^^^^^^^^^^^^^^^
Jenkins job builder supports globbed parameters to identify jobs from a set of
definition files. This feature only supports JJB managed jobs.
To update jobs that only have 'foo' in their name::
jenkins-jobs update ./myjobs \*foo\*
To delete jobs that only have 'foo' in their name::
jenkins-jobs delete --path ./myjobs \*foo\*
Command Reference
^^^^^^^^^^^^^^^^^
.. program-output:: jenkins-jobs --help
.. program-output:: jenkins-jobs test --help
.. program-output:: jenkins-jobs update --help
.. program-output:: jenkins-jobs delete-all --help
.. program-output:: jenkins-jobs delete --help
.. rubric:: Footnotes
.. [#f1] The cache default location is at ``~/.cache/jenkins_jobs``, which
can be overridden by setting the ``XDG_CACHE_HOME`` environment
variable.

View File

@ -0,0 +1,86 @@
.. _quick-start-guide:
Quick Start Guide
=================
This guide was made with the impatient in mind so explanation is sparse.
It will guide users through a set of typical use cases for JJB using the same
job definitions we use to test JJB.
#. Clone the repository_ to get the JJB job definition examples_
#. The :doc:`installation` can be either from pypi_ (released version) or from the clone (master).
Usage of the commands below assumes that you are at the root of the cloned directory.
.. _repository: http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/
.. _pypi: https://pypi.python.org/pypi/jenkins-job-builder/
.. _examples: http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/tests
.. _use-case-1:
Use Case 1: Test a job definition
---------------------------------
JJB creates Jenkins XML configuration file from a YAML/JSON definition file and
just uploads it to Jenkins. JJB provides a convenient ``test`` command to allow
you to validate the XML before you attempt to upload it to Jenkins.
Test a YAML job definition::
jenkins-jobs test tests/yamlparser/fixtures/templates002.yaml
The above command prints the generated Jenkins XML to the console. If you
prefer to send it to a directory::
jenkins-jobs test -o output tests/yamlparser/fixtures/templates002.yaml
The `output` directory will contain files with the XML configurations.
.. _use-case-2:
Use Case 2: Updating Jenkins Jobs
---------------------------------
Once you've tested your job definition and are happy with it then you can use the
``update`` command to deploy the job to Jenkins. The ``update`` command requires a
configuration file. An example file is supplied in the etc folder, you should
update it to match your Jenkins master::
jenkins-jobs --conf etc/jenkins_jobs.ini-sample update tests/yamlparser/fixtures/templates002.yaml
The above command will update your Jenkins master with the generated jobs.
**Caution**: JJB caches Jenkins job information locally. Changes
made using the Jenkins UI will not update that cache, which may
lead to confusion. See :ref:`updating-jobs` for more information.
.. _use-case-3:
Use Case 3: Working with JSON job definitions
---------------------------------------------
You can also define your jobs in json instead of yaml::
jenkins-jobs --config etc/jenkins_jobs.ini-sample update tests/jsonparser/fixtures/simple.json
The above command just uses a simple job definition. You can also convert any
of the YAML examples to JSON and feed that to JJB.
.. _use-case-4:
Use Case 4: Deleting a job
--------------------------
To delete a job::
jenkins-jobs --config etc/jenkins_jobs.ini-sample delete simple
The above command deletes the job `simple` from the Jenkins master.
Please refer to the jenkins-jobs :ref:`command-reference` and the
:doc:`definition` pages for more details.