diff --git a/doc/source/execution.rst b/doc/source/execution.rst new file mode 100644 index 000000000..034bfad45 --- /dev/null +++ b/doc/source/execution.rst @@ -0,0 +1,265 @@ +Configuration File +------------------ + +After installation, you will need to create a configuration file. By +default, ``jenkins-jobs`` looks in +``/etc/jenkins_jobs/jenkins_jobs.ini`` but you may specify an +alternative location when running ``jenkins-jobs``. The file should have +the following format: + +.. literalinclude:: ../../etc/jenkins_jobs.ini-sample + :language: ini + +job_builder section +^^^^^^^^^^^^^^^^^^^ + +**ignore_cache** + (Optional) If set to True, Jenkins Job Builder won't use any cache. + +**keep_descriptions** + By default `jenkins-jobs` will overwrite the jobs descriptions even if no + description has been defined explicitly. + When this option is set to True, that behavior changes and it will only + overwrite the description if you specified it in the yaml. False by default. + +**include_path** + (Optional) Can be set to a ':' delimited list of paths, which jenkins + job builder will search for any files specified by the custom application + yaml tags 'include', 'include-raw' and 'include-raw-escaped'. + +**recursive** + (Optional) If set to True, jenkins job builder will search for job + definition files recursively. + +**exclude** + (Optional) If set to a list of values separated by ':', these paths will be + excluded from the list of paths to be processed when searching recursively. + Values containing no ``/`` will be matched against directory names at all + levels, those starting with ``/`` will be considered absolute, while others + containing a ``/`` somewhere other than the start of the value will be + considered relative to the starting path. + +**allow_duplicates** + (Optional) By default `jenkins-jobs` will abort when a duplicate macro, + template, job-group or job name is encountered as it cannot establish the + correct one to use. When this option is set to True, only a warning is + emitted. + +**allow_empty_variables** + (Optional) When expanding strings, by default `jenkins-jobs` will raise an + exception if there's a key in the string, that has not been declared in the + input YAML files. Setting this option to True will replace it with the empty + string, allowing you to use those strings without having to define all the + keys it might be using. + + +jenkins section +^^^^^^^^^^^^^^^ + +**user** + This should be the name of a user previously defined in Jenkins. + Appropriate user permissions must be set under the Jenkins security + matrix: under the ``Global`` group of permissions, check ``Read``, + then under the ``Job`` group of permissions, check ``Create``, + ``Delete``, ``Configure`` and finally ``Read``. + +**password** + The API token for the user specified. You can get this through the + Jenkins management interface under ``People`` -> username -> + ``Configure`` and then click the ``Show API Token`` button. + +**url** + The base URL for your Jenkins installation. + +**query_plugins_info** + Whether to query the Jenkins instance for plugin info when a configuration + file is provided. If a configuration file is not provided `jenkins-jobs` will + ignore this setting and skip querying for plugin information. True by + default. + + +hipchat section +^^^^^^^^^^^^^^^ + +**send-as** + This is the hipchat user name that will be used when sending notifications. + +**authtoken** + The API token necessary to send messages to hipchat. This can be generated in + the hipchat web interface by a user with administrative access for your + organization. This authtoken is set for each job individually; the + JJB Hipchat Plugin does not currently support setting different tokens for + different projects, so the token you use will have to be scoped such that it + can be used for any room your jobs might be configured to notify. For more + information on this topic, please see the `Hipchat API Documentation`__ + +__ https://www.hipchat.com/docs/apiv2/auth + + + +Running +------- + +After it's installed and configured, you can invoke Jenkins Job +Builder by running ``jenkins-jobs``. You won't be able to do +anything useful just yet without a configuration; that is +discussed in the next section. + +Test Mode +^^^^^^^^^ +Once you have a configuration defined, you can run the job builder in test mode. + +If you want to run a simple test with just a single YAML job definition file +and see the XML output on stdout:: + + jenkins-jobs test /path/to/foo.yaml + +You can also pass JJB a directory containing multiple job definition files:: + + jenkins-jobs test /path/to/defs -o /path/to/output + +which will write XML files to the output directory for all of the jobs +defined in the defs directory. + +.. _updating-jobs: + +Updating Jobs +^^^^^^^^^^^^^ +When you're satisfied with the generated XML from the test, you can run:: + + jenkins-jobs update /path/to/defs + +which will upload the job definitions to Jenkins if needed. Jenkins Job +Builder maintains, for each host, a cache [#f1]_ of previously configured jobs, +so that you can run that command as often as you like, and it will only +update the jobs configurations in Jenkins if the defined definitions has +changed since the last time it was run. Note: if you modify a job +directly in Jenkins, jenkins-jobs will not know about it and will not +update it. + +To update a specific list of jobs, simply pass the job names as additional +arguments after the job definition path. To update Foo1 and Foo2 run:: + + jenkins-jobs update /path/to/defs Foo1 Foo2 + +Passing Multiple Paths +^^^^^^^^^^^^^^^^^^^^^^ +It is possible to pass multiple paths to JJB using colons as a path separator on +\*nix systems and semi-colons on Windows systems. For example:: + + jenkins-jobs test /path/to/global:/path/to/instance:/path/to/instance/project + +This helps when structuring directory layouts as you may selectively include +directories in different ways to suit different needs. If you maintain multiple +Jenkins instances suited to various needs you may want to share configuration +between those instances (global). Furthermore, there may be various ways you +would like to structure jobs within a given instance. + +Recursive Searching of Paths +---------------------------- + +In addition to passing multiple paths to JJB it is also possible to enable +recursive searching to process all yaml files in the tree beneath each path. +For example:: + + For a tree: + /path/ + to/ + defs/ + ci_jobs/ + release_jobs/ + globals/ + macros/ + templates/ + + jenkins-jobs update -r /path/to/defs:/path/to/globals + +JJB will search defs/ci_jobs, defs/release_jobs, globals/macros and +globals/templates in addition to the defs and globals trees. + +Excluding Paths +--------------- + +To allow a complex tree of jobs where some jobs are managed differently without +needing to explicitly provide each path, the recursive path processing supports +excluding paths based on absolute paths, relative paths and patterns. For +example:: + + For a tree: + /path/ + to/ + defs/ + ci_jobs/ + manual/ + release_jobs/ + manual/ + qa_jobs/ + globals/ + macros/ + templates/ + special/ + + jenkins-jobs update -r -x man*:./qa_jobs -x /path/to/defs/globals/special \ + /path/to/defs:/path/to/globals + +JJB will search the given paths, ignoring the directories qa_jobs, +ci_jobs/manual, release_jobs/manual, and globals/special when +building the list of yaml files to be processed. Absolute paths +are denoted by starting from the root, relative by containing +the path separator, and patterns by having neither. +Patterns use simple shell globing to match directories. + +Deleting Jobs +^^^^^^^^^^^^^ +Jenkins Job Builder supports deleting jobs from Jenkins. + +To delete a specific job:: + + jenkins-jobs delete Foo1 + +To delete a list of jobs, simply pass them as additional +arguments after the command:: + + jenkins-jobs delete Foo1 Foo2 + +The ``update`` command includes a ``delete-old`` option to remove obsolete +jobs:: + + jenkins-jobs update --delete-old /path/to/defs + +Obsolete jobs are *all* jobs not managed by JJB, even jobs which +were *never* managed by JJB. + +There is also a command to delete **all** jobs. +**WARNING**: Use with caution:: + + jenkins-jobs delete-all + +Globbed Parameters +^^^^^^^^^^^^^^^^^^ +Jenkins job builder supports globbed parameters to identify jobs from a set of +definition files. This feature only supports JJB managed jobs. + +To update jobs that only have 'foo' in their name:: + + jenkins-jobs update ./myjobs \*foo\* + +To delete jobs that only have 'foo' in their name:: + + jenkins-jobs delete --path ./myjobs \*foo\* + + +.. _command-reference: + +Command Reference +^^^^^^^^^^^^^^^^^ +.. program-output:: jenkins-jobs --help +.. program-output:: jenkins-jobs test --help +.. program-output:: jenkins-jobs update --help +.. program-output:: jenkins-jobs delete-all --help +.. program-output:: jenkins-jobs delete --help + +.. rubric:: Footnotes +.. [#f1] The cache default location is at ``~/.cache/jenkins_jobs``, which + can be overridden by setting the ``XDG_CACHE_HOME`` environment + variable. diff --git a/doc/source/index.rst b/doc/source/index.rst index f836c14a2..c0c5b8854 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -8,7 +8,9 @@ Contents .. toctree:: :maxdepth: 3 + quick-start installation + execution definition extending diff --git a/doc/source/installation.rst b/doc/source/installation.rst index 8cd3325c7..642578917 100644 --- a/doc/source/installation.rst +++ b/doc/source/installation.rst @@ -40,261 +40,3 @@ Test Coverage To measure test coverage, execute the command:: tox -e cover - -Configuration File ------------------- - -After installation, you will need to create a configuration file. By -default, ``jenkins-jobs`` looks in -``/etc/jenkins_jobs/jenkins_jobs.ini`` but you may specify an -alternative location when running ``jenkins-jobs``. The file should have -the following format: - -.. literalinclude:: ../../etc/jenkins_jobs.ini-sample - :language: ini - -job_builder section -^^^^^^^^^^^^^^^^^^^ - -**ignore_cache** - (Optional) If set to True, Jenkins Job Builder won't use any cache. - -**keep_descriptions** - By default `jenkins-jobs` will overwrite the jobs descriptions even if no - description has been defined explicitly. - When this option is set to True, that behavior changes and it will only - overwrite the description if you specified it in the yaml. False by default. - -**include_path** - (Optional) Can be set to a ':' delimited list of paths, which jenkins - job builder will search for any files specified by the custom application - yaml tags 'include', 'include-raw' and 'include-raw-escaped'. - -**recursive** - (Optional) If set to True, jenkins job builder will search for job - definition files recursively - -**exclude** - (Optional) If set to a list of values separated by ':', these paths will be - excluded from the list of paths to be processed when searching recursively. - Values containing no ``/`` will be matched against directory names at all - levels, those starting with ``/`` will be considered absolute, while others - containing a ``/`` somewhere other than the start of the value will be - considered relative to the starting path. - -**allow_duplicates** - (Optional) By default `jenkins-jobs` will abort any time a duplicate macro, - template, job-group or job name is encountered as it cannot establish the - correct one to use. When this option is set to True, only a warning is - emitted. - -**allow_empty_variables** - (Optional) When expanding strings, by default `jenkins-jobs` will raise an - exception if there's a key in the string, that has not been declared on the - yamls. Setting this options to True, will replace it with the empty string, - allowing you to use those strings without having to define all the keys it - might be using. - - -jenkins section -^^^^^^^^^^^^^^^ - -**user** - This should be the name of a user previously defined in Jenkins. - Appropriate user permissions must be set under the Jenkins security - matrix: under the ``Global`` group of permissions, check ``Read``, - then under the ``Job`` group of permissions, check ``Create``, - ``Delete``, ``Configure`` and finally ``Read``. - -**password** - The API token for the user specified. You can get this through the - Jenkins management interface under ``People`` -> username -> - ``Configure`` and then click the ``Show API Token`` button. - -**url** - The base URL for your Jenkins installation. - -**query_plugins_info** - Whether to query the Jenkins instance for plugin info when a configuration - file is provided. If a configuration file is not provided `jenkins-jobs` will - ignore this setting and skip querying for plugin information. True by - default. - - -hipchat section -^^^^^^^^^^^^^^^ - -**send-as** - This is the hipchat user name that will be used when sending notifications. - -**authtoken** - The API token necessary to send messages to hipchat. This can be generated in - the hipchat web interface by a user with administrative access for your - organization. This is the authtoken is set for each job individually; the - JJB Hipchat Plugin does not currently support setting different tokens for - different projects, so the token you use will have to be scoped such that it - can be used for any room your jobs might be configured to notify. For more - information on this topic, please see the `Hipchat API Documentation`__ - -__ https://www.hipchat.com/docs/apiv2/auth - - - -Running -------- - -After it's installed and configured, you can invoke Jenkins Job -Builder by running ``jenkins-jobs``. You won't be able to do anything -useful just yet without a configuration which is discussed in the next -section. - -Test Mode -^^^^^^^^^ -Once you have a configuration defined, you can run the job builder in test mode. - -If you want to run a simple test with just a single YAML job definition file -and see the XML output on stdout:: - - jenkins-jobs test /path/to/foo.yaml - -You can also pass JJB a directory containing multiple job definition files:: - - jenkins-jobs test /path/to/defs -o /path/to/output - -which will write XML files to the output directory for all of the jobs -defined in the defs directory. - -Updating Jobs -^^^^^^^^^^^^^ -When you're satisfied with the generated XML from the test, you can run:: - - jenkins-jobs update /path/to/defs - -which will upload the job definitions to Jenkins if needed. Jenkins Job -Builder maintains, for each host, a cache [#f1]_ of previously configured jobs, -so that you can run that command as often as you like, and it will only -update the jobs configurations in Jenkins if the defined definitions has -changed since the last time it was run. Note: if you modify a job -directly in Jenkins, jenkins-jobs will not know about it and will not -update it. - -To update a specific list of jobs, simply pass them as additional -arguments after the job definition path. To update Foo1 and Foo2 run:: - - jenkins-jobs update /path/to/defs Foo1 Foo2 - -Passing Multiple Paths -^^^^^^^^^^^^^^^^^^^^^^ -It is possible to pass multiple paths to JJB using colons as a path separator on -\*nix systems and semi-colons on Windows systems. For example:: - - jenkins-jobs test /path/to/global:/path/to/instance:/path/to/instance/project - -This helps when structuring directory layouts as you may selectively include -directories in different ways to suit different needs. If you maintain multiple -Jenkins instances suited to various needs you may want to share configuration -between those instances (global). Furthermore, there may be various ways you -would like to structure jobs within a given instance. - -Recursive Searching of Paths ----------------------------- - -In addition to passing multiple paths to JJB it is also possible to enable -recursive searching to process all yaml files in the tree beneath each path. -For example:: - - For a tree: - /path/ - to/ - defs/ - ci_jobs/ - release_jobs/ - globals/ - macros/ - templates/ - - jenkins-jobs update -r /path/to/defs:/path/to/globals - -JJB will search defs/ci_jobs, defs/release_jobs, globals/macros and -globals/templates in addition to the defs and globals trees. - -Excluding Paths ---------------- - -To allow a complex tree of jobs where some jobs are managed differently without -needing to explicitly provide each path, the recursive path processing supports -excluding paths based on absolute paths, relative paths and patterns. For -example:: - - For a tree: - /path/ - to/ - defs/ - ci_jobs/ - manual/ - release_jobs/ - manual/ - qa_jobs/ - globals/ - macros/ - templates/ - special/ - - jenkins-jobs update -r -x man*:./qa_jobs -x /path/to/defs/globals/special \ - /path/to/defs:/path/to/globals - -JJB search the given paths, ignoring the directories qa_jobs, ci_jobs/manual, -release_jobs/manual, and globals/special when building the list of yaml files -to be processed. Absolute paths are denoted by starting from the root, -relative by containing the path separator, and patterns by having neither. -Patterns use simple shell globing to match directories. - -Deleting Jobs -^^^^^^^^^^^^^ -Jenkins Job Builder supports deleting jobs from Jenkins. - -To delete a specific job:: - - jenkins-jobs delete Foo1 - -To delete a list of jobs, simply pass them as additional -arguments after the command:: - - jenkins-jobs delete Foo1 Foo2 - -The ``update`` command includes a ``delete-old`` option to remove obsolete -jobs. Obsolete jobs are *all* jobs not managed by JJB, even jobs which -were *never* managed by JJB:: - - jenkins-jobs update --delete-old /path/to/defs - -There is also a command to delete **all** jobs. -**WARNING**: Use with caution:: - - jenkins-jobs delete-all - -Globbed Parameters -^^^^^^^^^^^^^^^^^^ -Jenkins job builder supports globbed parameters to identify jobs from a set of -definition files. This feature only supports JJB managed jobs. - -To update jobs that only have 'foo' in their name:: - - jenkins-jobs update ./myjobs \*foo\* - -To delete jobs that only have 'foo' in their name:: - - jenkins-jobs delete --path ./myjobs \*foo\* - -Command Reference -^^^^^^^^^^^^^^^^^ -.. program-output:: jenkins-jobs --help -.. program-output:: jenkins-jobs test --help -.. program-output:: jenkins-jobs update --help -.. program-output:: jenkins-jobs delete-all --help -.. program-output:: jenkins-jobs delete --help - -.. rubric:: Footnotes -.. [#f1] The cache default location is at ``~/.cache/jenkins_jobs``, which - can be overridden by setting the ``XDG_CACHE_HOME`` environment - variable. diff --git a/doc/source/quick-start.rst b/doc/source/quick-start.rst new file mode 100644 index 000000000..69c8ea9fc --- /dev/null +++ b/doc/source/quick-start.rst @@ -0,0 +1,86 @@ +.. _quick-start-guide: + +Quick Start Guide +================= + +This guide was made with the impatient in mind so explanation is sparse. +It will guide users through a set of typical use cases for JJB using the same +job definitions we use to test JJB. + +#. Clone the repository_ to get the JJB job definition examples_ +#. The :doc:`installation` can be either from pypi_ (released version) or from the clone (master). + +Usage of the commands below assumes that you are at the root of the cloned directory. + +.. _repository: http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/ +.. _pypi: https://pypi.python.org/pypi/jenkins-job-builder/ +.. _examples: http://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/tests + + +.. _use-case-1: + +Use Case 1: Test a job definition +--------------------------------- + +JJB creates Jenkins XML configuration file from a YAML/JSON definition file and +just uploads it to Jenkins. JJB provides a convenient ``test`` command to allow +you to validate the XML before you attempt to upload it to Jenkins. + +Test a YAML job definition:: + + jenkins-jobs test tests/yamlparser/fixtures/templates002.yaml + + +The above command prints the generated Jenkins XML to the console. If you +prefer to send it to a directory:: + + jenkins-jobs test -o output tests/yamlparser/fixtures/templates002.yaml + + +The `output` directory will contain files with the XML configurations. + +.. _use-case-2: + +Use Case 2: Updating Jenkins Jobs +--------------------------------- + +Once you've tested your job definition and are happy with it then you can use the +``update`` command to deploy the job to Jenkins. The ``update`` command requires a +configuration file. An example file is supplied in the etc folder, you should +update it to match your Jenkins master:: + + jenkins-jobs --conf etc/jenkins_jobs.ini-sample update tests/yamlparser/fixtures/templates002.yaml + +The above command will update your Jenkins master with the generated jobs. + +**Caution**: JJB caches Jenkins job information locally. Changes +made using the Jenkins UI will not update that cache, which may +lead to confusion. See :ref:`updating-jobs` for more information. + +.. _use-case-3: + +Use Case 3: Working with JSON job definitions +--------------------------------------------- + +You can also define your jobs in json instead of yaml:: + + jenkins-jobs --config etc/jenkins_jobs.ini-sample update tests/jsonparser/fixtures/simple.json + +The above command just uses a simple job definition. You can also convert any +of the YAML examples to JSON and feed that to JJB. + +.. _use-case-4: + +Use Case 4: Deleting a job +-------------------------- + +To delete a job:: + + jenkins-jobs --config etc/jenkins_jobs.ini-sample delete simple + +The above command deletes the job `simple` from the Jenkins master. + + + +Please refer to the jenkins-jobs :ref:`command-reference` and the +:doc:`definition` pages for more details.