From 083cd44604005fefc03647f49d0c719857b76097 Mon Sep 17 00:00:00 2001
From: Iury Gregory Melo Ferreira <imelofer@redhat.com>
Date: Thu, 25 Jul 2019 18:14:06 +0200
Subject: [PATCH] CI Documentation

- Jobs description: Help people understand
what each job does in our CI.
- How to add a new Job
- How to initially debug failures in jobs.

Change-Id: I33b99bc7fc1033b693a3748322c1878e4c9231d7
---
 doc/source/contributor/adding-new-job.rst    |  73 ++++++++++++
 doc/source/contributor/debug-ci-failures.rst |  32 ++++++
 doc/source/contributor/index.rst             |  14 +++
 doc/source/contributor/jobs-description.rst  | 115 +++++++++++++++++++
 4 files changed, 234 insertions(+)
 create mode 100644 doc/source/contributor/adding-new-job.rst
 create mode 100644 doc/source/contributor/debug-ci-failures.rst
 create mode 100644 doc/source/contributor/jobs-description.rst

diff --git a/doc/source/contributor/adding-new-job.rst b/doc/source/contributor/adding-new-job.rst
new file mode 100644
index 0000000000..678c9b934c
--- /dev/null
+++ b/doc/source/contributor/adding-new-job.rst
@@ -0,0 +1,73 @@
+.. _adding-new-job:
+
+================
+Adding a new Job
+================
+
+Are you familiar with Zuul?
+===========================
+
+Before start trying to figure out how Zuul works, take some time and read
+about `Zuul Config <zuul_config_>`_ and the
+`Zuul Best Practices <zuul_best_practices_>`_.
+
+.. _zuul_config: https://zuul-ci.org/docs/zuul/user/config.html
+.. _zuul_best_practices: https://docs.openstack.org/infra/manual/creators.html#zuul-best-practices
+
+Where can I find the existing jobs?
+===================================
+
+The jobs for the Ironic project are defined under the zuul.d_ folder in the
+root directory, that contains three files, whose function is described
+below.
+
+* ironic-jobs.yaml_: Contains the configuration of each Ironic Job converted
+  to Zuul v3.
+
+* legacy-ironic-jobs.yaml_: Contains the configuration of each Ironic Job that
+  haven't been converted to Zuul v3 yet.
+
+* project.yaml_: Contains the jobs that will run during check and gate phase.
+
+
+.. _zuul.d: https://opendev.org/openstack/ironic/src/branch/master/zuul.d
+.. _ironic-jobs.yaml: https://opendev.org/openstack/ironic/src/branch/master/zuul.d/ironic-jobs.yaml
+.. _legacy-ironic-jobs.yaml: https://opendev.org/openstack/ironic/src/branch/master/zuul.d/legacy-ironic-jobs.yaml
+.. _project.yaml: https://opendev.org/openstack/ironic/src/branch/master/zuul.d/project.yaml
+
+
+Create a new Job
+================
+
+Identify among the existing jobs the one that most closely resembles the
+scenario you want to test, the existing job will be used as `parent` in your
+job definition.
+Now you will only need to either overwrite or add variables to your job
+definition under the `vars` section to represent the desired scenario.
+
+The code block below shows the minimal structure of a new job definition that
+you need to add to ironic-jobs.yaml_.
+
+.. code-block:: yaml
+
+   - job:
+       name: <name of the new job>
+       description: <what your job does>
+       parent: <Job that already exists>
+       vars:
+         <var1>: <new value>
+
+After having the definition of your new job you just need to add the job name
+to the project.yaml_ under `check` and `gate`. Only jobs that are voting
+should be in the `gate` section.
+
+.. code-block:: yaml
+
+   - project:
+       check:
+         jobs:
+           - <name of the new job>
+       gate:
+         queue: ironic
+         jobs:
+           - <name of the new job>
diff --git a/doc/source/contributor/debug-ci-failures.rst b/doc/source/contributor/debug-ci-failures.rst
new file mode 100644
index 0000000000..2770f9b427
--- /dev/null
+++ b/doc/source/contributor/debug-ci-failures.rst
@@ -0,0 +1,32 @@
+.. _debug-ci-failures:
+
+=====================
+Debugging CI failures
+=====================
+
+
+If you see `FAILURE` in one or more jobs for your patch please don't panic.
+This guide may help you to find the initial reason for the failure.
+When clicking in the failed job you will be redirect to a page that
+contains all the logs and configurations used to run the job.
+
+
+Using Ara Report
+================
+
+The `ara-report` folder will redirect you to a UI where you can see all the
+playbooks that were used to execute the job, and you will be able to find the
+playbook that failed. Click on the `Tasks` button for the playbook that failed
+and then click on the `Status` button for the task that has failed.
+
+You will be able to see what command was being executed and you can test
+locally to see if you can reproduce the failure locally.
+
+
+Looking at logs
+===============
+
+If you want to go more deep in your investigation you can look at the
+`job-output` file, it will give you an overall idea of the failures and you
+can identify services that may be involved. Under `controller/logs` you can
+find the the configuration and logs of those services.
diff --git a/doc/source/contributor/index.rst b/doc/source/contributor/index.rst
index 68646bec52..2a47b83363 100644
--- a/doc/source/contributor/index.rst
+++ b/doc/source/contributor/index.rst
@@ -99,3 +99,17 @@ Full Ironic Server Python API Reference
   :maxdepth: 1
 
   api/modules
+
+Understanding the Ironic's CI
+-----------------------------
+
+It's important to understand the role of each job in the CI, how to add new jobs
+and how to debug failures that may arise. To facilitate that, we have created
+the documentation below.
+
+.. toctree::
+  :maxdepth: 1
+
+  Job roles in the CI <jobs-description>
+  How to add a new job? <adding-new-job>
+  How to debug failures in CI jobs <debug-ci-failures>
diff --git a/doc/source/contributor/jobs-description.rst b/doc/source/contributor/jobs-description.rst
new file mode 100644
index 0000000000..88b2aec51e
--- /dev/null
+++ b/doc/source/contributor/jobs-description.rst
@@ -0,0 +1,115 @@
+.. _jobs-description:
+
+================
+Jobs description
+================
+
+The description of each jobs that runs in the CI when you submit a patch for
+`openstack/ironic` is visible in :ref:`table_jobs_description`.
+
+.. _table_jobs_description:
+
+.. list-table:: Table. OpenStack Ironic CI jobs description
+   :widths: 53 47
+   :header-rows: 1
+
+   * - Job name
+     - Description
+   * - ironic-tox-unit-with-driver-libs
+     - Runs Ironic unit tests with the driver dependencies installed under
+       Python2
+   * - ironic-tox-unit-with-driver-libs-python3
+     - Runs Ironic unit tests with the driver dependencies installed under
+       Python3
+   * - ironic-standalone
+     - Deploys Ironic in standalone mode and runs tempest tests that match
+       the regex `ironic_standalone`.
+   * - ironic-tempest-functional-python2
+     - Deploys Ironic in standalone mode and runs tempest functional tests
+       that matches the regex `ironic_tempest_plugin.tests.api` under Python2
+   * - ironic-tempest-functional-python3
+     - Deploys Ironic in standalone mode and runs tempest functional tests
+       that matches the regex `ironic_tempest_plugin.tests.api` under Python3
+   * - ironic-grenade-dsvm
+     - Deploys Ironic in a DevStack and runs upgrade for all enabled services.
+   * - ironic-grenade-dsvm-multinode-multitenant
+     - Deploys Ironic in a multinode DevStack and runs upgrade for all enabled
+       services.
+   * - ironic-tempest-ipa-partition-pxe_ipmitool-tinyipa-python3
+     - Deploys Ironic in DevStack under Python3, configured to use tinyipa
+       ramdisk partition image with `pxe` boot and `ipmi` driver.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario` and deploy 1 virtual baremetal.
+   * - ironic-tempest-ipa-partition-redfish-tinyipa
+     - Deploys Ironic in DevStack, configured to use tinyipa ramdisk partition
+       image with `pxe` boot and `redfish` driver.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario`, also deploys 1 virtual
+       baremetal.
+   * - ironic-tempest-ipa-partition-uefi-pxe_ipmitool-tinyipa
+     - Deploys Ironic in DevStack, configured to use tinyipa ramdisk partition
+       image with `uefi` boot and `ipmi` driver.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario`, also deploys 1 virtual
+       baremetal.
+   * - ironic-tempest-ipa-wholedisk-direct-tinyipa-multinode
+     - Deploys Ironic in a multinode DevStack, configured to use a pre-build
+       tinyipa ramdisk wholedisk image that is downloaded from a Swift
+       temporary url, `pxe` boot and `ipmi` driver.
+       Runs tempest tests that match the regex
+       `(ironic_tempest_plugin.tests.scenario|test_schedule_to_all_nodes)`
+       and deploys 7 virtual baremetal.
+   * - ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa
+     - Deploys Ironic in DevStack, configured to use a pre-build tinyipa
+       ramdisk wholedisk image that is downloaded from a Swift temporary url,
+       `pxe` boot and `ipmi` driver.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario` and deploys 1 virtual baremetal.
+   * - ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa-indirect
+     - Deploys Ironic in DevStack, configured to use a pre-build tinyipa
+       ramdisk wholedisk image that is downloaded from http url, `pxe` boot
+       and `ipmi` driver.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario` and deploys 1 virtual baremetal.
+   * - ironic-tempest-ipa-partition-bios-agent_ipmitool-tinyipa-indirect
+     - Deploys Ironic in DevStack, configured to use a pre-build tinyipa
+       ramdisk partition image that is downloaded from http url, `pxe` boot
+       and `ipmi` driver.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario` and deploys 1 virtual baremetal.
+   * - ironic-tempest-bfv
+     - Deploys Ironic in DevStack with cinder enabled, so it can deploy
+       baremetal using boot from volume.
+       Runs tempest tests that match the regex `baremetal_boot_from_volume`
+       and deploys 3 virtual baremetal nodes using boot from volume.
+   * - ironic-tempest-ipa-partition-uefi-pxe-grub2
+     - Deploys Ironic in DevStack, configured to use pxe with uefi and grub2
+       and `ipmi` driver.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario` and deploys 1 virtual baremetal.
+   * - ironic-tox-bandit
+     - Runs bandit security tests in a tox environment to find known issues in
+       the Ironic code.
+   * - ironic-tempest-ipa-wholedisk-bios-pxe_snmp-tinyipa
+     - Deploys Ironic in DevStack, configured to use a pre-build tinyipa
+       ramdisk wholedisk image that is downloaded from a Swift temporary url,
+       `pxe` boot and `snmp` driver.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario` and deploys 1 virtual baremetal.
+   * - ironic-inspector-tempest
+     - Deploys Ironic and Ironic Inspector in DevStack, configured to use a
+       pre-build tinyipa ramdisk wholedisk image that is downloaded from a
+       Swift temporary url, `pxe` boot and `ipmi` driver.
+       Runs tempest tests that match the regex `InspectorBasicTest` and
+       deploys 1 virtual baremetal.
+   * - bifrost-integration-tinyipa-ubuntu-xenial
+     - Tests the integration between Ironic and Bifrost.
+   * - metalsmith-integration-glance-localboot-centos7
+     - Tests the integration between Ironic and Metalsmith using Glance as
+       image source and CentOS7 with local boot.
+   * - ironic-tempest-pxe_ipmitool-postgres
+     - Deploys Ironic in DevStack, configured to use tinyipa ramdisk partition
+       image with `pxe` boot and `ipmi` driver and postgres instead of mysql.
+       Runs tempest tests that match the regex
+       `ironic_tempest_plugin.tests.scenario`, also deploys 1 virtual
+       baremetal.